samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Navina Ramesh <nram...@linkedin.com>
Subject Re: Review Request 45258: Abandon producer retry after a certain # of errors : SAMZA-911
Date Fri, 25 Mar 2016 22:44:37 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45258/#review125483
-----------------------------------------------------------


Ship it!




lgtm! +1 
On a side note, can you please document the existing race conditions with the producer in
a separate JIRA? You can assign that to me. Thanks!

- Navina Ramesh


On March 24, 2016, 1:10 a.m., Jagadish Venkatraman wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45258/
> -----------------------------------------------------------
> 
> (Updated March 24, 2016, 1:10 a.m.)
> 
> 
> Review request for samza, Jake Maes, Navina Ramesh, and Yi Pan (Data Infrastructure).
> 
> 
> Repository: samza
> 
> 
> Description
> -------
> 
> Currently, the KafkaSystemProducer's producer loop keeps retrying indefinitely when there
is an exception in the retryBackOff loop. If there are repeated exceptions, then it makes
sense to retry for awhile, and then fail the container.
> 
> Long term, we should focus on getting rid off the retryBackOff loop, and close the producer
object in the callback during failure. This will guarantee in-order delivery.
> 
> 1.Modified the KafkaSystemProducer to take a maxRetries. (currently, its set to 30).
> 2.Add tests to verify retry in case of RetriableExceptions.
> 
> 
> Diffs
> -----
> 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemProducer.scala
9a44d46d29a1997958a9d2bbf7be0bde860fff64 
>   samza-kafka/src/test/scala/org/apache/samza/system/kafka/TestKafkaSystemProducer.scala
39426d8cf64516ec4fdc0cb4ff60b1df3a757470 
> 
> Diff: https://reviews.apache.org/r/45258/diff/
> 
> 
> Testing
> -------
> 
> Added unit tests to verify functionality.
> 
> 
> Thanks,
> 
> Jagadish Venkatraman
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message