kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Viktor Kolodrevskiy <viktor.kolodrevs...@gmail.com>
Subject Re: Error handling in async Producer
Date Wed, 12 Mar 2014 15:18:35 GMT
I had similar questions some time ago.

Here's answer I've got:

> In Kafka, we detect failures using ZK. So, if the network connectivity btw
> the producer and the broker is down, but the one btw the broker and ZK is
> up, we assume the broker is still alive and will continue to send the data
> to it. Within the the same data center, we assume this is extremely rare.
> If the network connectivity btw the broker and ZK is also down, the
> producer will be able to automatically detect the failure and send data to
> other brokers.

> In 0.7, there is no way to check whether a message is really sent or not,
> since the producer requests don't receive any acknowledgement. This is
> changed in 0.8, where a producer can choose when to receive an ack (see
> request.required.acks in
> http://kafka.apache.org/documentation.html#producerconfigs).
> Thanks,
> Jun

2014-03-12 15:52 GMT+02:00 Yury Ruchin <yuri.ruchin@gmail.com>:
> Hi,
> Having Kafka 0.8, I send messages using Producer in async mode. I wonder
> what will happen if a message cannot be sent (e. g. all brokers get down).
> In sync mode, error handling is straightforward: after
> "message.send.max.retries" the send() method will throw
> FailedToSendMessageException. However, in async mode send() does not
> actually check whether message has been successfully sent. It just enqueues
> messages to be processed by ProducerSendThread in background. As I
> currently understand, possible errors will not be exposed to the sender
> then. Am I missing something? Is there any way to set error handler for
> async Producer?


View raw message