kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jiangjie Qin <j...@linkedin.com.INVALID>
Subject Re: New producer very slow to call Callback on error
Date Mon, 08 Jun 2015 17:48:36 GMT
KIP-19 should address this issue.

https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+ti
meout+to+NetworkClient

Thanks,

Jiangjie (Becket) Qin

On 6/8/15, 10:44 AM, "Christofer Hedbrandh" <christofer@knewton.com> wrote:

>I think this question might relate to the very recently posted "callback
>handler is not getting called if cluster is down" topic from "ankit
>tyagi".
>
>I am using the 0.8.2.1 new producer send(ProducerRecord<K,V> record,
>Callback callback) with a Callback and never calling .get() on the
>Future<RecordMetadata>. I have noticed that it takes ~10-15 minutes before
>Callback.onCompletion() is called with an Exception when I take down the
>Kafka cluster or block connections.
>
>From the documentation it is unclear what the expected producer behavior
>should be. I had assumed that the configuration "retries" and "timeout.ms"
>should control how long it would take for the Callback to be called with
>an
>Exception, but that does not seem to be the case.
>
>How do I know how long it will take for the Callback to be called with an
>Exception. And can I somehow configure this to be much shorter than ~10-15
>minutes.
>
>relevant broker configs:
>zookeeper.session.timeout.ms: 6000
>zookeeper.connection.timeout.m: 1000000
>
>relevant producer configs:
>retries: 0
>linger.ms: 0
>timeout.ms: 30000
>metadata.fetch.timeout.ms: 10000
>metadata.max.age.ms: 10000
>reconnect.backoff.ms: 10
>retry.backoff.ms: 100
>
>
>Thanks,
>Christofer


Mime
View raw message