kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: question about synchronous producer
Date Fri, 30 May 2014 15:02:24 GMT
I should mention that before each retry the producer will try to refresh
its metadata, so with replication factor >1 and when leader of the
partition has failed, it may find a different leader next time since it is
migrated by the controller, and retry will send to a different broker.

Guozhang


On Fri, May 30, 2014 at 8:00 AM, Guozhang Wang <wangguoz@gmail.com> wrote:

> That is correct.
>
>
> On Fri, May 30, 2014 at 6:20 AM, Libo Yu <yu_libo@hotmail.com> wrote:
>
>> Digged into kafka code and did some tests. Let me try to answer it by
>> myself. Please correct me if I am wrong.
>> While send() is being called, if brokers are down, send will throw
>> FailedToSendMessageException. Users can catch it and retry.
>>
>> > From: yu_libo@hotmail.com
>> > To: users@kafka.apache.org
>> > Subject: RE: question about synchronous producer
>> > Date: Fri, 30 May 2014 08:32:44 -0400
>> >
>> > Thanks for your help, Guozhen.
>> > Assume request.required.acks is set to 1 for the scenario I mentioned.
>>   While send() is being called, brokers are shutdown. What should I expect?
>> > 1 send() will retry 3 times with a 100ms interval and return.
>> > 2 send will not return until the brokers are back online.
>> >
>> > Thanks,
>> >
>> > Libo
>> >
>> > > Date: Thu, 29 May 2014 20:11:48 -0700
>> > > Subject: Re: question about synchronous producer
>> > > From: wangguoz@gmail.com
>> > > To: users@kafka.apache.org
>> > >
>> > > Libo,
>> > >
>> > > That is correct. You may want to increase the retry.backoff.ms in
>> this
>> > > case. In practice, if the brokers are down for too long, then data
>> loss is
>> > > usually inevitable.
>> > >
>> > > Guozhang
>> > >
>> > >
>> > > On Thu, May 29, 2014 at 2:55 PM, Libo Yu <yu_libo@hotmail.com> wrote:
>> > >
>> > > > Hi team,
>> > > >
>> > > > Assume I am using a synchronous producer and it has the following
>> default
>> > > > properties:
>> > > >
>> > > > message.send.max.retries
>> > > >       3
>> > > > retry.backoff.ms
>> > > >       100
>> > > >
>> > > > I use java api Producer.send(message) to send a message.
>> > > > While send() is being called, if the brokers are shutdown, what
>> happens?
>> > > > send() will retry 3 times with a 100ms interval and fail silently?
>> > > > If I don't want to lose any message when the brokers are back
>> online, what
>> > > > should I do? Thanks.
>> > > >
>> > > > Libo
>> > > >
>> > > >
>> > > >
>> > > >
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > > -- Guozhang
>> >
>>
>>
>
>
>
> --
> -- Guozhang
>



-- 
-- Guozhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message