kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiaoyu Wang <xw...@rocketfuel.com>
Subject Re: Producer can writes to a follower during preferred lead election?
Date Mon, 08 Dec 2014 21:54:46 GMT
Guozhang, thanks for the super clear answer. Yes, that's the scenario I was
wondering.

On Mon, Dec 8, 2014 at 1:01 PM, Guozhang Wang <wangguoz@gmail.com> wrote:

> Xiaoyu,
>
> I think your question is whether the following sequence can happen?
>
> 1. A received becomes-leader, stop fetching from B.
> 2. B received a producer request, and accepts it and append to log.
> 3. B received becomes-follower, cap its log and start fetching from A, and
> hence the message sent on step 2) will be lost even the producer gets acked
> from B.
>
> Since controller sends the leaderAndIsr request to all the replicas (both
> leader and followers) at the same time this is possible to happen with ack
> <=1. With ack > 1 since the produce request will not be returned to the
> producer until it has been replicated the above scenario will not happen.
>
> Guozhang
>
> On Mon, Dec 8, 2014 at 11:29 AM, Gwen Shapira <gshapira@cloudera.com>
> wrote:
>
> > I think that A will not be able to become a follower until B becomes a
> > leader.
> >
> > On Sun, Dec 7, 2014 at 11:07 AM, Xiaoyu Wang <xwang@rocketfuel.com>
> wrote:
> > > On preferred replica election, controller sends LeaderAndIsr requests
> to
> > > brokers. Broker will handle the LeaderAndIsr request by either become a
> > > leader or become a follower.
> > >
> > > In the previous case, when A receive the call, it will try to become
> the
> > > leader and stop fetching from B; when B receive the call, it will try
> to
> > > become a follower and stop receiving new requests. Is it possible that
> A
> > > stops fetching before B stops receiving new requests? If this is
> > possible,
> > > there still may be messages goes to B but not A, right?
> > >
> > > On Sun, Dec 7, 2014 at 7:20 AM, Thunder Stumpges <tstumpges@ntent.com>
> > > wrote:
> > >
> > >> In this case B will return "not leader for partition" error as soon as
> > the
> > >> leader is re-elected and I imagine the producer will correct itself.
> > >>
> > >> -Thunder
> > >>
> > >>
> > >> -----Original Message-----
> > >> From: Xiaoyu Wang [xwang@rocketfuel.com]
> > >> Received: Saturday, 06 Dec 2014, 6:49PM
> > >> To: users@kafka.apache.org [users@kafka.apache.org]
> > >> Subject: Producer can writes to a follower during preferred lead
> > election?
> > >>
> > >> Hello,
> > >>
> > >> I am looking at producer code and found that producer updates its
> > >> broker/partition info under the two conditions
> > >>
> > >>    1. has reached the topicMetadataRefreshInterval
> > >>    2. failed sending message, before retry
> > >>
> > >> So, assume we have broker A and B, B is the current lead and A is the
> > >> preferred lead and a producer is publishing to B. If someone execute
> > >> preferred lead election command now, A will become the new lead and
> the
> > >> producer won't know the lead is now A and will still writes to B until
> > the
> > >> metadata refresh interval has been reached. Is this correct? Or did I
> > did
> > >> miss anything.
> > >>
> > >>
> > >> Thanks.
> > >>
> >
>
>
>
> --
> -- Guozhang
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message