kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Brown <jbr...@easypost.com>
Subject Re: leader election bug
Date Fri, 29 Apr 2016 21:35:01 GMT
What version of ZooKeeper are you on? There have been a few bugs over
the years where ZK has lost ephemeral nodes (and spontaneously
de-registered brokers).

On Fri, Apr 29, 2016 at 11:30 AM, Kane Kim <kane.isturm@gmail.com> wrote:
> Any idea why it's happening? I'm sure rolling restart would fix it. Is it a
> bug?
>
> On Wed, Apr 27, 2016 at 5:42 PM, Kane Kim <kane.isturm@gmail.com> wrote:
>
>> Hello,
>>
>> Looks like we are hitting leader election bug. I've stopped one broker
>> (104224873) on other brokers I see following:
>>
>> WARN  kafka.controller.ControllerChannelManager  - [Channel manager on
>> controller 104224863]: Not sending request Name: StopReplicaRequest;
>> Version: 0; CorrelationId: 843100; ClientId: ; DeletePartitions: false;
>> ControllerId: 104224863; ControllerEpoch: 8; Partitions: [mp-auth,169] to
>> broker 104224873, since it is offline.
>>
>> Also describing topics returns this:
>> Topic: mp-unknown Partition: 597 Leader: 104224873 Replicas:
>> 104224874,104224873,104224875 Isr: 104224873,104224875
>>
>> broker 104224873 is shut down, but it's still leader for the partition (at
>> least for a couple of hours as I monitor it).
>> Zookeeper cluster is healthy.
>>
>> ls /brokers/ids
>> [104224874, 104224875, 104224863, 104224864, 104224871, 104224867,
>> 104224868, 104224865, 104224866, 104224876, 104224877, 104224869,
>> 104224878, 104224879]
>>
>> That broker is not registered in ZK.
>>



-- 
James Brown
Engineer

Mime
View raw message