kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Davison <ben.davi...@7digital.com>
Subject Re: Topic description by zookeeper after a kafka server dies
Date Tue, 10 May 2016 13:37:52 GMT
Hi Paolo,

We just jump on the box (don't use kuberenetes) and change the
metadata.properties file manually, then restart. We are only doing a small
amount of non prod traffic so it's easy for us to manage.

Thanks,

Ben

On Tuesday, 10 May 2016, Paolo Patierno <ppatierno@live.com> wrote:

> Hi Ben,
>
> in order to avoid conflicts I have auto generation for broker id in the
> server.properties file.
> Of course it's a workaround but I'm so curious how do you tell Kubernetes
> to start a new Kafka pod with a previous broker id ?
>
> Btw, waiting for an answer from the Kafka team.
>
> Paolo.
>
> Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat
> Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor
> Twitter : @ppatierno
> Linkedin : paolopatierno
> Blog : DevExperience
>
> > Date: Tue, 10 May 2016 22:27:13 +0900
> > Subject: Re: Topic description by zookeeper after a kafka server dies
> > From: ben.davison@7digital.com <javascript:;>
> > To: users@kafka.apache.org <javascript:;>
> >
> > Hi Paolo,
> >
> > Would be interested to hear the answer also. We've been getting around
> this
> > ourselves by setting the new broker that comes up back to id 1001.
> >
> > Thanks,
> >
> > Ben
> >
> > On Tuesday, 10 May 2016, Paolo Patierno <ppatierno@live.com
> <javascript:;>> wrote:
> >
> > > Hello,
> > >
> > > I'm experiencing the usage of Kafka with Kubernetes and I don't
> understand
> > > the reason of the following behavior.
> > >
> > > Starting with a pod with zookeeper and a pod with a kafka server (id =
> > > 1001). Create a topic named "test" ... producer and consumer can
> > > send/receive message without problems.
> > > I turn off the pod with kafka server and Kubernetes restarts another
> pod
> > > with a new kafka server (id = 1002).
> > > Now ... I know that the topic "test" is lost because the related log
> was
> > > in the previous kafka server (id = 1001).
> > > Anyway, the zookeeper server still have information about topic "test"
> and
> > > if I use "describe" command it says me that the leader is still the
> 1001
> > > server !!
> > >
> > > I tried the script for reassign replicas and the preferred replica
> > > election : now zookeeper shows me that the 1002 server is in the
> replicas
> > > list but the leader is still the 1001 and the only one in the ISR list.
> > > Why zookeeper doesn't know that kafka server 1001 is dead ??
> > >
> > > Thanks,
> > > Paolo.
> > >
> > > Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat
> > > Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor
> > > Twitter : @ppatierno
> > > Linkedin : paolopatierno
> > > Blog : DevExperience
> >
> > --
> >
> >
> > This email, including attachments, is private and confidential. If you
> have
> > received this email in error please notify the sender and delete it from
> > your system. Emails are not secure and may contain viruses. No liability
> > can be accepted for viruses that might be transferred by this email or
> any
> > attachment. Any unauthorised copying of this message or unauthorised
> > distribution and publication of the information contained herein are
> > prohibited.
> >
> > 7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
> > Registered in England and Wales. Registered No. 04843573.
>

-- 


This email, including attachments, is private and confidential. If you have 
received this email in error please notify the sender and delete it from 
your system. Emails are not secure and may contain viruses. No liability 
can be accepted for viruses that might be transferred by this email or any 
attachment. Any unauthorised copying of this message or unauthorised 
distribution and publication of the information contained herein are 
prohibited.

7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
Registered in England and Wales. Registered No. 04843573.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message