kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srinivas Rapolu <cnu.t...@gmail.com>
Subject Re: Kafka running on AWS - how to retain broker.id on new instance spun-up in-place of instance/broker failed
Date Wed, 14 Nov 2018 18:57:42 GMT
thanks Naresh for the quick response. But we don't want to make use of any
elastic IP in this case.

I found something like, we can manually get the broker-id using some script
mentioned  @ http://tech.gc.com/scaling-with-kafka/ while instance is
getting launched.

Trying to find, if there is any other/better way to
re-assingn/retain/impersonate the old broker.id.

Thanks and Regards,
Srinivas

On Wed, Nov 14, 2018 at 11:48 AM naresh Goud <nareshgoud.dulam@gmail.com>
wrote:

> Static IP. Buying static IP may help. I am not aws expert
>
> On Wed, Nov 14, 2018 at 12:47 PM Srinivas Rapolu <cnu.text@gmail.com>
> wrote:
>
> > Hello Kafka experts,
> >
> > We are running Kafka on AWS, main question is what is the best way to
> > retain broker.id on new instance spun-up in-place of instance/broker
> > failed.
> >
> > We are currently running Kafka in AWS with broker.id gets auto
> generated.
> > But we are having issues when a broker is failed, new broker/instance
> > spun-up in AWS get assigned with new broker.id. The issue is, with this
> > approach, we need to re-assign the topics/replications on to the new
> broker
> > manually.
> >
> > We learned that, replication can be auto resolved by Kafka, if we can
> > manage to get the same broker.id on the new AWS instance spun-up
> in-place
> > of failed broker/instance.
> >
> > I have read, we can set broker.id.generation.enable= false, but what is
> the
> > best way to identify and retain the broker.id? Any links/help is
> > appreciated.
> > Thanks and Regards,
> > Cnu
> >
> --
> Thanks,
> Naresh
> www.linkedin.com/in/naresh-dulam
> http://hadoopandspark.blogspot.com/
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message