kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: Broker ID
Date Fri, 09 Dec 2011 18:05:30 GMT
Right but if this is just part of the node allocation script that would
work. E.g. if your allocate_node.sh script was something like:

cp default_server_config.properties server.properties
echo "node.id=" random_number() >> server.properties
host=allocate_ec2_node()
scp -r kafka host:
scp -r server.properties host:
ssh host kafka/bin/kafka-server-start.sh server.properties

Bouncing that node would retain the node id from then on but it initially
starts with a random id which avoids needing to maintain a counter to know
which ids are currently in use.

-Jay

On Fri, Dec 9, 2011 at 9:55 AM, Neha Narkhede <neha.narkhede@gmail.com>wrote:

> Florian,
>
> >> When using Zookeeper, couldn't ZK just give each broker if registered as
> a
> SEQ node a generated ID that's guaranteed to be unique? Otherwise can this
> ID be a randomly generated ID?
>
> If you do that, then what happens when you merely bounce the broker ? It
> might end up getting a different id each time, which will break consumer
> logic.
>
> The purpose of a unique broker id is to be able to identify a Kafka broker,
> without having to know its host-name/IP. This allows you to switch the
> machine hosting a broker, without changing any client config.
>
> Thanks,
> Neha
>
> On Fri, Dec 9, 2011 at 9:37 AM, Jun Rao <junrao@gmail.com> wrote:
>
> > Florian,
> >
> > The broker id uniquely identifies a broker. Every time a broker is
> > restarted, we want the broker to come up with the same id. The consumer
> > relies on the broker id to resume consumption properly.
> >
> > Thanks,
> >
> > Jun
> >
> > On Fri, Dec 9, 2011 at 8:34 AM, Florian Leibert <flo@leibert.de> wrote:
> >
> > > Hi -
> > > I see there's a static broker-id in the properties files. Is this
> > optional?
> > > When using Zookeeper, couldn't ZK just give each broker if registered
> as
> > a
> > > SEQ node a generated ID that's guaranteed to be unique? Otherwise can
> > this
> > > ID be a randomly generated ID? Basically I'd like to bring up instances
> > in
> > > EC2 without having to worry about the state of other brokers...
> > >
> > > Thanks!
> > > Flo
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message