kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hogue <csho...@gmail.com>
Subject Re: [0.8] Can Kafka producer discover nodes from Zookeeper?
Date Fri, 02 Aug 2013 15:58:50 GMT
A couple of notes on name-spacing since there are a couple of gotchas when
trying to configure it:

It's specified via the zookeeper.connect property described here:

http://kafka.apache.org/08/configuration.html

If you have multiple servers in the zk cluster rather than a standalone
node the path is only specified once at the end, not on each node. So it
would look something like:

zookeeper.connect=myzk1.example.org:2181,myzk2.example.org:2181,
myzk3.example.org:2181/foo/bar

Also, the /foo/bar path must exist in the zk cluster before starting the
brokers or they will fail on startup. There is an error message but it's
not totally clear what the issue is. You can create the path via the
zookeeper shell if nothing else.

Hope that helps.

-Chris



On Fri, Aug 2, 2013 at 9:44 AM, Jun Rao <junrao@gmail.com> wrote:

> Kafka producers identify brokers registered in ZK through a getMetadata
> request to one of the brokers, not through ZK directly.
>
> Two Kafka clusters can share the same ZK cluster, as long as they use
> different ZK namespace.
>
> Thanks,
>
> Jun
>
>
> On Fri, Aug 2, 2013 at 8:23 AM, Haithem Jarraya <a-hjarraya@expedia.com
> >wrote:
>
> > Hi,
> >
> > I am new to Kafka, and I am a bit confused for this particular use case.
> > We are running a global Zookeeper cluster and I was wondering how I can
> > have my Kafka producer discover Kafka hostnames?
> > I saw that in the server.properties I can specify host.name and at that
> > point I will be able to see the list of hostnames in zookeeper
> > /brokers/ids/ so I can have my producer connect to zookeeper and resolve
> > Kafka hostnames.
> > Are there any other solutions?
> > Also, Can we have two separate Kafka Clusters using the same Zookeeper
> > cluster? Just to make sure that we do not collide with other teams. I
> guess
> > not given that the zookeeper path is hard coded. Am I wrong?
> >
> > Many Thanks,
> >
> > Haithem
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message