kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Haithem Jarraya <a-hjarr...@expedia.com>
Subject Re: [0.8] Can Kafka producer discover nodes from Zookeeper?
Date Fri, 02 Aug 2013 16:23:59 GMT
Thanks a lot Chris, you saved us a lot of time with this.

-Haithem
On 2 Aug 2013, at 16:58, Chris Hogue <cshogue@gmail.com> wrote:

> A couple of notes on name-spacing since there are a couple of gotchas when
> trying to configure it:
> 
> It's specified via the zookeeper.connect property described here:
> 
> http://kafka.apache.org/08/configuration.html
> 
> If you have multiple servers in the zk cluster rather than a standalone
> node the path is only specified once at the end, not on each node. So it
> would look something like:
> 
> zookeeper.connect=myzk1.example.org:2181,myzk2.example.org:2181,
> myzk3.example.org:2181/foo/bar
> 
> Also, the /foo/bar path must exist in the zk cluster before starting the
> brokers or they will fail on startup. There is an error message but it's
> not totally clear what the issue is. You can create the path via the
> zookeeper shell if nothing else.
> 
> Hope that helps.
> 
> -Chris
> 
> 
> 
> On Fri, Aug 2, 2013 at 9:44 AM, Jun Rao <junrao@gmail.com> wrote:
> 
>> Kafka producers identify brokers registered in ZK through a getMetadata
>> request to one of the brokers, not through ZK directly.
>> 
>> Two Kafka clusters can share the same ZK cluster, as long as they use
>> different ZK namespace.
>> 
>> Thanks,
>> 
>> Jun
>> 
>> 
>> On Fri, Aug 2, 2013 at 8:23 AM, Haithem Jarraya <a-hjarraya@expedia.com
>>> wrote:
>> 
>>> Hi,
>>> 
>>> I am new to Kafka, and I am a bit confused for this particular use case.
>>> We are running a global Zookeeper cluster and I was wondering how I can
>>> have my Kafka producer discover Kafka hostnames?
>>> I saw that in the server.properties I can specify host.name and at that
>>> point I will be able to see the list of hostnames in zookeeper
>>> /brokers/ids/ so I can have my producer connect to zookeeper and resolve
>>> Kafka hostnames.
>>> Are there any other solutions?
>>> Also, Can we have two separate Kafka Clusters using the same Zookeeper
>>> cluster? Just to make sure that we do not collide with other teams. I
>> guess
>>> not given that the zookeeper path is hard coded. Am I wrong?
>>> 
>>> Many Thanks,
>>> 
>>> Haithem
>> 


Mime
View raw message