kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Evan Chan ...@ooyala.com>
Subject Re: How to configure Kafka to use a running zookeeper instance?
Date Tue, 18 Sep 2012 20:56:04 GMT
So everything in ZooKeeper are nodes stored in a hierarchical, file-system
like path, right?

For example, everything in Kafka would be stored by default in paths like
this:

/kafka/brokers/id/NNN/....
/kafka/consumers/....

Everything in Storm, on the other hand, is stored by default in paths like
this:

/storm/.....

You can configure both Storm and Kafka to add ZooKeeper prefixes to the ZK
path, like this:

(for example if you add a prefix in the kafka server.properties):

/<prefixpath>/kafka/brokers/....

But even without any prefixes, if you only have one cluster of Kafkas and
one Storm cluster using that ZK, they won't share the same path and would
be OK.

One reason to use prefixes is if your ZK is shared amongst multiple Storm
topologies or multiple Kafka clusters.   For example, we have multiple
Storm clusters sharing the same ZK cluster, so we prefix by Storm cluster
name.

-Evan


On Tue, Sep 18, 2012 at 11:43 AM, Hussein Baghdadi
<hubaghdadi@hotmail.com>wrote:

>  Would you please post a sample of a different two namespaces for Storm
> and Kafka?
> Sorry but I'm still learning Kafka.
> Thanks all for help and time.
>
> > Date: Tue, 18 Sep 2012 10:21:53 -0700
> > Subject: Re: How to configure Kafka to use a running zookeeper instance?
> > From: ev@ooyala.com
> > To: kafka-users@incubator.apache.org
>
> >
> > No need, actually. Storm and kafka already use different paths (we do
> this
> > in our ZK setup).
> >
> > Of course it doesn't hurt to namespace things properly.
> >
> > On Tue, Sep 18, 2012 at 10:19 AM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
> >
> > > If you want to use the same zookeeper cluster for both Storm and
> > > kafka, you can use different namespaces in the zk.connect url.
> > >
> > > Thanks,
> > > Neha
> > >
> > > On Tue, Sep 18, 2012 at 6:47 AM, Evan chan <ev@ooyala.com> wrote:
> > > > Yes Hussein, you can definitely point Kafka at the same instance as
> > > Storm using ZK.connect
> > > >
> > > > -Evan
> > > > Carry your candle, run to the darkness
> > > > Seek out the helpless, deceived and poor
> > > > Hold out your candle for all to see it
> > > > Take your candle, and go light your world
> > > >
> > > >
> > > >
> > > > On Sep 18, 2012, at 3:52 AM, Hussein Baghdadi <
> hubaghdadi@hotmail.com>
> > > wrote:
> > > >
> > > >>
> > > >> Hi,
> > > >> I'm using Storm (https://github.com/nathanmarz/storm) that is
> already
> > > requires zookeeper instance to be running.Kafka also needs an instance
> of
> > > zookeeper.
> > > >> Is it possible to configure Kafka to make use of the zookeeper
> needed
> > > by Storm?No need to run two zookeepers, I think.
> > > >> Thanks.
> > >
> >
> >
> >
> > --
> > --
> > *Evan Chan*
> > Senior Software Engineer |
> > ev@ooyala.com | (650) 996-4600
> > www.ooyala.com | blog <http://www.ooyala.com/blog> |
> > @ooyala<http://www.twitter.com/ooyala>
>



-- 
--
*Evan Chan*
Senior Software Engineer |
ev@ooyala.com | (650) 996-4600
www.ooyala.com | blog <http://www.ooyala.com/blog> |
@ooyala<http://www.twitter.com/ooyala>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message