kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Evan Chan ...@ooyala.com>
Subject Re: Consumer re-design proposal
Date Wed, 20 Jun 2012 16:36:14 GMT
How would you accomplish automatic rebalancing without ZK?

On Wed, Jun 20, 2012 at 8:25 AM, Taylor Gautier <tgautier@tagged.com> wrote:

> Fwiw I think this is the right move. We don't use ZK in our Kafka
> installation.
>
> One reason is that we do not want or need the complexity of having the
> broker distribution in the clients.
>
> Moving towards this design in the core would help our installation be
> more "standard".
>
>
>
> On Jun 20, 2012, at 7:48 AM, Jun Rao <junrao@gmail.com> wrote:
>
> > Dave,
> >
> > Just to clarify. We are not removing ZK completely. The proposal is just
> > trying to see if we can remove the ZK dependency on the consumer client.
> ZK
> > will still be used at the broker.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Jun 19, 2012 at 10:49 PM, Dave Barr <dave.barr@gmail.com> wrote:
> >
> >> On Tue, Jun 19, 2012 at 10:09 AM, Neha Narkhede <
> neha.narkhede@gmail.com>
> >> wrote:
> >>> One of the goals of thinning the Kafka consumer client is removing the
> >>> zookeeper client from the consumer. Without this, Kafka consumer
> >>> client would depend on the stability of a zookeeper client.
> >>
> >> If there's a stability issue with the zookeeper client, then that
> >> should be addressed.
> >>
> >> ZK is a fine tool for service discovery and coordination.  It seems
> >> like any new system that forced me, as a consumer, to use yet another
> >> system to bootstrap and discover where my brokers are for a topic
> >> would be a step backward.
> >>
> >> I'm curious, why, specifically, is removing ZK a design goal
> >> (especially when it's such a core component of the broker)?  I think
> >> of other projects, like HBase, which seem to have no issue with using
> >> ZK in their client.
> >>
> >> --Dave
> >>
>



-- 
--
*Evan Chan*
Senior Software Engineer |
ev@ooyala.com | (650) 996-4600
www.ooyala.com | blog <http://www.ooyala.com/blog> |
@ooyala<http://www.twitter.com/ooyala>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message