kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Kafka consumer not reconnecting to restarted Kafka node.
Date Tue, 25 Oct 2011 19:22:44 GMT
Dan,

My understanding is that ZK 3.4 will be released soon, which will fix the
chroot issue. Will that fix your problem?

Thanks,

Jun

On Tue, Oct 25, 2011 at 11:52 AM, Dan Brown <dan@metamx.com> wrote:

> So would you all accept a patch that exposes the zk paths in the config?
>
>  Dan
>
> On Wed, Oct 19, 2011 at 14:20, Dan Brown <dan@metamx.com> wrote:
> > No, configurable kafka paths would be subsumed by a properly
> > functioning zk chroot. I can't imagine a situation where the brokers
> > and consumers could be safely decoupled.
> >
> >  Dan
> >
> > On Wed, Oct 19, 2011 at 13:19, Jay Kreps <jay.kreps@gmail.com> wrote:
> >> Adding a manual override path in the config seems like an acceptable
> thing
> >> to do, but it is not clear we will get 0.8 out before zookeeper gets
> their
> >> next release out. Would this have any use other than working around this
> zk
> >> problem?
> >>
> >> -Jay
> >>
> >> On Wed, Oct 19, 2011 at 1:02 PM, Mathias Herberts <
> >> mathias.herberts@gmail.com> wrote:
> >>
> >>> Not using a chroot is not a feasible thing when the ZK ensemble is
> >>> shared by several Kafka clusters.
> >>>
> >>> On Wed, Oct 19, 2011 at 21:54, Jun Rao <junrao@gmail.com> wrote:
> >>> > This particular ZK issue only exposes if the ZK connect string uses
> >>> chroot
> >>> > and there is a ZK disconnect. In the near term, you can choose not
to
> use
> >>> > chroot in your ZK connect string.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Jun
> >>> >
> >>> > On Wed, Oct 19, 2011 at 12:30 PM, Dan Brown <dan@metamx.com>
wrote:
> >>> >
> >>> >> On Tue, Oct 18, 2011 at 15:02, Mathias Herberts
> >>> >> <mathias.herberts@gmail.com> wrote:
> >>> >> >> I think Jun or Neha had tracked this one down at LinkedIn
and it
> is
> >>> >> actually
> >>> >> >> a zk bug. I think it is this one, but they could confirm:
> >>> >> >>
> >>> >> >> https://issues.apache.org/jira/browse/ZOOKEEPER-961
> >>> >> >
> >>> >> > Sure looks like it.
> >>> >>
> >>> >> Confirmed. Running the above repro against zookeeper trunk produces
> a
> >>> >> responsive consumer that successfully rebalances when the broker
> >>> >> rejoins.
> >>> >>
> >>> >> > So besides waiting for ZK 3.3.4 there is not much that can
be done
> >>> >> > since Kafka does not offer a way to change the root znode
of a
> >>> >> > cluster.
> >>> >>
> >>> >> Yeah, it's unclear that zk-3.3.4 will release anytime soon. Instead
> of
> >>> >> relying on zk chroot, why not just expose the zk paths as config
> >>> >> parameters for consumers, brokers, and producers?
> >>> >>
> >>> >> object ZkUtils {
> >>> >>  val ConsumersPath = "/consumers"
> >>> >>  val BrokerIdsPath = "/brokers/ids"
> >>> >>  val BrokerTopicsPath = "/brokers/topics"
> >>> >>  ...
> >>> >>
> >>> >
> >>>
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message