kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neha Narkhede <neha.narkh...@gmail.com>
Subject Re: Kafka consumer not reconnecting to restarted Kafka node.
Date Tue, 25 Oct 2011 22:33:39 GMT
>> My understanding is that ZK 3.4 will be released soon, which will fix the
chroot issue.

We are meeting with them tomorrow. I'm hoping we can get a precise
date for the 3.3.4 release from ZK team.

Thanks,
Neha


On Tue, Oct 25, 2011 at 12:22 PM, Jun Rao <junrao@gmail.com> wrote:
> Dan,
>
> My understanding is that ZK 3.4 will be released soon, which will fix the
> chroot issue. Will that fix your problem?
>
> Thanks,
>
> Jun
>
> On Tue, Oct 25, 2011 at 11:52 AM, Dan Brown <dan@metamx.com> wrote:
>
>> So would you all accept a patch that exposes the zk paths in the config?
>>
>>  Dan
>>
>> On Wed, Oct 19, 2011 at 14:20, Dan Brown <dan@metamx.com> wrote:
>> > No, configurable kafka paths would be subsumed by a properly
>> > functioning zk chroot. I can't imagine a situation where the brokers
>> > and consumers could be safely decoupled.
>> >
>> >  Dan
>> >
>> > On Wed, Oct 19, 2011 at 13:19, Jay Kreps <jay.kreps@gmail.com> wrote:
>> >> Adding a manual override path in the config seems like an acceptable
>> thing
>> >> to do, but it is not clear we will get 0.8 out before zookeeper gets
>> their
>> >> next release out. Would this have any use other than working around this
>> zk
>> >> problem?
>> >>
>> >> -Jay
>> >>
>> >> On Wed, Oct 19, 2011 at 1:02 PM, Mathias Herberts <
>> >> mathias.herberts@gmail.com> wrote:
>> >>
>> >>> Not using a chroot is not a feasible thing when the ZK ensemble is
>> >>> shared by several Kafka clusters.
>> >>>
>> >>> On Wed, Oct 19, 2011 at 21:54, Jun Rao <junrao@gmail.com> wrote:
>> >>> > This particular ZK issue only exposes if the ZK connect string
uses
>> >>> chroot
>> >>> > and there is a ZK disconnect. In the near term, you can choose
not to
>> use
>> >>> > chroot in your ZK connect string.
>> >>> >
>> >>> > Thanks,
>> >>> >
>> >>> > Jun
>> >>> >
>> >>> > On Wed, Oct 19, 2011 at 12:30 PM, Dan Brown <dan@metamx.com>
wrote:
>> >>> >
>> >>> >> On Tue, Oct 18, 2011 at 15:02, Mathias Herberts
>> >>> >> <mathias.herberts@gmail.com> wrote:
>> >>> >> >> I think Jun or Neha had tracked this one down at LinkedIn
and it
>> is
>> >>> >> actually
>> >>> >> >> a zk bug. I think it is this one, but they could confirm:
>> >>> >> >>
>> >>> >> >> https://issues.apache.org/jira/browse/ZOOKEEPER-961
>> >>> >> >
>> >>> >> > Sure looks like it.
>> >>> >>
>> >>> >> Confirmed. Running the above repro against zookeeper trunk
produces
>> a
>> >>> >> responsive consumer that successfully rebalances when the broker
>> >>> >> rejoins.
>> >>> >>
>> >>> >> > So besides waiting for ZK 3.3.4 there is not much that
can be done
>> >>> >> > since Kafka does not offer a way to change the root znode
of a
>> >>> >> > cluster.
>> >>> >>
>> >>> >> Yeah, it's unclear that zk-3.3.4 will release anytime soon.
Instead
>> of
>> >>> >> relying on zk chroot, why not just expose the zk paths as config
>> >>> >> parameters for consumers, brokers, and producers?
>> >>> >>
>> >>> >> object ZkUtils {
>> >>> >>  val ConsumersPath = "/consumers"
>> >>> >>  val BrokerIdsPath = "/brokers/ids"
>> >>> >>  val BrokerTopicsPath = "/brokers/topics"
>> >>> >>  ...
>> >>> >>
>> >>> >
>> >>>
>> >>
>> >
>>
>

Mime
View raw message