kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Henke <ghe...@cloudera.com>
Subject Re: Kafka Zookeeper Issues
Date Tue, 04 Aug 2015 16:54:35 GMT
The /brokers/ids nodes are ephemeral nodes that only exists while the
brokers maintain a session to zookeeper. There is more information on
Kafka's Zookeeper usage here:
   - http://kafka.apache.org/documentation.html
      - look for "Broker Node Registry"
   -
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper

Hopefully that helps debug your issue.

Thank you,
Grant

On Mon, Aug 3, 2015 at 5:20 AM, Wollert, Fabian <fabian.wollert@zalando.de>
wrote:

> hi everyone,
>
> we are trying to deploy Kafka 0.8.2.1 and Zookeeper on AWS using
> Cloudformation, ASG's and other Services. For Zookeeper we are using
> Netflix' Exhibitor (V 1.5.5) to ensure failover stability.
>
> What we are observing right now is that after some days our Brokers are not
> registered anymore in the "/brokers/ids" path in Zookeeper. I was trying to
> see when they get deleted to check the logs, but the ZK Transaction logs
> only shows the create stmt, no deletes or something (though deletes are
> written down there). Can someone explain me how the mechanism works with
> registering and deregistering in Zookeeper or point me to a doc or even
> source code, where this happens? Or some one has even some idea what
> happens there.
>
> Any experience on what to take care of deploying kafka on AWS (or generally
> a cloud env) would be also helpful.
>
> Cheers
>
> --
> *Fabian Wollert*
> Business Intelligence
>
> *POSTAL ADDRESS*
> Zalando SE
> 11501 Berlin
>
> *OFFICE*
> Zalando SE
> Mollstra├če 1
> 10178 Berlin
> Germany
>
> Phone: +49 30 20968 1819
> Fax:   +49 30 27594 693
> E-Mail: fabian.wollert@zalando.de
> Web: www.zalando.de
> Jobs: jobs.zalando.de
>
> Zalando SE, Tamara-Danz-Stra├če 1, 10243 Berlin
> Company registration: Amtsgericht Charlottenburg, HRB 158855 B
> Tax ID: 29/560/00596 * VAT registration number: DE 260543043
> Management Board: Robert Gentz, David Schneider, Rubin Ritter
> Chairperson of the Supervisory Board: Cristina Stenbeck
> Registered office: Berlinn
>



-- 
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message