kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wollert, Fabian" <fabian.woll...@zalando.de>
Subject Re: Kafka Zookeeper Issues
Date Wed, 05 Aug 2015 08:20:25 GMT
thanks Grant, i also found some more information (also the one your
referring to). I think it is a kafka -> zookeeper connection issue on AWS.
We are investigating further on this.

Cheers
Fabian

2015-08-04 18:54 GMT+02:00 Grant Henke <ghenke@cloudera.com>:

> The /brokers/ids nodes are ephemeral nodes that only exists while the
> brokers maintain a session to zookeeper. There is more information on
> Kafka's Zookeeper usage here:
>    - http://kafka.apache.org/documentation.html
>       - look for "Broker Node Registry"
>    -
>
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper
>
> Hopefully that helps debug your issue.
>
> Thank you,
> Grant
>
> On Mon, Aug 3, 2015 at 5:20 AM, Wollert, Fabian <fabian.wollert@zalando.de
> >
> wrote:
>
> > hi everyone,
> >
> > we are trying to deploy Kafka 0.8.2.1 and Zookeeper on AWS using
> > Cloudformation, ASG's and other Services. For Zookeeper we are using
> > Netflix' Exhibitor (V 1.5.5) to ensure failover stability.
> >
> > What we are observing right now is that after some days our Brokers are
> not
> > registered anymore in the "/brokers/ids" path in Zookeeper. I was trying
> to
> > see when they get deleted to check the logs, but the ZK Transaction logs
> > only shows the create stmt, no deletes or something (though deletes are
> > written down there). Can someone explain me how the mechanism works with
> > registering and deregistering in Zookeeper or point me to a doc or even
> > source code, where this happens? Or some one has even some idea what
> > happens there.
> >
> > Any experience on what to take care of deploying kafka on AWS (or
> generally
> > a cloud env) would be also helpful.
> >
> > Cheers
> >
> > --
> > *Fabian Wollert*
> > Business Intelligence
> >
> > *POSTAL ADDRESS*
> > Zalando SE
> > 11501 Berlin
> >
> > *OFFICE*
> > Zalando SE
> > Mollstraße 1
> > 10178 Berlin
> > Germany
> >
> > Phone: +49 30 20968 1819
> > Fax:   +49 30 27594 693
> > E-Mail: fabian.wollert@zalando.de
> > Web: www.zalando.de
> > Jobs: jobs.zalando.de
> >
> > Zalando SE, Tamara-Danz-Straße 1, 10243 Berlin
> > Company registration: Amtsgericht Charlottenburg, HRB 158855 B
> > Tax ID: 29/560/00596 * VAT registration number: DE 260543043
> > Management Board: Robert Gentz, David Schneider, Rubin Ritter
> > Chairperson of the Supervisory Board: Cristina Stenbeck
> > Registered office: Berlinn
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>



-- 
*Fabian Wollert*
Business Intelligence



*POSTAL ADDRESS*
Zalando SE
11501 Berlin

*OFFICE*
Zalando SE
Mollstraße 1
10178 Berlin
Germany

Phone: +49 30 20968 1819
Fax:   +49 30 27594 693
E-Mail: fabian.wollert@zalando.de
Web: www.zalando.de
Jobs: jobs.zalando.de

Zalando SE, Tamara-Danz-Straße 1, 10243 Berlin
Company registration: Amtsgericht Charlottenburg, HRB 158855 B
Tax ID: 29/560/00596 * VAT registration number: DE 260543043
Management Board: Robert Gentz, David Schneider, Rubin Ritter
Chairperson of the Supervisory Board: Cristina Stenbeck
Registered office: Berlinn

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message