kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stevo Slavić <ssla...@gmail.com>
Subject Re: Unable to create new topics, "no brokers available"
Date Wed, 28 Dec 2016 10:23:33 GMT
Hello Alex,

ZooKeeper nodes that Kafka brokers create in /brokers/ids path, upon
registering themselves that they are available, are ephemeral, so not
persistent. Ephemeral ZooKeeper nodes live as long as ZooKeeper session
that created them is active. The /broker/ids child nodes are gone by design
when broker to ZooKeeper connection is down long enough (e.g. due to
network partition, or broker actually stopped/crashed). They are meant to
signal availability of particular broker in the Kafka cluster. Broker
announces its availability so cluster members can be discovered dynamically
(as brokers join or leave the cluster) by other Kafka brokers in the
cluster and by Kafka clients. So that data is meant to be lost by design.

Maybe ZooKeeper node type could be mentioned in
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper

Kind regards,
Stevo Slavic.

On Wed, Dec 28, 2016 at 10:57 AM, Alex Eftimie <
alex.eftimie@getyourguide.com> wrote:

> Quick update: digging into Zookeeper data, we observed that the
> /brokers/ids path was empty. Restarting the kafka nodes repopulated
> zookeeper data, and now the error is gone (we are able to create new
> topics).
>
> We didn’t alter data in Zookeeper manually, but recently we added 2 nodes
> to a 3 nodes zookeeper cluster. Also, we had a network partitioning issue
> in the past. Could any of these be the reason for the lost ZK data?
>
> Any other hints on what may have generated the data loss in ZK?
>
> Thanks,
> Alex
>
> > On 28 Dec 2016, at 10:00, Alex Eftimie <alex.eftimie@getyourguide.com>
> wrote:
> >
> > Hello,
> >
> > We recently migrated from Kafka 0.8 to 0.10, while keeping the log
> format and internal communication at 0.9 version[1]. We have a cluster of
> two nodes which is working correctly for 10 topics (producers/consumers
> work fine).
> >
> > Trying to create a new topic raises:
> >     kafka-topics.sh --zookeeper zknode:2181 --create
> --replication-factor 1 --partitions 2 --topic newtopic
> >     Error while executing topic command : replication factor: 1 larger
> than available brokers: 0
> >     [2016-12-28 09:30:02,177] ERROR org.apache.kafka.common.errors.
> InvalidReplicationFactorException: replication factor: 1 larger than
> available brokers: 0
> >  (kafka.admin.TopicCommand$)
> >
> > There are two available brokers, so this must be a different error
> hidden behind this error message. There’s nothing in the log file.
> >
> > Where should we start investigating?
> >
> > Thanks,
> >
> > Alex Eftimie
> > Software Engineer DevOps
> >
> > [1]
> > kafka version: 0.10.1.0
> > # config:
> > inter.broker.protocol.version=0.9.0.1
> > log.message.format.version=0.9.0.1
> >
>
>
> --
>
>
> ------------------------------
>
> [image: email GYG_Logo__Standart_Transparent_400x600_WEB copy.png]
> <http://www.getyourguide.com>
>
> GetYourGuide AG
>
> Technoparkstrasse 1
>
> 8005 Zürich
>
> Switzerland
>
> <https://www.facebook.com/GetYourGuide> <https://twitter.com/GetYourGuide>
> <https://www.instagram.com/getyourguide/>
> <https://www.linkedin.com/company/getyourguide-ag> [image: Icon_4.png]
> <http://www.getyourguide.com>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message