kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Clasen <sc...@heroku.com>
Subject Re: Inconsistency with Zookeeper
Date Sat, 08 Aug 2015 01:04:02 GMT
each zk needs a myid file in the data dir, with a different number 1,2,3

http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html

You can find the meanings of these and other configuration settings in the
section Configuration Parameters
<http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_configuration>.
A word though about a few here:

Every machine that is part of the ZooKeeper ensemble should know about
every other machine in the ensemble. You accomplish this with the series of
lines of the form*server.id <http://server.id>=host:port:port*. The
parameters *host* and *port* are straightforward. You attribute the server
id to each machine by creating a file named myid, one for each server,
which resides in that server's data directory, as specified by the
configuration file parameter *dataDir*.

On Fri, Aug 7, 2015 at 5:59 PM, Hemanth Abbina <HemanthA@eiqnetworks.com>
wrote:

> Yes. I have set the unique broker ids (as 0, 1, 2) in the
> server.properties file
>
> I did not get the " set broker id  in zookeeper data directory".  We don't
> set any broker id in zookeeper.
>
> We provide only "zookeeper.connect=ZK:2181,ZK2:2181,ZK3:2181" in
> server.properties file. Right ?
>
> -----Original Message-----
> From: Prabhjot Bharaj [mailto:prabhbharaj@gmail.com]
> Sent: Friday, August 7, 2015 8:44 PM
> To: users@kafka.apache.org
> Subject: Re: Inconsistency with Zookeeper
>
> Have you set broker id  in zookeeper data directory and some unique broker
> Id in server.properties ?
>
> Regards,
> Prabhjot
> On Aug 6, 2015 1:43 PM, "Hemanth Abbina" <HemanthA@eiqnetworks.com> wrote:
>
> > Hi,
> >
> > I am running a Kafka POC with below details
> >
> > *         3 Node cluster (4 Core, 16 GB RAM each) running Kafka 0.8.2.1.
> >
> > *         Each node running Kafka & Zookeeper instances. (So total of 3
> > Kafka brokers & 3 zookeepers)
> >
> > When I tried to create a topic using the kafka-topics.sh, observing
> > the below exceptions on the console. If I try multiple times, the
> > topic is getting created at some time. Please suggest a workaround to
> > make this work every time.
> >
> > Thanks in advance.
> >
> > [ec2-user@ip-10-20-0-196 kafka_2.10-0.8.2.1]$ ./bin/kafka-topics.sh
> > --zookeeper 10.20.0.196:2181,10.20.0.197:2181,10.20.0.198:2181 --topic
> > reportspoc4 --create --partitions 1 --replication-factor 2 Error while
> > executing topic command replication factor: 2 larger than available
> > brokers: 1
> > kafka.admin.AdminOperationException: replication factor: 2 larger than
> > available brokers: 1
> >         at
> > kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
> >         at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:171)
> >         at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:93)
> >         at kafka.admin.TopicCommand$.main(TopicCommand.scala:55)
> >         at kafka.admin.TopicCommand.main(TopicCommand.scala)
> >
> >
> > [ec2-user@ip-10-20-0-196 kafka_2.10-0.8.2.1]$ ./bin/kafka-topics.sh
> > --zookeeper 10.20.0.196:2181,10.20.0.197:2181,10.20.0.198:2181 --topic
> > reportspoc4 --create --partitions 1 --replication-factor 2 Error while
> > executing topic command
> > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode
> > = NoNode for /brokers/ids
> > org.I0Itec.zkclient.exception.ZkNoNodeException:
> > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode
> > = NoNode for /brokers/ids
> >         at
> > org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47)
> >         at
> > org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685)
> >         at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:413)
> >         at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:409)
> >         at kafka.utils.ZkUtils$.getChildren(ZkUtils.scala:462)
> >         at kafka.utils.ZkUtils$.getSortedBrokerList(ZkUtils.scala:78)
> >         at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:170)
> >         at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:93)
> >         at kafka.admin.TopicCommand$.main(TopicCommand.scala:55)
> >         at kafka.admin.TopicCommand.main(TopicCommand.scala)
> > Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
> > KeeperErrorCode = NoNode for /brokers/ids
> >         at
> > org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> >         at
> > org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> >         at
> org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
> >         at
> org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1500)
> >         at
> > org.I0Itec.zkclient.ZkConnection.getChildren(ZkConnection.java:99)
> >         at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:416)
> >         at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:413)
> >         at
> > org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
> >         ... 8 more
> >
> > --regards
> > Hemanth
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message