kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brett Hoerner <br...@bretthoerner.com>
Subject Producer only finding partition on 1 of 2 Brokers, even though ZK shows 1 partition exists on both Brokers?
Date Tue, 04 Jun 2013 23:21:25 GMT
(version 0.7.2)

For some reason, my producers are only picking up the partition on 1 of my
2 brokers. I've been digging through the code, and I don't see any issues
given the state of my ZK nodes. The producer never seems to locate a
partition on Broker0, even though ZK clearly states that it has 1 (just
like Broker1 does).

The result of this is that the producer works, but only sends data to one

Has anyone seen something like this before? I'm stumped. Thanks.

# zk information follows:
[zk: localhost:2181(CONNECTED) 5] ls /kafka/brokers/ids
[1, 0]

[zk: localhost:2181(CONNECTED) 6] get /kafka/brokers/ids/0

[zk: localhost:2181(CONNECTED) 7] get /kafka/brokers/ids/1

[zk: localhost:2181(CONNECTED) 10] ls /kafka/brokers/topics/test
[1, 0]

# this is the most confusing one
[zk: localhost:2181(CONNECTED) 11] get /kafka/brokers/topics/test/0

[zk: localhost:2181(CONNECTED) 12] get /kafka/brokers/topics/test/1

# kafka producer ZK information DEBUG log, as you can see it finds 0
partitions on Broker0:
Broker Topic Path => /brokers/topics
DEBUG [2013-06-04 23:14:30,689] kafka.producer.ZKBrokerPartitionInfo:
Broker ids and # of partitions on each for topic: test = ArrayBuffer((0,0),
DEBUG [2013-06-04 23:14:30,690] kafka.producer.ZKBrokerPartitionInfo:
Sorted list of broker ids and partition ids on each for topic: test =
DEBUG [2013-06-04 23:14:30,819]
[BrokerTopicsListener] Creating broker topics listener to watch the
following paths -
/broker/topics, /broker/topics/topic, /broker/ids
DEBUG [2013-06-04 23:14:30,823]
[BrokerTopicsListener] Initialized this broker topics listener with initial
mapping of broker id to partition id per topic with Map(test ->
DEBUG [2013-06-04 23:14:30,904] kafka.producer.ZKBrokerPartitionInfo:
Registering listener on path: /brokers/topics/test

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message