kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yonghui Zhao <zhaoyong...@gmail.com>
Subject owner info in zk is not correct
Date Mon, 12 May 2014 07:48:13 GMT
Hi,

We are using kafka 0.7.

2 brokers, each broker has 10 partitions for one topic
3 consumers in one consumer group, each consumer create 10 streams.


Today, when we want to rollout new service.
After we restart one consumer we find exceptions and warning.

kafka.common.ConsumerRebalanceFailedException:
RecommendEvent_sd-sns-relation01.bj-1399630465426-53d3aefc can't rebalance
after 4 retries


[INFO  2014-05-12 15:17:47.364]
kafka.utils.Logging$class.info(Logging.scala:61)
[conflict in /consumers/RecommendEvent/owners/sensei/1-2 data:
RecommendEvent_sd-sns-relation01.bj-1399879066480-5426fb5e-2 stored data:
RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-1]
[INFO  2014-05-12 15:17:47.366]
kafka.utils.Logging$class.info(Logging.scala:61)
[RecommendEvent_sd-sns-relation01.bj-1399879066480-5426fb5e waiting for the
partition ownership to be deleted: 1-2]
[INFO  2014-05-12 15:17:47.375]
kafka.utils.Logging$class.info(Logging.scala:61)
[conflict in /consumers/RecommendEvent/owners/sensei/1-3 data:
RecommendEvent_sd-sns-relation01.bj-1399879066480-5426fb5e-3 stored data:
RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-1]
[INFO  2014-05-12 15:17:47.375]
kafka.utils.Logging$class.info(Logging.scala:61)
[RecommendEvent_sd-sns-relation01.bj-1399879066480-5426fb5e waiting for the
partition ownership to be deleted: 1-3]
[INFO  2014-05-12 15:17:47.385]
kafka.utils.Logging$class.info(Logging.scala:61)
[conflict in /consumers/RecommendEvent/owners/sensei/1-5 data:
RecommendEvent_sd-sns-relation01.bj-1399879066480-5426fb5e-5 stored data:
RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-2]
[INFO  2014-05-12 15:17:47.386]
kafka.utils.Logging$class.info(Logging.scala:61)
[RecommendEvent_sd-sns-relation01.bj-1399879066480-5426fb5e waiting for the
partition ownership to be deleted: 1-5]



And I opened zk viewer.

In zk, we found 2 consumers in ConsumerGroup/ids:

RecommendEvent_sd-sns-relation02.bj-1399635256619-5d8123c6
RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3


And in owners/topic/ we found all partitions are assigned to
sd-sns-relation03.bj:

Here is the owner info:
1:0  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-0
1:1  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-0
1:2  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-1
1:3  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-1
1:4  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-2
1:5  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-2
1:6  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-3
1:7  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-3
1:8  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-4
1:9  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-4

2:0  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-0
2:1  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-1
2:2  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-2
2:3  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-3
2:4  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-4
2:5  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-5
2:6  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-6
2:7  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-7
2:8  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-8
2:9  RecommendEvent_sd-sns-relation03.bj-1399635121250-487bdbb3-9


So all partitions are assigned to sd-sns-relation03.bj,  but from logs and
counter, we are sure sd-sns-relation02.bj has input too.


My question is:

1. why rebalance failed?
2. why owner info is wrong?  btw: zkclient is 0.2

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message