kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Hinojosa <dh.evolutionn...@gmail.com>
Subject Are consumer coordinators not replicated in case of failure?
Date Fri, 07 Apr 2017 00:16:06 GMT
Hey all,

Question. I have three brokers.  I also have 3 consumers on their own
thread consuming 3 partitions of the same topic "scaled-states".  Here are
the configs when I run:

kafka-topics.sh --describe --topic 'scaled-cities' --zookeeper zoo2:2181

Topic:scaled-cities PartitionCount:3 ReplicationFactor:3 Configs:

Topic: scaled-cities Partition: 0 Leader: 1 Replicas: 0,1,2 Isr: 1,2,0

Topic: scaled-cities Partition: 1 Leader: 1 Replicas: 1,2,0 Isr: 1,2,0

Topic: scaled-cities Partition: 2 Leader: 2 Replicas: 2,0,1 Isr: 1,2,0

My consumer loops uses all three brokers for failover

props.put("bootstrap.servers", "kaf0:9092, kaf1:9092, kaf2:9092");

But when I stop a broker where the consumer coordinator happens to be,
in this case kaf0, the consumers will stop and no longer consume the
messages, and the following is displayed

[pool-1-thread-1] INFO
org.apache.kafka.clients.consumer.internals.AbstractCoordinator -
Marking the coordinator kaf0:9092 (id: 2147483647 rack: null) dead for
group consumerGroupAlpha

Anyone know why the consumer coordinator is not on another broker to
help out.  These consumers will not work and process any other
messages for this group until I turn that one broker back and this
doesn't feel like this should be by design.

Thanks and Appreciate the Help

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message