kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Otto <ao...@wikimedia.org>
Subject Re: Kafka Simple Consumer Replicas versus ISR
Date Wed, 05 Aug 2015 18:09:51 GMT

I’m not sure, but it is possible the discrepancy you are seeing is related to this:

I’m pretty sure the CLI talks to Zookeeper directly, whereas likely the SimpleConsumer talks
to the Brokers.

> On Aug 5, 2015, at 13:13, dave@ariens.ca wrote:
> Using ( kafka.javaapi.consumer.SimpleConsumer and examining a
> partition's metadata I'm seeing conflicting information for replicas and
> ISR compared to the CLI /opt/kafka/bin/kafka-topics.sh --describe for
> the same topic/partition. The CLI command shows replicas are equal to
> ISR, whereas the simple consumer usually shows only 2/3 ISR.
> Here's a gist [1] that I'm using for my testing. Note, it doesn't
> actively determine the leader to fetch the metadata from, however when I
> execute it against a seed broker list that only contains the leader of
> the partition I'm interested in, the results are the same. This leads me
> to expect that the metadata is the same regardless of which broker you
> query (as opposed to consuming offsets which require you request from
> only the leader). 
> Links:
> ------
> [1] https://gist.github.com/ariens/f9e5f70ac4a5a139eac0

View raw message