kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Myles <bmy...@salesforce.com>
Subject Unexpected 0.9 Consumer Behavior
Date Tue, 05 Jan 2016 23:07:45 GMT
Hi,

Wondering if anyone can provide some insight into some unexpected
behavior we're seeing with the 0.9 consumer:

1) We create two consumer instances, each with the same group.id and
subscribe them to the same topic.

2) We would expect each consumer to be assigned 1/2 the topic
partitions, but they both end up being assigned *all* the partitions.

3) The consumers constantly rebalance, which leads to a
"CommitFailedException: Commit cannot be completed due to group
rebalance".

Here's the simple code to replicate:
https://gist.github.com/benmyles/a275ffeccc64442a836a

If we box each consumer inside a separate thread the problem goes
away, however we'd like to understand why it doesn't work as-is. I
understand consumers are not thread-safe, however we're not doing any
concurrent access here, everything is sequential.

Thanks,
Ben Myles

Mime
View raw message