kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Gustafson <ja...@confluent.io>
Subject Re: New consumer - partitions auto assigned only on poll
Date Wed, 22 Jul 2015 16:18:42 GMT
Hey Stevo,

That's a good point. I think the javadoc is pretty clear that this could
return no partitions when the consumer has no active assignment, but it may
be a little unintuitive to have to call poll() after subscribing before you
can get the assigned partitions. I can't think of a strong reason not to go
ahead with the assignment in subscriptions() other than to keep it
non-blocking. Perhaps you can open a ticket and we can get feedback from
some other devs?

Thanks,
Jason

On Wed, Jul 22, 2015 at 2:09 AM, Stevo Slavić <sslavic@gmail.com> wrote:

> Hello Apache Kafka community,
>
> In the new consumer I encountered unexpected behavior. After constructing
> KafakConsumer instance with configured consumer rebalance callback handler,
> and subscribing to a topic with "consumer.subscribe(topic)", retrieving
> subscriptions would return empty set and callback handler would not get
> called (no partitions ever assigned or revoked), no matter how long
> instance was up.
>
> Then I found by inspecting KafkaConsumer code that partition assignment
> will only be triggered on first poll, pollOnce has:
>
> // ensure we have partitions assigned if we expect to
> if (subscriptions.partitionsAutoAssigned())
>     coordinator.ensurePartitionAssignment();
>
> Would it make sense to include this fragment in KafkaConsumer.subscriptions
> accessor as well?
>
> Kind regards,
> Stevo Slavic.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message