kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stevo Slavić <ssla...@gmail.com>
Subject Re: New consumer - partitions auto assigned only on poll
Date Fri, 24 Jul 2015 16:21:41 GMT
Hello Jason,

Thanks for feedback. I've created ticket for this
https://issues.apache.org/jira/browse/KAFKA-2359

Kind regards,
Stevo Slavic.

On Wed, Jul 22, 2015 at 6:18 PM, Jason Gustafson <jason@confluent.io> wrote:

> Hey Stevo,
>
> That's a good point. I think the javadoc is pretty clear that this could
> return no partitions when the consumer has no active assignment, but it may
> be a little unintuitive to have to call poll() after subscribing before you
> can get the assigned partitions. I can't think of a strong reason not to go
> ahead with the assignment in subscriptions() other than to keep it
> non-blocking. Perhaps you can open a ticket and we can get feedback from
> some other devs?
>
> Thanks,
> Jason
>
> On Wed, Jul 22, 2015 at 2:09 AM, Stevo Slavić <sslavic@gmail.com> wrote:
>
> > Hello Apache Kafka community,
> >
> > In the new consumer I encountered unexpected behavior. After constructing
> > KafakConsumer instance with configured consumer rebalance callback
> handler,
> > and subscribing to a topic with "consumer.subscribe(topic)", retrieving
> > subscriptions would return empty set and callback handler would not get
> > called (no partitions ever assigned or revoked), no matter how long
> > instance was up.
> >
> > Then I found by inspecting KafkaConsumer code that partition assignment
> > will only be triggered on first poll, pollOnce has:
> >
> > // ensure we have partitions assigned if we expect to
> > if (subscriptions.partitionsAutoAssigned())
> >     coordinator.ensurePartitionAssignment();
> >
> > Would it make sense to include this fragment in
> KafkaConsumer.subscriptions
> > accessor as well?
> >
> > Kind regards,
> > Stevo Slavic.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message