kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ewen Cheslack-Postava <e...@confluent.io>
Subject Re: Partition ownership with high-level consumer
Date Thu, 08 Oct 2015 01:43:17 GMT
And you can get the current assignment in the new consumer after the
rebalance completes too:
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L593

On Tue, Oct 6, 2015 at 5:27 PM, Gwen Shapira <gwen@confluent.io> wrote:

> Ah, so the new consumer got that one:
>
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceListener.java#L88
>
> On Tue, Oct 6, 2015 at 3:55 PM, Joey Echeverria <joey@rocana.com> wrote:
>
> > I really only want them for the partitions I own. The client should know
> > that in order to acquire the zookeeper locks and could potentially
> execute
> > a callback to tell me the partitions I own after a rebalance.
> >
> > -Joey
> >
> > On Tue, Oct 6, 2015 at 4:08 PM, Gwen Shapira <gwen@confluent.io> wrote:
> >
> > > I don't think so. AFAIK, even the new API won't send this information
> to
> > > every consumer, because in some cases it can be huge.
> > >
> > >
> > >
> > > On Tue, Oct 6, 2015 at 1:44 PM, Joey Echeverria <joey@rocana.com>
> wrote:
> > >
> > > > But nothing in the API?
> > > >
> > > > -Joey
> > > >
> > > > On Tue, Oct 6, 2015 at 3:43 PM, Gwen Shapira <gwen@confluent.io>
> > wrote:
> > > >
> > > > > Zookeeper will have this information under /consumers/<group.id
> > > >/owners
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Oct 6, 2015 at 12:22 PM, Joey Echeverria <joey@rocana.com>
> > > > wrote:
> > > > >
> > > > > > Hi!
> > > > > >
> > > > > > Is there a way to track current partition ownership when using
> the
> > > > > > high-level consumer? It looks like the rebalance callback only
> > tells
> > > me
> > > > > the
> > > > > > partitions I'm (potentially) losing.
> > > > > >
> > > > > > -Joey
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Joey Echeverria
> > > > Director of Engineering
> > > >
> > >
> >
> >
> >
> > --
> > Joey Echeverria
> > Director of Engineering
> >
>



-- 
Thanks,
Ewen

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message