kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "hsy541@gmail.com" <hsy...@gmail.com>
Subject Re: Questions about new consumer API
Date Tue, 02 Dec 2014 22:32:07 GMT
Thanks Neha, another question, so if offsets are stored under group.id,
dose it mean in one group, there should be at most one subscriber for each
topic partition?

Best,
Siyuan

On Tue, Dec 2, 2014 at 12:55 PM, Neha Narkhede <neha.narkhede@gmail.com>
wrote:

> 1. In this doc it says kafka consumer will automatically do load balance.
> Is it based on throughtput or same as what we have now balance the
> cardinality among all consumers in same ConsumerGroup? In a real case
> different partitions could have different peak time.
>
> Load balancing is still based on # of partitions for the subscribed topics
> and
> ensuring that each partition has exactly one consumer as the owner.
>
> 2. In the API, threre is subscribe(partition...) method saying not using
> group management, does it mean the group.id property will be discarded and
> developer has full control of distributing partitions to consumers?
>
> group.id is also required for offset management, if the user chooses to
> use
> Kafka based offset management. The user will have full control over
> distribution
> of partitions to consumers.
>
> 3. Is new API compatible with old broker?
>
> Yes, it will.
>
> 4. Will simple consumer api and high-level consumer api still be supported?
>
> Over time, we will phase out the current high-level and simple consumer
> since the
> 0.9 API supports both.
>
> Thanks,
> Neha
>
> On Tue, Dec 2, 2014 at 12:07 PM, hsy541@gmail.com <hsy541@gmail.com>
> wrote:
>
> > Hi guys,
> >
> > I'm interested in the new Consumer API.
> > http://people.apache.org/~nehanarkhede/kafka-0.9-consumer-javadoc/doc/
> >
> > I have couple of question.
> > 1. In this doc it says kafka consumer will automatically do load balance.
> > Is it based on throughtput or same as what we have now balance the
> > cardinality among all consumers in same ConsumerGroup? In a real case
> > different partitions could have different peak time.
> > 2. In the API, threre is subscribe(partition...) method saying not using
> > group management, does it mean the group.id property will be discarded
> and
> > developer has full control of distributing partitions to consumers?
> > 3. Is new API compatible with old broker?
> > 4. Will simple consumer api and high-level consumer api still be
> supported?
> >
> > Thanks!
> >
> > Best,
> > Siyuan
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message