kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: High level zookeeper consumer life
Date Sat, 01 Jun 2013 03:47:29 GMT
Consumers are not auto deleted from ZK. So, you will need to reuse the
group ID and existing consumer instances on the group. For the relationship
btw partitions and consumers, see #4 in http://kafka.apache.org/faq.html.
Rebalance failures are typically transient, but are more likely with more
consumers in the same group.

Thanks,

Jun


On Fri, May 31, 2013 at 12:47 PM, arathi maddula <arathimaddula@gmail.com>wrote:

> Hi,
> I use a high level consumer inside a servlet which reads from Kafka stream.
> Every time I send an HTTP request, I use a different group ID. This results
> in lots of consumers on zookeeper node( ls /consumers on zookeeper client
> shows list of all consumers).
> Please help me clarify these:
>
> 1)Is there a time after which zookeeper auto deletes these consumers? If
> yes, can that be configured.
> 2)If I reuse a group ID later on, will an existing consumer be used?
> 3)Can there be issues if the no. of group IDs/consumers (due to
> simultaneous hits or separate hits) for a topic exceeds the total no. of
> partitions for a topic?
> 4)Sometimes I get a ConsumerRebalanceException. Could this be due to many
> consumers? Can this exception occur if two servlet users use the same
> groupID/consumer for the same topic simultaneously.
>
> Thanks,
> Arathi
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message