kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ming Li <lm0...@gmail.com>
Subject Re: Is there a limitation on the number of simultaneous consumer connections to the same topic
Date Tue, 14 May 2013 03:08:59 GMT
Got it and thanks for your explanation~~

Best Regards,
Li Ming



On Mon, May 13, 2013 at 9:10 PM, Chris Curtin <curtin.chris@gmail.com>wrote:

> Yes. However be aware that starting and stopping processes will cause a
> rebalance of the consumers, so your code may find itself receiving events
> from a different partition suddenly (so don't assume the partition you are
> reading isn't going to change!)  Also as things are starting up you may
> find a process receives many partitions at first, but as the other
> processes are started the partitions get reassigned.
>
> Finally, running more processes than partitions will mean those other
> processes are idle.
>
>
>
>
> On Mon, May 13, 2013 at 8:49 AM, Ming Li <lm0909@gmail.com> wrote:
>
> > Hi Andrea,
> >
> > Thanks for your reply~~ you mean, it is no difference between
> > having N threads share the same ConsumerConnector created by
> > Consumer.createJavaConsumerConnector,
> > and
> > having N consumer process which has its own ConsumerConnector in every
> one
> > of them?
> >
> > Best Regards,
> > Li Ming
> >
> >
> >
> > On Mon, May 13, 2013 at 6:43 PM, Andrea Gazzarini <
> > andrea.gazzarini@gmail.com> wrote:
> >
> > > It shouldn't.
> > > Creating several listener / consumer processes belonging to the same
> > group
> > > means you are working with a point-to-point message channel so incoming
> > > messages will be delivered only to one consumer.
> > >
> > > Maybe I'm wrong but I believe in that scenario there's no difference
> > (from
> > > broker perspective) between threads and processes.
> > >
> > > Regards,
> > > Andrea
> > >
> > >
> > > On 05/13/2013 12:15 PM, Ming Li wrote:
> > >
> > >> Hi,
> > >>
> > >> Does Kafka have a limitation on the simultaneous connections (created
> > with
> > >> Consumer.**createJavaConsumerConnector) for the same topic within the
> > >> same
> > >> group?
> > >>
> > >> My scenario is I need to consume a topic from different process (not
> > >> thread), so I need to create lots of high level consumers.
> > >>
> > >>
> > >> Best Regards,
> > >> Li Ming
> > >>
> > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message