kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avi Levi <...@indeni.com>
Subject Re: What are reasonable limits for max number of consumer groups per partition and per broker?
Date Thu, 16 Nov 2017 12:46:57 GMT
I think you will find this article
https://www.confluent.io/blog/how-to-choose-the-number-of-topicspartitions-in-a-kafka-cluster/
  useful

On Tue, Nov 14, 2017 at 6:49 PM, Viktor Somogyi <viktorsomogyi@gmail.com>
wrote:

> Hi Jeff,
>
> I think it's also worth considering that 1K consumer (implying that you
> have at least 1 consumer per group) for 1 partition means 1K TCP connection
> which means that they have to share the available bandwidth.
> Why do you have so many consumer groups? Do you basically want to
> multicast?
>
> Viktor
>
>
> On Tue, Nov 14, 2017 at 4:18 PM, Arunkumar <pm_arunkumar@yahoo.com.
> invalid>
> wrote:
>
> > Hi Jeff
> > Number of partition depends on number of consumers on that particular
> > consumer group. So you may have to create your partitions based on that.
> >
> > ThanksArunkumar Pichaimuthu, PMP
> >
> >     On Monday, November 13, 2017, 5:25:35 PM CST, Jeff Widman <
> > jeff@jeffwidman.com> wrote:
> >
> >  We're considering an architecture that would result in 5K-10K consumer
> > groups consuming from a single topic that has one partition.
> >
> > What are the reasonable limits for the max number of consumer groups per
> > partition and per broker?
> >
> > Can a single broker be the group coordinator for 1K+ consumer groups?
> >
> > --
> >
> > *Jeff Widman*
> > jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265)
> > <><
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message