kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiangyuan LI <flashmouse2...@gmail.com>
Subject Re: Kakfa Broker scaling tips for high number of consumers for a single topic
Date Fri, 15 Nov 2019 03:39:01 GMT
hi
I want to know what's the kafka server & client version you're using?
And I want to know ,you said you have 2000 consumer,is that mean 2000
consumer groups or 2000 consumer in one consumer group?


aravind s <sumoo6649@gmail.com> 于2019年11月15日周五 上午3:12写道:

> Hi,
>
> We have a use-case where there are close to 40 producers to a topic with 2
> replicas and consumers for this topic are around 2000. We have seen that
> producer latency goes by 3 times when consumers grew from 500 to 2000. We
> changed the following properties. The machine has sufficient memory. The
> rate is around 100k we are targeting.
>
>    - queued.max.requests
>    - num.io.threads
>    - num.network.threads
>
> But there is no impact. Please, could you suggest how to go about this?
>
> Regards,
> Aravind
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message