kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aravind s <sumoo6...@gmail.com>
Subject Kakfa Broker scaling tips for high number of consumers for a single topic
Date Thu, 14 Nov 2019 07:03:29 GMT
Hi,

We have a use-case where there are close to 40 producers to a topic with 2
replicas and consumers for this topic are around 2000. We have seen that
producer latency goes by 3 times when consumers grew from 500 to 2000. We
changed the following properties. The machine has sufficient memory. The
rate is around 100k we are targeting.

   - queued.max.requests
   - num.io.threads
   - num.network.threads

But there is no impact. Please, could you suggest how to go about this?

Regards,
Aravind

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message