kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vijay Patil <vijay2110.t...@gmail.com>
Subject Re: Recommended number of partitions on each broker
Date Thu, 02 Mar 2017 06:54:58 GMT
300 partitions is at lower side, surely that won't be the root cause.
How about usage of network bandwidth for your nodes?
Are they reachable to zookeeper?
Are you executing some partition-rebalancing jobs (or re-assigning
partitions) in parallel?

On 2 March 2017 at 10:57, Jun MA <mj.saber1990@gmail.com> wrote:

> Hi,
> I’m curious what’s the recommended number of partitions running on each
> individual broker? We have a 3 nodes clusters running, each one has
> 24 cores, 1.1T ssd, 48G ram, 10G NIC. There’s about 300 partitions running
> on each broker and the resource usage is pretty low (5% cpu, 50% ram, 2%
> disk). But we see a lot random ISR expand/shrink happens to the cluster.
> Our networking is pretty stable, so I’m wondering if that’s because more
> partitions than kafka could handle? Is there any kafka parameters that we
> could tuning?
> Thanks,
> Jun

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message