kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brett Rann <br...@zendesk.com.INVALID>
Subject Re: Disk-size aware partitioning
Date Wed, 10 Oct 2018 03:22:14 GMT
LInkedin's cruise-control  https://github.com/linkedin/cruise-control

has numerous goals, including disk, network, cpu, rack awareness,
leadership distribution etc.

You can have separate disk/network limits per broker (ours are all the same
fwiw)

We use it and it does a stellar job of keeping a cluster balanced.

On Tue, Oct 9, 2018 at 8:11 PM Vincent Bernardi <vincent@kameleoon.com>
wrote:

> Hello everyone,
> I couldn't find an answer to this on the web so:
> If I have a Kafka cluster where nodes have different disk size, is there a
> way to have an automatic partitioning aware of these disk sizes (i.e.
> allocating 4 partitions to the 4TB node and 1 partition to the 1TB node)? I
> know I can partition by hand, but it would be much easier if I could just
> declare a max disk space to the partitioner one way or the other.
>
> Thanks in advance,
> Vincent Bernardi
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message