kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <Manoj.Agraw...@cognizant.com>
Subject Re: Kafka topic partition distributing evenly on disks
Date Thu, 06 Aug 2020 18:34:23 GMT
Kafka  evenly distributed number of partition on each disk so in your case every disk should
have 3/2 topic partitions .
It is producer job to evenly produce data by partition key  to topic partition .
How it partition key , it is auto generated or producer sending key along with message .


On 8/6/20, 7:29 AM, "Péter Nagykátai" <st4r.f1sch@gmail.com> wrote:

    [External]


    Hello,

    I have a Kafka cluster with 3 brokers (v2.3.0) and each broker has 2 disks
    attached. I added a new topic (heavyweight) and was surprised that even if
    the topic has 15 partitions, those weren't distributed evenly on the disks.
    Thus I got one disk that's almost empty and the other almost filled up. Is
    there any way to have Kafka evenly distribute data on its disks?

    Thank you!


This e-mail and any files transmitted with it are for the sole use of the intended recipient(s)
and may contain confidential and privileged information. If you are not the intended recipient(s),
please reply to the sender and destroy all copies of the original message. Any unauthorized
review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or
any action taken in reliance on the contents of this e-mail is strictly prohibited and may
be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications
sent to and from Cognizant e-mail addresses may be monitored.
This e-mail and any files transmitted with it are for the sole use of the intended recipient(s)
and may contain confidential and privileged information. If you are not the intended recipient(s),
please reply to the sender and destroy all copies of the original message. Any unauthorized
review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or
any action taken in reliance on the contents of this e-mail is strictly prohibited and may
be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications
sent to and from Cognizant e-mail addresses may be monitored.
Mime
View raw message