kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <Manoj.Agraw...@cognizant.com>
Subject Re: Kafka topic partition distributing evenly on disks
Date Thu, 06 Aug 2020 20:12:25 GMT
What do you mean older disk ?

On 8/6/20, 12:05 PM, "Péter Nagykátai" <st4r.f1sch@gmail.com> wrote:

    [External]


    Yeah, but it doesn't do that. My "older" disks have ~70 partitions, the
    newer ones ~5 partitions. That's why I'm asking what went wrong.

    On Thu, Aug 6, 2020 at 8:35 PM <Manoj.Agrawal2@cognizant.com> wrote:

    > Kafka  evenly distributed number of partition on each disk so in your case
    > every disk should have 3/2 topic partitions .
    > It is producer job to evenly produce data by partition key  to topic
    > partition .
    > How it partition key , it is auto generated or producer sending key along
    > with message .
    >
    >
    > On 8/6/20, 7:29 AM, "Péter Nagykátai" <st4r.f1sch@gmail.com> wrote:
    >
    >     [External]
    >
    >
    >     Hello,
    >
    >     I have a Kafka cluster with 3 brokers (v2.3.0) and each broker has 2
    > disks
    >     attached. I added a new topic (heavyweight) and was surprised that
    > even if
    >     the topic has 15 partitions, those weren't distributed evenly on the
    > disks.
    >     Thus I got one disk that's almost empty and the other almost filled
    > up. Is
    >     there any way to have Kafka evenly distribute data on its disks?
    >
    >     Thank you!
    >
    >
    > This e-mail and any files transmitted with it are for the sole use of the
    > intended recipient(s) and may contain confidential and privileged
    > information. If you are not the intended recipient(s), please reply to the
    > sender and destroy all copies of the original message. Any unauthorized
    > review, use, disclosure, dissemination, forwarding, printing or copying of
    > this email, and/or any action taken in reliance on the contents of this
    > e-mail is strictly prohibited and may be unlawful. Where permitted by
    > applicable law, this e-mail and other e-mail communications sent to and
    > from Cognizant e-mail addresses may be monitored.
    > This e-mail and any files transmitted with it are for the sole use of the
    > intended recipient(s) and may contain confidential and privileged
    > information. If you are not the intended recipient(s), please reply to the
    > sender and destroy all copies of the original message. Any unauthorized
    > review, use, disclosure, dissemination, forwarding, printing or copying of
    > this email, and/or any action taken in reliance on the contents of this
    > e-mail is strictly prohibited and may be unlawful. Where permitted by
    > applicable law, this e-mail and other e-mail communications sent to and
    > from Cognizant e-mail addresses may be monitored.
    >


This e-mail and any files transmitted with it are for the sole use of the intended recipient(s)
and may contain confidential and privileged information. If you are not the intended recipient(s),
please reply to the sender and destroy all copies of the original message. Any unauthorized
review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or
any action taken in reliance on the contents of this e-mail is strictly prohibited and may
be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications
sent to and from Cognizant e-mail addresses may be monitored.
This e-mail and any files transmitted with it are for the sole use of the intended recipient(s)
and may contain confidential and privileged information. If you are not the intended recipient(s),
please reply to the sender and destroy all copies of the original message. Any unauthorized
review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or
any action taken in reliance on the contents of this e-mail is strictly prohibited and may
be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications
sent to and from Cognizant e-mail addresses may be monitored.
Mime
View raw message