kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Péter Nagykátai <st4r.f1...@gmail.com>
Subject Re: Kafka topic partition distributing evenly on disks
Date Thu, 06 Aug 2020 19:05:12 GMT
Yeah, but it doesn't do that. My "older" disks have ~70 partitions, the
newer ones ~5 partitions. That's why I'm asking what went wrong.

On Thu, Aug 6, 2020 at 8:35 PM <Manoj.Agrawal2@cognizant.com> wrote:

> Kafka  evenly distributed number of partition on each disk so in your case
> every disk should have 3/2 topic partitions .
> It is producer job to evenly produce data by partition key  to topic
> partition .
> How it partition key , it is auto generated or producer sending key along
> with message .
>
>
> On 8/6/20, 7:29 AM, "Péter Nagykátai" <st4r.f1sch@gmail.com> wrote:
>
>     [External]
>
>
>     Hello,
>
>     I have a Kafka cluster with 3 brokers (v2.3.0) and each broker has 2
> disks
>     attached. I added a new topic (heavyweight) and was surprised that
> even if
>     the topic has 15 partitions, those weren't distributed evenly on the
> disks.
>     Thus I got one disk that's almost empty and the other almost filled
> up. Is
>     there any way to have Kafka evenly distribute data on its disks?
>
>     Thank you!
>
>
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to the
> sender and destroy all copies of the original message. Any unauthorized
> review, use, disclosure, dissemination, forwarding, printing or copying of
> this email, and/or any action taken in reliance on the contents of this
> e-mail is strictly prohibited and may be unlawful. Where permitted by
> applicable law, this e-mail and other e-mail communications sent to and
> from Cognizant e-mail addresses may be monitored.
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to the
> sender and destroy all copies of the original message. Any unauthorized
> review, use, disclosure, dissemination, forwarding, printing or copying of
> this email, and/or any action taken in reliance on the contents of this
> e-mail is strictly prohibited and may be unlawful. Where permitted by
> applicable law, this e-mail and other e-mail communications sent to and
> from Cognizant e-mail addresses may be monitored.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message