kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manoj Khangaonkar <khangaon...@gmail.com>
Subject Re: Kafka ingestion data not equally distribute among brokers
Date Mon, 05 Nov 2018 20:36:40 GMT
Hi

In Kafka topic data is split into partitions. Partitions are assigned to
brokers.

I assume what you are trying to says is that the distribution of messages
across partitions is not balanced.

Messages are written to topics as key,value. The messages are distributed
across partitions based on key.

Perhaps you need to pick a better key that gives balanced distribution.

regards



On Mon, Nov 5, 2018 at 9:22 AM yong <shiuhrong@gmail.com> wrote:

> Hi Kafka experts,
>
> We have a 3 nodes kafka setup with data streams from mysql binlog, we
> noticed that from grafana monitoring in and out, some broker nodes are
> having lower data ingested in and out, is there anyway to balance it? Or it
> is nature of Kafka?
>
> Thanks!
>


-- 
http://khangaonkar.blogspot.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message