kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shiuh Rong Yong <shiuhrongy...@cirosolution.com>
Subject Re: Kafka ingestion data not equally distribute among brokers
Date Wed, 07 Nov 2018 01:46:44 GMT
Hi Manoj,

I am new on Kafka, is it something mentioned on this site? https://kafka.apache.org/0100/javadoc/org/apache/kafka/streams/state/KeyValueStore.html

Where can we set the key and value in Kafka?

Thanks!


----- Original Message -----
From: "Manoj Khangaonkar" <khangaonkar@gmail.com>
To: "users" <users@kafka.apache.org>
Sent: Tuesday, November 6, 2018 4:36:40 AM
Subject: Re: Kafka ingestion data not equally distribute among brokers

Hi

In Kafka topic data is split into partitions. Partitions are assigned to
brokers.

I assume what you are trying to says is that the distribution of messages
across partitions is not balanced.

Messages are written to topics as key,value. The messages are distributed
across partitions based on key.

Perhaps you need to pick a better key that gives balanced distribution.

regards



On Mon, Nov 5, 2018 at 9:22 AM yong <shiuhrong@gmail.com> wrote:

> Hi Kafka experts,
>
> We have a 3 nodes kafka setup with data streams from mysql binlog, we
> noticed that from grafana monitoring in and out, some broker nodes are
> having lower data ingested in and out, is there anyway to balance it? Or it
> is nature of Kafka?
>
> Thanks!
>


-- 
http://khangaonkar.blogspot.com/

Mime
View raw message