kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manoj Khangaonkar <khangaon...@gmail.com>
Subject Re: Kafka ingestion data not equally distribute among brokers
Date Wed, 07 Nov 2018 21:51:21 GMT
Hi

That is part of the Producer API.

For Java see
https://kafka.apache.org/20/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html
ProducerRecord<K,V>

regards



On Tue, Nov 6, 2018 at 5:46 PM Shiuh Rong Yong <
shiuhrongyong@cirosolution.com> wrote:

> Hi Manoj,
>
> I am new on Kafka, is it something mentioned on this site?
> https://kafka.apache.org/0100/javadoc/org/apache/kafka/streams/state/KeyValueStore.html
>
> Where can we set the key and value in Kafka?
>
> Thanks!
>
>
> ----- Original Message -----
> From: "Manoj Khangaonkar" <khangaonkar@gmail.com>
> To: "users" <users@kafka.apache.org>
> Sent: Tuesday, November 6, 2018 4:36:40 AM
> Subject: Re: Kafka ingestion data not equally distribute among brokers
>
> Hi
>
> In Kafka topic data is split into partitions. Partitions are assigned to
> brokers.
>
> I assume what you are trying to says is that the distribution of messages
> across partitions is not balanced.
>
> Messages are written to topics as key,value. The messages are distributed
> across partitions based on key.
>
> Perhaps you need to pick a better key that gives balanced distribution.
>
> regards
>
>
>
> On Mon, Nov 5, 2018 at 9:22 AM yong <shiuhrong@gmail.com> wrote:
>
> > Hi Kafka experts,
> >
> > We have a 3 nodes kafka setup with data streams from mysql binlog, we
> > noticed that from grafana monitoring in and out, some broker nodes are
> > having lower data ingested in and out, is there anyway to balance it? Or
> it
> > is nature of Kafka?
> >
> > Thanks!
> >
>
>
> --
> http://khangaonkar.blogspot.com/
>


-- 
http://khangaonkar.blogspot.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message