kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shiuh Rong Yong <shiuhr...@gmail.com>
Subject Re: Kafka ingestion data not equally distribute among brokers
Date Thu, 08 Nov 2018 01:30:12 GMT
Thank you,

Let me have a look into it.



On 2018/11/07 21:51:21, Manoj Khangaonkar <k...@gmail.com> wrote: 
> Hi> 
> 
> That is part of the Producer API.> 
> 
> For Java see> 
> https://kafka.apache.org/20/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html>

> ProducerRecord<K,V>> 
> 
> regards> 
> 
> 
> 
> On Tue, Nov 6, 2018 at 5:46 PM Shiuh Rong Yong <> 
> shiuhrongyong@cirosolution.com> wrote:> 
> 
> > Hi Manoj,> 
> >> 
> > I am new on Kafka, is it something mentioned on this site?> 
> > https://kafka.apache.org/0100/javadoc/org/apache/kafka/streams/state/KeyValueStore.html>

> >> 
> > Where can we set the key and value in Kafka?> 
> >> 
> > Thanks!> 
> >> 
> >> 
> > ----- Original Message -----> 
> > From: "Manoj Khangaonkar" <kh...@gmail.com>> 
> > To: "users" <us...@kafka.apache.org>> 
> > Sent: Tuesday, November 6, 2018 4:36:40 AM> 
> > Subject: Re: Kafka ingestion data not equally distribute among brokers> 
> >> 
> > Hi> 
> >> 
> > In Kafka topic data is split into partitions. Partitions are assigned to> 
> > brokers.> 
> >> 
> > I assume what you are trying to says is that the distribution of messages> 
> > across partitions is not balanced.> 
> >> 
> > Messages are written to topics as key,value. The messages are distributed> 
> > across partitions based on key.> 
> >> 
> > Perhaps you need to pick a better key that gives balanced distribution.> 
> >> 
> > regards> 
> >> 
> >> 
> >> 
> > On Mon, Nov 5, 2018 at 9:22 AM yong <sh...@gmail.com> wrote:> 
> >> 
> > > Hi Kafka experts,> 
> > >> 
> > > We have a 3 nodes kafka setup with data streams from mysql binlog, we> 
> > > noticed that from grafana monitoring in and out, some broker nodes are>

> > > having lower data ingested in and out, is there anyway to balance it? Or>

> > it> 
> > > is nature of Kafka?> 
> > >> 
> > > Thanks!> 
> > >> 
> >> 
> >> 
> > --> 
> > http://khangaonkar.blogspot.com/> 
> >> 
> 
> 
> -- > 
> http://khangaonkar.blogspot.com/> 
> 
Mime
View raw message