kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karthik Selva <karth...@julysystems.com>
Subject Topic/Partition Assignment in streams application cluster
Date Sat, 06 Jan 2018 12:38:34 GMT

I am using Kafka Streams for one of our application. The application has
several type of topics like the initial ingestion topics and the live
stream topic, all are sharing the same state in continuous fashion.

My problem is that the the assignment of these topics/partitions where I am
observing all the ingestion topics are assigned to the instance A of the
cluster(A,B,C) and live stream topics are assigned to the instance B and
other type topics are assigned to instance C. Due to this, the load is not
distributed to these 3 instances and it always goes to any one of the
instance the instance in the cluster.

Is there any way for me to give weightage for the topics or always
distribute the equivalent amount of partitions(of each topic) to each
instance. Also, how will I handle this scenario when I need to scale
up/down the cluster?




This email message along with attachments, email thread if any, is for the 
sole use of the intended recipients and for the intended purpose and may 
contain July Systems confidential and privileged information.Any 
unauthorized review, use, disclosure, copying, distribution or any other 
form of unintended use is strictly prohibited. If you are not the intended 
recipient, please contact the sender by reply email and destroy all copies 
/ forms of the original document.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message