Hi All,
How does kafka streams partition assignment work for sources? I have a
stream application reading from a topic which has 24 partitions. There are
6 application containers with 4 stream tasks in each container running but
only 2 instances are assigned with partitions and even within the two, one
has 21 partitions and the other has 3 partitions assigned. There is
something wrong in the way the partitions are assigned. Is there a way to
evenly distribute partitions?
*Group Assignment:*
userapp user-data 5 1716495057 1716504638
9581
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-1-consumer-428625d0-da9c-44d0-b499-79e698f5add0
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-1-consumer
userapp user-data 13 1735721509 1735730754
9245
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-1-consumer-428625d0-da9c-44d0-b499-79e698f5add0
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-1-consumer
userapp user-data 9 1729975595 1729984597
9002
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-1-consumer-428625d0-da9c-44d0-b499-79e698f5add0
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-1-consumer
userapp user-data 17 1736457935 1736466997
9062
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-1-consumer-428625d0-da9c-44d0-b499-79e698f5add0
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-1-consumer
userapp user-data 0 1917953948 1917963103
9155
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-1-consumer-428625d0-da9c-44d0-b499-79e698f5add0
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-1-consumer
userapp user-data 11 1719082482 1719085979
3497
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-4-consumer-a97f7c1f-f4a1-4b39-bd80-9b8c18dfd62a
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-4-consumer
userapp user-data 19 1755067810 1755071386
3576
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-4-consumer-a97f7c1f-f4a1-4b39-bd80-9b8c18dfd62a
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-4-consumer
userapp user-data 7 1729598367 1729601982
3615
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-4-consumer-a97f7c1f-f4a1-4b39-bd80-9b8c18dfd62a
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-4-consumer
userapp user-data 1 1949943357 1949947252
3895
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-4-consumer-a97f7c1f-f4a1-4b39-bd80-9b8c18dfd62a
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-4-consumer
userapp user-data 15 1751889619 1751893116
3497
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-4-consumer-a97f7c1f-f4a1-4b39-bd80-9b8c18dfd62a
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-4-consumer
userapp user-data 3 1752352594 1752356255
3661
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-4-consumer-a97f7c1f-f4a1-4b39-bd80-9b8c18dfd62a
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-4-consumer
userapp user-data 6 1693612741 1693621514
8773
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-2-consumer-59fe9d37-cd32-4a96-867b-c13e85cba398
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-2-consumer
userapp user-data 2 1923482321 1923490826
8505
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-2-consumer-59fe9d37-cd32-4a96-867b-c13e85cba398
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-2-consumer
userapp user-data 14 1677960791 1677969039
8248
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-2-consumer-59fe9d37-cd32-4a96-867b-c13e85cba398
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-2-consumer
userapp user-data 10 1722133802 1722142388
8586
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-2-consumer-59fe9d37-cd32-4a96-867b-c13e85cba398
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-2-consumer
userapp user-data 18 1739152927 1739161360
8433
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-2-consumer-59fe9d37-cd32-4a96-867b-c13e85cba398
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-2-consumer
userapp user-data 20 1715428183 1715431991
3808
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-3-consumer-21543ce6-dcd3-4bf5-992a-faf3af8c5895
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-3-consumer
userapp user-data 8 1738854860 1738858541
3681
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-3-consumer-21543ce6-dcd3-4bf5-992a-faf3af8c5895
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-3-consumer
userapp user-data 4 1704061352 1704065142
3790
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-3-consumer-21543ce6-dcd3-4bf5-992a-faf3af8c5895
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-3-consumer
userapp user-data 12 1743370130 1743373727
3597
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-3-consumer-21543ce6-dcd3-4bf5-992a-faf3af8c5895
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-3-consumer
userapp user-data 16 1696027581 1696031399
3818
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-3-consumer-21543ce6-dcd3-4bf5-992a-faf3af8c5895
/172.21.82.140
userapp-aa8f851b-26b9-4dad-ad3c-333caeae388d-StreamThread-3-consumer
userapp user-data 21 1722643635 1722647766
4131
userapp-dc160b83-62db-41ea-9266-ffefe233fa83-StreamThread-1-consumer-19fdbf9f-fb65-45fb-a020-aa16e9da76c5
/172.21.64.86
userapp-dc160b83-62db-41ea-9266-ffefe233fa83-StreamThread-1-consumer
userapp user-data 23 1675792619 1675800530
7911
userapp-dc160b83-62db-41ea-9266-ffefe233fa83-StreamThread-3-consumer-e98dae60-89b9-44b0-a366-d533b288fcf8
/172.21.64.86
userapp-dc160b83-62db-41ea-9266-ffefe233fa83-StreamThread-3-consumer
userapp user-data 22 1735482901 1735490720
7819
userapp-dc160b83-62db-41ea-9266-ffefe233fa83-StreamThread-2-consumer-3553c1a0-def4-453d-a4fb-a6c4f08ad3f2
/172.21.64.86
userapp-dc160b83-62db-41ea-9266-ffefe233fa83-StreamThread-2-consumer
Regards,
Navneeth
|