kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Asaf Mesika <asaf.mes...@gmail.com>
Subject Partition size skew using default partitioner (without key)
Date Thu, 16 Jun 2016 09:39:21 GMT
Hi,

We've noticed that we have some partitions receiving more messages than
others. What I've done to learn that is:
* In Kafka Manager, per a given topic, the list of Partition Information is
displayed.
* For each partition there's a column called Latest Offset - which I assume
is the producer offset. As I understand, this is the total number of
messages written to this partition since the topic was created (and/or the
partition was added)
* I plotted the two columns: Partition number and Latest Offset.

This is what I got:
[image: pasted1]

I'm using Kafka 0.8.2.1 with the new Producer API. We've using default
partitioner, and we're *not* supplying partition number nor a key, thus it
should be round-robin.

>From some reason, we see this.

I was wondering if someone from the community ever encountered such a
behaviour?

Thanks!

Asaf Mesika
Logz.io

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message