kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Asaf Mesika <asaf.mes...@gmail.com>
Subject Re: Partition size skew using default partitioner (without key)
Date Tue, 05 Jul 2016 04:01:30 GMT
As we continue to track down the cause I'm trying to ping back here in case
someone new might have an answer to the question below?

On Thu, Jun 16, 2016 at 12:39 PM Asaf Mesika <asaf.mesika@gmail.com> wrote:

> Hi,
>
> We've noticed that we have some partitions receiving more messages than
> others. What I've done to learn that is:
> * In Kafka Manager, per a given topic, the list of Partition Information
> is displayed.
> * For each partition there's a column called Latest Offset - which I
> assume is the producer offset. As I understand, this is the total number of
> messages written to this partition since the topic was created (and/or the
> partition was added)
> * I plotted the two columns: Partition number and Latest Offset.
>
> This is what I got:
> [image: pasted1]
>
> I'm using Kafka 0.8.2.1 with the new Producer API. We've using default
> partitioner, and we're *not* supplying partition number nor a key, thus
> it should be round-robin.
>
> From some reason, we see this.
>
> I was wondering if someone from the community ever encountered such a
> behaviour?
>
> Thanks!
>
> Asaf Mesika
> Logz.io
>
>

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message