kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Noll <mich...@confluent.io>
Subject Re: Kafka null keys - OK or a problem?
Date Mon, 10 Oct 2016 06:48:00 GMT
FYI: Kafka's new Java producer (which ships with Kafka) the behavior is as
follows:  If no partition is explicitly specified (to send the message to)
AND the key is null, then the DefaultPartitioner [1] will assign messages
to topic partitions in a round-robin fashion.  See the javadoc and also the
little bit of code in [1] for details.

Not sure which Go client you're using exactly so I can't tell whether your
Go client follows the behavior of Kafka's Java producer.

-Michael




[1]
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/producer/internals/DefaultPartitioner.java


On Mon, Oct 10, 2016 at 7:53 AM, Ali Akhtar <ali.rac200@gmail.com> wrote:

> If keys are null, what happens in terms of partitioning, is the load spread
> evenly..?
>
> On Mon, Oct 10, 2016 at 7:59 AM, Gwen Shapira <gwen@confluent.io> wrote:
>
> > Kafka itself supports null keys. I'm not sure about the Go client you
> > use, but Confluent's Go client also supports null keys
> > (https://github.com/confluentinc/confluent-kafka-go/).
> >
> > If you decide to generate keys and you want even spread, a random
> > number generator is probably your best bet.
> >
> > Gwen
> >
> > On Sun, Oct 9, 2016 at 6:05 PM, Ali Akhtar <ali.rac200@gmail.com> wrote:
> > > A kafka producer written elsewhere that I'm using, which uses the Go
> > kafka
> > > driver, is sending messages where the key is null.
> > >
> > > Is this OK - or will this cause issues due to partitioning not
> happening
> > > correctly?
> > >
> > > What would be a good way to generate keys in this case, to ensure even
> > > partition spread?
> > >
> > > Thanks.
> >
> >
> >
> > --
> > Gwen Shapira
> > Product Manager | Confluent
> > 650.450.2760 | @gwenshap
> > Follow us: Twitter | blog
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message