kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Kathuria <mkathu...@sprinklr.com>
Subject Re: Does kafka write key to broker?
Date Wed, 24 Jun 2015 16:45:40 GMT
Thanks Jason.

Hi Liquan,

I was doing this initially using KeyedMessage without setting the key. But
if the key is null, the custom partitioner won't be used and kafka will
write to random partition which is cached for some time within the
producer. I have a custom partitioner to have fairness between the
partitions.

-Thanks,
Mohit

On Tue, Jun 23, 2015 at 10:34 PM, Liquan Pei <liquanpei@gmail.com> wrote:

> Hi Mohit,
>
> If you instantiate the keyed message with
> val topic = "topic"
> val value = "value"
> val message = new KeyedMessage[String, String](topic,  value);
>
> Then the key in the KeyedMessage will be null.
>
> Hope this helps!
>
> Thanks,
> Liquan
>
> On Tue, Jun 23, 2015 at 8:18 AM, Mohit Kathuria <mkathuria@sprinklr.com>
> wrote:
>
> > Hi,
> >
> > We are using kafka 0.8.1.1 in our production cluster. I recently started
> > specifying key as the message itself. I just realised that the key is
> also
> > written to the broker which means that the data is duplicated within a
> > keyed message. I am going to change the key. Stupid mistake.
> >
> > However, just out of anxiety, I want to know whether we can turn off
> > writing the key to the broker. Any configuration I can change to achieve
> > this?
> >
> > -Thanks,
> > Mohit Kathuria
> >
>
>
>
> --
> Liquan Pei
> Department of Physics
> University of Massachusetts Amherst
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message