kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Storing data in kafka keys
Date Thu, 06 Nov 2014 23:43:34 GMT
The keyed messages are typically used in two cases (1) you want messages
with the same key to be in the same partition and therefore those messages
with the same key will be consumed by the same consumer instance; (2) you
want to enable the log compaction feature for retention such that the
broker will over time garbage collect all messages with the same key,
except for the most recent message.

Thanks,

Jun

On Wed, Nov 5, 2014 at 10:15 AM, Ivan Balashov <ibalashov@gmail.com> wrote:

> Hi,
>
> It looks like it is a general practice to avoid storing data in kafka
> keys. Some examples of this: Camus, Secor both not using keys. Even
> such a swiss-army tool as kafkacat doesn't seem to have the ability to
> display key (although I might be wrong). Also, console producer does
> not display keys by default, which makes it confusing in initial quick
> checks for data.
>
> What's the story behind this? Should one think twice before tying
> business data to kafka keys?
>
> Thanks,
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message