kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damian Guy <damian....@gmail.com>
Subject Re: what is the key used for change log topic backed by windowed store
Date Fri, 23 Dec 2016 17:29:48 GMT
Hi Sachin,

The key is a combination of the record key + window start time + a sequence
number. The timestamp is 8 bytes and the sequence number is 4 bytes.


On Thu, 22 Dec 2016 at 15:26 Sachin Mittal <sjmittal@gmail.com> wrote:

> Hi All,
> Our stream is something like
> builder.stream()
>         .groupByKey()
>         .aggregate(Initializer, Aggregator, TimeWindows, valueSerde,
> "table-name')
> So This creates a changelog topic.
> I was wondering what would be the key used for this topic.
> Would it be they key we use to group by or a compounded key of (our key,
> window key).
> I am asking this because what is observed that when stream runs for few
> days we start getting the exception where message size is greater than the
> allowable message size.
> However our messages are windowed on hourly windows, so size of a message
> should be far less than the max message size.
> Thanks
> Sachin

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message