kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eno Thereska <eno.there...@gmail.com>
Subject Re: KTable#through from window store to key value store
Date Sat, 17 Dec 2016 10:41:03 GMT
Ok, makes sense. We'll be putting out a KIP on updating the KTable semantics and one of the
things we'll add is the ability to materialize any KTable without explicitely needing to call
through. So you could do a .mapValues, you'll get a KTable back and then we'll probably have
a .materialize API to allow you to query that. Stay tuned.

Thanks
Eno


> On 16 Dec 2016, at 18:44, Mikael Högqvist <hoegqvist@gmail.com> wrote:
> 
> Hi Eno,
> 
> in this example it doesn't make much sense, but I could add a .mapValues
> that transforms the values and the use .through to materialize a table with
> the updated values.
> 
> It is possible to the access the new table as a KeyValueStore also, but it
> is much less convenient and I also expected that if I have a window store
> it would still be a window store after transformation/through.
> 
> Thanks,
> Mikael
> 
> On Fri, Dec 16, 2016 at 6:20 PM Eno Thereska <eno.thereska@gmail.com> wrote:
> 
>> Hi Mikael,
>> 
>> Currently that is not possible. Could you elaborate why you'd need that
>> since you can query from tableOne.
>> 
>> Thanks
>> Eno
>>> On 16 Dec 2016, at 10:45, Mikael Högqvist <hoegqvist@gmail.com> wrote:
>>> 
>>> Hi,
>>> 
>>> I have a small example topology that count words per minute (scala):
>>> 
>>>   words
>>>     .map { (key, word) =>
>>>       new KeyValue(word, Long.box(1L))
>>>     }
>>>     .groupByKey(Serdes.String, Serdes.Long)
>>>     .count(TimeWindows.of(5 * 60 * 1000L), tableOne)
>>>     .through(new WindowedSerde, Serdes.Long, s"$appId-count-topic",
>>> tableTwo)
>>> 
>>> The first table, tableOne, is a WindowStore and can be accessed using
>> fetch
>>> on the key and time range. After using .through to forward the data to
>>> another topic and table, tableTwo becomes a KeyValueStore. Is it possible
>>> to keep tableTwo as a WindowStore also?
>>> 
>>> Best,
>>> Mikael
>> 
>> 


Mime
View raw message