kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Praveen <praveev...@gmail.com>
Subject Re: Writing streams to kafka topic
Date Sun, 03 Sep 2017 01:24:37 GMT
Thanks. I was able to quickly build a simple example out of this. Also saw
the issue with punctuate and your “tick” feed recommendation for now.

- Praveen

On Fri, Sep 1, 2017 at 9:48 AM, Matthias J. Sax <matthias@confluent.io>
wrote:

> Hi,
>
> this is not supported by the DSL layer. What you would need to do, is to
> add a custom stateful transform() operator after there window
> (`stream.groupByKey().aggregate().toStream().transform().to()`), that
> buffers the output and remembers the latest result. Second, you would
> schedule a punctuation that emit the data whenever you want.
>
> Hope this helps.
>
>
> -Matthias
>
> On 8/31/17 9:52 PM, Praveen wrote:
> > Hi,
> >
> > I have a use case where I want to schedule processing of events in the
> > future. I am not really sure if this a proper use of stream processing
> > application. But I was looking at KTable and kafka streams api to see if
> > this was possible.
> >
> > So far the pattern I have is:
> >     FEED -> changelog stream -> groupByKey() -> window -> write to
> > different kafka topic
> >
> > The window here i believe would be the TumblingWindow for my use case.
> I'd
> > like to write back to a kafka topic only after the window retention ends.
> > The documentation
> > <http://docs.confluent.io/current/streams/developer-
> guide.html#writing-streams-back-to-kafka>
> > says that streams may only be written "continuously" to the kafka topic.
> Is
> > that the case?
> >
> > - Praveen
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message