samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yi Pan <nickpa...@gmail.com>
Subject Re: Batch processing stream-stream joins
Date Mon, 04 Jan 2016 18:13:16 GMT
Hi, Rick,

Please refer to the whole discussion on SAMZA-552, which exactly targets
the issues that you are considering. We have been working on the design and
proto-type since last year. The work was paused for the last few months due
to other priorities. We are planning to resume the work this Q1. Feel free
to read and comment on SAMZA-552.

Thanks!

-Yi

On Sat, Jan 2, 2016 at 7:00 PM, rick bolkey <rick@bolkey.org> wrote:

> Hi all,
>
> I'm looking for advice in how to set up a samza job that does a
> stream-stream join in a way that the code can be re-used in both streaming
> and batch (by re-hydrating our kafka queue with historical data).
>
> It seems we need a way to inform our job that there is "no more data" via a
> end of stream message. It also seems like any windowing aggregation that
> assumes there is no more data while streaming would need to be disabled. I
> wasn't able to find much discussion on the topic, so looking for some
> pointers.
>
> Thanks
> Rick
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message