samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mayur Rustagi" <mayur.rust...@gmail.com>
Subject Re: Do we want to have the sliding window implementation?
Date Wed, 10 Sep 2014 05:31:23 GMT
Slide window is another dimension to processing

say my batch is 3 sec  & window is 9 sec then this is what I get

[ x1 x2 x3][x4 x5 x6][x7 x8 x9]

This is using slide but slide is equal to window size, but i may want to get last 3 elements
at any point of time then that would be


[ x1 x2 x3]  after3sec  [ x2 x3 x4] after3sec  [ x3 x4 x5] after3sec  [ x4 x5 x6]

to implement this  you use batch of 3 sec, window of 9 sec & slide duration of 3 sec.
So we are sliding every 3 sec & also getting a batch every 3 sec. 





-- 
Regards,
Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rustagi

On Wed, Sep 10, 2014 at 5:25 AM, Yan Fang <yanfang724@gmail.com> wrote:

> Hi guys,
> realize that both Storm and Spark Streaming have sliding window
> implementation while Samza only has the fixed window (not sure if it's a
> correct name). I think you guys must consider this idea at the beginning of
> designing the Samza. What was the thought? Thank you.
> Cheers,
> Fang, Yan
> yanfang724@gmail.com
> +1 (206) 849-4108
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message