spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hao Ren <inv...@gmail.com>
Subject Re: [Streaming] Difference between windowed stream and stream with large batch size?
Date Wed, 16 Mar 2016 09:00:00 GMT
Any ideas ?

Feel free to ask me more details, if my questions are not clear.

Thank you.

On Mon, Mar 7, 2016 at 3:38 PM, Hao Ren <invkrh@gmail.com> wrote:

> I want to understand the advantage of using windowed stream.
>
> For example,
>
> Stream 1:
> initial duration = 5 s,
> and then transformed into a stream windowed by (*windowLength = *30s, *slideInterval
> = *30s)
>
> Stream 2:
> Duration = 30 s
>
> Questions:
>
> 1. Is Stream 1 equivalent to Stream 2 on behavior ? Do users observe the
> same result ?
> 2. If yes, what is the advantage of one vs. another ? Performance or
> something else ?
> 3. Is a stream with large batch reasonable , say 30 mins or even an hour ?
>
> Thank you.
>
> --
> Hao Ren
>
> Data Engineer @ leboncoin
>
> Paris, France
>



-- 
Hao Ren

Data Engineer @ leboncoin

Paris, France

Mime
View raw message