spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hao Ren <inv...@gmail.com>
Subject [Streaming] Difference between windowed stream and stream with large batch size?
Date Mon, 07 Mar 2016 14:38:32 GMT
I want to understand the advantage of using windowed stream.

For example,

Stream 1:
initial duration = 5 s,
and then transformed into a stream windowed by (*windowLength = *30s,
*slideInterval
= *30s)

Stream 2:
Duration = 30 s

Questions:

1. Is Stream 1 equivalent to Stream 2 on behavior ? Do users observe the
same result ?
2. If yes, what is the advantage of one vs. another ? Performance or
something else ?
3. Is a stream with large batch reasonable , say 30 mins or even an hour ?

Thank you.

-- 
Hao Ren

Data Engineer @ leboncoin

Paris, France

Mime
View raw message