spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hoai-Thu Vuong <thuv...@gmail.com>
Subject Re: about window length and slide interval
Date Wed, 14 Jan 2015 10:21:53 GMT
Oh I see, thank you all guys.

On Wed Jan 14 2015 at 4:35:16 PM Sean Owen <sowen@cloudera.com> wrote:

> You're just describing the normal operation of Spark Streaming them.
> Windowing is for when you want overlapping intervals. Here you simply
> do not window, and ask for intervals of whatever time you want. You
> get non-overlapping RDDs.
>
> On Wed, Jan 14, 2015 at 9:26 AM, Hoai-Thu Vuong <thuvh87@gmail.com> wrote:
> > I would like to counting value in non overlap window, so that I think, I
> can
> > do it with same value of window length and slide interval, (note that,
> this
> > value is multiple of batch interval). However, nothing is calculated.
> >
> > Thanks!
> >
> > On Wed Jan 14 2015 at 4:16:27 PM <francois.garillot@typesafe.com> wrote:
> >>
> >> Why would you want to ? That would be equivalent to setting your batch
> >> interval to the same duration you’re suggesting to specify for both
> window
> >> length and slide interval, IIUC.
> >>
> >> Here’s a nice explanation of windowing by TD :
> >> https://groups.google.com/forum/#!topic/spark-users/GQoxJHAAtX4
> >>
> >> —
> >> FG
> >>
> >>
> >> On Wed, Jan 14, 2015 at 9:58 AM, Hoai-Thu Vuong <thuvh87@gmail.com>
> wrote:
> >>>
> >>> Could we run spark streaming and reducebywindow with same window length
> >>> and slide interval?
>

Mime
View raw message