spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From salemi <>
Subject Spark Streaming + reduceByWindow(reduceFunc, invReduceFunc, windowDuration, slideDuration
Date Wed, 06 Aug 2014 22:43:04 GMT
I have a  DStream called eventData and it contains set of  Data objects
defined as followed:

case class Data(startDate: Long, endDate: Long, className: String, id:
String, state: String)

How would the reducer and inverse reducer functions look like if I would
like to add the data for current 3 second and filter out the last 3 second

eventData.reduceByWindow(/reduceFunc/, /invReduceFunc/, Minutes(15),


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message