spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikhil Goyal <nownik...@gmail.com>
Subject Timed aggregation in Spark
Date Mon, 23 May 2016 20:28:03 GMT
Hi all,

I want to aggregate my data for 5-10 min and then flush the aggregated data
to some database like vertica. updateStateByKey is not exactly helpful in
this scenario as I can't flush all the records at once, neither can I clear
the state. I wanted to know if anyone else has faced a similar issue and
how did they handle it.

Thanks
Nikhil

Mime
View raw message