spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Silvio Fiorito <silvio.fior...@granturing.com>
Subject Re: clean up of state in State Dstream
Date Sat, 13 Dec 2014 03:32:08 GMT
If you no longer need to maintain state for a key, just return None for that value and it gets
removed.

From: Sunil Yarram <yvsunil@gmail.com<mailto:yvsunil@gmail.com>>
Date: Friday, December 12, 2014 at 9:44 PM
To: "user@spark.apache.org<mailto:user@spark.apache.org>" <user@spark.apache.org<mailto:user@spark.apache.org>>
Subject: clean up of state in State Dstream

I am using updateStateByKey to maintain state in my streaming application, the state gets
accumulated over time.
Is there a way i can delete the old state data or put a limit on the amount of state the State
Dstream can keep in the system.

Thanks,
Sunil.

Mime
View raw message