spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Silvio Fiorito <>
Subject Re: clean up of state in State Dstream
Date Sat, 13 Dec 2014 03:32:08 GMT
If you no longer need to maintain state for a key, just return None for that value and it gets

From: Sunil Yarram <<>>
Date: Friday, December 12, 2014 at 9:44 PM
To: "<>" <<>>
Subject: clean up of state in State Dstream

I am using updateStateByKey to maintain state in my streaming application, the state gets
accumulated over time.
Is there a way i can delete the old state data or put a limit on the amount of state the State
Dstream can keep in the system.


View raw message