spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Asim Jalis <>
Subject Streaming Checkpointing
Date Fri, 09 Jan 2015 09:19:31 GMT
In Spark Streaming apps if I enable ssc.checkpoint(dir) does this
checkpoint all RDDs? Or is it just checkpointing windowing and state RDDs?

For example, if in a DStream I am using an iterative algorithm on a
non-state non-window RDD, do I have to checkpoint it explicitly myself, or
can I assume that ssc.checkpoint has taken care of checkpointing it?

View raw message