spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yan Fang <>
Subject Does RDD checkpointing store the entire state in HDFS?
Date Thu, 17 Jul 2014 00:38:06 GMT
Hi guys,

am wondering how the RDD checkpointing
Checkpointing> works in Spark Streaming. When I use updateStateByKey, does
the Spark store the entire state (at one time point) into the HDFS or only
put the transformation into the HDFS? Thank you.


Fang, Yan
+1 (206) 849-4108

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message