spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheuk Lam <chl...@hotmail.com>
Subject How to delete graph checkpoints?
Date Wed, 21 Jan 2015 17:25:39 GMT
This is a question about checkpointing on GraphX.

We'd like to automate deleting checkpoint files of old graphs.  The RDD
class has a getCheckpointFile() function, which allows us to retrieve the
checkpoint file of an old RDD and then delete it.  However, I couldn't find
a way to get hold of the corresponding checkpointed RDDs given the graph
reference; because the checkpoint of a GraphImpl is really done to the
underlying partitionsRDD in both VertexRDD and EdgeRDD, and that
partitionsRDD as defined today doesn't seem to be accessible from outside of
graphx.

Below is the declaration in VertexRDD.scala:
private[graphx] def partitionsRDD: RDD[ShippableVertexPartition[VD]]

We would really appreciate it if anyone could shed some light on solving
this problem, or anyone who has come across a similar problem could share a
solution or workaround.

Thank you,
Cheuk Lam





--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-delete-graph-checkpoints-tp21296.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message