spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Pivovarov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-17877) Can not checkpoint connectedComponents resulting graph
Date Tue, 11 Oct 2016 19:47:20 GMT

    [ https://issues.apache.org/jira/browse/SPARK-17877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15566385#comment-15566385
] 

Alexander Pivovarov commented on SPARK-17877:
---------------------------------------------

Another open issue with checkpointing is SPARK-14804

> Can not checkpoint connectedComponents resulting graph
> ------------------------------------------------------
>
>                 Key: SPARK-17877
>                 URL: https://issues.apache.org/jira/browse/SPARK-17877
>             Project: Spark
>          Issue Type: Bug
>          Components: GraphX
>    Affects Versions: 1.5.2, 1.6.2, 2.0.1
>            Reporter: Alexander Pivovarov
>            Priority: Minor
>
> The following code demonstrates the issue
> {code}
> import org.apache.spark.graphx._
> val users = sc.parallelize(List(3L -> "lucas", 7L -> "john", 5L -> "matt", 2L
-> "kelly"))
> val rel = sc.parallelize(List(Edge(3L, 7L, "collab"), Edge(5L, 3L, "advisor"), Edge(2L,
5L, "colleague"), Edge(5L, 7L, "pi")))
> sc.setCheckpointDir("/tmp/check")
> val g = Graph(users, rel)
> g.checkpoint
> val gg = g.connectedComponents
> gg.checkpoint
> gg.vertices.collect
> gg.edges.collect
> gg.isCheckpointed  // res5: Boolean = false
> {code}
> I think the last line should return true instead of false



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message