spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alessandro Lulli <alessandro.lu...@gmail.com>
Subject Computation time increasing every super-step
Date Tue, 11 Mar 2014 15:07:03 GMT
Hi All,

I'm facing a performance degradation running an iterative algorithm built
using Spark 0.9 and GraphX.
I'm using org.apache.spark.graphx.Pregel to run the iterative algorithm.

My graph has 2395 vertex 7462 edges.

Every super step the computation time increase significantly. The steps 1-5
are executed in the order of seconds instead steps > 10 are executed in the
order of tens of minutes and always increasing.

In every step each vertex executes always the same actions and sends a
message to all of its neighbor. The graph doesn't change topology during
execution.

I tried also to perform a checkpoint of vertices, edges and triplets at the
end of each step but i'm encountering the same issue.

Could you please help me solve this issue?
Please let me know if i'm missing something or you need additional details.

Thanks
Alessandro

Mime
View raw message