spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DanielH <dhurw...@ebay.com>
Subject Re: debugger
Date Tue, 10 Jun 2014 07:36:59 GMT
Hi Josh,

I came across this post when looking for a debugger or RDD visualization
tool for Spark. I am using Spark 0.9.1 and upgrading soon to Spark 1.0. The
links you posted are dead. Can you please direct me to how I can debug my
existing Spark job.

Will I need to edit my existing job's code in addition to setting any
environment variables/parameters.

The problem: I am running Bagel on a very large graph and when the job gets
to the final step (saveAsTextFile) it will hang for up to many days until I
kill it. Oftentimes if I simply rerun the job, it will finish in an hour
which is the expected amount of time it should take.

Thanks!



--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/debugger-tp284p6982.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

Mime
View raw message