spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DanielH <>
Subject Re: debugger
Date Tue, 10 Jun 2014 07:36:59 GMT
Hi Josh,

I came across this post when looking for a debugger or RDD visualization
tool for Spark. I am using Spark 0.9.1 and upgrading soon to Spark 1.0. The
links you posted are dead. Can you please direct me to how I can debug my
existing Spark job.

Will I need to edit my existing job's code in addition to setting any
environment variables/parameters.

The problem: I am running Bagel on a very large graph and when the job gets
to the final step (saveAsTextFile) it will hang for up to many days until I
kill it. Oftentimes if I simply rerun the job, it will finish in an hour
which is the expected amount of time it should take.


View this message in context:
Sent from the Apache Spark Developers List mailing list archive at

View raw message