spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghava <>
Subject DAG visualization: no visualization information available with history server
Date Mon, 01 Feb 2016 04:38:37 GMT
Hello All,

I am running the history server for a completed application. This
application was run with the following parameters

bin/spark-submit --class <class_name> --master local[2] --conf
spark.local.dir=/mnt/ --conf spark.eventLog.dir=/mnt/sparklog/ --conf
spark.eventLog.enabled=true --conf spark.ui.retainedJobs=10000 --conf
spark.ui.retainedStages=10000 <jar_name>

In the Spark Web UI (http://localhost:18080/), the DAG visualization of only
the most recent job is available. For rest of the jobs, I get the following

No visualization information available for this job!
If this is an old job, its visualization metadata may have been cleaned up
over time.
You may consider increasing the value of spark.ui.retainedJobs and

I did increase the retainedJobs and retainedStages value to 10,000. What
else should be done to retain the visualization information of all the

Thanks in advance.


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message