spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <>
Subject Re: No event logs in yarn-cluster mode
Date Sat, 01 Aug 2015 20:05:45 GMT
On Sat, Aug 1, 2015 at 9:25 AM, Akmal Abbasov <>

> When I running locally(./run-example SparkPi), the event logs are being
> created, and I can start history server.
> But when I am trying
> ./spark-submit --class org.apache.spark.examples.SparkPi --master
> yarn-cluster file:///opt/hadoop/spark/examples/src/main/python/

Did you look for the event log on the machine where the Spark driver ran?
You're using a "file:" URL and on yarn-cluster, that is in some random
machine in the cluster, not your local machine launching the job.

Which is why you should probably write these logs to HDFS.

View raw message