spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <van...@cloudera.com>
Subject Re: No event logs in yarn-cluster mode
Date Sat, 01 Aug 2015 20:05:45 GMT
On Sat, Aug 1, 2015 at 9:25 AM, Akmal Abbasov <akmal.abbasov@icloud.com>
wrote:

> When I running locally(./run-example SparkPi), the event logs are being
> created, and I can start history server.
> But when I am trying
> ./spark-submit --class org.apache.spark.examples.SparkPi --master
> yarn-cluster file:///opt/hadoop/spark/examples/src/main/python/pi.py
>

Did you look for the event log on the machine where the Spark driver ran?
You're using a "file:" URL and on yarn-cluster, that is in some random
machine in the cluster, not your local machine launching the job.

Which is why you should probably write these logs to HDFS.

Mime
View raw message