spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zhen <z...@latrobe.edu.au>
Subject Re: problem starting the history server on EC2
Date Thu, 12 Jun 2014 00:18:32 GMT
I tried everything including sudo, but it still did not work using the local
directory.
However, I finally got it working by getting the history server to log into
hdfs.
I first created a directory in hdfs like the following:
./ephemeral-hdfs/bin/hadoop fs -mkdir /spark_logs
Then I started the started the history server like the following.

./start-history-server.sh hdfs:///spark_logs --port 18080

In order to see the history server UI I needed to open up inbound traffic
for the port 18080 in AWS. As follows

custom TCP  port 18080 from anywhere

Hope this will help others.

Zhen



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/problem-starting-the-history-server-on-EC2-tp7361p7436.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message