spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From AlexG <swift...@gmail.com>
Subject using pyspark with standalone cluster
Date Mon, 01 Jun 2015 18:18:00 GMT
I've followed the instructions for setting up a standalone spark cluster (on
EC2):
- install spark on all the machines
- enabled passwordless ssh
- setup the conf/slaves file
- start the master and slaves with the provided scripts

The status on the 8080 port of the master tells me that the master and
executors are all running. I can successfully use pyspark from the master. 

However, if I try to call pyspark remotely from my laptop, with 
MASTER=spark://<ip>:7077 pyspark,
I get these errors:
15/06/01 10:02:14 INFO AppClient$ClientActor: Connecting to master
akka.tcp://sparkMaster@<IP>:7077/user/Master...
15/06/01 10:02:34 INFO AppClient$ClientActor: Connecting to master
akka.tcp://sparkMaster@<IP>:7077/user/Master...
15/06/01 10:02:54 INFO AppClient$ClientActor: Connecting to master
akka.tcp://sparkMaster@<IP>:7077/user/Master...
15/06/01 10:03:14 ERROR SparkDeploySchedulerBackend: Application has been
killed. Reason: All masters are unresponsive! Giving up.
15/06/01 10:03:14 ERROR TaskSchedulerImpl: Exiting due to error from cluster
scheduler: All masters are unresponsive! Giving up.

Any idea what's going on here? I set port 7077 to be publicly accessible...




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/using-pyspark-with-standalone-cluster-tp23099.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message