spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy Davidson <>
Subject Re: Trouble while running spark at ec2 cluster
Date Mon, 18 Jul 2016 16:11:26 GMT
Hi Hassan

Typically I log on to my master to submit my app.

[ec2-user@ip-172-31-11-222 bin]$ echo $SPARK_ROOT


[ec2-user@ip-172-31-11-222 bin]$echo $MASTER_URL


[ec2-user@ip-172-31-11-222 bin]$ $SPARK_ROOT/bin/spark-submit \

--class "com.pws.sparkStreaming.collector.StreamingKafkaCollector" \

--master $MASTER_URL

I think you might be trying to launch your application from a  machine
outside of your ec2 cluster. I do not think that is going to work when you
submit to port 7077 because the driver is going to be on your local machine.
Also you probably have a file wall issue


From:  Hassaan Chaudhry <>
Date:  Friday, July 15, 2016 at 9:32 PM
To:  "user @spark" <>
Subject:  Trouble while running spark at ec2 cluster

> Hi 
> I have launched my cluster and I am trying to submit my application to run on
> cluster but its not allowing me to connect . It prompts  the following error
> "Master endpoint 
> spark://
> <>  was not a
> REST server." The command I use to run my application on cluster is
> " /spark-1.6.1/bin/spark-submit --master spark://
> <>
> --deploy-mode cluster --class BFS target/scala-
> 2.10/scalaexample_2.10-1.0.jar "
> Am i missing something ? Your help will be highly appreciated .
> P.S  I have even tried adding inbound rule to my master node but still no
> success.
> Thanks 

View raw message