spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robin East <robin.e...@xense.co.uk>
Subject Re: Spark ec2 lunch problem
Date Mon, 24 Aug 2015 12:58:20 GMT
spark-ec2 is the way to go however you may need to debug connectivity issues. For example do
you know that the servers were correctly setup in AWS and can you access each node using ssh?
If no then you need to work out why (it’s not a spark issue). If yes then you will need
to work out why ssh via the spark-ec2 script is not working.

I’ve used spark-ec2 successfully many times but have never used the —vpc-id and —subnet-id
options and that may be the source of your problems, especially since it appears to be a hostname
resolution issue. If you could confirm the above questions then maybe someone on the list
can help diagnose the specific problem.


-------------------------------------------------------------------------------
Robin East
Spark GraphX in Action Michael Malak and Robin East
Manning Publications Co.
http://www.manning.com/malak/ <http://www.manning.com/malak/>

> On 24 Aug 2015, at 13:45, Garry Chen <gc92@cornell.edu> wrote:
> 
> So what is the best way to deploy spark cluster in EC2 environment any suggestions?
>  
> Garry
>  
> From: Akhil Das [mailto:akhil@sigmoidanalytics.com <mailto:akhil@sigmoidanalytics.com>]

> Sent: Friday, August 21, 2015 4:27 PM
> To: Garry Chen <gc92@cornell.edu <mailto:gc92@cornell.edu>>
> Cc: user@spark.apache.org <mailto:user@spark.apache.org>
> Subject: Re: Spark ec2 lunch problem
>  
> It may happen that the version of spark-ec2 script you are using is buggy or sometime
AWS have problem provisioning machines.
> 
> On Aug 21, 2015 7:56 AM, "Garry Chen" <gc92@cornell.edu <mailto:gc92@cornell.edu>>
wrote:
> Hi All,
>                 I am trying to lunch a spark ec2 cluster by running  spark-ec2 --key-pair=key
--identity-file=my.pem --vpc-id=myvpc --subnet-id=subnet-011 --spark-version=1.4.1 launch
spark-cluster but getting following message endless.  Please help.
>  
>  
> Warning: SSH connection error. (This could be temporary.)
> Host:
> SSH return code: 255
> SSH output: ssh: Could not resolve hostname : Name or service not known


Mime
View raw message