spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Or <and...@databricks.com>
Subject Re: Spark ec2 lunch problem
Date Mon, 24 Aug 2015 21:51:45 GMT
Hey Garry,

Have you verified that your particular VPC and subnet are open to the
world? In particular, have you verified the route table attached to your
VPC / subnet contains an internet gateway open to the public?

I've run into this issue myself recently and that was the problem for me.

-Andrew

2015-08-24 5:58 GMT-07:00 Robin East <robin.east@xense.co.uk>:

> spark-ec2 is the way to go however you may need to debug connectivity
> issues. For example do you know that the servers were correctly setup in
> AWS and can you access each node using ssh? If no then you need to work out
> why (it’s not a spark issue). If yes then you will need to work out why ssh
> via the spark-ec2 script is not working.
>
> I’ve used spark-ec2 successfully many times but have never used the
> —vpc-id and —subnet-id options and that may be the source of your problems,
> especially since it appears to be a hostname resolution issue. If you could
> confirm the above questions then maybe someone on the list can help
> diagnose the specific problem.
>
>
>
> -------------------------------------------------------------------------------
> Robin East
> *Spark GraphX in Action* Michael Malak and Robin East
> Manning Publications Co.
> http://www.manning.com/malak/
>
> On 24 Aug 2015, at 13:45, Garry Chen <gc92@cornell.edu> wrote:
>
> So what is the best way to deploy spark cluster in EC2 environment any
> suggestions?
>
> Garry
>
> *From:* Akhil Das [mailto:akhil@sigmoidanalytics.com
> <akhil@sigmoidanalytics.com>]
> *Sent:* Friday, August 21, 2015 4:27 PM
> *To:* Garry Chen <gc92@cornell.edu>
> *Cc:* user@spark.apache.org
> *Subject:* Re: Spark ec2 lunch problem
>
>
> It may happen that the version of spark-ec2 script you are using is buggy
> or sometime AWS have problem provisioning machines.
> On Aug 21, 2015 7:56 AM, "Garry Chen" <gc92@cornell.edu> wrote:
>
> Hi All,
>                 I am trying to lunch a spark ec2 cluster by running
>  spark-ec2 --key-pair=key --identity-file=my.pem --vpc-id=myvpc
> --subnet-id=subnet-011 --spark-version=1.4.1 launch spark-cluster but
> getting following message endless.  Please help.
>
>
> Warning: SSH connection error. (This could be temporary.)
> Host:
> SSH return code: 255
> SSH output: ssh: Could not resolve hostname : Name or service not known
>
>
>

Mime
View raw message