spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-3405) EC2 cluster creation on VPC
Date Tue, 21 Oct 2014 06:30:34 GMT

    [ https://issues.apache.org/jira/browse/SPARK-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178024#comment-14178024
] 

Apache Spark commented on SPARK-3405:
-------------------------------------

User 'mvj101' has created a pull request for this issue:
https://github.com/apache/spark/pull/2872

> EC2 cluster creation on VPC
> ---------------------------
>
>                 Key: SPARK-3405
>                 URL: https://issues.apache.org/jira/browse/SPARK-3405
>             Project: Spark
>          Issue Type: New Feature
>          Components: EC2
>    Affects Versions: 1.0.2
>         Environment: Ubuntu 12.04
>            Reporter: Dawson Reid
>            Priority: Minor
>
> It would be very useful to be able to specify the EC2 VPC in which the Spark cluster
should be created. 
> When creating a Spark cluster on AWS via the spark-ec2 script there is no way to specify
a VPC id of the VPC you would like the cluster to be created in. The script always creates
the cluster in the default VPC. 
> In my case I have deleted the default VPC and the spark-ec2 script errors out with the
following : 
> Setting up security groups...
> Creating security group test-master
> ERROR:boto:400 Bad Request
> ERROR:boto:<?xml version="1.0" encoding="UTF-8"?>
> <Response><Errors><Error><Code>VPCIdNotSpecified</Code><Message>No
default VPC for this user</Message></Error></Errors><RequestID>312a2281-81a1-4d3c-ba10-0593a886779d</RequestID></Response>
> Traceback (most recent call last):
>   File "./spark_ec2.py", line 860, in <module>
>     main()
>   File "./spark_ec2.py", line 852, in main
>     real_main()
>   File "./spark_ec2.py", line 735, in real_main
>     conn, opts, cluster_name)
>   File "./spark_ec2.py", line 247, in launch_cluster
>     master_group = get_or_make_group(conn, cluster_name + "-master")
>   File "./spark_ec2.py", line 143, in get_or_make_group
>     return conn.create_security_group(name, "Spark EC2 group")
>   File "/home/dawson/Develop/spark-1.0.2/ec2/third_party/boto-2.4.1.zip/boto-2.4.1/boto/ec2/connection.py",
line 2011, in create_security_group
>   File "/home/dawson/Develop/spark-1.0.2/ec2/third_party/boto-2.4.1.zip/boto-2.4.1/boto/connection.py",
line 925, in get_object
> boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request
> <?xml version="1.0" encoding="UTF-8"?>
> <Response><Errors><Error><Code>VPCIdNotSpecified</Code><Message>No
default VPC for this user</Message></Error></Errors><RequestID>312a2281-81a1-4d3c-ba10-0593a886779d</RequestID></Response>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message