spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicholas Chammas (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-6220) Allow extended EC2 options to be passed through spark-ec2
Date Mon, 09 Mar 2015 03:15:39 GMT
Nicholas Chammas created SPARK-6220:
---------------------------------------

             Summary: Allow extended EC2 options to be passed through spark-ec2
                 Key: SPARK-6220
                 URL: https://issues.apache.org/jira/browse/SPARK-6220
             Project: Spark
          Issue Type: Improvement
          Components: EC2
            Reporter: Nicholas Chammas
            Priority: Minor


There are many EC2 options exposed by the boto library that spark-ec2 uses. 

Over time, many of these EC2 options have been bubbled up here and there to become spark-ec2
options.

Examples:
* spot prices
* placement groups
* VPC, subnet, and security group assignments

It's likely that more and more EC2 options will trickle up like this to become spark-ec2 options.

While major options are well suited to this type of promotion, we should probably allow users
to pass through EC2 options they want to use through spark-ec2 in some generic way.

Let's add two options:
* {{--ec2-instance-option}} -> [{{boto::run_instances}}|http://boto.readthedocs.org/en/latest/ref/ec2.html#boto.ec2.connection.EC2Connection.run_instances]
* {{--ec2-spot-instance-option}} -> [{{boto::request_spot_instances}}|http://boto.readthedocs.org/en/latest/ref/ec2.html#boto.ec2.connection.EC2Connection.request_spot_instances]

Each option can be specified multiple times and is simply passed directly to the underlying
boto call.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message