spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Vogel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-6403) Launch master as spot instance on EC2
Date Fri, 20 Mar 2015 17:45:38 GMT

    [ https://issues.apache.org/jira/browse/SPARK-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371715#comment-14371715
] 

Adam Vogel commented on SPARK-6403:
-----------------------------------

I'm working on a pull request for this. From the user perspective, it will add a --master-spot-price
command line argument. Similar to --spot-price, it will indicate both the bid and the fact
that master should be requested as a spot instance.

> Launch master as spot instance on EC2
> -------------------------------------
>
>                 Key: SPARK-6403
>                 URL: https://issues.apache.org/jira/browse/SPARK-6403
>             Project: Spark
>          Issue Type: New Feature
>          Components: EC2
>    Affects Versions: 1.2.1
>            Reporter: Adam Vogel
>            Priority: Minor
>
> Currently the spark_ec2.py script only supports requesting slaves as spot instances.
Launching the master as a spot instance has potential cost savings, at the risk of losing
the Spark cluster without warning. Unless users include logic for relaunching slaves when
lost, it is usually the case that all slaves are lost simultaneously. Thus, for jobs which
do not require resilience to losing spot instances, being able to launch the master as a spot
instance saves money.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message