spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Graves <tgraves...@gmail.com>
Subject Re: [VOTE] [SPARK-27495] SPIP: Support Stage level resource configuration and scheduling
Date Fri, 13 Sep 2019 16:59:39 GMT
Thanks everyone so far for the voting and the feedback, bumping this
up as vote is scheduling to end today.

Tom

On Wed, Sep 11, 2019 at 1:10 PM Bryan Cutler <cutlerb@gmail.com> wrote:
>
> +1 (non-binding), looks good!
>
> On Wed, Sep 11, 2019 at 10:05 AM Ryan Blue <rblue@netflix.com.invalid> wrote:
>>
>> +1
>>
>> This is going to be really useful. Thanks for working on it!
>>
>> On Wed, Sep 11, 2019 at 9:38 AM Felix Cheung <felixcheung_m@hotmail.com> wrote:
>>>
>>> +1
>>>
>>> ________________________________
>>> From: Thomas graves <tgraves@apache.org>
>>> Sent: Wednesday, September 4, 2019 7:24:26 AM
>>> To: dev <dev@spark.apache.org>
>>> Subject: [VOTE] [SPARK-27495] SPIP: Support Stage level resource configuration
and scheduling
>>>
>>> Hey everyone,
>>>
>>> I'd like to call for a vote on SPARK-27495 SPIP: Support Stage level
>>> resource configuration and scheduling
>>>
>>> This is for supporting stage level resource configuration and
>>> scheduling.  The basic idea is to allow the user to specify executor
>>> and task resource requirements for each stage to allow the user to
>>> control the resources required at a finer grain. One good example here
>>> is doing some ETL to preprocess your data in one stage and then feed
>>> that data into an ML algorithm (like tensorflow) that would run as a
>>> separate stage.  The ETL could need totally different resource
>>> requirements for the executors/tasks than the ML stage does.
>>>
>>> The text for the SPIP is in the jira description:
>>>
>>> https://issues.apache.org/jira/browse/SPARK-27495
>>>
>>> I split the API and Design parts into a google doc that is linked to
>>> from the jira.
>>>
>>> This vote is open until next Fri (Sept 13th).
>>>
>>> [ ] +1: Accept the proposal as an official SPIP
>>> [ ] +0
>>> [ ] -1: I don't think this is a good idea because ...
>>>
>>> I'll start with my +1
>>>
>>> Thanks,
>>> Tom
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>
>>
>>
>> --
>> Ryan Blue
>> Software Engineer
>> Netflix

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Mime
View raw message