spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dongjoon Hyun <dongjoon.h...@gmail.com>
Subject Re: [VOTE] [SPARK-27495] SPIP: Support Stage level resource configuration and scheduling
Date Fri, 13 Sep 2019 19:26:52 GMT
+1

Thanks,
Dongjoon.

On Fri, Sep 13, 2019 at 9:59 AM Thomas Graves <tgravescs1@gmail.com> wrote:

> Thanks everyone so far for the voting and the feedback, bumping this
> up as vote is scheduling to end today.
>
> Tom
>
> On Wed, Sep 11, 2019 at 1:10 PM Bryan Cutler <cutlerb@gmail.com> wrote:
> >
> > +1 (non-binding), looks good!
> >
> > On Wed, Sep 11, 2019 at 10:05 AM Ryan Blue <rblue@netflix.com.invalid>
> wrote:
> >>
> >> +1
> >>
> >> This is going to be really useful. Thanks for working on it!
> >>
> >> On Wed, Sep 11, 2019 at 9:38 AM Felix Cheung <felixcheung_m@hotmail.com>
> wrote:
> >>>
> >>> +1
> >>>
> >>> ________________________________
> >>> From: Thomas graves <tgraves@apache.org>
> >>> Sent: Wednesday, September 4, 2019 7:24:26 AM
> >>> To: dev <dev@spark.apache.org>
> >>> Subject: [VOTE] [SPARK-27495] SPIP: Support Stage level resource
> configuration and scheduling
> >>>
> >>> Hey everyone,
> >>>
> >>> I'd like to call for a vote on SPARK-27495 SPIP: Support Stage level
> >>> resource configuration and scheduling
> >>>
> >>> This is for supporting stage level resource configuration and
> >>> scheduling.  The basic idea is to allow the user to specify executor
> >>> and task resource requirements for each stage to allow the user to
> >>> control the resources required at a finer grain. One good example here
> >>> is doing some ETL to preprocess your data in one stage and then feed
> >>> that data into an ML algorithm (like tensorflow) that would run as a
> >>> separate stage.  The ETL could need totally different resource
> >>> requirements for the executors/tasks than the ML stage does.
> >>>
> >>> The text for the SPIP is in the jira description:
> >>>
> >>> https://issues.apache.org/jira/browse/SPARK-27495
> >>>
> >>> I split the API and Design parts into a google doc that is linked to
> >>> from the jira.
> >>>
> >>> This vote is open until next Fri (Sept 13th).
> >>>
> >>> [ ] +1: Accept the proposal as an official SPIP
> >>> [ ] +0
> >>> [ ] -1: I don't think this is a good idea because ...
> >>>
> >>> I'll start with my +1
> >>>
> >>> Thanks,
> >>> Tom
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
> >>>
> >>
> >>
> >> --
> >> Ryan Blue
> >> Software Engineer
> >> Netflix
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>

Mime
View raw message