spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix Cheung <>
Subject Re: [VOTE] [SPARK-24615] SPIP: Accelerator-aware Scheduling
Date Sun, 03 Mar 2019 03:35:39 GMT
I’m very hesitant with this.

I don’t want to vote -1, because I personally think it’s important to do, but I’d like
to see more discussion points addressed and not voting completely on the spirit of it.

First, SPIP doesn’t match the format of SPIP proposed and agreed on. (Maybe this is a minor
point and perhaps we should also vote to update the SPIP format)

Second, there are multiple pdf/google doc and JIRA. And I think for example the design sketch
is not covering the same points as the updated SPIP doc? It would help to make them align
before moving forward.

Third, the proposal touches on some fairly core and sensitive components, like the scheduler,
and I think more discussions are necessary. We have a few comments there and in the JIRA.

From: Marco Gaido <>
Sent: Saturday, March 2, 2019 4:18 AM
To: Weichen Xu
Cc: Yinan Li; Tom Graves; dev; Xingbo Jiang
Subject: Re: [VOTE] [SPARK-24615] SPIP: Accelerator-aware Scheduling

+1, a critical feature for AI/DL!

Il giorno sab 2 mar 2019 alle ore 05:14 Weichen Xu <<>>
ha scritto:
+1, nice feature!

On Sat, Mar 2, 2019 at 6:11 AM Yinan Li <<>>

On Fri, Mar 1, 2019 at 12:37 PM Tom Graves <> wrote:
+1 for the SPIP.


On Friday, March 1, 2019, 8:14:43 AM CST, Xingbo Jiang <<>>

Hi all,

I want to call for a vote of SPARK-24615<>.
It improves Spark by making it aware of GPUs exposed by cluster managers, and hence Spark
can match GPU resources with user task requests properly. The proposal<>
and production doc<>
was made available on dev@ to collect input. Your can also find a design sketch at SPARK-27005<>.

The vote will be up for the next 72 hours. Please reply with your vote:

+1: Yeah, let's go forward and implement the SPIP.
+0: Don't really care.
-1: I don't think this is a good idea because of the following technical reasons.

Thank you!


View raw message