spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Garlapati, Suryanarayana (Nokia - IN/Bangalore)" <>
Subject RE: External shuffle service on K8S
Date Sat, 27 Oct 2018 08:11:00 GMT
There is an unmerged PR which can be used against spark 2.4(if you are interested) or master
branch(3.0). Spark 2.3 K8S lacks lot of features. I suggest you upgrade to 2.4 which will
be released in few days from now.


From: Matt Cheah <>
Sent: Saturday, October 27, 2018 6:12 AM
To: Li Gao <>;
Subject: Re: External shuffle service on K8S

Hi there,

Please see for more discussion around this

-Matt Cheah

From: Li Gao <<>>
Date: Friday, October 26, 2018 at 9:10 AM
To: "<>" <<>>
Cc: "<>" <<>>,
"<>" <<>>
Subject: Re: External shuffle service on K8S

There are existing 2.2 based ext shuffle on the fork: []<>

You can modify it to suit your needs.


On Fri, Oct 26, 2018 at 3:22 AM vincent gromakowski <<>>
No it's on the roadmap >2.4

Le ven. 26 oct. 2018 à 11:15, 曹礼俊 <<>>
a écrit :
Hi all:

Does Spark 2.3.2 supports external shuffle service on Kubernetes?

I have looked up the documentation(
but couldn't find related suggestions.

If suppports, how can I enable it?

Best Regards

Lijun Cao

View raw message