spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yifei Huang (PD)" <yif...@palantir.com>
Subject [DISCUSS][SPARK-25299] SPIP: Shuffle storage API
Date Wed, 08 May 2019 18:15:06 GMT
Hi everyone,

For the past several months, we have been working on an API for pluggable storage of shuffle
data. In this SPIP, we describe the proposed API, its implications, and how it fits into other
work being done in the Spark shuffle space. If you're interested in Spark shuffle, and especially
if you have done some work in this area already, please take a look at the SPIP and give us
your thoughts and feedback.

Jira Ticket: https://issues.apache.org/jira/browse/SPARK-25299
SPIP: https://docs.google.com/document/d/1d6egnL6WHOwWZe8MWv3m8n4PToNacdx7n_0iMSWwhCQ/edit

Thank you!

Yifei Huang and Matt Cheah

 


Mime
View raw message