spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shao, Saisai" <>
Subject RE: Shuffle to HDFS
Date Mon, 26 Jan 2015 05:44:16 GMT
Hi Larry,

I don’t think current Spark’s shuffle can support HDFS as a shuffle output. Anyway, is
there any specific reason to spill shuffle data to HDFS or NFS, this will severely increase
the shuffle time.


From: Larry Liu []
Sent: Sunday, January 25, 2015 4:45 PM
Subject: Shuffle to HDFS

How to change shuffle output to HDFS or NFS?
View raw message