spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aniket Mokashi <aniket...@gmail.com>
Subject Re: Shuffle Files
Date Tue, 04 Mar 2014 08:32:18 GMT
>From BlockManager code + ShuffleMapTask code, it writes under
spark.local.dir or java.io.tmpdir.

val diskBlockManager = new DiskBlockManager(shuffleBlockManager,
    conf.get("spark.local.dir",  System.getProperty("java.io.tmpdir")))




On Mon, Mar 3, 2014 at 10:45 PM, Usman Ghani <usman@platfora.com> wrote:

> Where on the filesystem does spark write the shuffle files?
>



-- 
"...:::Aniket:::... Quetzalco@tl"

Mime
View raw message