spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <matei.zaha...@gmail.com>
Subject Re: Spark to utilize HDFS's mmap caching
Date Mon, 12 May 2014 06:04:10 GMT
Yes, Spark goes through the standard HDFS client and will automatically benefit from this.

Matei

On May 8, 2014, at 4:43 AM, Chanwit Kaewkasi <chanwit@gmail.com> wrote:

> Hi all,
> 
> Can Spark (0.9.x) utilize the caching feature in HDFS 2.3 via
> sc.textFile() and other HDFS-related APIs?
> 
> http://hadoop.apache.org/docs/r2.3.0/hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html
> 
> Best regards,
> 
> -chanwit
> 
> --
> Chanwit Kaewkasi
> linkedin.com/in/chanwit


Mime
View raw message