spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haopu Wang" <HW...@qilinsoft.com>
Subject data locality
Date Fri, 18 Jul 2014 07:35:07 GMT
I have a standalone spark cluster and a HDFS cluster which share some of nodes.

 

When reading HDFS file, how does spark assign tasks to nodes? Will it ask HDFS the location
for each file block in order to get a right worker node?

 

How about a spark cluster on Yarn?

 

Thank you very much!

 

Mime
View raw message