spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From qinwei <wei....@dewmobile.net>
Subject problem with data locality api
Date Sun, 28 Sep 2014 06:05:05 GMT






Hi, everyone? ? I come across with a problem about data locality, i found these?example?code
in 《Spark-on-YARN-A-Deep-Dive-Sandy-Ryza.pdf》? ??? ??val locData = InputFormatInfo.computePreferredLocations(Seq(new
InputFormatInfo(conf, classOf[TextInputFormat], new Path(“myfile.txt”)))?? ??? ??val sc
= new SparkContext(conf, locData)? ? but i found the two confs above are of different types,
conf in the first line if of type?org.apache.hadoop.conf.Configuration, and conf in the second
line is of type SparkConf, ?can anyone explain that to me or give me some example code?? ??

qinwei

Mime
View raw message