spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dino Fancellu <>
Subject Local Spark talking to remote HDFS?
Date Mon, 24 Aug 2015 18:46:05 GMT
I have a file in HDFS inside my HortonWorks HDP 2.3_1 VirtualBox VM.

If I go into the guest spark-shell and refer to the file thus, it works fine

  val words=sc.textFile("hdfs:///tmp/people.txt")

However if I try to access it from a local Spark app on my Windows host, it
doesn't work

  val conf = new SparkConf().setMaster("local").setAppName("My App")
  val sc = new SparkContext(conf)
  val words=sc.textFile("hdfs://localhost:8020/tmp/people.txt")


The port 8020 is open, and if I choose the wrong file name, it will tell me

My pom has


Am I doing something wrong?


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message