spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naveen Madhire <>
Subject pyspark issue
Date Mon, 27 Jul 2015 21:40:46 GMT

I am running pyspark in windows and I am seeing an error while adding
pyfiles to the sparkcontext. below is the example,

sc = SparkContext("local","Sample",pyFiles="C:/sample/")

this fails with no file found error for "C"

The below logic is treating the path as individual files like "C", ":" "/" etc.

It works if I use Spark Conf,

sc = SparkContext("local","Sample",conf=sparkConf)

Is this an existing issue or I am not including the files in correct
way in Spark Context?


when I run this, I am getting

View raw message