You want to write it to a local file on the machine? Try using "file:///path/to/target/mydir/" instead
I'm not sure what behavior would be if you did this on a multi-machine cluster though -- you may get a bit of data on each machine in that local directory.
On Thu, Jan 2, 2014 at 12:22 PM, Philip Ogren <email@example.com> wrote:
I have a very simple Spark application that looks like the following:
var myRdd: RDD[Array[String]] = initMyRdd()
The println statements work as expected. The first saveAsTextFile statement also works as expected. The second saveAsTextFile statement does not (even if the first is commented out.) I get the exception pasted below. If I inspect "target/mydir" I see that there is a directory called _temporary/0/_temporary/attempt_201401020953_0000_m_000000_1 which contains an empty part-00000 file. It's curious because this code worked before with Spark 0.8.0 and now I am running on Spark 0.8.1. I happen to be running this on Windows in "local" mode at the moment. Perhaps I should try running it on my linux box.
Exception in thread "main" org.apache.spark.SparkException: Job aborted: Task 2.0:0 failed more than 0 times; aborting job java.lang.NullPointerException