spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gtinside <gtins...@gmail.com>
Subject unread block data when reading from NFS
Date Sat, 13 Dec 2014 14:23:46 GMT
Hi ,

I am trying to read a csv file in the following way :
val csvData = sc.textFile("file:///tmp/sample.csv") 
csvData.collect().length

This works file on spark-shell but when I try to do spark-submit of the jar,
I get the following exceptions :
java.lang.IllegalStateException: unread block data
       
java.io.ObjectInputStream$BlockDataInputStream.setBlockDataMode(ObjectInputStream.java:2420)
        java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1380)
       
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
       
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
       
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
        java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
        java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
       
org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:62)
       
org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:87)
       
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:159)
       
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
       
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        java.lang.Thread.run(Thread.java:724)

Can you please help me here ?

Regards,
Gaurav




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/unread-block-data-when-reading-from-NFS-tp20672.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message