spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sung Hwan Chung <coded...@cs.stanford.edu>
Subject Spark executor error
Date Thu, 26 Jun 2014 03:08:38 GMT
I'm seeing the following message in the log of an executor. Anyone
seen this error? After this, the executor seems to lose the cache, and
but besides that the whole thing slows down drastically - I.e. it gets
stuck in a reduce phase for 40+ minutes, whereas before it was
finishing reduces in 2~3 seconds.



14/06/25 19:22:31 WARN SendingConnection: Error writing in connection
to ConnectionManagerId(alpinenode7.alpinenow.local,46251)
java.lang.NullPointerException
	at org.apache.spark.network.MessageChunkHeader.buffer$lzycompute(MessageChunkHeader.scala:35)
	at org.apache.spark.network.MessageChunkHeader.buffer(MessageChunkHeader.scala:32)
	at org.apache.spark.network.MessageChunk.buffers$lzycompute(MessageChunk.scala:31)
	at org.apache.spark.network.MessageChunk.buffers(MessageChunk.scala:29)
	at org.apache.spark.network.SendingConnection.write(Connection.scala:349)
	at org.apache.spark.network.ConnectionManager$$anon$5.run(ConnectionManager.scala:142)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:724)

Mime
View raw message