spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: java.io.IOException: Filesystem closed
Date Tue, 02 Dec 2014 06:34:47 GMT
What is the application that you are submitting? Looks like you might have
invoked fs inside the app and then closed it within it.

Thanks
Best Regards

On Tue, Dec 2, 2014 at 11:59 AM, rapelly kartheek <kartheek.mbms@gmail.com>
wrote:

> Hi,
>
> I face the following exception when submit a spark application. The log
> file shows:
>
> 14/12/02 11:52:58 ERROR LiveListenerBus: Listener EventLoggingListener
> threw an exception
> java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:689)
> at
> org.apache.hadoop.hdfs.DFSOutputStream.flushOrSync(DFSOutputStream.java:1668)
> at org.apache.hadoop.hdfs.DFSOutputStream.hflush(DFSOutputStream.java:1629)
> at org.apache.hadoop.hdfs.DFSOutputStream.sync(DFSOutputStream.java:1614)
> at
> org.apache.hadoop.fs.FSDataOutputStream.sync(FSDataOutputStream.java:120)
> at
> org.apache.spark.util.FileLogger$$anonfun$flush$2.apply(FileLogger.scala:158)
> at
> org.apache.spark.util.FileLogger$$anonfun$flush$2.apply(FileLogger.scala:158)
> at scala.Option.foreach(Option.scala:236)
> at org.apache.spark.util.FileLogger.flush(FileLogger.scala:158)
> at
> org.apache.spark.scheduler.EventLoggingListener.logEvent(EventLoggingListener.scala:87)
> at
> org.apache.spark.scheduler.EventLoggingListener.onJobEnd(EventLoggingListener.scala:112)
> at
> org.apache.spark.scheduler.SparkListenerBus$$anonfun$postToAll$4.apply(SparkListenerBus.scala:52)
> at
> org.apache.spark.scheduler.SparkListenerBus$$anonfun$postToAll$4.apply(SparkListenerBus.scala:52)
> at
> org.apache.spark.scheduler.SparkListenerBus$$anonfun$foreachListener$1.apply(SparkListenerBus.scala:81)
> at
> org.apache.spark.scheduler.SparkListenerBus$$anonfun$foreachListener$1.apply(SparkListenerBus.scala:79)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
> at
> org.apache.spark.scheduler.SparkListenerBus$class.foreachListener(SparkListenerBus.scala:79)
> at
> org.apache.spark.scheduler.SparkListenerBus$class.postToAll(SparkListenerBus.scala:52)
> at
> org.apache.spark.scheduler.LiveListenerBus.postToAll(LiveListenerBus.scala:32)
> at
> org.apache.spark.scheduler.LiveListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(LiveListenerBus.scala:56)
> at
> org.apache.spark.scheduler.LiveListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(LiveListenerBus.scala:56)
> at scala.Option.foreach(Option.scala:236)
> at
> org.apache.spark.scheduler.LiveListenerBus$$anon$1$$anonfun$run$1.apply$mcV$sp(LiveListenerBus.scala:56)
> at
> org.apache.spark.scheduler.LiveListenerBus$$anon$1$$anonfun$run$1.apply(LiveListenerBus.scala:47)
> at
> org.apache.spark.scheduler.LiveListenerBus$$anon$1$$anonfun$run$1.apply(LiveListenerBus.scala:47)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1160)
> at
> org.apache.spark.scheduler.LiveListenerBus$$anon$1.run(LiveListenerBus.scala:46)
>
> Someone please help me resolve this!!
>
> Thanks
>
>

Mime
View raw message