spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Wendell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-1407) EventLogging to HDFS doesn't work properly on yarn
Date Tue, 08 Apr 2014 17:55:16 GMT

    [ https://issues.apache.org/jira/browse/SPARK-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963231#comment-13963231
] 

Patrick Wendell commented on SPARK-1407:
----------------------------------------

[~tgraves] Did you test this with an application that explicitly calls stop() instead of relying
on the shutdown hook in YARN? 

> EventLogging to HDFS doesn't work properly on yarn
> --------------------------------------------------
>
>                 Key: SPARK-1407
>                 URL: https://issues.apache.org/jira/browse/SPARK-1407
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 1.0.0
>            Reporter: Thomas Graves
>            Priority: Blocker
>
> When running on spark on yarn and accessing an HDFS file (like in the SparkHdfsLR example)
while using the event logging configured to write logs to HDFS, it throws an exception at
the end of the application. 
> SPARK_JAVA_OPTS=-Dspark.eventLog.enabled=true -Dspark.eventLog.dir=hdfs:///history/spark/
> 14/04/03 13:41:31 INFO yarn.ApplicationMaster$$anon$1: Invoking sc stop from shutdown
hook
> Exception in thread "Thread-41" java.io.IOException: Filesystem closed
>         at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:398)
>         at org.apache.hadoop.hdfs.DFSOutputStream.hflush(DFSOutputStream.java:1465)
>         at org.apache.hadoop.hdfs.DFSOutputStream.sync(DFSOutputStream.java:1450)
>         at org.apache.hadoop.fs.FSDataOutputStream.sync(FSDataOutputStream.java:116)
>         at org.apache.spark.util.FileLogger$$anonfun$flush$2.apply(FileLogger.scala:137)
>         at org.apache.spark.util.FileLogger$$anonfun$flush$2.apply(FileLogger.scala:137)
>         at scala.Option.foreach(Option.scala:236)
>         at org.apache.spark.util.FileLogger.flush(FileLogger.scala:137)
>         at org.apache.spark.scheduler.EventLoggingListener.logEvent(EventLoggingListener.scala:69)
>         at org.apache.spark.scheduler.EventLoggingListener.onApplicationEnd(EventLoggingListener.scala:101)
>         at org.apache.spark.scheduler.SparkListenerBus$$anonfun$postToAll$13.apply(SparkListenerBus.scala:67)
>         at org.apache.spark.scheduler.SparkListenerBus$$anonfun$postToAll$13.apply(SparkListenerBus.scala:67)
>         at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
>         at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
>         at org.apache.spark.scheduler.SparkListenerBus$class.postToAll(SparkListenerBus.scala:67)
>         at org.apache.spark.scheduler.LiveListenerBus.postToAll(LiveListenerBus.scala:31)
>         at org.apache.spark.scheduler.LiveListenerBus.post(LiveListenerBus.scala:78)
>         at org.apache.spark.SparkContext.postApplicationEnd(SparkContext.scala:1081)
>         at org.apache.spark.SparkContext.stop(SparkContext.scala:828)
>         at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$1.run(ApplicationMaster.scala:460)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message