spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-6197) handle json parse exception for eventlog file not finished writing
Date Fri, 06 Mar 2015 05:37:39 GMT

    [ https://issues.apache.org/jira/browse/SPARK-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349984#comment-14349984
] 

Apache Spark commented on SPARK-6197:
-------------------------------------

User 'liyezhang556520' has created a pull request for this issue:
https://github.com/apache/spark/pull/4927

> handle json parse exception for eventlog file not finished writing 
> -------------------------------------------------------------------
>
>                 Key: SPARK-6197
>                 URL: https://issues.apache.org/jira/browse/SPARK-6197
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core, Web UI
>    Affects Versions: 1.4.0, 1.3.1
>            Reporter: Zhang, Liye
>
> This is a following JIRA for [SPARK-6107|https://issues.apache.org/jira/browse/SPARK-6107].
In  [SPARK-6107|https://issues.apache.org/jira/browse/SPARK-6107], webUI can display event
log files that with suffix *.inprogress*. However, the eventlog file may be not finished writing
for some abnormal cases (e.g. Ctrl+C), In which case, the file maybe  truncated in the last
line, leading to the line being not in valid Json format. Which will cause Json parse exception
when reading the file. 
> For this case, we can just ignore the last line content, since the history for abnormal
cases showed on web is only a reference for user, it can demonstrate the past status of the
app before terminated abnormally (we can not guarantee the history can show exactly the last
moment when app encounter the abnormal situation). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message