spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-6197) handle json parse exception for eventlog file not finished writing
Date Sat, 02 Jan 2016 14:22:39 GMT

     [ https://issues.apache.org/jira/browse/SPARK-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sean Owen updated SPARK-6197:
-----------------------------
    Labels:   (was: backport-needed)

I don't think this will ever be back-ported to 1.3.x at this point

> handle json parse exception for eventlog file not finished writing 
> -------------------------------------------------------------------
>
>                 Key: SPARK-6197
>                 URL: https://issues.apache.org/jira/browse/SPARK-6197
>             Project: Spark
>          Issue Type: Bug
>          Components: Web UI
>    Affects Versions: 1.3.0
>            Reporter: Zhang, Liye
>            Assignee: Zhang, Liye
>            Priority: Minor
>             Fix For: 1.4.0
>
>
> This is a following JIRA for [SPARK-6107|https://issues.apache.org/jira/browse/SPARK-6107].
In  [SPARK-6107|https://issues.apache.org/jira/browse/SPARK-6107], webUI can display event
log files that with suffix *.inprogress*. However, the eventlog file may be not finished writing
for some abnormal cases (e.g. Ctrl+C), In which case, the file maybe  truncated in the last
line, leading to the line being not in valid Json format. Which will cause Json parse exception
when reading the file. 
> For this case, we can just ignore the last line content, since the history for abnormal
cases showed on web is only a reference for user, it can demonstrate the past status of the
app before terminated abnormally (we can not guarantee the history can show exactly the last
moment when app encounter the abnormal situation). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message