spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-18671) Add tests to ensure stability of that all Structured Streaming log formats
Date Wed, 07 Dec 2016 01:46:58 GMT

    [ https://issues.apache.org/jira/browse/SPARK-18671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15727365#comment-15727365
] 

Apache Spark commented on SPARK-18671:
--------------------------------------

User 'tdas' has created a pull request for this issue:
https://github.com/apache/spark/pull/16183

> Add tests to ensure stability of that all Structured Streaming log formats
> --------------------------------------------------------------------------
>
>                 Key: SPARK-18671
>                 URL: https://issues.apache.org/jira/browse/SPARK-18671
>             Project: Spark
>          Issue Type: Bug
>          Components: Structured Streaming
>            Reporter: Tathagata Das
>            Assignee: Tathagata Das
>             Fix For: 2.1.0
>
>
> To be able to restart StreamingQueries across Spark version, we have already made the
logs (offset log, file source log, file sink log) use json. We should added tests with actual
json files in the Spark such that any incompatible changes in reading the logs is immediately
caught. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message