spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Zhuge (JIRA)" <>
Subject [jira] [Resolved] (SPARK-20314) Inconsistent error handling in JSON parsing SQL functions
Date Fri, 22 Feb 2019 01:38:00 GMT


John Zhuge resolved SPARK-20314.
    Resolution: Duplicate

Resolve it as duplicate.

> Inconsistent error handling in JSON parsing SQL functions
> ---------------------------------------------------------
>                 Key: SPARK-20314
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.1.0
>            Reporter: Eric Wasserman
>            Priority: Major
> Most parse errors in the JSON parsing SQL functions (e.g. json_tuple, get_json_object)
will return a null(s) if the JSON is badly formed. However, if Jackson determines that the
string includes invalid characters it will throw an exception (
Invalid UTF-32 character) that Spark does not catch. This creates a robustness problem in
that these functions cannot be used at all when there may be dirty data as these exceptions
will kill the jobs.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message