Huon Wilson created SPARK-26964:
-----------------------------------
Summary: to_json/from_json do not match JSON spec due to not supporting scalars
Key: SPARK-26964
URL: https://issues.apache.org/jira/browse/SPARK-26964
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.4.0, 2.3.2
Reporter: Huon Wilson
Spark SQL's {{to_json}} and {{from_json}} currently support arrays and objects, but not the
scalar/primitive types. This doesn't match the JSON spec on https://www.json.org: a JSON document
({{json: element}}) consists of a value surrounded by whitespace ({{element: ws value ws}}),
where a value is an object or array _or_ a number or string etc.:
{code:none}
value
object
array
string
number
"true"
"false"
"null"
{code}
Having {{to_json}} and {{from_json}} support scalars would make them flexible enough for a
library I'm working on, where an arbitrary (user-supplied) column needs to be turned into
JSON.
This is related to SPARK-24391 and SPARK-25252, which added support for arrays of scalars.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org
|