spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "kant kodali (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-22228) Add support for Array<primitive_type> so from_json can parse
Date Mon, 09 Oct 2017 18:47:01 GMT
kant kodali created SPARK-22228:
-----------------------------------

             Summary: Add support for Array<primitive_type> so from_json can parse
                 Key: SPARK-22228
                 URL: https://issues.apache.org/jira/browse/SPARK-22228
             Project: Spark
          Issue Type: Improvement
          Components: Java API
    Affects Versions: 2.2.0
            Reporter: kant kodali


[val inputDS = Seq("""["foo", "bar"]""").toDF]

inputDS.printSchema()

root
 |-- value: string (nullable = true)
Input Dataset inputDS

inputDS.show(false)

value
-----
["foo", "bar"]
Expected output dataset outputDS

value
-------
"foo" |
"bar" |

Tried explode function like below but it doesn't quite work

inputDS.select(explode(from_json(col("value"), ArrayType(StringType))))
and I get the following error

org.apache.spark.sql.AnalysisException: cannot resolve 'jsontostructs(`value`)' due to data
type mismatch: Input schema string must be a struct or an array of structs




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message