Brad Willard created SPARK-5092:
-----------------------------------
Summary: Selecting from a nested structure with SparkSQL should return a nested
structure
Key: SPARK-5092
URL: https://issues.apache.org/jira/browse/SPARK-5092
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 1.2.0
Reporter: Brad Willard
Priority: Minor
When running a sparksql query like this (at least on a json dataset)
select
rid,
meta_data.name
from
a_table
The rows returned lose the nested structure. I receive a row like
Row(rid='123', name='delete')
instead of
Row(rid='123', meta_data=Row(name='data'))
I personally think this is confusing especially when programmatically building and executing
queries and then parsing it to find your data in a new structure. I could understand how that's
less desirable in some situations, but you could get around it by supporting 'as'. If you
wanted to skip the nested structure simply write.
select
rid,
meta_data.name as name
from
a_table
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org
|