spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brad Willard (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-5092) Selecting from a nested structure with SparkSQL should return a nested structure
Date Mon, 05 Jan 2015 20:29:34 GMT
Brad Willard created SPARK-5092:
-----------------------------------

             Summary: Selecting from a nested structure with SparkSQL should return a nested
structure
                 Key: SPARK-5092
                 URL: https://issues.apache.org/jira/browse/SPARK-5092
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 1.2.0
            Reporter: Brad Willard
            Priority: Minor


When running a sparksql query like this (at least on a json dataset)

select
   rid,
   meta_data.name
from
   a_table

The rows returned lose the nested structure. I receive a row like

Row(rid='123', name='delete')

instead of

Row(rid='123', meta_data=Row(name='data'))

I personally think this is confusing especially when programmatically building and executing
queries and then parsing it to find your data in a new structure. I could understand how that's
less desirable in some situations, but you could get around it by supporting 'as'. If you
wanted to skip the nested structure simply write.

select
   rid,
   meta_data.name as name
from
   a_table




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message