spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bob Tiernay (JIRA)" <>
Subject [jira] [Created] (SPARK-5302) Add support for SQLContext "partition" columns
Date Sun, 18 Jan 2015 00:24:34 GMT
Bob Tiernay created SPARK-5302:

             Summary: Add support for SQLContext "partition" columns
                 Key: SPARK-5302
             Project: Spark
          Issue Type: New Feature
          Components: SQL
            Reporter: Bob Tiernay

For {{SQLContext}} (not {{HiveContext}}) it would be very convenient to support a virtual
column that maps to part of the the file path, similar to what is done in Hive for partitions.
The API could allow the user to type the column using an appropriate {{DataType}} instance.
This new field could be addressed in SQL statements much the same as is done in Hive. As a
consequence, this would provide an nice interop and migration strategy for Hive users who
may one day use {{SQLContext}} directly.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message