spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wenchen Fan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-12474) support deserialization for physical plan and hive logical plan from JSON string
Date Tue, 22 Dec 2015 03:03:46 GMT
Wenchen Fan created SPARK-12474:
-----------------------------------

             Summary: support deserialization for physical plan and hive logical plan from
JSON string
                 Key: SPARK-12474
                 URL: https://issues.apache.org/jira/browse/SPARK-12474
             Project: Spark
          Issue Type: New Feature
            Reporter: Wenchen Fan


SPARK-12321 add a framework based on reflection that can serialize {{TreeNode}} to JSON and
deserialize it back. However, it can't handle all corner cases and we bypass them in the test,
see https://github.com/apache/spark/pull/10311/files#diff-238d584c15e16c24f49a40bcf163fe13R190
and https://github.com/apache/spark/pull/10311/files#diff-238d584c15e16c24f49a40bcf163fe13R212.

known corner cases:
1. ExpressionEncoder
2. BaseRelation
3. hive logical plan
4. physical plan

The framework is in catalyst module and may not be able to handle corner cases from other
modules, an idea is defining a {{JsonSerializable}} trait and implement it for corner cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message