spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Reynold Xin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-15956) When unwrapping ORC avoid pattern matching at runtime
Date Sun, 02 Oct 2016 22:48:20 GMT

     [ https://issues.apache.org/jira/browse/SPARK-15956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Reynold Xin updated SPARK-15956:
--------------------------------
    Fix Version/s: 2.1.0

> When unwrapping ORC avoid pattern matching at runtime
> -----------------------------------------------------
>
>                 Key: SPARK-15956
>                 URL: https://issues.apache.org/jira/browse/SPARK-15956
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>            Reporter: Brian Cho
>            Assignee: Apache Spark
>            Priority: Minor
>             Fix For: 2.1.0
>
>
> When unwrapping ORC values, pattern matching for each data value at runtime hurts performance.
This should be avoided.
> Instead, we can run pattern matching once and return a function that is subsequently
used to unwrap each data value. This is already implemented for certain primitive types. We
should implement for the remaining types, including complex types (e.g, list, map).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message