hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shubham Sharma (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HAWQ-1527) Implement partition pushdown for integral data types when accessing hive through pxf
Date Fri, 15 Sep 2017 00:25:00 GMT
Shubham Sharma created HAWQ-1527:
------------------------------------

             Summary: Implement partition pushdown for integral data types when accessing
hive through pxf
                 Key: HAWQ-1527
                 URL: https://issues.apache.org/jira/browse/HAWQ-1527
             Project: Apache HAWQ
          Issue Type: Improvement
          Components: PXF
            Reporter: Shubham Sharma
            Assignee: Ed Espino


Hive, when accessed through hcatalog,  currently supports partition filtering using columns
whose datatype is either string or one of the integral data types (which are TINYINT, SMALLINT,
INT, BIGINT). However pxf currently ignores non string partition columns([code reference |
https://github.com/apache/incubator-hawq/blob/master/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveDataFragmenter.java#L427-#L431])

Hive supports only two operations while filtering partition using integral columns. these
operations are "=" and "!=" . [Code reference|https://github.com/apache/hive/blob/branch-1.2/metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java#L433-#L457]
for hive ExpressionTree. Hive introduced a parameter hive.metastore.integral.jdo.pushdown
which must be set to true in hive-site.xml to enable filtering on integral datatypes.

Logging this JIRA to implement this feature and enable partition filtering using Integral
data types from pxf.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message