hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <>
Subject [jira] [Commented] (HIVE-10289) Support filter on non-first partition key and non-string partition key
Date Fri, 14 Aug 2015 18:23:45 GMT


Daniel Dai commented on HIVE-10289:

I am doing comparison in actual datatype. Hbase bytes are converted to actual datatype using
BinarySortableSerDe. Do you mean Operator.val? That is string and means different thing according
to operators it handles. For LIKE, it is the regex string. For NOTEQUALS, it is the value
to compare against, and yes this can be optimized by converting to actual type at init time
rather than in compareTo(). But that's operator dependent.

> Support filter on non-first partition key and non-string partition key
> ----------------------------------------------------------------------
>                 Key: HIVE-10289
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>          Components: HBase Metastore, Metastore
>    Affects Versions: hbase-metastore-branch
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>         Attachments: HIVE-10289.1.patch, HIVE-10289.2.patch
> Currently, partition filtering only handles the first partition key and the type for
this partition key must be string. In order to break this limitation, several improvements
are required:
> 1. Change serialization format for partition key. Currently partition keys are serialized
into delimited string, which sorted on string order not with regard to the actual type of
the partition key. We use BinarySortableSerDe for this purpose.
> 2. For filter condition not on the initial partition keys, push it into HBase RowFilter.
RowFilter will deserialize the partition key and evaluate the filter condition.

This message was sent by Atlassian JIRA

View raw message