hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharathkrishna Guruvayoor Murali (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20545) Ability to exclude potentially large parameters in HMS Notifications
Date Fri, 05 Oct 2018 18:52:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16640211#comment-16640211
] 

Bharathkrishna Guruvayoor Murali commented on HIVE-20545:
---------------------------------------------------------

Hi [~anishek] ,

An example would be when Impala writes stats information to Partition objects and accesses
it, as shown :  [Impala reading stats
|http://github.mtv.cloudera.com/CDH/Impala/blob/6f2d928734a33ace15ec6abd5659651173b9e69e/fe/src/main/java/org/apache/impala/catalog/PartitionStatsUtil.java#L45]

But this information is not relevant as a Partition metadata for any other purpose. Also,
these parameters have considerably large-size. On running some preliminary tests, we observe
that there is a performance hit on increasing message size of HMS Notifications, so we can
use this configuration to filter parameters like above mentioned.

> Ability to exclude potentially large parameters in HMS Notifications
> --------------------------------------------------------------------
>
>                 Key: HIVE-20545
>                 URL: https://issues.apache.org/jira/browse/HIVE-20545
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore
>    Affects Versions: 3.1.0, 4.0.0
>            Reporter: Bharathkrishna Guruvayoor Murali
>            Assignee: Bharathkrishna Guruvayoor Murali
>            Priority: Major
>         Attachments: HIVE-20545.1.patch, HIVE-20545.2.patch, HIVE-20545.3.branch-3.patch,
HIVE-20545.3.patch, HIVE-20545.4.patch, HIVE-20545.6.patch, HIVE-20545.7.patch
>
>
> Clients can add large-sized parameters in Table/Partition objects. So we need to enable adding
regex patterns through HiveConf to match parameters to be filtered from table and partition
objects before serialization in HMS notifications.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message