hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15670) column_stats_accurate may not fit in PARTITION_PARAMS.VALUE
Date Wed, 25 Oct 2017 01:07:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16217948#comment-16217948
] 

Sergey Shelukhin commented on HIVE-15670:
-----------------------------------------

Beats me... the current implementation is as such.

> column_stats_accurate may not fit in PARTITION_PARAMS.VALUE
> -----------------------------------------------------------
>
>                 Key: HIVE-15670
>                 URL: https://issues.apache.org/jira/browse/HIVE-15670
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>
> The JSON can be too big with many columns (see setColumnStatsState method).
> We can make JSON more compact by only storing the list of columns with true values. Or
we can even store a bitmask in a dedicated column, and adjust it when altering table (rare
enough). Or we can just change the VALUE column to text blob (might be a painful change wrt
upgrade scripts, and supporting all the DBs' varied blob implementations, esp. in directsql).
> Storing denormalized flags in a separate table will probably be slow, comparatively.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message