hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (HIVE-12052) automatically populate file metadata to HBase metastore based on config or table properties
Date Sat, 05 Mar 2016 02:25:40 GMT

     [ https://issues.apache.org/jira/browse/HIVE-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sergey Shelukhin reassigned HIVE-12052:
---------------------------------------

    Assignee: Sergey Shelukhin

> automatically populate file metadata to HBase metastore based on config or table properties
> -------------------------------------------------------------------------------------------
>
>                 Key: HIVE-12052
>                 URL: https://issues.apache.org/jira/browse/HIVE-12052
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>
> As discussed in HIVE-11500
> Should use a table property similar to auto.purge.
> Then, when this setting is set, partitions are added (convertToMPart is a good source
to find all the paths for that), after compactions, after load/non-ACID insert, and periodically
(configurable), the storage locations should be scanned for new files and cache updated accordingly.
All the updates should probably be in the background thread and taken from queue (high pri
from most ops, low pri from enabling the property and from periodic updates) to avoid high
load on HDFS from metastore.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message