hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harel Ben Attia (JIRA)" <>
Subject [jira] [Commented] (HIVE-6897) Allow overwrite/append to external Hive table (with partitions) via HCatStorer
Date Sun, 19 Apr 2015 07:35:58 GMT


Harel Ben Attia commented on HIVE-6897:

We have the same kind of issue, due to late rewrites to partitions because of late-data arrivals.

Adding a manual removal of the hive and hdfs data is not only cumbersone and error prone,
but leads to an intermediate state where the old data does not exist.

To complicate things more, we have multiple hadoop clusters, so just running "hadoop fs" or
"hive" from inside the pig script requires complicated tricks in order to make sure that things
are performed against the proper hadoop cluster.

We'd be very glad to see this supported.


> Allow overwrite/append to external Hive table (with partitions) via HCatStorer
> ------------------------------------------------------------------------------
>                 Key: HIVE-6897
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: HCatalog, HiveServer2
>    Affects Versions: 0.12.0
>            Reporter: Dip Kharod
> I'm using HCatStorer to write to external Hive table with partition from Pig and have
the following different use cases:
> 1) Need to overwrite (aka, refresh) data into table: Currently I end up doing this outside
(drop partition and delete HDFS folder) of Pig which is very painful and error-prone
> 2) Need to append (aka, add new file) data to the Hive external table/partition: Again,
I end up doing this outside of Pig by copying file in appropriate folder
> It would be very productive for the developers to have both options in HCatStorer.

This message was sent by Atlassian JIRA

View raw message