hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Kolbasov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-19253) HMS ignores tableType property for external tables
Date Thu, 16 Aug 2018 18:11:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-19253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582885#comment-16582885
] 

Alexander Kolbasov commented on HIVE-19253:
-------------------------------------------

A bit more fun. In {{HiveAlterHandler.alterPartition}}:

{code}
      // when renaming a partition, we should update
      // 1) partition SD Location
      // 2) partition column stats if there are any because of part_name field in HMS table
PART_COL_STATS
      // 3) rename the partition directory if it is not an external table
      if (!tbl.getTableType().equals(TableType.EXTERNAL_TABLE.toString())) {
        // TODO: refactor this into a separate method after master merge, this one is too
big.
{code}

So here the property is ignored altogether and tableType is used instead, so it is quite likely
that this is currently done incorrectly since this doesn't look at the property.

> HMS ignores tableType property for external tables
> --------------------------------------------------
>
>                 Key: HIVE-19253
>                 URL: https://issues.apache.org/jira/browse/HIVE-19253
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 3.0.0, 3.1.0, 4.0.0
>            Reporter: Alexander Kolbasov
>            Assignee: Alexander Kolbasov
>            Priority: Major
>              Labels: newbie
>         Attachments: HIVE-19253.01.patch, HIVE-19253.02.patch, HIVE-19253.03.patch, HIVE-19253.03.patch,
HIVE-19253.04.patch, HIVE-19253.05.patch
>
>
> When someone creates a table using Thrift API they may think that setting tableType to
{{EXTERNAL_TABLE}} creates an external table. And boom - their table is gone later because
HMS will silently change it to managed table.
> here is the offending code:
> {code:java}
>   private MTable convertToMTable(Table tbl) throws InvalidObjectException,
>       MetaException {
>     ...
>     // If the table has property EXTERNAL set, update table type
>     // accordingly
>     String tableType = tbl.getTableType();
>     boolean isExternal = Boolean.parseBoolean(tbl.getParameters().get("EXTERNAL"));
>     if (TableType.MANAGED_TABLE.toString().equals(tableType)) {
>       if (isExternal) {
>         tableType = TableType.EXTERNAL_TABLE.toString();
>       }
>     }
>     if (TableType.EXTERNAL_TABLE.toString().equals(tableType)) {
>       if (!isExternal) { // Here!
>         tableType = TableType.MANAGED_TABLE.toString();
>       }
>     }
> {code}
> So if the EXTERNAL parameter is not set, table type is changed to managed even if it
was external in the first place - which is wrong.
> More over, in other places code looks at the table property to decide table type and
some places look at parameter. HMS should really make its mind which one to use.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message