hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-22826) ALTER TABLE RENAME COLUMN doesn't update list of bucketed column names
Date Thu, 01 Oct 2020 11:14:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-22826?focusedWorklogId=493429&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-493429
]

ASF GitHub Bot logged work on HIVE-22826:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 01/Oct/20 11:13
            Start Date: 01/Oct/20 11:13
    Worklog Time Spent: 10m 
      Work Description: adesh-rao commented on a change in pull request #1528:
URL: https://github.com/apache/hive/pull/1528#discussion_r497652090



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/column/change/AlterTableChangeColumnOperation.java
##########
@@ -33,6 +35,7 @@
 import org.apache.hadoop.hive.ql.metadata.Partition;
 import org.apache.hadoop.hive.ql.metadata.Table;
 import org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils;
+import org.apache.spark.util.CollectionsUtils;

Review comment:
       required?

##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
##########
@@ -63,6 +64,8 @@
 import org.datanucleus.api.jdo.JDOPersistenceManagerFactory;
 import org.junit.Assert;
 import org.junit.Before;
+import org.junit.experimental.categories.Category;

Review comment:
       required?

##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java
##########
@@ -1554,4 +1555,30 @@ public static Partition createMetaPartitionObject(Table tbl, Map<String,
String>
     }
     return tpart;
   }
+
+  /**
+   * Validate bucket columns should belong to table columns.
+   * @param sd StorageDescriptor of given table
+   * @return true if bucket columns are empty or belong to table columns else false
+   */
+  public static boolean validateBucketColumns(StorageDescriptor sd) {
+    if (CollectionUtils.isNotEmpty(sd.getBucketCols())) {
+      List<String> columnNames =
+          getColumnNames(sd.getCols()).stream().map(String::toLowerCase).collect(Collectors.toList());
+      return columnNames.containsAll(sd.getBucketCols().stream().map(String::toLowerCase).collect(Collectors.toList()));

Review comment:
       add a null check for columnNames? getColumnNames returns null too.

##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
##########
@@ -3428,4 +3432,44 @@ public void testUpdatePartitionStat_doesNotUpdateStats() throws Exception
{
     m.invoke(hms, tbl, part, false, null);
     verify(wh, never()).getFileStatusesForLocation(part.getSd().getLocation());
   }
+
+  @Test(expected = InvalidOperationException.class)

Review comment:
       Also, Is it a positive test case or negative? I see that there is an expected error
but we are doing the positive validations too. We should break this into two, right?

##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
##########
@@ -3428,4 +3432,44 @@ public void testUpdatePartitionStat_doesNotUpdateStats() throws Exception
{
     m.invoke(hms, tbl, part, false, null);
     verify(wh, never()).getFileStatusesForLocation(part.getSd().getLocation());
   }
+
+  @Test(expected = InvalidOperationException.class)

Review comment:
       Won't adding a qtest be better?
   
   You are explicitly doing `tblUpdated.getSd().getBucketCols()` and updating them in the
test. qtest will check end to end flow and you don't need to bother about getting Sd and updating
the cols explicitly (Just a single alter table will do the trick). 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 493429)
    Time Spent: 20m  (was: 10m)

>  ALTER TABLE RENAME COLUMN doesn't update list of bucketed column names
> -----------------------------------------------------------------------
>
>                 Key: HIVE-22826
>                 URL: https://issues.apache.org/jira/browse/HIVE-22826
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Planning
>    Affects Versions: 4.0.0
>            Reporter: Karen Coppage
>            Assignee: Ashish Sharma
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: unitTest.patch
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> Compaction for tables where a bucketed column has been renamed fails since the list of
bucketed columns in the StorageDescriptor doesn't get updated when the column is renamed,
therefore we can't recreate the table correctly during compaction.
> Attached a unit test that fails.
> NO PRECOMMIT TESTS



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message