carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jack...@apache.org
Subject [carbondata] branch master updated: [HOTFIX] Fix After alter table add column schemaOrdinal is not continuous
Date Wed, 15 Jan 2020 07:32:09 GMT
This is an automated email from the ASF dual-hosted git repository.

jackylk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/master by this push:
     new 82347da  [HOTFIX] Fix After alter table add column schemaOrdinal is not continuous
82347da is described below

commit 82347dadee8bd524d6f97c7ca8316718f4f68261
Author: ajantha-bhat <ajanthabhat@gmail.com>
AuthorDate: Tue Jan 14 10:54:02 2020 +0800

    [HOTFIX] Fix After alter table add column schemaOrdinal is not continuous
    
    Why is this PR needed?
    
    Consider a scenario with complex columns[c1 int, c2 array, c3 array] here schema ordinal
of child columns are -1. so c1, c2, c3 has ordinal 0, 1, 2. After alter table add column "c4
int", its ordinal should be 3. But currently it is 6 (due to flat column schema).
    This will give problem when this ordinal is referred for some other functionality like
rearrange projection columns.
    
    What changes were proposed in this PR?
    
    make schema ordinal continuous by collecting previous schema ordinal max.
    
    Does this PR introduce any user interface change?
    
    No
    
    Is any new testcase added?
    
    No
    
    This closes #3577
---
 .../spark/sql/execution/command/carbonTableSchemaCommon.scala      | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/integration/spark-common/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
b/integration/spark-common/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
index 523ec9e..a7e6f63 100644
--- a/integration/spark-common/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
+++ b/integration/spark-common/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
@@ -240,7 +240,8 @@ class AlterTableColumnSchemaGenerator(
   def process: Seq[ColumnSchema] = {
     val tableSchema = tableInfo.getFactTable
     val tableCols = tableSchema.getListOfColumns.asScala
-    val existingColsSize = tableCols.size
+    // previous maximum column schema ordinal + 1 is the current column schema ordinal
+    val currentSchemaOrdinal = tableCols.map(col => col.getSchemaOrdinal).max + 1
     var longStringCols = Seq[ColumnSchema]()
     // get all original dimension columns
     // but exclude complex type columns and long string columns
@@ -263,7 +264,7 @@ class AlterTableColumnSchemaGenerator(
         isDimensionCol = true,
         field.precision,
         field.scale,
-        field.schemaOrdinal + existingColsSize,
+        field.schemaOrdinal + currentSchemaOrdinal,
         alterTableModel.highCardinalityDims,
         alterTableModel.databaseName.getOrElse(dbName),
         isSortColumn(field.name.getOrElse(field.column)),
@@ -296,7 +297,7 @@ class AlterTableColumnSchemaGenerator(
         isDimensionCol = false,
         field.precision,
         field.scale,
-        field.schemaOrdinal + existingColsSize,
+        field.schemaOrdinal + currentSchemaOrdinal,
         alterTableModel.highCardinalityDims,
         alterTableModel.databaseName.getOrElse(dbName)
       )


Mime
View raw message