carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jack...@apache.org
Subject carbondata git commit: [CARBONDATA-2681][32K] Fix loading problem using global/batch sort fails when table has long string columns
Date Mon, 09 Jul 2018 07:13:22 GMT
Repository: carbondata
Updated Branches:
  refs/heads/master f0c77e5f5 -> c2c5b18ba


[CARBONDATA-2681][32K] Fix loading problem using global/batch sort fails when table has long
string columns

In SortStepRowHandler, global/batch sort use convertRawRowTo3Parts instead of convertIntermediateSortTempRowTo3Parted.

varcharDimCnt was not add up to noDictArray cause error: Problem while converting row to 3
parts.

This closes #2435


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/c2c5b18b
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/c2c5b18b
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/c2c5b18b

Branch: refs/heads/master
Commit: c2c5b18baa09c082de0e99050fde10b85516c38f
Parents: f0c77e5
Author: Manhua <kevinjmh@qq.com>
Authored: Mon Jul 2 17:26:49 2018 +0800
Committer: Jacky Li <jacky.likun@qq.com>
Committed: Mon Jul 9 15:13:10 2018 +0800

----------------------------------------------------------------------
 .../carbondata/processing/loading/sort/SortStepRowHandler.java    | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/carbondata/blob/c2c5b18b/processing/src/main/java/org/apache/carbondata/processing/loading/sort/SortStepRowHandler.java
----------------------------------------------------------------------
diff --git a/processing/src/main/java/org/apache/carbondata/processing/loading/sort/SortStepRowHandler.java
b/processing/src/main/java/org/apache/carbondata/processing/loading/sort/SortStepRowHandler.java
index bcf8a39..1c6d8b2 100644
--- a/processing/src/main/java/org/apache/carbondata/processing/loading/sort/SortStepRowHandler.java
+++ b/processing/src/main/java/org/apache/carbondata/processing/loading/sort/SortStepRowHandler.java
@@ -104,7 +104,8 @@ public class SortStepRowHandler implements Serializable {
     try {
       int[] dictDims
           = new int[this.dictSortDimCnt + this.dictNoSortDimCnt];
-      byte[][] nonDictArray = new byte[this.noDictSortDimCnt + this.noDictNoSortDimCnt][];
+      byte[][] nonDictArray
+          = new byte[this.noDictSortDimCnt + this.noDictNoSortDimCnt + this.varcharDimCnt][];
       Object[] measures = new Object[this.measureCnt];
 
       // convert dict & data


Mime
View raw message