carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jack...@apache.org
Subject [carbondata] branch master updated: [CARBONDATA-3756] Fix stage query bug it only read the first blocklet of each carbondata file
Date Sat, 28 Mar 2020 12:44:56 GMT
This is an automated email from the ASF dual-hosted git repository.

jackylk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/master by this push:
     new 994034b  [CARBONDATA-3756] Fix stage query bug it only read the first blocklet of
each carbondata file
994034b is described below

commit 994034b0477fe2b846699421161d6c5fcd5120be
Author: QiangCai <qiangcai@qq.com>
AuthorDate: Fri Mar 27 16:35:38 2020 +0800

    [CARBONDATA-3756] Fix stage query bug it only read the first blocklet of each carbondata
file
    
    Why is this PR needed?
    The query of stage files only read the first blocklet of each carbondata file.
    So when the file contains multiple blocklets, the query result will be wrong.
    
    What changes were proposed in this PR?
    The query of stage files should read the all blocklets of all carbondata files.
    
    Does this PR introduce any user interface change?
    No
    
    Is any new testcase added?
    No
    
    This closes #3684
---
 .../org/apache/carbondata/core/util/CarbonUtil.java    | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java b/core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java
index 41902b6..e56b8b1 100644
--- a/core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java
+++ b/core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java
@@ -927,14 +927,22 @@ public final class CarbonUtil {
     AbstractDataFileFooterConverter footerConverter =
         DataFileFooterConverterFactory.getInstance().getDataFileFooterConverter(version);
     List<DataFileFooter> footers = footerConverter.getIndexInfo(indexFilePath, null,
true);
-
+    DataFileFooter blockFooter = null;
     // find the footer of the input data file (tableBlockInfo)
-    for (DataFileFooter footer : footers) {
-      if (footer.getBlockInfo().getFilePath().equals(dataFilePath)) {
-        return footer;
+    for (DataFileFooter blockletFooter : footers) {
+      if (blockletFooter.getBlockInfo().getFilePath().equals(dataFilePath)) {
+        if (blockFooter == null) {
+          blockFooter = blockletFooter;
+        } else {
+          blockFooter.getBlockletList().addAll(blockletFooter.getBlockletList());
+        }
       }
     }
-    throw new RuntimeException("Footer not found in index file");
+    if (blockFooter == null) {
+      throw new RuntimeException("Footer not found in index file");
+    } else {
+      return blockFooter;
+    }
   }
 
   /**


Mime
View raw message