carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From manishgupt...@apache.org
Subject carbondata git commit: [CARBONDATA-1865] Remove unnecessary condition to check first pass dataload
Date Wed, 13 Dec 2017 08:54:01 GMT
Repository: carbondata
Updated Branches:
  refs/heads/master c08fe933c -> 63afc00f6


[CARBONDATA-1865] Remove unnecessary condition to check first pass dataload

As high cardinality check for the first load is removed for converting dictionary to no dictionary
column condition to check first single pass dataload is not required

This closes #1622


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/63afc00f
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/63afc00f
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/63afc00f

Branch: refs/heads/master
Commit: 63afc00f6582cbb81412208155114c9191ad88a8
Parents: c08fe93
Author: SangeetaGulia <sangeeta.gulia@knoldus.in>
Authored: Wed Dec 6 15:18:00 2017 +0530
Committer: manishgupta88 <tomanishgupta18@gmail.com>
Committed: Wed Dec 13 14:26:59 2017 +0530

----------------------------------------------------------------------
 .../execution/command/management/CarbonLoadDataCommand.scala  | 7 -------
 1 file changed, 7 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/carbondata/blob/63afc00f/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala
----------------------------------------------------------------------
diff --git a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala
b/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala
index 9f6fce1..38c34dd 100644
--- a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala
+++ b/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala
@@ -165,13 +165,6 @@ case class CarbonLoadDataCommand(
         if (isOverwriteTable) {
           LOGGER.info(s"Overwrite of carbon table with $dbName.$tableName is in progress")
         }
-        if (carbonLoadModel.getLoadMetadataDetails.isEmpty && carbonLoadModel.getUseOnePass
&&
-            StringUtils.isEmpty(carbonLoadModel.getColDictFilePath) &&
-            StringUtils.isEmpty(carbonLoadModel.getAllDictPath)) {
-          LOGGER.info(s"Cannot use single_pass=true for $dbName.$tableName during the first
load")
-          LOGGER.audit(s"Cannot use single_pass=true for $dbName.$tableName during the first
load")
-          carbonLoadModel.setUseOnePass(false)
-        }
         // if table is an aggregate table then disable single pass.
         if (carbonLoadModel.isAggLoadRequest) {
           carbonLoadModel.setUseOnePass(false)


Mime
View raw message