carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From chenliang...@apache.org
Subject carbondata git commit: [CARBONDATA-1921] Documentation - Merge Index
Date Fri, 22 Dec 2017 14:48:32 GMT
Repository: carbondata
Updated Branches:
  refs/heads/master 4b0eee3b4 -> 406d75442


[CARBONDATA-1921] Documentation - Merge Index

add Merge Index document

This closes #1717


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/406d7544
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/406d7544
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/406d7544

Branch: refs/heads/master
Commit: 406d754424fdd36c2710fa928afe9ced904b6805
Parents: 4b0eee3
Author: ajantha-bhat <ajanthabhat@gmail.com>
Authored: Fri Dec 22 16:17:43 2017 +0530
Committer: chenliang613 <chenliang613@huawei.com>
Committed: Fri Dec 22 22:48:21 2017 +0800

----------------------------------------------------------------------
 docs/configuration-parameters.md      |  4 ++--
 docs/data-management-on-carbondata.md | 13 ++++++++++++-
 2 files changed, 14 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/carbondata/blob/406d7544/docs/configuration-parameters.md
----------------------------------------------------------------------
diff --git a/docs/configuration-parameters.md b/docs/configuration-parameters.md
index 5875529..39f1e3b 100644
--- a/docs/configuration-parameters.md
+++ b/docs/configuration-parameters.md
@@ -70,7 +70,7 @@ This section provides the details of all the configurations required for
CarbonD
 | carbon.horizontal.UPDATE.compaction.threshold | 1 | This property specifies the threshold
limit on number of UPDATE delta files within a segment. In case the number of delta files
goes beyond the threshold, the UPDATE delta files within the segment becomes eligible for
horizontal compaction and compacted into single UPDATE delta file. | Values between 1 to 10000.
|
 | carbon.horizontal.DELETE.compaction.threshold | 1 | This property specifies the threshold
limit on number of DELETE delta files within a block of a segment. In case the number of delta
files goes beyond the threshold, the DELETE delta files for the particular block of the segment
becomes eligible for horizontal compaction and compacted into single DELETE delta file. |
Values between 1 to 10000. |
 | carbon.update.segment.parallelism | 1 | This property specifies the parallelism for each
segment during update. If there are segments that contain too many records to update and the
spark job encounter data-spill related errors, it is better to increase this property value.
It is recommended to set this value to a multiple of the number of executors for balance.
| Values between 1 to 1000. |
-  
+| carbon.merge.index.in.segment | true | This property is used to merge all CarbonData index
files (.carbonindex) inside a segment to a sinle CarbonData index merge file (.carbonindexmerge).|
Values true or false |  
 
 * **Query Configuration**
   
@@ -222,4 +222,4 @@ RESET
 
 * Success will be recorded in the driver log.
 
-* Failure will be displayed in the UI.
\ No newline at end of file
+* Failure will be displayed in the UI.

http://git-wip-us.apache.org/repos/asf/carbondata/blob/406d7544/docs/data-management-on-carbondata.md
----------------------------------------------------------------------
diff --git a/docs/data-management-on-carbondata.md b/docs/data-management-on-carbondata.md
index 94820ad..90afb4e 100644
--- a/docs/data-management-on-carbondata.md
+++ b/docs/data-management-on-carbondata.md
@@ -231,6 +231,17 @@ This tutorial is going to introduce all commands and data operations
on CarbonDa
      ```
      ALTER TABLE test_db.carbon CHANGE a1 a1 DECIMAL(18,2)
      ```
+- **MERGE INDEX**
+   
+     This command is used to merge all the CarbonData index files (.carbonindex) inside a
segment to a single CarbonData index merge file (.carbonindexmerge). This enhances the first
query performance.
+     ```
+      ALTER TABLE [db_name.]table_name COMPACT 'SEGMENT_INDEX'
+      ```
+      
+      Examples:
+      ```
+      ALTER TABLE test_db.carbon COMPACT 'SEGMENT_INDEX'
+      ```
 
 ### DROP TABLE
   
@@ -746,4 +757,4 @@ This tutorial is going to introduce all commands and data operations on
CarbonDa
   Example:
   ```
   DELETE FROM TABLE CarbonDatabase.CarbonTable WHERE SEGMENT.STARTTIME BEFORE '2017-06-01
12:05:06' 
-  ```
\ No newline at end of file
+  ```


Mime
View raw message