carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From chenliang...@apache.org
Subject [27/35] incubator-carbondata-site git commit: Updated website for CarbonData release 1.0.0
Date Sat, 04 Feb 2017 02:38:26 GMT
http://git-wip-us.apache.org/repos/asf/incubator-carbondata-site/blob/0d4cdb1c/content/docs/latest/ddl-operation-on-carbondata.html
----------------------------------------------------------------------
diff --git a/content/docs/latest/ddl-operation-on-carbondata.html b/content/docs/latest/ddl-operation-on-carbondata.html
new file mode 100644
index 0000000..3a5b4b5
--- /dev/null
+++ b/content/docs/latest/ddl-operation-on-carbondata.html
@@ -0,0 +1,247 @@
+<!--
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+-->
+<h1>DDL Operations on CarbonData</h1><p>This tutorial guides you through
the data definition language support provided by CarbonData.</p><h2>Overview</h2><p>The
following DDL operations are supported in CarbonData :</p>
+<ul>
+  <li><a href="#create-table">CREATE TABLE</a></li>
+  <li><a href="#show-table">SHOW TABLE</a></li>
+  <li><a href="#drop-table">DROP TABLE</a></li>
+  <li><a href="#compaction">COMPACTION</a></li>
+  <li><a href="#bucketing">BUCKETING</a></li>
+</ul><h2 id="create-table">CREATE TABLE</h2><p>This command can be
used to create a CarbonData table by specifying the list of fields along with the table properties.</p><p><pre><code>
+  CREATE TABLE [IF NOT EXISTS] [db_name.]table_name
+  [(col_name data_type, ...)]
+  STORED BY &#39;carbondata&#39;
+  [TBLPROPERTIES (property_name=property_value, ...)]
+  // All Carbon&#39;s additional table options will go into properties
+</code></pre></p><h3>Parameter Description</h3>
+<table class="table table-striped table-bordered">
+  <thead>
+  <tr>
+    <th>Parameter </th>
+    <th>Description </th>
+    <th>Optional </th>
+  </tr>
+  </thead>
+  <tbody>
+  <tr>
+    <td>db_name </td>
+    <td>Name of the database. Database name should consist of alphanumeric characters
and underscore(_) special character. </td>
+    <td>Yes </td>
+  </tr>
+  <tr>
+    <td>field_list </td>
+    <td>Comma separated List of fields with data type. The field names should consist
of alphanumeric characters and underscore(_) special character. </td>
+    <td>No </td>
+  </tr>
+  <tr>
+    <td>table_name </td>
+    <td>The name of the table in Database. Table Name should consist of alphanumeric
characters and underscore(_) special character. </td>
+    <td>No </td>
+  </tr>
+  <tr>
+    <td>STORED BY </td>
+    <td>"org.apache.carbondata.format", identifies and creates a CarbonData table.
</td>
+    <td>No </td>
+  </tr>
+  <tr>
+    <td>TBLPROPERTIES </td>
+    <td>List of CarbonData table properties. </td>
+    <td> </td>
+  </tr>
+  </tbody>
+</table><h3>Usage Guidelines</h3><p>Following are the guidelines
for using table properties.</p>
+<ul>
+  <li><p><strong>Dictionary Encoding Configuration</strong></p><p>Dictionary
encoding is enabled by default for all String columns, and disabled for non-String columns.
You can include and exclude columns for dictionary encoding.</p></li>
+</ul><p><code>
+  TBLPROPERTIES (&quot;DICTIONARY_EXCLUDE&quot;=&quot;column1, column2&quot;)
+  TBLPROPERTIES (&quot;DICTIONARY_INCLUDE&quot;=&quot;column1, column2&quot;)
+</code></p><p>Here, DICTIONARY_EXCLUDE will exclude dictionary creation.
This is applicable for high-cardinality columns and is an optional parameter. DICTIONARY_INCLUDE
will generate dictionary for the columns specified in the list.</p>
+<ul>
+  <li><p><strong>Row/Column Format Configuration</strong></p><p>Column
groups with more than one column are stored in row format, instead of columnar format. By
default, each column is a separate column group.</p></li>
+</ul><p><code>
+  TBLPROPERTIES (&quot;COLUMN_GROUPS&quot;=&quot;(column1, column3),
+  (Column4,Column5,Column6)&quot;)
+</code></p>
+<ul>
+  <li><p><strong>Table Block Size Configuration</strong></p><p>The
block size of table files can be defined using the property TABLE_BLOCKSIZE. It accepts only
integer values. The default value is 1024 MB and supports a range of 1 MB to 2048 MB.  If
you do not specify this value in the DDL command, default value is used.</p></li>
+</ul><p><code>
+  TBLPROPERTIES (&quot;TABLE_BLOCKSIZE&quot;=&quot;512 MB&quot;)
+</code></p><p>Here 512 MB means the block size of this table is 512 MB,
you can also set it as 512M or 512.</p>
+<ul>
+  <li><p><strong>Inverted Index Configuration</strong></p><p>Inverted
index is very useful to improve compression ratio and query speed, especially for those low-cardinality
columns who are in reward position.  By default inverted index is enabled. The user can disable
the inverted index creation for some columns.</p></li>
+</ul><p><code>
+  TBLPROPERTIES (&quot;NO_INVERTED_INDEX&quot;=&quot;column1, column3&quot;)
+</code></p><p>No inverted index shall be generated for the columns specified
in NO_INVERTED_INDEX. This property is applicable on columns with high-cardinality and is
an optional parameter.</p><p>NOTE:</p>
+<ul>
+  <li><p>By default all columns other than numeric datatype are treated as dimensions
and all columns of numeric datatype are treated as measures.</p></li>
+  <li><p>All dimensions except complex datatype columns are part of multi dimensional
key(MDK). This behavior can be overridden by using TBLPROPERTIES. If the user wants to keep
any column (except columns of complex datatype) in multi dimensional key then he can keep
the columns either in DICTIONARY_EXCLUDE or DICTIONARY_INCLUDE.</p><h3>Example:</h3>
+    <p><pre><code>
+    CREATE TABLE IF NOT EXISTS productSchema.productSalesTable (
+                                                                productNumber Int,
+                                                                productName String,
+                                                                storeCity String,
+                                                                storeProvince String,
+                                                                productCategory String,
+                                                                productBatch String,
+                                                                saleQuantity Int,
+                                                                revenue Int)
+                                                                STORED BY &#39;carbondata&#39;
+                                                                TBLPROPERTIES (&#39;COLUMN_GROUPS&#39;=&#39;(productName,productCategory)&#39;,
+                                                                &#39;DICTIONARY_EXCLUDE&#39;=&#39;productName&#39;,
+                                                                &#39;DICTIONARY_INCLUDE&#39;=&#39;productNumber&#39;,
+                                                                &#39;NO_INVERTED_INDEX&#39;=&#39;productBatch&#39;)
+  </code></pre></p></li>
+</ul><h2 id="show-table">SHOW TABLE</h2><p>This command can be used
to list all the tables in current database or all the tables of a specific database. <code>
+  SHOW TABLES [IN db_Name];
+</code></p><h3>Parameter Description</h3>
+<table class="table table-striped table-bordered">
+  <thead>
+  <tr>
+    <th>Parameter </th>
+    <th>Description </th>
+    <th>Optional </th>
+  </tr>
+  </thead>
+  <tbody>
+  <tr>
+    <td>IN db_Name </td>
+    <td>Name of the database. Required only if tables of this specific database are
to be listed. </td>
+    <td>Yes </td>
+  </tr>
+  </tbody>
+</table><h3>Example:</h3><p><code>
+  SHOW TABLES IN ProductSchema;
+</code></p><h2 id="drop-table">DROP TABLE</h2><p>This command
is used to delete an existing table.</p><p><code>
+  DROP TABLE [IF EXISTS] [db_name.]table_name;
+</code></p><h3>Parameter Description</h3>
+<table class="table table-striped table-bordered">
+  <thead>
+  <tr>
+    <th>Parameter </th>
+    <th>Description </th>
+    <th>Optional </th>
+  </tr>
+  </thead>
+  <tbody>
+  <tr>
+    <td>db_Name </td>
+    <td>Name of the database. If not specified, current database will be selected.
</td>
+    <td>YES </td>
+  </tr>
+  <tr>
+    <td>table_name </td>
+    <td>Name of the table to be deleted. </td>
+    <td>NO </td>
+  </tr>
+  </tbody>
+</table><h3>Example:</h3><p><code>
+  DROP TABLE IF EXISTS productSchema.productSalesTable;
+</code></p><h2 id="compaction">COMPACTION</h2><p>This command
merges the specified number of segments into one segment. This enhances the query performance
of the table.</p><p><code>
+  ALTER TABLE [db_name.]table_name COMPACT &#39;MINOR/MAJOR&#39;;
+</code></p><p>To get details about Compaction refer to Data Management</p><h3>Parameter
Description</h3>
+<table class="table table-striped table-bordered">
+  <thead>
+  <tr>
+    <th>Parameter </th>
+    <th>Description </th>
+    <th>Optional </th>
+  </tr>
+  </thead>
+  <tbody>
+  <tr>
+    <td>db_name </td>
+    <td>Database name, if it is not specified then it uses current database. </td>
+    <td>YES </td>
+  </tr>
+  <tr>
+    <td>table_name </td>
+    <td>The name of the table in provided database.</td>
+    <td>NO </td>
+  </tr>
+  </tbody>
+</table><h3>Syntax</h3>
+<ul>
+  <li><strong>Minor Compaction</strong></li>
+</ul><p><code>
+  ALTER TABLE table_name COMPACT &#39;MINOR&#39;;
+</code>
+<ul>
+  <li><strong>Major Compaction</strong></li></p><p><code>
+  ALTER TABLE table_name COMPACT &#39;MAJOR&#39;;
+</code></p>
+</ul>
+ <h2 id="bucketing">BUCKETING</h2>
+<p>Bucketing feature can be used to distribute/organize the table/partition data into
multiple files such that similar records are present in the same file. While creating a table,
a user needs to specify the columns to be used for bucketing and the number of buckets. For
the selction of bucket the Hash value of columns is used.</p><p>
+  <pre>
+  <code>CREATE TABLE [IF NOT EXISTS] [db_name.]table_name  [(col_name data_type, ...)]
+        STORED BY 'carbondata'  TBLPROPERTIES('BUCKETNUMBER'='noOfBuckets', 'BUCKETCOLUMNS'='columnname',
'TABLENAME'='tablename')
+  </code>
+</pre>
+    </p><p></p><h2>Parameter Description</h2>
+<table class="table table-striped table-bordered">
+  <thead>
+  <tr>
+    <th>Parameter </th>
+    <th>Description </th>
+    <th>Optional </th>
+  </tr>
+  </thead>
+  <tbody>
+  <tr>
+    <td>BUCKETNUMBER </td>
+    <td>Specifies the number of Buckets to be created. </td>
+    <td>No </td>
+  </tr>
+  <tr>
+    <td>BUCKETCOLUMNS </td>
+    <td>Specify the columns to be considered for Bucketing </td>
+    <td>No </td>
+  </tr>
+  <tr>
+    <td>TABLENAME </td>
+    <td>The name of the table in Database. Table Name should consist of alphanumeric
characters and underscore(_) special character. </td>
+    <td>Yes </td>
+  </tr>
+  </tbody>
+</table><h2>Usage Guidelines</h2>
+<ul>
+  <li><p>The feature is supported for Spark 1.6.2 onwards, but the performance
optimization is evident from Spark 2.1 onwards.</p></li>
+  <li><p>Bucketing can not be performed for columns of Complex Data Types.</p></li>
+  <li><p>Columns in the BUCKETCOLUMN parameter must be either a dimension or
a measure but combination of both is not supported.</p></li>
+</ul><h2>Example :</h2>
+<pre>
+  <code>CREATE TABLE IF NOT EXISTS productSchema.productSalesTable (
+                                                                productNumber Int,
+                                                                productName String,
+                                                                storeCity String,
+                                                                storeProvince String,
+                                                                productCategory String,
+                                                                productBatch String,
+                                                                saleQuantity Int,
+                                                                revenue Int)
+                                                                STORED BY 'carbondata'
+                                                                TBLPROPERTIES ('COLUMN_GROUPS'='(productName,productCategory)',
+                                                                'DICTIONARY_EXCLUDE'='productName',
+                                                                'DICTIONARY_INCLUDE'='productNumber',
+                                                                'NO_INVERTED_INDEX'='productBatch',
+                                                                'BUCKETNUMBER'='4',
+                                                                'BUCKETCOLUMNS'='productNumber,saleQuantity',
+                                                                'TABLENAME'='productSalesTable')
+  </code>
+</pre>
\ No newline at end of file


Mime
View raw message