orc-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From omal...@apache.org
Subject orc git commit: Push site updates to deploy.
Date Wed, 02 May 2018 15:35:26 GMT
Repository: orc
Updated Branches:
  refs/heads/asf-site da021334f -> be01d14ac

Push site updates to deploy.

Signed-off-by: Owen O'Malley <omalley@apache.org>

Project: http://git-wip-us.apache.org/repos/asf/orc/repo
Commit: http://git-wip-us.apache.org/repos/asf/orc/commit/be01d14a
Tree: http://git-wip-us.apache.org/repos/asf/orc/tree/be01d14a
Diff: http://git-wip-us.apache.org/repos/asf/orc/diff/be01d14a

Branch: refs/heads/asf-site
Commit: be01d14ac25f6c241ff131afb3893a618fefa6b1
Parents: da02133
Author: Owen O'Malley <omalley@apache.org>
Authored: Wed May 2 08:34:33 2018 -0700
Committer: Owen O'Malley <omalley@apache.org>
Committed: Wed May 2 08:34:33 2018 -0700

 docs/hive-config.html    |  7 +++++++
 security.html            |  4 ++--
 specification/ORCv2.html | 21 ++++++---------------
 3 files changed, 15 insertions(+), 17 deletions(-)

diff --git a/docs/hive-config.html b/docs/hive-config.html
index bc2f68c..159cdf7 100644
--- a/docs/hive-config.html
+++ b/docs/hive-config.html
@@ -886,6 +886,13 @@ with the same options.</p>
       the compression level of higher level compression codec. Value can be
       SPEED or COMPRESSION.</td>
+  <td>orc.write.variable.length.blocks</td>
+  <td>false</td>
+  <td>Should the ORC writer use HDFS variable length blocks, if they are
+      available? If the new stripe would straddle a block, Hadoop is &ge; 2.7,
+      and this is enabled, it will end the block before the new stripe.</td>

diff --git a/security.html b/security.html
index 56e06ed..0db181b 100644
--- a/security.html
+++ b/security.html
@@ -102,12 +102,12 @@ address. All mail sent to this address that does not relate to an
 undisclosed security problem in Apache ORC will be ignored.</p>
 <p>The ORC security mailing list address is:
-<a href="mailto:private@orc.apache.org">private@orc.apache.org</a>.
+<a href="mailto:security@orc.apache.org">security@orc.apache.org</a>.
 This is a private mailing list and only members of the ORC project
 are subscribed.</p>
 <p>Please note that we do not use a team GnuPG key. If you wish to
-encrypt your e-mail to private@orc.apache.org then please use the GnuPG
+encrypt your e-mail to security@orc.apache.org then please use the GnuPG
 keys from <a href="https://people.apache.org/keys/group/orc.asc">ORC GPG keys</a>
 the members of the
 <a href="https://people.apache.org/phonebook.html?ctte=orc">ORC PMC</a>.</p>

diff --git a/specification/ORCv2.html b/specification/ORCv2.html
index b78fc0a..db8aecf 100644
--- a/specification/ORCv2.html
+++ b/specification/ORCv2.html
@@ -92,7 +92,7 @@ developers on the project.</p>
 <p>The list of things that we plan to change:</p>
-  <li>Create a decimal representation with fixed scale using rle.</li>
+  <li>Move decimal encoding to RLEv3 and remove variable length encoding.</li>
   <li>Create a better float/double encoding that splits mantissa and
   <li>Create a dictionary encoding for float, double, and decimal.</li>
@@ -1283,13 +1283,10 @@ value.</p>
 <h2 id="decimal-columns">Decimal Columns</h2>
-<p>Decimal was introduced in Hive 0.11 with infinite precision (the total
-number of digits). In Hive 0.13, the definition was change to limit
-the precision to a maximum of 38 digits, which conveniently uses 127
-bits plus a sign bit. The current encoding of decimal columns stores
-the integer representation of the value as an unbounded length zigzag
-encoded base 128 varint. The scale is stored in the SECONDARY stream
-as an signed integer.</p>
+<p>Since Hive 0.13, all decimals have had fixed precision and scale.
+The goal is to use RLEv3 for the value and use the fixed scale from
+the type. As an interim solution, we are using RLE v2 for short decimals
+(precision &lt;= 18) and the old encoding for long decimals.</p>
@@ -1311,13 +1308,7 @@ as an signed integer.</p>
       <td style="text-align: left"> </td>
       <td style="text-align: left">DATA</td>
       <td style="text-align: left">No</td>
-      <td style="text-align: left">Unbounded base 128 varints</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">SECONDARY</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
+      <td style="text-align: left">Signed Integer RLE v2</td>
       <td style="text-align: left">DIRECT_V2</td>

View raw message