drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bridg...@apache.org
Subject [drill-site] branch asf-site updated: edit s3 SP doc
Date Thu, 21 Jun 2018 23:30:10 GMT
This is an automated email from the ASF dual-hosted git repository.

bridgetb pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/drill-site.git


The following commit(s) were added to refs/heads/asf-site by this push:
     new 55efe5a  edit s3 SP doc
55efe5a is described below

commit 55efe5a8af8e7d0a96eb7ce8fde12fe1905ab7a2
Author: Bridget Bevens <bbevens@maprtech.com>
AuthorDate: Thu Jun 21 16:29:54 2018 -0700

    edit s3 SP doc
---
 docs/s3-storage-plugin/index.html | 61 +++++++++++++++++----------------------
 feed.xml                          |  4 +--
 2 files changed, 28 insertions(+), 37 deletions(-)

diff --git a/docs/s3-storage-plugin/index.html b/docs/s3-storage-plugin/index.html
index 30c5919..8663260 100644
--- a/docs/s3-storage-plugin/index.html
+++ b/docs/s3-storage-plugin/index.html
@@ -1230,36 +1230,37 @@
 
     </div>
 
-     Jun 19, 2018
+     Jun 21, 2018
 
     <link href="/css/docpage.css" rel="stylesheet" type="text/css">
 
     <div class="int_text" align="left">
       
-        <p>Drill works with data stored in the cloud. With a few simple steps, you
can configure the S3 storage plugin for Drill and be off to the races running queries.</p>
+        <p>Drill works with data stored in the cloud. With a few simple steps, you
can configure the S3 storage plugin for Drill and be off to the races running queries. Drill
has the ability to query files stored on Amazon&#39;s S3 cloud storage using the HDFS
s3a library. The HDFS s3a library adds support for files larger than 5 gigabytes (these were
unsupported using the older HDFS s3n library).</p>
 
-<h2 id="connecting-drill-to-s3">Connecting Drill to S3</h2>
+<p>To connect Drill to S3, you must provide your AWS credentials, and configure the
S3 storage plugin with an S3 bucket.  </p>
 
-<p>Drill has the ability to query files stored on Amazon&#39;s S3 cloud storage
using the HDFS s3a library. The HDFS s3a library adds support for files larger than 5 gigabytes
(these were unsupported using the older HDFS s3n library).</p>
+<p>For additional information, refer to the <a href="https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html">HDFS
S3 documentation</a>.   </p>
 
-<p>To connect Drill to S3:  </p>
+<p><strong>Note:</strong> Drill does not use HDFS 3.x, therefore Drill
does not support AWS temporary credentials, as described in the s3a documentation.</p>
 
-<ul>
-<li>Provide your AWS credentials.<br></li>
-<li>Configure the S3 storage plugin with an S3 bucket.<br></li>
-</ul>
+<h2 id="providing-aws-credentials">Providing AWS Credentials</h2>
 
-<p>For additional information, you can refer to the <a href="https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html">HDFS
S3 documentation</a>.   </p>
+<p>Your environment determines where you provide your AWS credentials. You can define
your AWS credentials one of three ways:  </p>
 
-<p><strong>Note:</strong> Drill does not use HDFS 3.x, therefore Drill
does not support AWS temporary credentials, as described in the s3a documentation.</p>
+<ul>
+<li>Directly in the S3 storage plugin. Note that this method is the least secure, but
sufficient for use on a single machine, such as a laptop.<br></li>
+<li>In a non-Hadoop environment, you can use the Drill-specific core-site.xml file
to provide the AWS credentials.<br></li>
+<li>In a Hadoop environment, you can use the existing S3 configuration for Hadoop.
The S3 access keys should already be defined. All you need to do is <a href="/docs/s3-storage-plugin/#configuring-the-s3-storage-plugin">configure
the S3 storage plugin</a>.<br></li>
+</ul>
 
-<h3 id="providing-aws-credentials">Providing AWS Credentials</h3>
+<h3 id="defining-access-keys-in-the-s3-storage-plugin">Defining Access Keys in the
S3 Storage Plugin</h3>
 
-<p>Define the AWS credentials (access keys) in the core-site.xml file or as storage
plugin configuration properties in the S3 storage plugin configuration in the Drill Web UI.
However, configuring credentials in the S3 storage plugin in not very secure, but sufficient
for use on a single machine, such as a laptop.  </p>
+<p>Refer to <a href="/docs/s3-storage-plugin/#configuring-the-s3-storage-plugin">Configuring
the S3 Storage Plugin</a>. </p>
 
-<p><strong>Configuring Access Keys in core-site.xml</strong></p>
+<h3 id="defining-access-keys-in-the-drill-core-site.xml-file">Defining Access Keys
in the Drill core-site.xml File</h3>
 
-<p>To configure the access keys in the core-site.xml file, navigate to the <code>$DRILL_HOME/conf</code>
or <code>$DRILL_SITE</code> directory, and rename the core-site-example.xml file
to core-site.xml. Replace the text <code>ENTER_YOUR_ACESSKEY</code> and <code>ENTER_YOUR_SECRETKEY</code>
with your AWS credentials and also include the endpoint, as shown in the following example:
  </p>
+<p>To configure the access keys in Drill&#39;s core-site.xml file, navigate to
the <code>$DRILL_HOME/conf</code> or <code>$DRILL_SITE</code> directory,
and rename the core-site-example.xml file to core-site.xml. Replace the text <code>ENTER_YOUR_ACESSKEY</code>
and <code>ENTER_YOUR_SECRETKEY</code> with your AWS credentials and also include
the endpoint, as shown in the following example:   </p>
 <div class="highlight"><pre><code class="language-text" data-lang="text">
  &lt;configuration&gt;
        &lt;property&gt;
            &lt;name&gt;fs.s3a.access.key&lt;/name&gt;
@@ -1277,34 +1278,24 @@
 </code></pre></div>
 <p><strong>Note:</strong> When you rename the file, Hadoop support breaks
if <code>$HADOOP_HOME</code> was in the path because Drill pulls in the Drill
core-site.xml file instead of the Hadoop core-site.xml file. In this situation, make the changes
in the Hadoop core-site.xml file. Do not create a core-site.xml file for Drill.  </p>
 
-<p><strong>Configuring Access Keys in the S3 Storage Plugin</strong> </p>
-
-<p>If you choose to configure the access keys in the S3 plugin, include the access
keys and the endpoint in the S3 storage plugin configuration, as shown in the following example:
 </p>
-<div class="highlight"><pre><code class="language-text" data-lang="text">
    &quot;connection&quot;: &quot;s3a://&lt;bucket-name&gt;/&quot;,
-     &quot;config&quot;: {
-       &quot;fs.s3a.access.key&quot;: &quot;&lt;key&gt;&quot;,
-       &quot;fs.s3a.secret.key&quot;: &quot;&lt;key&gt;&quot;,
-       &quot;fs.s3a.endpoint&quot;: &quot;s3.us-west-1.amazonaws.com&quot;
-     },  
-</code></pre></div>
-<p><strong>Note:</strong> General instructions for configuring the S3 storage
plugin are provided in the next section of this document.   </p>
+<h2 id="configuring-the-s3-storage-plugin">Configuring the S3 Storage Plugin</h2>
 
-<h3 id="configuring-the-s3-storage-plugin">Configuring the S3 Storage Plugin</h3>
+<p>The Storage page in the Drill Web UI provides an S3 storage plugin that you configure
to connect Drill to the S3 distributed file system registered in core-site.xml. If you did
not define your AWS credentials in the core-site.xml file, you can define them in the storage
plugin configuration.   </p>
 
-<p>The Storage page in the Drill Web UI provides an S3 storage plugin that you configure
to connect Drill to the S3 distributed file system registered in core-site.xml. Once configured,
you can query the files stored in S3. To configure the S3 storage plugin, log in to the Drill
Web UI and then update the S3 configuration, as described in the following steps:   </p>
+<p>To configure the S3 storage plugin, log in to the Drill Web UI and then update the
S3 configuration with the bucket name, as described in the following steps:   </p>
 
 <p>1- To access the Drill Web UI, enter the following URL in the address bar of your
web browser:  </p>
 <div class="highlight"><pre><code class="language-text" data-lang="text">
  http://&lt;drill-hostname&gt;:8047  
 
    //The drill-hostname is a node on which Drill is running.  
 </code></pre></div>
-<p>2-To configure the S3 storage plugin, complete the following steps:  </p>
+<p>2-To configure the S3 storage plugin in Drill, complete the following steps:  </p>
 
-<p>a. Click on the Storage page.<br>
+<p>a. Click on the <strong>Storage</strong> page.<br>
    b. Find the S3 option on the page and then click <strong>Update</strong> next
to the option.<br>
-   c. Configure the S3 storage plugin, specifying the bucket in the &quot;connection&quot;
property, as shown in the following example:  </p>
+   c. Configure the S3 storage plugin, specifying the bucket in the <code>&quot;connection&quot;</code>
property, as shown in the following example:  </p>
 
-<p><strong>Note:</strong> The &quot;config&quot; section with the
access key and endpoint properties are included in the following configuration as an example.
Do not include the &quot;config&quot; section in your S3 configuration if you used
core-site.xml to store your access keys.   </p>
+<p><strong>Note:</strong> The <code>&quot;config&quot;</code>
block in the following S3 storage plugin configuration contains the access key and endpoint
properties required if you want to define your AWS credentials in the storage plugin configuration.
Do not include the <code>&quot;config&quot;</code> block in your S3 storage
plugin configuration if you defined your AWS credentials in the core-site.xml file.   </p>
 <div class="highlight"><pre><code class="language-text" data-lang="text">
  {
 &quot;type&quot;: &quot;file&quot;,
 &quot;enabled&quot;: true,
@@ -1345,10 +1336,10 @@
 }
 </code></pre></div>
 <p>4-Click <strong>Update</strong> to save the configuration.<br>
-5-Navigate back to the Storage page.<br>
-6-On the Storage page, click <strong>Enable</strong> next to the S3 option. 
</p>
+5-Navigate back to the <strong>Storage</strong> page.<br>
+6-On the <strong>Storage</strong> page, click <strong>Enable</strong>
next to the S3 option.  </p>
 
-<p>Drill should now be able to use the HDFS s3a library to access data in S3. </p>
+<p>Drill should now be able to use the HDFS s3a library to access data in S3.</p>
 
 <h2 id="quering-parquet-format-files-on-s3">Quering Parquet Format Files On S3</h2>
 
diff --git a/feed.xml b/feed.xml
index 7cfe89a..fe30a50 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
 </description>
     <link>/</link>
     <atom:link href="/feed.xml" rel="self" type="application/rss+xml"/>
-    <pubDate>Wed, 20 Jun 2018 18:52:18 -0700</pubDate>
-    <lastBuildDate>Wed, 20 Jun 2018 18:52:18 -0700</lastBuildDate>
+    <pubDate>Thu, 21 Jun 2018 16:28:11 -0700</pubDate>
+    <lastBuildDate>Thu, 21 Jun 2018 16:28:11 -0700</lastBuildDate>
     <generator>Jekyll v2.5.2</generator>
     
       <item>


Mime
View raw message