knox-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From m...@apache.org
Subject svn commit: r1832208 - in /knox: site/ site/books/knox-1-1-0/ trunk/books/1.1.0/
Date Thu, 24 May 2018 21:45:27 GMT
Author: more
Date: Thu May 24 21:45:27 2018
New Revision: 1832208

URL: http://svn.apache.org/viewvc?rev=1832208&view=rev
Log:
KNOX-1332 - Document ElasticSearch service (Dequan Chen via Sandeep More)

Added:
    knox/trunk/books/1.1.0/service_elasticsearch.md
Modified:
    knox/site/books/knox-1-1-0/user-guide.html
    knox/site/index.html
    knox/site/issue-tracking.html
    knox/site/license.html
    knox/site/mail-lists.html
    knox/site/project-info.html
    knox/site/team-list.html
    knox/trunk/books/1.1.0/book.md
    knox/trunk/books/1.1.0/book_service-details.md

Modified: knox/site/books/knox-1-1-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-1-1-0/user-guide.html?rev=1832208&r1=1832207&r2=1832208&view=diff
==============================================================================
--- knox/site/books/knox-1-1-0/user-guide.html (original)
+++ knox/site/books/knox-1-1-0/user-guide.html Thu May 24 21:45:27 2018
@@ -109,6 +109,7 @@
     <li><a href="#SOLR">SOLR</a></li>
     <li><a href="#Avatica">Avatica</a></li>
     <li><a href="#Livy+Server">Livy Server</a></li>
+    <li><a href="#ElasticSearch">ElasticSearch</a></li>
     <li><a href="#Common+Service+Config">Common Service Config</a></li>
     <li><a href="#Default+Service+HA+support">Default Service HA support</a></li>
   </ul></li>
@@ -4664,6 +4665,7 @@ dep/commons-codec-1.7.jar
   <li><a href="#SOLR">SOLR</a></li>
   <li><a href="#Avatica">Avatica</a></li>
   <li><a href="#Livy+Server">Livy Server</a></li>
+  <li><a href="#ElasticSearch">ElasticSearch</a></li>
 </ul><h3><a id="Assumptions">Assumptions</a> <a href="#Assumptions"><img
src="markbook-section-link.png"/></a></h3><p>This document assumes a
few things about your environment in order to simplify the examples.</p>
 <ul>
   <li>The JVM is executable as simply <code>java</code>.</li>
@@ -6397,7 +6399,99 @@ DriverManager.getConnection(url, props);
   &lt;role&gt;LIVYSERVER&lt;/role&gt;
   &lt;url&gt;http://&lt;livy-server&gt;:8998&lt;/url&gt;
 &lt;/service&gt;
-</code></pre><p>Livy server will use proxyUser to run the Spark session.
To avoid that a user can provide here any user (e.g. a more privileged), Knox will need to
rewrite the the json body to replace what so ever is the value of proxyUser is with the username
of the authenticated user.</p><p>{<br/> &ldquo;driverMemory&rdquo;:&ldquo;2G&rdquo;,
 &ldquo;executorCores&rdquo;:4,  &ldquo;executorMemory&rdquo;:&ldquo;8G&rdquo;,
 &ldquo;proxyUser&rdquo;:&ldquo;bernhard&rdquo;,  &ldquo;conf&rdquo;:{<br/>
&ldquo;spark.master&rdquo;:&ldquo;yarn-cluster&rdquo;,  &ldquo;spark.jars.packages&rdquo;:&ldquo;com.databricks:spark-csv_2.10:1.5.0&rdquo;
 }  } </p><p>The above is an example request body to be used to create a Spark
session via Livy server and illustrates the &ldquo;proxyUser&rdquo; that requires
rewrite.</p><h3><a id="Service+Test+API">Service Test API</a> <a
href="#Service+Test+API"><img src="markbook-section-link.png"/></a></h3><p>The
gateway supports a Service Test API that can be us
 ed to test Knox&rsquo;s ability to connect to each of the different Hadoop services via
a simple HTTP GET request. To be able to access this API one must add the following line into
the topology for which you wish to run the service test.</p>
+</code></pre><p>Livy server will use proxyUser to run the Spark session.
To avoid that a user can provide here any user (e.g. a more privileged), Knox will need to
rewrite the the json body to replace what so ever is the value of proxyUser is with the username
of the authenticated user.</p><p>{<br/> &ldquo;driverMemory&rdquo;:&ldquo;2G&rdquo;,
 &ldquo;executorCores&rdquo;:4,  &ldquo;executorMemory&rdquo;:&ldquo;8G&rdquo;,
 &ldquo;proxyUser&rdquo;:&ldquo;bernhard&rdquo;,  &ldquo;conf&rdquo;:{<br/>
&ldquo;spark.master&rdquo;:&ldquo;yarn-cluster&rdquo;,  &ldquo;spark.jars.packages&rdquo;:&ldquo;com.databricks:spark-csv_2.10:1.5.0&rdquo;
 }  } </p><p>The above is an example request body to be used to create a Spark
session via Livy server and illustrates the &ldquo;proxyUser&rdquo; that requires
rewrite.</p><h3><a id="ElasticSearch">ElasticSearch</a> <a href="#ElasticSearch"><img
src="markbook-section-link.png"/></a></h3><p>ElasticSearch provides a
REST API for communicating with Elasti
 cSearch via JSON over HTTP. ElasticSearch uses X-Pack to do its own security (authentication
and authorization). Therefore, the Knox Gateway is to forward the user credentials to ElasticSearch,
and treats the ElasticSearch-authenticated user as &ldquo;anonymous&rdquo; to the
backend service via a doas query param while Knox will authenticate to backend services as
itself.</p><h4><a id="Gateway+configuration">Gateway configuration</a>
<a href="#Gateway+configuration"><img src="markbook-section-link.png"/></a></h4><p>The
Gateway can be configured for ElasticSearch by modifying the topology XML file and providing
a new service XML file.</p><p>In the topology XML file, add the following new
service named &ldquo;ELASTICSEARCH&rdquo; with the correct elasticsearch-rest-server
hostname and port number (e.g., 9200):</p>
+<pre><code> &lt;service&gt;
+   &lt;role&gt;ELASTICSEARCH&lt;/role&gt;
+   &lt;url&gt;http://&lt;elasticsearch-rest-server&gt;:9200/&lt;/url&gt;
+   &lt;name&gt;elasticsearch&lt;/name&gt;
+ &lt;/service&gt;
+</code></pre><h4><a id="ElasticSearch+via+Knox+Gateway">ElasticSearch
via Knox Gateway</a> <a href="#ElasticSearch+via+Knox+Gateway"><img src="markbook-section-link.png"/></a></h4><p>After
adding the above to a topology, you can make a cURL request similar to the following structures:</p><h5><a
id="1.++ElasticSearch+Node+Root+Query">1. ElasticSearch Node Root Query</a> <a
href="#1.++ElasticSearch+Node+Root+Query"><img src="markbook-section-link.png"/></a></h5>
+<pre><code>curl -i -k -u username:password -H &quot;Accept: application/json&quot;
 -X GET  &quot;https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch&quot;
+
+or
+
+curl -i -k -u username:password -H &quot;Accept: application/json&quot;  -X GET 
&quot;https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/&quot;
+</code></pre><p>The quotation marks around the URL, can be single quotes
or double quotes on both sides, and can also be omitted (Note: This is true for all other
ElasticSearch queries via Knox). Below is an example response:</p>
+<pre><code> HTTP/1.1 200 OK
+ Date: Wed, 23 May 2018 16:36:34 GMT
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 356
+ Server: Jetty(9.2.15.v20160210)
+
+ {&quot;name&quot;:&quot;w0A80p0&quot;,&quot;cluster_name&quot;:&quot;elasticsearch&quot;,&quot;cluster_uuid&quot;:&quot;poU7j48pSpu5qQONr64HLQ&quot;,&quot;version&quot;:{&quot;number&quot;:&quot;6.2.4&quot;,&quot;build_hash&quot;:&quot;ccec39f&quot;,&quot;build_date&quot;:&quot;2018-04-12T20:37:28.497551Z&quot;,&quot;build_snapshot&quot;:false,&quot;lucene_version&quot;:&quot;7.2.1&quot;,&quot;minimum_wire_compatibility_version&quot;:&quot;5.6.0&quot;,&quot;minimum_index_compatibility_version&quot;:&quot;5.0.0&quot;},&quot;tagline&quot;:&quot;You
Know, for Search&quot;}
+</code></pre><h5><a id="2.++ElasticSearch+Index+-+Creation,+Deletion,+Refreshing+and+Data+Operations+-+Writing,+Updating+and+Retrieval">2.
ElasticSearch Index - Creation, Deletion, Refreshing and Data Operations - Writing, Updating
and Retrieval</a> <a href="#2.++ElasticSearch+Index+-+Creation,+Deletion,+Refreshing+and+Data+Operations+-+Writing,+Updating+and+Retrieval"><img
src="markbook-section-link.png"/></a></h5><h6><a id="(1)+Index+Creation">(1)
Index Creation</a> <a href="#(1)+Index+Creation"><img src="markbook-section-link.png"/></a></h6>
+<pre><code>curl -i -k -u username:password -H &quot;Content-Type: application/json&quot;
 -X PUT  &quot;https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}&quot;
 -d &#39;{
+&quot;settings&quot; : {
+    &quot;index&quot; : {
+        &quot;number_of_shards&quot; : {index-shards-number},
+        &quot;number_of_replicas&quot; : {index-replicas-number}
+    }
+  }
+}&#39;
+</code></pre><p>Below is an example response:</p>
+<pre><code> HTTP/1.1 200 OK
+ Date: Wed, 23 May 2018 16:51:31 GMT
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 65
+ Server: Jetty(9.2.15.v20160210)
+
+ {&quot;acknowledged&quot;:true,&quot;shards_acknowledged&quot;:true,&quot;index&quot;:&quot;estest&quot;}
+</code></pre><h6><a id="(2)+Index+Data+Writing">(2) Index Data Writing</a>
<a href="#(2)+Index+Data+Writing"><img src="markbook-section-link.png"/></a></h6><p>For
adding a &ldquo;Hello Joe Smith&rdquo; document:</p>
+<pre><code>curl -i -k -u username:password -H &quot;Content-Type: application/json&quot;
 -X PUT  &quot;https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}/{document-type-name}/{document-id}&quot;
 -d &#39;{
+    &quot;title&quot;:&quot;Hello Joe Smith&quot; 
+}&#39;
+</code></pre><p>Below is an example response:</p>
+<pre><code> HTTP/1.1 201 Created
+ Date: Wed, 23 May 2018 17:00:17 GMT
+ Location: /estest/greeting/1
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 158
+ Server: Jetty(9.2.15.v20160210)
+
+ {&quot;_index&quot;:&quot;estest&quot;,&quot;_type&quot;:&quot;greeting&quot;,&quot;_id&quot;:&quot;1&quot;,&quot;_version&quot;:1,&quot;result&quot;:&quot;created&quot;,&quot;_shards&quot;:{&quot;total&quot;:1,&quot;successful&quot;:1,&quot;failed&quot;:0},&quot;_seq_no&quot;:0,&quot;_primary_term&quot;:1}
+</code></pre><h6><a id="(3)+Index+Refreshing">(3) Index Refreshing</a>
<a href="#(3)+Index+Refreshing"><img src="markbook-section-link.png"/></a></h6>
+<pre><code>curl -i -k -u username:password  -X POST  &quot;https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}/_refresh&quot;

+</code></pre><p>Below is an example response:</p>
+<pre><code> HTTP/1.1 200 OK
+ Date: Wed, 23 May 2018 17:02:32 GMT
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 49
+ Server: Jetty(9.2.15.v20160210)
+
+ {&quot;_shards&quot;:{&quot;total&quot;:1,&quot;successful&quot;:1,&quot;failed&quot;:0}}
+</code></pre><h6><a id="(4)+Index+Data+Upgrading">(4) Index Data
Upgrading</a> <a href="#(4)+Index+Data+Upgrading"><img src="markbook-section-link.png"/></a></h6><p>For
changing the Person Joe Smith to Tom Smith:</p>
+<pre><code>curl -i -k -u username:password -H &quot;Content-Type: application/json&quot;
 -X PUT  &quot;https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}/{document-type-name}/{document-id}&quot;
 -d &#39;{ 
+&quot;title&quot;:&quot;Hello Tom Smith&quot; 
+}&#39;
+</code></pre><p>Below is an example response:</p>
+<pre><code> HTTP/1.1 200 OK
+ Date: Wed, 23 May 2018 17:09:59 GMT
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 158
+ Server: Jetty(9.2.15.v20160210)
+
+ {&quot;_index&quot;:&quot;estest&quot;,&quot;_type&quot;:&quot;greeting&quot;,&quot;_id&quot;:&quot;1&quot;,&quot;_version&quot;:2,&quot;result&quot;:&quot;updated&quot;,&quot;_shards&quot;:{&quot;total&quot;:1,&quot;successful&quot;:1,&quot;failed&quot;:0},&quot;_seq_no&quot;:1,&quot;_primary_term&quot;:1}
+</code></pre><h6><a id="(5)+Index+Data+Retrieval+or+Search">(5) Index
Data Retrieval or Search</a> <a href="#(5)+Index+Data+Retrieval+or+Search"><img
src="markbook-section-link.png"/></a></h6><p>For finding documents with
&ldquo;title&rdquo;:&ldquo;Hello&rdquo; in a specified document-type:</p>
+<pre><code>curl -i -k -u username:password -H &quot;Accept: application/json&quot;
-X GET  &quot;https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}/{document-type-name}/
_search?pretty=true;q=title:Hello&quot;
+</code></pre><p>Below is an example response:</p>
+<pre><code> HTTP/1.1 200 OK
+ Date: Wed, 23 May 2018 17:13:08 GMT
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 244
+ Server: Jetty(9.2.15.v20160210)
+
+ {&quot;took&quot;:0,&quot;timed_out&quot;:false,&quot;_shards&quot;:{&quot;total&quot;:1,&quot;successful&quot;:1,&quot;skipped&quot;:0,&quot;failed&quot;:0},&quot;hits&quot;:{&quot;total&quot;:1,&quot;max_score&quot;:0.2876821,&quot;hits&quot;:[{&quot;_index&quot;:&quot;estest&quot;,&quot;_type&quot;:&quot;greeting&quot;,&quot;_id&quot;:&quot;1&quot;,&quot;_score&quot;:0.2876821,&quot;_source&quot;:{&quot;title&quot;:&quot;Hello
Tom Smith&quot;}}]}}
+</code></pre><h6><a id="(6)+Index+Deleting">(6) Index Deleting</a>
<a href="#(6)+Index+Deleting"><img src="markbook-section-link.png"/></a></h6>
+<pre><code>curl -i -k -u username:password  -X DELETE  &quot;https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}&quot;
+</code></pre><p>Below is an example response:</p>
+<pre><code> HTTP/1.1 200 OK
+ Date: Wed, 23 May 2018 17:20:19 GMT
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 21
+ Server: Jetty(9.2.15.v20160210)
+
+ {&quot;acknowledged&quot;:true}
+</code></pre><h3><a id="Service+Test+API">Service Test API</a>
<a href="#Service+Test+API"><img src="markbook-section-link.png"/></a></h3><p>The
gateway supports a Service Test API that can be used to test Knox&rsquo;s ability to connect
to each of the different Hadoop services via a simple HTTP GET request. To be able to access
this API one must add the following line into the topology for which you wish to run the service
test.</p>
 <pre><code>&lt;service&gt;
   &lt;role&gt;SERVICE-TEST&lt;/role&gt;
 &lt;/service&gt;

Modified: knox/site/index.html
URL: http://svn.apache.org/viewvc/knox/site/index.html?rev=1832208&r1=1832207&r2=1832208&view=diff
==============================================================================
--- knox/site/index.html (original)
+++ knox/site/index.html Thu May 24 21:45:27 2018
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-05-18
+ | Generated by Apache Maven Doxia at 2018-05-24
  | Rendered using Apache Maven Fluido Skin 1.3.0
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180518" />
+    <meta name="Date-Revision-yyyymmdd" content="20180524" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Knox Gateway &#x2013; Announcing Apache Knox 1.0.0!</title>
     <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
               
                 
                     
-                  <li id="publishDate" class="pull-right">Last Published: 2018-05-18</li>

+                  <li id="publishDate" class="pull-right">Last Published: 2018-05-24</li>

             
                             </ul>
       </div>

Modified: knox/site/issue-tracking.html
URL: http://svn.apache.org/viewvc/knox/site/issue-tracking.html?rev=1832208&r1=1832207&r2=1832208&view=diff
==============================================================================
--- knox/site/issue-tracking.html (original)
+++ knox/site/issue-tracking.html Thu May 24 21:45:27 2018
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-05-18
+ | Generated by Apache Maven Doxia at 2018-05-24
  | Rendered using Apache Maven Fluido Skin 1.3.0
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180518" />
+    <meta name="Date-Revision-yyyymmdd" content="20180524" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Knox Gateway &#x2013; Issue Tracking</title>
     <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
               
                 
                     
-                  <li id="publishDate" class="pull-right">Last Published: 2018-05-18</li>

+                  <li id="publishDate" class="pull-right">Last Published: 2018-05-24</li>

             
                             </ul>
       </div>

Modified: knox/site/license.html
URL: http://svn.apache.org/viewvc/knox/site/license.html?rev=1832208&r1=1832207&r2=1832208&view=diff
==============================================================================
--- knox/site/license.html (original)
+++ knox/site/license.html Thu May 24 21:45:27 2018
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-05-18
+ | Generated by Apache Maven Doxia at 2018-05-24
  | Rendered using Apache Maven Fluido Skin 1.3.0
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180518" />
+    <meta name="Date-Revision-yyyymmdd" content="20180524" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Knox Gateway &#x2013; Project License</title>
     <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
               
                 
                     
-                  <li id="publishDate" class="pull-right">Last Published: 2018-05-18</li>

+                  <li id="publishDate" class="pull-right">Last Published: 2018-05-24</li>

             
                             </ul>
       </div>

Modified: knox/site/mail-lists.html
URL: http://svn.apache.org/viewvc/knox/site/mail-lists.html?rev=1832208&r1=1832207&r2=1832208&view=diff
==============================================================================
--- knox/site/mail-lists.html (original)
+++ knox/site/mail-lists.html Thu May 24 21:45:27 2018
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-05-18
+ | Generated by Apache Maven Doxia at 2018-05-24
  | Rendered using Apache Maven Fluido Skin 1.3.0
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180518" />
+    <meta name="Date-Revision-yyyymmdd" content="20180524" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Knox Gateway &#x2013; Project Mailing Lists</title>
     <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
               
                 
                     
-                  <li id="publishDate" class="pull-right">Last Published: 2018-05-18</li>

+                  <li id="publishDate" class="pull-right">Last Published: 2018-05-24</li>

             
                             </ul>
       </div>

Modified: knox/site/project-info.html
URL: http://svn.apache.org/viewvc/knox/site/project-info.html?rev=1832208&r1=1832207&r2=1832208&view=diff
==============================================================================
--- knox/site/project-info.html (original)
+++ knox/site/project-info.html Thu May 24 21:45:27 2018
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-05-18
+ | Generated by Apache Maven Doxia at 2018-05-24
  | Rendered using Apache Maven Fluido Skin 1.3.0
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180518" />
+    <meta name="Date-Revision-yyyymmdd" content="20180524" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Knox Gateway &#x2013; Project Information</title>
     <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
               
                 
                     
-                  <li id="publishDate" class="pull-right">Last Published: 2018-05-18</li>

+                  <li id="publishDate" class="pull-right">Last Published: 2018-05-24</li>

             
                             </ul>
       </div>

Modified: knox/site/team-list.html
URL: http://svn.apache.org/viewvc/knox/site/team-list.html?rev=1832208&r1=1832207&r2=1832208&view=diff
==============================================================================
--- knox/site/team-list.html (original)
+++ knox/site/team-list.html Thu May 24 21:45:27 2018
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2018-05-18
+ | Generated by Apache Maven Doxia at 2018-05-24
  | Rendered using Apache Maven Fluido Skin 1.3.0
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20180518" />
+    <meta name="Date-Revision-yyyymmdd" content="20180524" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Knox Gateway &#x2013; Team list</title>
     <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
               
                 
                     
-                  <li id="publishDate" class="pull-right">Last Published: 2018-05-18</li>

+                  <li id="publishDate" class="pull-right">Last Published: 2018-05-24</li>

             
                             </ul>
       </div>

Modified: knox/trunk/books/1.1.0/book.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/1.1.0/book.md?rev=1832208&r1=1832207&r2=1832208&view=diff
==============================================================================
--- knox/trunk/books/1.1.0/book.md (original)
+++ knox/trunk/books/1.1.0/book.md Thu May 24 21:45:27 2018
@@ -100,6 +100,7 @@
     * #[SOLR]
     * #[Avatica]
     * #[Livy Server]
+    * #[ElasticSearch]
     * #[Common Service Config]
     * #[Default Service HA support]
 * #[UI Service Details]

Modified: knox/trunk/books/1.1.0/book_service-details.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/1.1.0/book_service-details.md?rev=1832208&r1=1832207&r2=1832208&view=diff
==============================================================================
--- knox/trunk/books/1.1.0/book_service-details.md (original)
+++ knox/trunk/books/1.1.0/book_service-details.md Thu May 24 21:45:27 2018
@@ -40,6 +40,7 @@ These are the current Hadoop services wi
 * #[SOLR]
 * #[Avatica]
 * #[Livy Server]
+* #[ElasticSearch]
 
 ### Assumptions
 
@@ -92,6 +93,7 @@ Therefore each request via cURL will res
 <<service_default_ha.md>>
 <<service_avatica.md>>
 <<service_livy.md>>
+<<service_elasticsearch.md>>
 
 ### Service Test API
 

Added: knox/trunk/books/1.1.0/service_elasticsearch.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/1.1.0/service_elasticsearch.md?rev=1832208&view=auto
==============================================================================
--- knox/trunk/books/1.1.0/service_elasticsearch.md (added)
+++ knox/trunk/books/1.1.0/service_elasticsearch.md Thu May 24 21:45:27 2018
@@ -0,0 +1,159 @@
+<!---
+   Licensed to the Apache Software Foundation (ASF) under one or more
+   contributor license agreements.  See the NOTICE file distributed with
+   this work for additional information regarding copyright ownership.
+   The ASF licenses this file to You under the Apache License, Version 2.0
+   (the "License"); you may not use this file except in compliance with
+   the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+--->
+
+### ElasticSearch ###
+
+ElasticSearch provides a REST API for communicating with ElasticSearch via JSON over HTTP.
ElasticSearch uses X-Pack to do its own security (authentication and authorization). Therefore,
the Knox Gateway is to forward the user credentials to ElasticSearch, and treats the ElasticSearch-authenticated
user as "anonymous" to the backend service via a doas query param while Knox will authenticate
to backend services as itself.
+
+#### Gateway configuration ####
+
+The Gateway can be configured for ElasticSearch by modifying the topology XML file and providing
a new service XML file.
+
+In the topology XML file, add the following new service named "ELASTICSEARCH" with the correct
elasticsearch-rest-server hostname and port number (e.g., 9200):
+
+     <service>
+       <role>ELASTICSEARCH</role>
+       <url>http://<elasticsearch-rest-server>:9200/</url>
+       <name>elasticsearch</name>
+     </service>
+
+#### ElasticSearch via Knox Gateway ####
+
+After adding the above to a topology, you can make a cURL request similar to the following
structures:
+
+##### 1.  ElasticSearch Node Root Query #####
+
+    curl -i -k -u username:password -H "Accept: application/json"  -X GET  "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch"
+
+    or
+
+    curl -i -k -u username:password -H "Accept: application/json"  -X GET  "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/"
+
+The quotation marks around the URL, can be single quotes or double quotes on both sides,
and can also be omitted (Note: This is true for all other ElasticSearch queries via Knox).
Below is an example response:
+
+     HTTP/1.1 200 OK
+     Date: Wed, 23 May 2018 16:36:34 GMT
+     Content-Type: application/json; charset=UTF-8
+     Content-Length: 356
+     Server: Jetty(9.2.15.v20160210)
+     
+     {"name":"w0A80p0","cluster_name":"elasticsearch","cluster_uuid":"poU7j48pSpu5qQONr64HLQ","version":{"number":"6.2.4","build_hash":"ccec39f","build_date":"2018-04-12T20:37:28.497551Z","build_snapshot":false,"lucene_version":"7.2.1","minimum_wire_compatibility_version":"5.6.0","minimum_index_compatibility_version":"5.0.0"},"tagline":"You
Know, for Search"}
+    
+##### 2.  ElasticSearch Index - Creation, Deletion, Refreshing and Data Operations - Writing,
Updating and Retrieval #####
+
+###### (1) Index Creation ######
+
+    curl -i -k -u username:password -H "Content-Type: application/json"  -X PUT  "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}"
 -d '{
+    "settings" : {
+        "index" : {
+            "number_of_shards" : {index-shards-number},
+            "number_of_replicas" : {index-replicas-number}
+        }
+      }
+    }'
+
+Below is an example response:
+
+     HTTP/1.1 200 OK
+     Date: Wed, 23 May 2018 16:51:31 GMT
+     Content-Type: application/json; charset=UTF-8
+     Content-Length: 65
+     Server: Jetty(9.2.15.v20160210)
+     
+     {"acknowledged":true,"shards_acknowledged":true,"index":"estest"}
+
+###### (2) Index Data Writing ######
+
+For adding a "Hello Joe Smith" document:
+
+    curl -i -k -u username:password -H "Content-Type: application/json"  -X PUT  "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}/{document-type-name}/{document-id}"
 -d '{
+        "title":"Hello Joe Smith" 
+    }'
+
+Below is an example response:
+
+     HTTP/1.1 201 Created
+     Date: Wed, 23 May 2018 17:00:17 GMT
+     Location: /estest/greeting/1
+     Content-Type: application/json; charset=UTF-8
+     Content-Length: 158
+     Server: Jetty(9.2.15.v20160210)
+     
+     {"_index":"estest","_type":"greeting","_id":"1","_version":1,"result":"created","_shards":{"total":1,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
+
+###### (3) Index Refreshing ######
+
+    curl -i -k -u username:password  -X POST  "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}/_refresh"

+
+Below is an example response:
+
+     HTTP/1.1 200 OK
+     Date: Wed, 23 May 2018 17:02:32 GMT
+     Content-Type: application/json; charset=UTF-8
+     Content-Length: 49
+     Server: Jetty(9.2.15.v20160210)
+     
+     {"_shards":{"total":1,"successful":1,"failed":0}}
+
+###### (4) Index Data Upgrading ######
+
+For changing the Person Joe Smith to Tom Smith:
+
+    curl -i -k -u username:password -H "Content-Type: application/json"  -X PUT  "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}/{document-type-name}/{document-id}"
 -d '{ 
+    "title":"Hello Tom Smith" 
+    }'
+
+Below is an example response:
+
+     HTTP/1.1 200 OK
+     Date: Wed, 23 May 2018 17:09:59 GMT
+     Content-Type: application/json; charset=UTF-8
+     Content-Length: 158
+     Server: Jetty(9.2.15.v20160210)
+     
+     {"_index":"estest","_type":"greeting","_id":"1","_version":2,"result":"updated","_shards":{"total":1,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1}
+
+###### (5) Index Data Retrieval or Search ######
+
+For finding documents with "title":"Hello" in a specified document-type:
+
+    curl -i -k -u username:password -H "Accept: application/json" -X GET  "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}/{document-type-name}/
_search?pretty=true;q=title:Hello"
+
+Below is an example response:
+
+     HTTP/1.1 200 OK
+     Date: Wed, 23 May 2018 17:13:08 GMT
+     Content-Type: application/json; charset=UTF-8
+     Content-Length: 244
+     Server: Jetty(9.2.15.v20160210)
+     
+     {"took":0,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":1,"max_score":0.2876821,"hits":[{"_index":"estest","_type":"greeting","_id":"1","_score":0.2876821,"_source":{"title":"Hello
Tom Smith"}}]}}
+
+###### (6) Index Deleting ######
+
+    curl -i -k -u username:password  -X DELETE  "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}"
+
+Below is an example response:
+
+     HTTP/1.1 200 OK
+     Date: Wed, 23 May 2018 17:20:19 GMT
+     Content-Type: application/json; charset=UTF-8
+     Content-Length: 21
+     Server: Jetty(9.2.15.v20160210)
+     
+     {"acknowledged":true}
+



Mime
View raw message