knox-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From su...@apache.org
Subject svn commit: r1704665 - in /knox: site/books/knox-0-7-0/ trunk/books/0.7.0/
Date Tue, 22 Sep 2015 16:22:26 GMT
Author: sumit
Date: Tue Sep 22 16:22:23 2015
New Revision: 1704665

URL: http://svn.apache.org/viewvc?rev=1704665&view=rev
Log:
KNOX-580 docs for default HA support

Added:
    knox/trunk/books/0.7.0/service_default_ha.md
Modified:
    knox/site/books/knox-0-7-0/user-guide.html
    knox/trunk/books/0.7.0/book_service-details.md
    knox/trunk/books/0.7.0/service_hbase.md
    knox/trunk/books/0.7.0/service_hive.md
    knox/trunk/books/0.7.0/service_oozie.md
    knox/trunk/books/0.7.0/service_webhcat.md

Modified: knox/site/books/knox-0-7-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/user-guide.html?rev=1704665&r1=1704664&r2=1704665&view=diff
==============================================================================
--- knox/site/books/knox-0-7-0/user-guide.html (original)
+++ knox/site/books/knox-0-7-0/user-guide.html Tue Sep 22 16:22:23 2015
@@ -2478,7 +2478,7 @@ session.shutdown()
   <ul>
     <li><code>Job.queryStatus(session).jobId(jobId).now().string</code></li>
   </ul></li>
-</ul><h3><a id="Oozie"></a>Oozie</h3><p>Oozie is a Hadoop
component provides complex job workflows to be submitted and managed. Please refer to the
latest <a href="http://oozie.apache.org/docs/4.0.0/">Oozie documentation</a> for
details.</p><p>In order to make Oozie accessible via the gateway there are several
important Hadoop configuration settings. These all relate to the network endpoint exposed
by various Hadoop services.</p><p>The HTTP endpoint at which Oozie is running
can be found via the oozie.base.url property in the oozie-site.xml file. In a Sandbox installation
this can typically be found in /etc/oozie/conf/oozie-site.xml.</p>
+</ul><h3><a id="WebHCat+HA"></a>WebHCat HA</h3><p>Please
look at <a href="#Default+Service+HA+support">Default Service HA support</a></p><h3><a
id="Oozie"></a>Oozie</h3><p>Oozie is a Hadoop component provides complex
job workflows to be submitted and managed. Please refer to the latest <a href="http://oozie.apache.org/docs/4.0.0/">Oozie
documentation</a> for details.</p><p>In order to make Oozie accessible via
the gateway there are several important Hadoop configuration settings. These all relate to
the network endpoint exposed by various Hadoop services.</p><p>The HTTP endpoint
at which Oozie is running can be found via the oozie.base.url property in the oozie-site.xml
file. In a Sandbox installation this can typically be found in /etc/oozie/conf/oozie-site.xml.</p>
 <pre><code>&lt;property&gt;
     &lt;name&gt;oozie.base.url&lt;/name&gt;
     &lt;value&gt;http://sandbox.hortonworks.com:11000/oozie&lt;/value&gt;
@@ -2603,7 +2603,7 @@ curl -i -k -u guest:guest-password -X DE
   <ul>
     <li><code>Workflow.status(session).jobId(jobId).now().string</code></li>
   </ul></li>
-</ul><h3><a id="HBase"></a>HBase</h3><p>The HBase REST
API is provided by the Stargate service for HBase. See the HBase Stargate Setup section below
for getting started with stargate and Knox with the Hortonworks Sandbox environment.</p><h4><a
id="HBase+URL+Mapping"></a>HBase URL Mapping</h4>
+</ul><h3><a id="Oozie+HA"></a>Oozie HA</h3><p>Please
look at <a href="#Default+Service+HA+support">Default Service HA support</a></p><h3><a
id="HBase"></a>HBase</h3><p>The HBase REST API is provided by the Stargate
service for HBase. See the HBase Stargate Setup section below for getting started with stargate
and Knox with the Hortonworks Sandbox environment.</p><h4><a id="HBase+URL+Mapping"></a>HBase
URL Mapping</h4>
 <table>
   <tbody>
     <tr>
@@ -3177,7 +3177,7 @@ session.shutdown(10, SECONDS)
 </code></pre><h4><a id="Delete+table"></a>Delete table</h4>
 <pre><code>curl -ik -u guest:guest-password\
  -X DELETE &#39;https://localhost:8443/gateway/sandbox/hbase/table1/schema&#39;
-</code></pre><h3><a id="Hive"></a>Hive</h3><p>The
<a href="https://cwiki.apache.org/confluence/display/Hive/Home">Hive wiki pages</a>
describe Hive installation and configuration processes. In sandbox configuration file for
Hive is located at /etc/hive/hive-site.xml. Hive Server has to be started in HTTP mode. Note
the properties shown below as they are related to configuration required by the gateway.</p>
+</code></pre><h3><a id="HBase+REST+HA"></a>HBase REST HA</h3><p>Please
look at <a href="#Default+Service+HA+support">Default Service HA support</a></p><h3><a
id="Hive"></a>Hive</h3><p>The <a href="https://cwiki.apache.org/confluence/display/Hive/Home">Hive
wiki pages</a> describe Hive installation and configuration processes. In sandbox configuration
file for Hive is located at /etc/hive/hive-site.xml. Hive Server has to be started in HTTP
mode. Note the properties shown below as they are related to configuration required by the
gateway.</p>
 <pre><code>&lt;property&gt;
     &lt;name&gt;hive.server2.thrift.http.port&lt;/name&gt;
     &lt;value&gt;10001&lt;/value&gt;
@@ -3410,7 +3410,7 @@ connection.close();
   <li><p>enabled - Flag to turn the particular service on or off for HA.</p></li>
   <li><p>zookeeperEnsemble - A comma separated list of host names (or IP addresses)
of the zookeeper hosts that consist of the ensemble that the Hive servers register their information
with. This value can be obtained from Hive&rsquo;s config file hive-site.xml as the value
for the parameter &lsquo;hive.zookeeper.quorum&rsquo;.</p></li>
   <li><p>zookeeperNamespace - This is the namespace under which HiveServer2 information
is registered in the Zookeeper ensemble. This value can be obtained from Hive&rsquo;s
config file hive-site.xml as the value for the parameter &lsquo;hive.server2.zookeeper.namespace&rsquo;.</p></li>
-</ul><p>And for the service configuration itself the additional URLs that standby
nodes should be added to the list. The active URL (at the time of configuration) should ideally
be added to the top of the list.</p>
+</ul><p>And for the service configuration itself the URLs need not be added to
the list. For example.</p>
 <pre><code>&lt;service&gt;
     &lt;role&gt;HIVE&lt;/role&gt;
 &lt;/service&gt;
@@ -3582,6 +3582,56 @@ curl -ik -b ~/cookiejar.txt -c ~/cookiej
 
 curl -ik -b ~/cookiejar.txt -c ~/cookiejar.txt -u guest:guest-password -H &#39;x-csrf-token:{token-value}&#39;
-X POST \
  http://localhost:8744/api/v1/topology/{id}/kill/0
+</code></pre><h3><a id="Default+Service+HA+support"></a>Default
Service HA support</h3><p>Knox provides connectivity based failover functionality
for service calls that can be made to more than one server instance in a cluster. To enable
this functionality HaProvider configuration needs to be enabled for the service and the service
itself needs to be configured with more than one URL in the topology file.</p><p>The
default HA functionality works on a simple round robin algorithm, where the top of the list
of URLs is always used to route all of a service&rsquo;s REST calls until a connection
error occurs. The top URL is then put at the bottom of the list and the next URL is attempted.
This goes on until the setting of &lsquo;maxFailoverAttempts&rsquo; is reached.</p><p>At
present the following services can avail this default High Availability functionality and
have been tested for the same:</p>
+<ul>
+  <li>WEBHCAT</li>
+  <li>HBASE</li>
+  <li>OOZIE</li>
+</ul><p>To enable HA functionality for a service in Knox the following configuration
has to be added to the topology file.</p>
+<pre><code>&lt;provider&gt;
+   &lt;role&gt;ha&lt;/role&gt;
+   &lt;name&gt;HaProvider&lt;/name&gt;
+   &lt;enabled&gt;true&lt;/enabled&gt;
+   &lt;param&gt;
+       &lt;name&gt;{SERVICE}&lt;/name&gt;
+       &lt;value&gt;maxFailoverAttempts=3;failoverSleep=1000;enabled=true&lt;/value&gt;
+   &lt;/param&gt;
+&lt;/provider&gt;
+</code></pre><p>The role and name of the provider above must be as shown.
The name in the &lsquo;param&rsquo; section i.e. {SERVICE} must match that of the
service role name that is being configured for HA and the value in the &lsquo;param&rsquo;
section is the configuration for that particular service in HA mode. For example, the value
of {SERVICE} can be &lsquo;WEBHCAT&rsquo;, &lsquo;HBASE&rsquo; or &lsquo;OOZIE&rsquo;.</p><p>To
configure multiple services in HA mode, additional &lsquo;param&rsquo; sections can
be added.</p><p>For example,</p>
+<pre><code>&lt;provider&gt;
+   &lt;role&gt;ha&lt;/role&gt;
+   &lt;name&gt;HaProvider&lt;/name&gt;
+   &lt;enabled&gt;true&lt;/enabled&gt;
+   &lt;param&gt;
+       &lt;name&gt;OOZIE&lt;/name&gt;
+       &lt;value&gt;maxFailoverAttempts=3;failoverSleep=1000;enabled=true&lt;/value&gt;
+   &lt;/param&gt;
+   &lt;param&gt;
+       &lt;name&gt;HBASE&lt;/name&gt;
+       &lt;value&gt;maxFailoverAttempts=3;failoverSleep=1000;enabled=true&lt;/value&gt;
+   &lt;/param&gt;
+   &lt;param&gt;
+       &lt;name&gt;WEBHCAT&lt;/name&gt;
+       &lt;value&gt;maxFailoverAttempts=3;failoverSleep=1000;enabled=true&lt;/value&gt;
+   &lt;/param&gt;
+&lt;/provider&gt;
+</code></pre><p>The various configuration parameters are described below:</p>
+<ul>
+  <li><p>maxFailoverAttempts - This is the maximum number of times a failover
will be attempted. The failover strategy at this time is very simplistic in that the next
URL in the list of URLs provided for the service is used and the one that failed is put at
the bottom of the list. If the list is exhausted and the maximum number of attempts is not
reached then the first URL will be tried again.</p></li>
+  <li><p>failoverSleep - The amount of time in millis that the process will wait
or sleep before attempting to failover.</p></li>
+  <li><p>enabled - Flag to turn the particular service on or off for HA.</p></li>
+</ul><p>And for the service configuration itself the additional URLs should be
added to the list.</p>
+<pre><code>&lt;service&gt;
+    &lt;role&gt;{SERVICE}&lt;/role&gt;
+    &lt;url&gt;http://host1:port1&lt;/url&gt;
+    &lt;url&gt;http://host2:port2&lt;/url&gt;
+&lt;/service&gt;
+</code></pre><p>For example,</p>
+<pre><code>&lt;service&gt;
+    &lt;role&gt;OOZIE&lt;/role&gt;
+    &lt;url&gt;http://sandbox1:11000/oozie&lt;/url&gt;
+    &lt;url&gt;http://sandbox2:11000/oozie&lt;/url&gt;
+&lt;/service&gt;
 </code></pre><h3><a id="Service+Test+API"></a>Service Test
API</h3><p>The gateway supports a Service Test API that can be used to test Knox&rsquo;s
ability to connect to each of the different Hadoop services via a simeple HTTP GET request.
To be able to access this API one must add the following line into the topology for which
you wish to run the service test.</p>
 <pre><code>&lt;service&gt;
     &lt;role&gt;SERVICE-TEST&lt;/role&gt;

Modified: knox/trunk/books/0.7.0/book_service-details.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/book_service-details.md?rev=1704665&r1=1704664&r2=1704665&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/book_service-details.md (original)
+++ knox/trunk/books/0.7.0/book_service-details.md Tue Sep 22 16:22:23 2015
@@ -82,6 +82,7 @@ Therefore each request via cURL will res
 <<service_hive.md>>
 <<service_yarn.md>>
 <<service_storm.md>>
+<<service_default_ha.md>>
 
 
 ### Service Test API

Added: knox/trunk/books/0.7.0/service_default_ha.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/service_default_ha.md?rev=1704665&view=auto
==============================================================================
--- knox/trunk/books/0.7.0/service_default_ha.md (added)
+++ knox/trunk/books/0.7.0/service_default_ha.md Tue Sep 22 16:22:23 2015
@@ -0,0 +1,102 @@
+<!---
+   Licensed to the Apache Software Foundation (ASF) under one or more
+   contributor license agreements.  See the NOTICE file distributed with
+   this work for additional information regarding copyright ownership.
+   The ASF licenses this file to You under the Apache License, Version 2.0
+   (the "License"); you may not use this file except in compliance with
+   the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+--->
+
+### Default Service HA support ###
+
+Knox provides connectivity based failover functionality for service calls that can be made
to more than one server
+instance in a cluster. To enable this functionality HaProvider configuration needs to be
enabled for the service and
+the service itself needs to be configured with more than one URL in the topology file.
+
+The default HA functionality works on a simple round robin algorithm, where the top of the
list of URLs is always used
+to route all of a service's REST calls until a connection error occurs. The top URL is then
put at the bottom of the
+list and the next URL is attempted. This goes on until the setting of 'maxFailoverAttempts'
is reached.
+
+At present the following services can avail this default High Availability functionality
and have been tested for the
+same:
+
+* WEBHCAT
+* HBASE
+* OOZIE
+
+To enable HA functionality for a service in Knox the following configuration has to be added
to the topology file.
+
+    <provider>
+       <role>ha</role>
+       <name>HaProvider</name>
+       <enabled>true</enabled>
+       <param>
+           <name>{SERVICE}</name>
+           <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+       </param>
+    </provider>
+
+The role and name of the provider above must be as shown. The name in the 'param' section
i.e. {SERVICE} must match
+that of the service role name that is being configured for HA and the value in the 'param'
section is the configuration
+for that particular service in HA mode. For example, the value of {SERVICE} can be 'WEBHCAT',
'HBASE' or 'OOZIE'.
+
+To configure multiple services in HA mode, additional 'param' sections can be added.
+
+For example,
+
+
+    <provider>
+       <role>ha</role>
+       <name>HaProvider</name>
+       <enabled>true</enabled>
+       <param>
+           <name>OOZIE</name>
+           <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+       </param>
+       <param>
+           <name>HBASE</name>
+           <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+       </param>
+       <param>
+           <name>WEBHCAT</name>
+           <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+       </param>
+    </provider>
+
+The various configuration parameters are described below:
+
+* maxFailoverAttempts -
+This is the maximum number of times a failover will be attempted. The failover strategy at
this time is very simplistic
+in that the next URL in the list of URLs provided for the service is used and the one that
failed is put at the bottom
+of the list. If the list is exhausted and the maximum number of attempts is not reached then
the first URL will be tried
+again.
+
+* failoverSleep -
+The amount of time in millis that the process will wait or sleep before attempting to failover.
+
+* enabled -
+Flag to turn the particular service on or off for HA.
+
+And for the service configuration itself the additional URLs should be added to the list.
+
+    <service>
+        <role>{SERVICE}</role>
+        <url>http://host1:port1</url>
+        <url>http://host2:port2</url>
+    </service>
+
+For example,
+
+    <service>
+        <role>OOZIE</role>
+        <url>http://sandbox1:11000/oozie</url>
+        <url>http://sandbox2:11000/oozie</url>
+    </service>

Modified: knox/trunk/books/0.7.0/service_hbase.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/service_hbase.md?rev=1704665&r1=1704664&r2=1704665&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/service_hbase.md (original)
+++ knox/trunk/books/0.7.0/service_hbase.md Tue Sep 22 16:22:23 2015
@@ -649,3 +649,9 @@ Scanner URL will be in Location response
 
     curl -ik -u guest:guest-password\
      -X DELETE 'https://localhost:8443/gateway/sandbox/hbase/table1/schema'
+
+
+### HBase REST HA ###
+
+Please look at #[Default Service HA support]
+

Modified: knox/trunk/books/0.7.0/service_hive.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/service_hive.md?rev=1704665&r1=1704664&r2=1704665&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/service_hive.md (original)
+++ knox/trunk/books/0.7.0/service_hive.md Tue Sep 22 16:22:23 2015
@@ -315,9 +315,7 @@ This is the namespace under which HiveSe
 obtained from Hive's config file hive-site.xml as the value for the parameter 'hive.server2.zookeeper.namespace'.
 
 
-And for the service configuration itself the additional URLs that standby nodes should be
added to the list. The active
-URL (at the time of configuration) should ideally be added to the top of the list.
-
+And for the service configuration itself the URLs need not be added to the list. For example.
 
     <service>
         <role>HIVE</role>

Modified: knox/trunk/books/0.7.0/service_oozie.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/service_oozie.md?rev=1704665&r1=1704664&r2=1704665&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/service_oozie.md (original)
+++ knox/trunk/books/0.7.0/service_oozie.md Tue Sep 22 16:22:23 2015
@@ -190,4 +190,9 @@ These replacement values are identified
 * Example
     * `Workflow.status(session).jobId(jobId).now().string`
 
+### Oozie HA ###
+
+Please look at #[Default Service HA support]
+
+
 

Modified: knox/trunk/books/0.7.0/service_webhcat.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/service_webhcat.md?rev=1704665&r1=1704664&r2=1704665&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/service_webhcat.md (original)
+++ knox/trunk/books/0.7.0/service_webhcat.md Tue Sep 22 16:22:23 2015
@@ -150,3 +150,8 @@ Each line from the file `samples/Example
     * BasicResponse
 * Example
     * `Job.queryStatus(session).jobId(jobId).now().string`
+
+### WebHCat HA ###
+
+Please look at #[Default Service HA support]
+



Mime
View raw message