knox-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From su...@apache.org
Subject svn commit: r1703239 - in /knox: site/books/knox-0-4-0/ site/books/knox-0-5-0/ site/books/knox-0-6-0/ site/books/knox-0-7-0/ trunk/books/0.7.0/
Date Tue, 15 Sep 2015 15:58:36 GMT
Author: sumit
Date: Tue Sep 15 15:58:36 2015
New Revision: 1703239

URL: http://svn.apache.org/r1703239
Log:
KNOX-570 docs HS2 HA support

Modified:
    knox/site/books/knox-0-4-0/deployment-overview.png
    knox/site/books/knox-0-4-0/deployment-provider.png
    knox/site/books/knox-0-4-0/deployment-service.png
    knox/site/books/knox-0-4-0/runtime-overview.png
    knox/site/books/knox-0-4-0/runtime-request-processing.png
    knox/site/books/knox-0-5-0/deployment-overview.png
    knox/site/books/knox-0-5-0/deployment-provider.png
    knox/site/books/knox-0-5-0/deployment-service.png
    knox/site/books/knox-0-5-0/runtime-overview.png
    knox/site/books/knox-0-5-0/runtime-request-processing.png
    knox/site/books/knox-0-6-0/deployment-overview.png
    knox/site/books/knox-0-6-0/deployment-provider.png
    knox/site/books/knox-0-6-0/deployment-service.png
    knox/site/books/knox-0-6-0/runtime-overview.png
    knox/site/books/knox-0-6-0/runtime-request-processing.png
    knox/site/books/knox-0-7-0/deployment-overview.png
    knox/site/books/knox-0-7-0/deployment-provider.png
    knox/site/books/knox-0-7-0/deployment-service.png
    knox/site/books/knox-0-7-0/runtime-overview.png
    knox/site/books/knox-0-7-0/runtime-request-processing.png
    knox/site/books/knox-0-7-0/user-guide.html
    knox/trunk/books/0.7.0/service_hive.md

Modified: knox/site/books/knox-0-4-0/deployment-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/deployment-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-4-0/deployment-provider.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/deployment-provider.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-4-0/deployment-service.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/deployment-service.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-4-0/runtime-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/runtime-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-4-0/runtime-request-processing.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/runtime-request-processing.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-5-0/deployment-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/deployment-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-5-0/deployment-provider.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/deployment-provider.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-5-0/deployment-service.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/deployment-service.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-5-0/runtime-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/runtime-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-5-0/runtime-request-processing.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/runtime-request-processing.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-6-0/deployment-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/deployment-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-6-0/deployment-provider.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/deployment-provider.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-6-0/deployment-service.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/deployment-service.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-6-0/runtime-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/runtime-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-6-0/runtime-request-processing.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/runtime-request-processing.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-7-0/deployment-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/deployment-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-7-0/deployment-provider.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/deployment-provider.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-7-0/deployment-service.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/deployment-service.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-7-0/runtime-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/runtime-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-7-0/runtime-request-processing.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/runtime-request-processing.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-7-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/user-guide.html?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
--- knox/site/books/knox-0-7-0/user-guide.html (original)
+++ knox/site/books/knox-0-7-0/user-guide.html Tue Sep 15 15:58:36 2015
@@ -3392,7 +3392,29 @@ connection.close();
 2012-02-03 --- 18:35:34 --- SampleClass6 --- [TRACE]
 2012-02-03 --- 18:35:34 --- SampleClass2 --- [DEBUG]
 ...
-</code></pre><h3><a id="Yarn"></a>Yarn</h3><p>Knox
provides gateway functionality for the REST APIs of the ResourceManager. The ResourceManager
REST API&rsquo;s allow the user to get information about the cluster - status on the cluster,
metrics on the cluster, scheduler information, information about nodes in the cluster, and
information about applications on the cluster. Also as of hadoop version 2.5.0, the user can
submit a new application as well as kill it (or get state) using the &lsquo;Writable&rsquo;
APIs.</p><p>The docs for this can be found here</p><p><a href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html">http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html</a></p><p>To
enable this functionality, a topology file needs to have the following configuration:</p>
+</code></pre><h3><a id="HIVE+SERVER2+HA"></a>HIVE SERVER2 HA</h3><p>Knox
provides basic failover functionality for calls made to Hive Server when more than one HiveServer2
instance is installed in the cluster and registered with the same Zookeeper ensemble. The
HA functionality in this case fetches the HiveServer2 URL information from a Zookeeper ensemble,
so the user need only supply the necessary Zookeeper configuration and not the Hive connection
URLs.</p><p>To enable HA functionality for Hive in Knox the following configuration
has to be added to the topology file.</p>
+<pre><code>&lt;provider&gt;
+   &lt;role&gt;ha&lt;/role&gt;
+   &lt;name&gt;HaProvider&lt;/name&gt;
+   &lt;enabled&gt;true&lt;/enabled&gt;
+   &lt;param&gt;
+       &lt;name&gt;HIVE&lt;/name&gt;
+       &lt;value&gt;maxFailoverAttempts=3;failoverSleep=1000;enabled=true;zookeeperEnsemble=machine1:2181,machine2:2181,machine3:2181;
+       zookeeperNamespace=hiveserver2&lt;/value&gt;
+   &lt;/param&gt;
+&lt;/provider&gt;
+</code></pre><p>The role and name of the provider above must be as shown.
The name in the &lsquo;param&rsquo; section must match that of the service role name
that is being configured for HA and the value in the &lsquo;param&rsquo; section is
the configuration for that particular service in HA mode. In this case the name is &lsquo;HIVE&rsquo;.</p><p>The
various configuration parameters are described below:</p>
+<ul>
+  <li><p>maxFailoverAttempts - This is the maximum number of times a failover
will be attempted. The failover strategy at this time is very simplistic in that the next
URL in the list of URLs provided for the service is used and the one that failed is put at
the bottom of the list. If the list is exhausted and the maximum number of attempts is not
reached then the first URL will be tried again after the list is fetched again from Zookeeper
(a refresh of the list is done at this point)</p></li>
+  <li><p>failoverSleep - The amount of time in millis that the process will wait
or sleep before attempting to failover.</p></li>
+  <li><p>enabled - Flag to turn the particular service on or off for HA.</p></li>
+  <li><p>zookeeperEnsemble - A comma separated list of host names (or IP addresses)
of the zookeeper hosts that consist of the ensemble that the Hive servers register their information
with. This value can be obtained from Hive&rsquo;s config file hive-site.xml as the value
for the parameter &lsquo;hive.zookeeper.quorum&rsquo;.</p></li>
+  <li><p>zookeeperNamespace - This is the namespace under which HiveServer2 information
is registered in the Zookeeper ensemble. This value can be obtained from Hive&rsquo;s
config file hive-site.xml as the value for the parameter &lsquo;hive.server2.zookeeper.namespace&rsquo;.</p></li>
+</ul><p>And for the service configuration itself the additional URLs that standby
nodes should be added to the list. The active URL (at the time of configuration) should ideally
be added to the top of the list.</p>
+<pre><code>&lt;service&gt;
+    &lt;role&gt;HIVE&lt;/role&gt;
+&lt;/service&gt;
+</code></pre><p>Please note that there is no <url> tag specified
here as the URLs for the Hive servers are obtained from Zookeeper.</p><h3><a
id="Yarn"></a>Yarn</h3><p>Knox provides gateway functionality for the
REST APIs of the ResourceManager. The ResourceManager REST API&rsquo;s allow the user
to get information about the cluster - status on the cluster, metrics on the cluster, scheduler
information, information about nodes in the cluster, and information about applications on
the cluster. Also as of hadoop version 2.5.0, the user can submit a new application as well
as kill it (or get state) using the &lsquo;Writable&rsquo; APIs.</p><p>The
docs for this can be found here</p><p><a href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html">http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html</a></p><p>To
enable this functionality, a topology file needs to have the following configuration:</p>
 <pre><code>&lt;service&gt;
         &lt;role&gt;RESOURCEMANAGER&lt;/role&gt;
         &lt;url&gt;http://&lt;hostname&gt;:&lt;port&gt;/ws&lt;/url&gt;

Modified: knox/trunk/books/0.7.0/service_hive.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/service_hive.md?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/service_hive.md (original)
+++ knox/trunk/books/0.7.0/service_hive.md Tue Sep 15 15:58:36 2015
@@ -266,3 +266,62 @@ Expected output:
     2012-02-03 --- 18:35:34 --- SampleClass6 --- [TRACE]
     2012-02-03 --- 18:35:34 --- SampleClass2 --- [DEBUG]
     ...
+
+### HIVE SERVER2 HA ###
+
+Knox provides basic failover functionality for calls made to Hive Server when more than one
HiveServer2 instance is
+installed in the cluster and registered with the same Zookeeper ensemble. The HA functionality
in this case fetches the
+HiveServer2 URL information from a Zookeeper ensemble, so the user need only supply the necessary
Zookeeper
+configuration and not the Hive connection URLs.
+
+To enable HA functionality for Hive in Knox the following configuration has to be added to
the topology file.
+
+    <provider>
+       <role>ha</role>
+       <name>HaProvider</name>
+       <enabled>true</enabled>
+       <param>
+           <name>HIVE</name>
+           <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true;zookeeperEnsemble=machine1:2181,machine2:2181,machine3:2181;
+           zookeeperNamespace=hiveserver2</value>
+       </param>
+    </provider>
+
+The role and name of the provider above must be as shown. The name in the 'param' section
must match that of the service
+role name that is being configured for HA and the value in the 'param' section is the configuration
for that particular
+service in HA mode. In this case the name is 'HIVE'.
+
+The various configuration parameters are described below:
+
+* maxFailoverAttempts -
+This is the maximum number of times a failover will be attempted. The failover strategy at
this time is very simplistic
+in that the next URL in the list of URLs provided for the service is used and the one that
failed is put at the bottom
+of the list. If the list is exhausted and the maximum number of attempts is not reached then
the first URL will be tried
+again after the list is fetched again from Zookeeper (a refresh of the list is done at this
point)
+
+* failoverSleep -
+The amount of time in millis that the process will wait or sleep before attempting to failover.
+
+* enabled -
+Flag to turn the particular service on or off for HA.
+
+* zookeeperEnsemble -
+A comma separated list of host names (or IP addresses) of the zookeeper hosts that consist
of the ensemble that the Hive
+servers register their information with. This value can be obtained from Hive's config file
hive-site.xml as the value
+for the parameter 'hive.zookeeper.quorum'.
+
+* zookeeperNamespace -
+This is the namespace under which HiveServer2 information is registered in the Zookeeper
ensemble. This value can be
+obtained from Hive's config file hive-site.xml as the value for the parameter 'hive.server2.zookeeper.namespace'.
+
+
+And for the service configuration itself the additional URLs that standby nodes should be
added to the list. The active
+URL (at the time of configuration) should ideally be added to the top of the list.
+
+
+    <service>
+        <role>HIVE</role>
+    </service>
+
+Please note that there is no <url> tag specified here as the URLs for the Hive servers
are obtained from Zookeeper.
+



Mime
View raw message