Author: more Date: Fri Nov 10 16:08:01 2017 New Revision: 1814879 URL: http://svn.apache.org/viewvc?rev=1814879&view=rev Log: KNOX-1083 - Document HttpClient default timeout for hive HA Modified: knox/site/books/knox-0-14-0/user-guide.html knox/site/index.html knox/site/issue-tracking.html knox/site/license.html knox/site/mail-lists.html knox/site/project-info.html knox/site/team-list.html knox/trunk/books/0.14.0/config.md knox/trunk/books/0.14.0/service_hive.md Modified: knox/site/books/knox-0-14-0/user-guide.html URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-14-0/user-guide.html?rev=1814879&r1=1814878&r2=1814879&view=diff ============================================================================== --- knox/site/books/knox-0-14-0/user-guide.html (original) +++ knox/site/books/knox-0-14-0/user-guide.html Fri Nov 10 16:08:01 2017 @@ -472,7 +472,23 @@ https://{gateway-host}:{gateway-port}/{g <value>false</value> <description>Enable/Disable port mapping feature.</description> </property> -

If a topology mapped port is in use by another topology or process then an ERROR message is logged and gateway startup continues as normal.

Configuration

Configuration for Apache Knox includes:

+

If a topology mapped port is in use by another topology or process then an ERROR message is logged and gateway startup continues as normal.

+

Configuration

Configuration for Apache Knox includes:

  1. Related Cluster Configuration that must be done within the Hadoop cluster to allow Knox to communicate with various services
  2. Gateway Server Configuration - which is the configurable elements of the server itself which applies to behavior that spans all topologies or managed Hadoop clusters
    3. @@ -615,12 +631,12 @@ https://{gateway-host}:{gateway-port}/{g gateway.httpclient.connectionTimeout The amount of time to wait when attempting a connection. The natural unit is milliseconds but a ‘s’ or ‘m’ suffix may be used for seconds or minutes respectively. The default timeout is system dependent. - System Dependent + System Dependent (-1) gateway.httpclient.socketTimeout The amount of time to wait for data on a socket before aborting the connection. The natural unit is milliseconds but a ‘s’ or ‘m’ suffix may be used for seconds or minutes respectively. The default timeout is system dependent but is likely to be indefinite. - System Dependent + System Dependent (-1) gateway.httpserver.requestBuffer @@ -5687,7 +5703,18 @@ connection.close(); 
    <service>
     <role>HIVE</role>
 </service>
-

    Please note that there is no <url> tag specified here as the URLs for the Hive servers are obtained from Zookeeper.

    Yarn

    Knox provides gateway functionality for the REST APIs of the ResourceManager. The ResourceManager REST APIs allow the user to get information about the cluster - status on the cluster, metrics on the cluster, scheduler information, information about nodes in the cluster, and information about applications on the cluster. Also as of Hadoop version 2.5.0, the user can submit a new application as well as kill it (or get state) using the ‘Writable’ APIs.

    The docs for this can be found here

    http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html

    To enable this functionality, a topology file needs to have the following configuration:

    +

    Please note that there is no <url> tag specified here as the URLs for the Hive servers are obtained from Zookeeper.

    Also make sure the properties gateway.httpclient.connectionTimeout and gateway.httpclient.socketTimeout are set to acceptable defaults in gateway-site.xml config file e.g.

    +
     <property>
+     <name>gateway.httpclient.connectionTimeout</name>
+     <value>3000</value>
+     <description>Connection timeout after 30 sec</description>
+ </property>
+ <property>
+     <name>gateway.httpclient.socketTimeout</name>
+     <value>3000</value>
+     <description>Socket timeout after 30 sec</description>
+ </property>
+

    Yarn

    Knox provides gateway functionality for the REST APIs of the ResourceManager. The ResourceManager REST APIs allow the user to get information about the cluster - status on the cluster, metrics on the cluster, scheduler information, information about nodes in the cluster, and information about applications on the cluster. Also as of Hadoop version 2.5.0, the user can submit a new application as well as kill it (or get state) using the ‘Writable’ APIs.

    The docs for this can be found here

    http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html

    To enable this functionality, a topology file needs to have the following configuration:

    <service>
     <role>RESOURCEMANAGER</role>
     <url>http://<hostname>:<port>/ws</url>

Modified: knox/site/index.html
URL: http://svn.apache.org/viewvc/knox/site/index.html?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/site/index.html (original)
+++ knox/site/index.html Fri Nov 10 16:08:01 2017
@@ -1,13 +1,13 @@
 
 
 
   
     
     
-    
+    
     
     Knox Gateway – REST API and Application Gateway for the Apache Hadoop Ecosystem
     
@@ -58,7 +58,7 @@
               
                 
                     
-                  
  3. Last Published: 2017-11-06
    4.  
+                  
  4. Last Published: 2017-11-10
    5.  
             
                             
       

Modified: knox/site/issue-tracking.html
URL: http://svn.apache.org/viewvc/knox/site/issue-tracking.html?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/site/issue-tracking.html (original)
+++ knox/site/issue-tracking.html Fri Nov 10 16:08:01 2017
@@ -1,13 +1,13 @@
 
 
 
   
     
     
-    
+    
     
     Knox Gateway – Issue Tracking
     
@@ -58,7 +58,7 @@
               
                 
                     
-                  
  5. Last Published: 2017-11-06
    6.  
+                  
  6. Last Published: 2017-11-10
    7.  
             
                             
       

Modified: knox/site/license.html
URL: http://svn.apache.org/viewvc/knox/site/license.html?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/site/license.html (original)
+++ knox/site/license.html Fri Nov 10 16:08:01 2017
@@ -1,13 +1,13 @@
 
 
 
   
     
     
-    
+    
     
     Knox Gateway – Project License
     
@@ -58,7 +58,7 @@
               
                 
                     
-                  
  7. Last Published: 2017-11-06
    8.  
+                  
  8. Last Published: 2017-11-10
    9.  
             
                             
       

Modified: knox/site/mail-lists.html
URL: http://svn.apache.org/viewvc/knox/site/mail-lists.html?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/site/mail-lists.html (original)
+++ knox/site/mail-lists.html Fri Nov 10 16:08:01 2017
@@ -1,13 +1,13 @@
 
 
 
   
     
     
-    
+    
     
     Knox Gateway – Project Mailing Lists
     
@@ -58,7 +58,7 @@
               
                 
                     
-                  
  9. Last Published: 2017-11-06
    10.  
+                  
  10. Last Published: 2017-11-10
    11.  
             
                             
       

Modified: knox/site/project-info.html
URL: http://svn.apache.org/viewvc/knox/site/project-info.html?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/site/project-info.html (original)
+++ knox/site/project-info.html Fri Nov 10 16:08:01 2017
@@ -1,13 +1,13 @@
 
 
 
   
     
     
-    
+    
     
     Knox Gateway – Project Information
     
@@ -58,7 +58,7 @@
               
                 
                     
-                  
  11. Last Published: 2017-11-06
    12.  
+                  
  12. Last Published: 2017-11-10
    13.  
             
                             
       

Modified: knox/site/team-list.html
URL: http://svn.apache.org/viewvc/knox/site/team-list.html?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/site/team-list.html (original)
+++ knox/site/team-list.html Fri Nov 10 16:08:01 2017
@@ -1,13 +1,13 @@
 
 
 
   
     
     
-    
+    
     
     Knox Gateway – Team list
     
@@ -58,7 +58,7 @@
               
                 
                     
-                  
  13. Last Published: 2017-11-06
    14.  
+                  
  14. Last Published: 2017-11-10
    15.  
             
                             
       

Modified: knox/trunk/books/0.14.0/config.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.14.0/config.md?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/trunk/books/0.14.0/config.md (original)
+++ knox/trunk/books/0.14.0/config.md Fri Nov 10 16:08:01 2017
@@ -1,4 +1,4 @@
-
+-->
 
 ### Configuration ###
 
@@ -128,8 +128,8 @@ gateway.keystore.type|Indicates the type
 gateway.jdk.tls.ephemeralDHKeySize|jdk.tls.ephemeralDHKeySize, is defined to customize the ephemeral DH key sizes. The minimum acceptable DH key size is 1024 bits, except for exportable cipher suites or legacy mode (jdk.tls.ephemeralDHKeySize=legacy)|2048
 gateway.threadpool.max|The maximum concurrent requests the server will process.  The default is 254.  Connections beyond this will be queued.|254
 gateway.httpclient.maxConnections|The maximum number of connections that a single httpclient will maintain to a single host:port.  The default is 32.|32
-gateway.httpclient.connectionTimeout|The amount of time to wait when attempting a connection. The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds or minutes respectively. The default timeout is system dependent. | System Dependent
-gateway.httpclient.socketTimeout|The amount of time to wait for data on a socket before aborting the connection. The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds or minutes respectively. The default timeout is system dependent but is likely to be indefinite. | System Dependent
+gateway.httpclient.connectionTimeout|The amount of time to wait when attempting a connection. The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds or minutes respectively. The default timeout is system dependent. | System Dependent (-1)
+gateway.httpclient.socketTimeout|The amount of time to wait for data on a socket before aborting the connection. The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds or minutes respectively. The default timeout is system dependent but is likely to be indefinite. | System Dependent (-1)
 gateway.httpserver.requestBuffer|The size of the HTTP server request buffer.  The default is 16K.|16384
 gateway.httpserver.requestHeaderBuffer|The size of the HTTP server request header buffer.  The default is 8K.|8192
 gateway.httpserver.responseBuffer|The size of the HTTP server response buffer.  The default is 32K.|32768

Modified: knox/trunk/books/0.14.0/service_hive.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.14.0/service_hive.md?rev=1814879&r1=1814878&r2=1814879&view=diff
==============================================================================
--- knox/trunk/books/0.14.0/service_hive.md (original)
+++ knox/trunk/books/0.14.0/service_hive.md Fri Nov 10 16:08:01 2017
@@ -322,3 +322,17 @@ And for the service configuration itself
 
 Please note that there is no `` tag specified here as the URLs for the Hive servers are obtained from Zookeeper.
 
+Also make sure the properties `gateway.httpclient.connectionTimeout` and `gateway.httpclient.socketTimeout` are set to acceptable defaults in `gateway-site.xml` config file e.g.
+  
+     
+         gateway.httpclient.connectionTimeout
+         3000
+         Connection timeout after 30 sec
+     
+     
+         gateway.httpclient.socketTimeout
+         3000
+         Socket timeout after 30 sec
+     
+
+