knox-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kmin...@apache.org
Subject svn commit: r1735602 - in /knox: site/ site/books/knox-0-9-0/ trunk/books/0.9.0/
Date Fri, 18 Mar 2016 13:23:32 GMT
Author: kminder
Date: Fri Mar 18 13:23:31 2016
New Revision: 1735602

URL: http://svn.apache.org/viewvc?rev=1735602&view=rev
Log:
[KNOX-695] - Expose configuration of HttpClient's connection and socket timeout settings

Added:
    knox/trunk/books/0.9.0/service_config.md
      - copied, changed from r1735430, knox/trunk/books/0.9.0/service_default_ha.md
Modified:
    knox/site/books/knox-0-9-0/user-guide.html
    knox/site/index.html
    knox/site/issue-tracking.html
    knox/site/license.html
    knox/site/mail-lists.html
    knox/site/project-info.html
    knox/site/team-list.html
    knox/trunk/books/0.9.0/book.md
    knox/trunk/books/0.9.0/book_service-details.md
    knox/trunk/books/0.9.0/config.md

Modified: knox/site/books/knox-0-9-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-9-0/user-guide.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/books/knox-0-9-0/user-guide.html (original)
+++ knox/site/books/knox-0-9-0/user-guide.html Fri Mar 18 13:23:31 2016
@@ -55,6 +55,7 @@
     <li><a href="#Hive">Hive</a></li>
     <li><a href="#Yarn">Yarn</a></li>
     <li><a href="#Storm">Storm</a></li>
+    <li><a href="#Common+Service+Config">Common Service Config</a></li>
     <li><a href="#Default+Service+HA+support">Default Service HA support</a></li>
   </ul></li>
   <li><a href="#UI+Service+Details">UI Service Details</a></li>
@@ -510,6 +511,16 @@ https://{gateway-host}:{gateway-port}/{g
       <td>32</td>
     </tr>
     <tr>
+      <td>gateway.httpclient.connectionTimeout</td>
+      <td>The amount of time to wait when attempting a connection. The natural unit
is milliseconds but a &lsquo;s&rsquo; or &lsquo;m&rsquo; suffix may be used
for seconds or minutes respectively. The default timeout is system dependent. </td>
+      <td>System Dependent</td>
+    </tr>
+    <tr>
+      <td>gateway.httpclient.socketTimeout</td>
+      <td>The amount of time to wait for data on a socket before aborting the connection.
The natural unit is milliseconds but a &lsquo;s&rsquo; or &lsquo;m&rsquo;
suffix may be used for seconds or minutes respectively. The default timeout is system dependent
but is likely to be indefinite. </td>
+      <td>System Dependent</td>
+    </tr>
+    <tr>
       <td>gateway.httpserver.requestBuffer</td>
       <td>The size of the HTTP server request buffer. The default is 16K.</td>
       <td>16384</td>
@@ -4324,6 +4335,40 @@ curl -ik -b ~/cookiejar.txt -c ~/cookiej
 
 curl -ik -b ~/cookiejar.txt -c ~/cookiejar.txt -u guest:guest-password -H &#39;x-csrf-token:{token-value}&#39;
-X POST \
  http://localhost:8744/api/v1/topology/{id}/kill/0
+</code></pre><h3><a id="Common+Service+Config">Common Service Config</a>
<a href="#Common+Service+Config"><img src="markbook-section-link.png"/></a></h3><p>It
is possible to override a few of the global configuration settings provided in gateway-site.xml
at the service level. These overrides are specified as name/value pairs within the &lt;service&gt;
elements of a particular service. The overidden settings apply only to that service.</p><p>The
following table shows the common configuration settings available at the service level via
service level parameters. Individual services may support additional service level parameters.</p>
+<table>
+  <thead>
+    <tr>
+      <th>Property </th>
+      <th>Description </th>
+      <th>Default</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td>httpclient.maxConnections</td>
+      <td>The maximum number of connections that a single httpclient will maintain
to a single host:port. The default is 32.</td>
+      <td>32</td>
+    </tr>
+    <tr>
+      <td>httpclient.connectionTimeout</td>
+      <td>The amount of time to wait when attempting a connection. The natural unit
is milliseconds but a &lsquo;s&rsquo; or &lsquo;m&rsquo; suffix may be used
for seconds or minutes respectively. The default timeout is system dependent. </td>
+      <td>System Dependent</td>
+    </tr>
+    <tr>
+      <td>httpclient.socketTimeout</td>
+      <td>The amount of time to wait for data on a socket before aborting the connection.
The natural unit is milliseconds but a &lsquo;s&rsquo; or &lsquo;m&rsquo;
suffix may be used for seconds or minutes respectively. The default timeout is system dependent
but is likely to be indefinite. </td>
+      <td>System Dependent</td>
+    </tr>
+  </tbody>
+</table><p>The example below demonstrates how these service level parameters
are used.</p>
+<pre><code>&lt;service&gt;
+     &lt;role&gt;HIVE&lt;/role&gt;
+     &lt;param&gt;
+         &lt;name&gt;httpclient.socketTimeout&lt;/name&gt;
+         &lt;value&gt;180s&lt;/value&gt;
+     &lt;/param&gt;
+&lt;/service&gt;
 </code></pre><h3><a id="Default+Service+HA+support">Default Service
HA support</a> <a href="#Default+Service+HA+support"><img src="markbook-section-link.png"/></a></h3><p>Knox
provides connectivity based failover functionality for service calls that can be made to more
than one server instance in a cluster. To enable this functionality HaProvider configuration
needs to be enabled for the service and the service itself needs to be configured with more
than one URL in the topology file.</p><p>The default HA functionality works on
a simple round robin algorithm, where the top of the list of URLs is always used to route
all of a service&rsquo;s REST calls until a connection error occurs. The top URL is then
put at the bottom of the list and the next URL is attempted. This goes on until the setting
of &lsquo;maxFailoverAttempts&rsquo; is reached.</p><p>At present the
following services can use this default High Availability functionality and have been tested
for the same:</p>
 <ul>
   <li>WEBHCAT</li>

Modified: knox/site/index.html
URL: http://svn.apache.org/viewvc/knox/site/index.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/index.html (original)
+++ knox/site/index.html Fri Mar 18 13:23:31 2016
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2016-03-17
+ | Generated by Apache Maven Doxia at 2016-03-18
  | Rendered using Apache Maven Fluido Skin 1.3.0
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20160317" />
+    <meta name="Date-Revision-yyyymmdd" content="20160318" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Knox Gateway &#x2013; REST API Gateway for the Hadoop Ecosystem</title>
     <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
               
                 
                     
-                  <li id="publishDate" class="pull-right">Last Published: 2016-03-17</li>

+                  <li id="publishDate" class="pull-right">Last Published: 2016-03-18</li>

             
                             </ul>
       </div>

Modified: knox/site/issue-tracking.html
URL: http://svn.apache.org/viewvc/knox/site/issue-tracking.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/issue-tracking.html (original)
+++ knox/site/issue-tracking.html Fri Mar 18 13:23:31 2016
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2016-03-17
+ | Generated by Apache Maven Doxia at 2016-03-18
  | Rendered using Apache Maven Fluido Skin 1.3.0
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20160317" />
+    <meta name="Date-Revision-yyyymmdd" content="20160318" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Knox Gateway &#x2013; Issue Tracking</title>
     <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
               
                 
                     
-                  <li id="publishDate" class="pull-right">Last Published: 2016-03-17</li>

+                  <li id="publishDate" class="pull-right">Last Published: 2016-03-18</li>

             
                             </ul>
       </div>

Modified: knox/site/license.html
URL: http://svn.apache.org/viewvc/knox/site/license.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/license.html (original)
+++ knox/site/license.html Fri Mar 18 13:23:31 2016
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2016-03-17
+ | Generated by Apache Maven Doxia at 2016-03-18
  | Rendered using Apache Maven Fluido Skin 1.3.0
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20160317" />
+    <meta name="Date-Revision-yyyymmdd" content="20160318" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Knox Gateway &#x2013; Project License</title>
     <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
               
                 
                     
-                  <li id="publishDate" class="pull-right">Last Published: 2016-03-17</li>

+                  <li id="publishDate" class="pull-right">Last Published: 2016-03-18</li>

             
                             </ul>
       </div>

Modified: knox/site/mail-lists.html
URL: http://svn.apache.org/viewvc/knox/site/mail-lists.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/mail-lists.html (original)
+++ knox/site/mail-lists.html Fri Mar 18 13:23:31 2016
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2016-03-17
+ | Generated by Apache Maven Doxia at 2016-03-18
  | Rendered using Apache Maven Fluido Skin 1.3.0
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20160317" />
+    <meta name="Date-Revision-yyyymmdd" content="20160318" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Knox Gateway &#x2013; Project Mailing Lists</title>
     <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
               
                 
                     
-                  <li id="publishDate" class="pull-right">Last Published: 2016-03-17</li>

+                  <li id="publishDate" class="pull-right">Last Published: 2016-03-18</li>

             
                             </ul>
       </div>

Modified: knox/site/project-info.html
URL: http://svn.apache.org/viewvc/knox/site/project-info.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/project-info.html (original)
+++ knox/site/project-info.html Fri Mar 18 13:23:31 2016
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2016-03-17
+ | Generated by Apache Maven Doxia at 2016-03-18
  | Rendered using Apache Maven Fluido Skin 1.3.0
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20160317" />
+    <meta name="Date-Revision-yyyymmdd" content="20160318" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Knox Gateway &#x2013; Project Information</title>
     <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
               
                 
                     
-                  <li id="publishDate" class="pull-right">Last Published: 2016-03-17</li>

+                  <li id="publishDate" class="pull-right">Last Published: 2016-03-18</li>

             
                             </ul>
       </div>

Modified: knox/site/team-list.html
URL: http://svn.apache.org/viewvc/knox/site/team-list.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/team-list.html (original)
+++ knox/site/team-list.html Fri Mar 18 13:23:31 2016
@@ -1,13 +1,13 @@
 <!DOCTYPE html>
 <!--
- | Generated by Apache Maven Doxia at 2016-03-17
+ | Generated by Apache Maven Doxia at 2016-03-18
  | Rendered using Apache Maven Fluido Skin 1.3.0
 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <meta name="Date-Revision-yyyymmdd" content="20160317" />
+    <meta name="Date-Revision-yyyymmdd" content="20160318" />
     <meta http-equiv="Content-Language" content="en" />
     <title>Knox Gateway &#x2013; Team list</title>
     <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
               
                 
                     
-                  <li id="publishDate" class="pull-right">Last Published: 2016-03-17</li>

+                  <li id="publishDate" class="pull-right">Last Published: 2016-03-18</li>

             
                             </ul>
       </div>

Modified: knox/trunk/books/0.9.0/book.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.9.0/book.md?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/trunk/books/0.9.0/book.md (original)
+++ knox/trunk/books/0.9.0/book.md Fri Mar 18 13:23:31 2016
@@ -60,6 +60,7 @@
     * #[Hive]
     * #[Yarn]
     * #[Storm]
+    * #[Common Service Config]
     * #[Default Service HA support]
 * #[UI Service Details]
 * #[Limitations]

Modified: knox/trunk/books/0.9.0/book_service-details.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.9.0/book_service-details.md?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/trunk/books/0.9.0/book_service-details.md (original)
+++ knox/trunk/books/0.9.0/book_service-details.md Fri Mar 18 13:23:31 2016
@@ -82,6 +82,7 @@ Therefore each request via cURL will res
 <<service_hive.md>>
 <<service_yarn.md>>
 <<service_storm.md>>
+<<service_config.md>>
 <<service_default_ha.md>>
 
 

Modified: knox/trunk/books/0.9.0/config.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.9.0/config.md?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/trunk/books/0.9.0/config.md (original)
+++ knox/trunk/books/0.9.0/config.md Fri Mar 18 13:23:31 2016
@@ -128,6 +128,8 @@ gateway.keystore.type|Indicates the type
 gateway.jdk.tls.ephemeralDHKeySize|jdk.tls.ephemeralDHKeySize, is defined to customize the
ephemeral DH key sizes. The minimum acceptable DH key size is 1024 bits, except for exportable
cipher suites or legacy mode (jdk.tls.ephemeralDHKeySize=legacy)|2048
 gateway.threadpool.max|The maximum concurrent requests the server will process.  The default
is 254.  Connections beyond this will be queued.|254
 gateway.httpclient.maxConnections|The maximum number of connections that a single httpclient
will maintain to a single host:port.  The default is 32.|32
+gateway.httpclient.connectionTimeout|The amount of time to wait when attempting a connection.
The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds or minutes
respectively. The default timeout is system dependent. | System Dependent
+gateway.httpclient.socketTimeout|The amount of time to wait for data on a socket before aborting
the connection. The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds
or minutes respectively. The default timeout is system dependent but is likely to be indefinite.
| System Dependent
 gateway.httpserver.requestBuffer|The size of the HTTP server request buffer.  The default
is 16K.|16384
 gateway.httpserver.requestHeaderBuffer|The size of the HTTP server request header buffer.
 The default is 8K.|8192
 gateway.httpserver.responseBuffer|The size of the HTTP server response buffer.  The default
is 32K.|32768

Copied: knox/trunk/books/0.9.0/service_config.md (from r1735430, knox/trunk/books/0.9.0/service_default_ha.md)
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.9.0/service_config.md?p2=knox/trunk/books/0.9.0/service_config.md&p1=knox/trunk/books/0.9.0/service_default_ha.md&r1=1735430&r2=1735602&rev=1735602&view=diff
==============================================================================
--- knox/trunk/books/0.9.0/service_default_ha.md (original)
+++ knox/trunk/books/0.9.0/service_config.md Fri Mar 18 13:23:31 2016
@@ -15,87 +15,28 @@
    limitations under the License.
 --->
 
-### Default Service HA support ###
+### Common Service Config ###
 
-Knox provides connectivity based failover functionality for service calls that can be made
to more than one server
-instance in a cluster. To enable this functionality HaProvider configuration needs to be
enabled for the service and
-the service itself needs to be configured with more than one URL in the topology file.
+It is possible to override a few of the global configuration settings provided in gateway-site.xml
at the service level.
+These overrides are specified as name/value pairs within the \<service> elements of
a particular service.
+The overidden settings apply only to that service.
+
+The following table shows the common configuration settings available at the service level
via service level parameters.
+Individual services may support additional service level parameters.
+
+Property | Description | Default
+---------|-------------|---------
+httpclient.maxConnections|The maximum number of connections that a single httpclient will
maintain to a single host:port.  The default is 32.|32
+httpclient.connectionTimeout|The amount of time to wait when attempting a connection. The
natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds or minutes respectively.
The default timeout is system dependent. | System Dependent
+httpclient.socketTimeout|The amount of time to wait for data on a socket before aborting
the connection. The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds
or minutes respectively. The default timeout is system dependent but is likely to be indefinite.
| System Dependent
 
-The default HA functionality works on a simple round robin algorithm, where the top of the
list of URLs is always used
-to route all of a service's REST calls until a connection error occurs. The top URL is then
put at the bottom of the
-list and the next URL is attempted. This goes on until the setting of 'maxFailoverAttempts'
is reached.
+The example below demonstrates how these service level parameters are used.
 
-At present the following services can use this default High Availability functionality and
have been tested for the
-same:
-
-* WEBHCAT
-* HBASE
-* OOZIE
-
-To enable HA functionality for a service in Knox the following configuration has to be added
to the topology file.
-
-    <provider>
-         <role>ha</role>
-         <name>HaProvider</name>
-         <enabled>true</enabled>
-         <param>
-             <name>{SERVICE}</name>
-             <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
-         </param>
-    </provider>
-
-The role and name of the provider above must be as shown. The name in the 'param' section
i.e. `{SERVICE}` must match
-that of the service role name that is being configured for HA and the value in the 'param'
section is the configuration
-for that particular service in HA mode. For example, the value of `{SERVICE}` can be 'WEBHCAT',
'HBASE' or 'OOZIE'.
-
-To configure multiple services in HA mode, additional 'param' sections can be added.
-
-For example,
-
-    <provider>
-         <role>ha</role>
-         <name>HaProvider</name>
-         <enabled>true</enabled>
-         <param>
-             <name>OOZIE</name>
-             <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
-         </param>
-         <param>
-             <name>HBASE</name>
-             <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
-         </param>
+    <service>
+         <role>HIVE</role>
          <param>
-             <name>WEBHCAT</name>
-             <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+             <name>httpclient.socketTimeout</name>
+             <value>180s</value>
          </param>
-    </provider>
-
-The various configuration parameters are described below:
-
-* maxFailoverAttempts -
-This is the maximum number of times a failover will be attempted. The failover strategy at
this time is very simplistic
-in that the next URL in the list of URLs provided for the service is used and the one that
failed is put at the bottom
-of the list. If the list is exhausted and the maximum number of attempts is not reached then
the first URL will be tried
-again.
-
-* failoverSleep -
-The amount of time in millis that the process will wait or sleep before attempting to failover.
-
-* enabled -
-Flag to turn the particular service on or off for HA.
-
-And for the service configuration itself the additional URLs should be added to the list.
-
-    <service>
-        <role>{SERVICE}</role>
-        <url>http://host1:port1</url>
-        <url>http://host2:port2</url>
     </service>
 
-For example,
-
-    <service>
-        <role>OOZIE</role>
-        <url>http://sandbox1:11000/oozie</url>
-        <url>http://sandbox2:11000/oozie</url>
-    </service>



Mime
View raw message