Author: kminder
Date: Fri Mar 18 13:23:31 2016
New Revision: 1735602
URL: http://svn.apache.org/viewvc?rev=1735602&view=rev
Log:
[KNOX-695] - Expose configuration of HttpClient's connection and socket timeout settings
Added:
knox/trunk/books/0.9.0/service_config.md
- copied, changed from r1735430, knox/trunk/books/0.9.0/service_default_ha.md
Modified:
knox/site/books/knox-0-9-0/user-guide.html
knox/site/index.html
knox/site/issue-tracking.html
knox/site/license.html
knox/site/mail-lists.html
knox/site/project-info.html
knox/site/team-list.html
knox/trunk/books/0.9.0/book.md
knox/trunk/books/0.9.0/book_service-details.md
knox/trunk/books/0.9.0/config.md
Modified: knox/site/books/knox-0-9-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-9-0/user-guide.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/books/knox-0-9-0/user-guide.html (original)
+++ knox/site/books/knox-0-9-0/user-guide.html Fri Mar 18 13:23:31 2016
@@ -55,6 +55,7 @@
<li><a href="#Hive">Hive</a></li>
<li><a href="#Yarn">Yarn</a></li>
<li><a href="#Storm">Storm</a></li>
+ <li><a href="#Common+Service+Config">Common Service Config</a></li>
<li><a href="#Default+Service+HA+support">Default Service HA support</a></li>
</ul></li>
<li><a href="#UI+Service+Details">UI Service Details</a></li>
@@ -510,6 +511,16 @@ https://{gateway-host}:{gateway-port}/{g
<td>32</td>
</tr>
<tr>
+ <td>gateway.httpclient.connectionTimeout</td>
+ <td>The amount of time to wait when attempting a connection. The natural unit
is milliseconds but a ‘s’ or ‘m’ suffix may be used
for seconds or minutes respectively. The default timeout is system dependent. </td>
+ <td>System Dependent</td>
+ </tr>
+ <tr>
+ <td>gateway.httpclient.socketTimeout</td>
+ <td>The amount of time to wait for data on a socket before aborting the connection.
The natural unit is milliseconds but a ‘s’ or ‘m’
suffix may be used for seconds or minutes respectively. The default timeout is system dependent
but is likely to be indefinite. </td>
+ <td>System Dependent</td>
+ </tr>
+ <tr>
<td>gateway.httpserver.requestBuffer</td>
<td>The size of the HTTP server request buffer. The default is 16K.</td>
<td>16384</td>
@@ -4324,6 +4335,40 @@ curl -ik -b ~/cookiejar.txt -c ~/cookiej
curl -ik -b ~/cookiejar.txt -c ~/cookiejar.txt -u guest:guest-password -H 'x-csrf-token:{token-value}'
-X POST \
http://localhost:8744/api/v1/topology/{id}/kill/0
+</code></pre><h3><a id="Common+Service+Config">Common Service Config</a>
<a href="#Common+Service+Config"><img src="markbook-section-link.png"/></a></h3><p>It
is possible to override a few of the global configuration settings provided in gateway-site.xml
at the service level. These overrides are specified as name/value pairs within the <service>
elements of a particular service. The overidden settings apply only to that service.</p><p>The
following table shows the common configuration settings available at the service level via
service level parameters. Individual services may support additional service level parameters.</p>
+<table>
+ <thead>
+ <tr>
+ <th>Property </th>
+ <th>Description </th>
+ <th>Default</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td>httpclient.maxConnections</td>
+ <td>The maximum number of connections that a single httpclient will maintain
to a single host:port. The default is 32.</td>
+ <td>32</td>
+ </tr>
+ <tr>
+ <td>httpclient.connectionTimeout</td>
+ <td>The amount of time to wait when attempting a connection. The natural unit
is milliseconds but a ‘s’ or ‘m’ suffix may be used
for seconds or minutes respectively. The default timeout is system dependent. </td>
+ <td>System Dependent</td>
+ </tr>
+ <tr>
+ <td>httpclient.socketTimeout</td>
+ <td>The amount of time to wait for data on a socket before aborting the connection.
The natural unit is milliseconds but a ‘s’ or ‘m’
suffix may be used for seconds or minutes respectively. The default timeout is system dependent
but is likely to be indefinite. </td>
+ <td>System Dependent</td>
+ </tr>
+ </tbody>
+</table><p>The example below demonstrates how these service level parameters
are used.</p>
+<pre><code><service>
+ <role>HIVE</role>
+ <param>
+ <name>httpclient.socketTimeout</name>
+ <value>180s</value>
+ </param>
+</service>
</code></pre><h3><a id="Default+Service+HA+support">Default Service
HA support</a> <a href="#Default+Service+HA+support"><img src="markbook-section-link.png"/></a></h3><p>Knox
provides connectivity based failover functionality for service calls that can be made to more
than one server instance in a cluster. To enable this functionality HaProvider configuration
needs to be enabled for the service and the service itself needs to be configured with more
than one URL in the topology file.</p><p>The default HA functionality works on
a simple round robin algorithm, where the top of the list of URLs is always used to route
all of a service’s REST calls until a connection error occurs. The top URL is then
put at the bottom of the list and the next URL is attempted. This goes on until the setting
of ‘maxFailoverAttempts’ is reached.</p><p>At present the
following services can use this default High Availability functionality and have been tested
for the same:</p>
<ul>
<li>WEBHCAT</li>
Modified: knox/site/index.html
URL: http://svn.apache.org/viewvc/knox/site/index.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/index.html (original)
+++ knox/site/index.html Fri Mar 18 13:23:31 2016
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2016-03-17
+ | Generated by Apache Maven Doxia at 2016-03-18
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20160317" />
+ <meta name="Date-Revision-yyyymmdd" content="20160318" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – REST API Gateway for the Hadoop Ecosystem</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2016-03-17</li>
+ <li id="publishDate" class="pull-right">Last Published: 2016-03-18</li>
</ul>
</div>
Modified: knox/site/issue-tracking.html
URL: http://svn.apache.org/viewvc/knox/site/issue-tracking.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/issue-tracking.html (original)
+++ knox/site/issue-tracking.html Fri Mar 18 13:23:31 2016
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2016-03-17
+ | Generated by Apache Maven Doxia at 2016-03-18
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20160317" />
+ <meta name="Date-Revision-yyyymmdd" content="20160318" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Issue Tracking</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2016-03-17</li>
+ <li id="publishDate" class="pull-right">Last Published: 2016-03-18</li>
</ul>
</div>
Modified: knox/site/license.html
URL: http://svn.apache.org/viewvc/knox/site/license.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/license.html (original)
+++ knox/site/license.html Fri Mar 18 13:23:31 2016
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2016-03-17
+ | Generated by Apache Maven Doxia at 2016-03-18
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20160317" />
+ <meta name="Date-Revision-yyyymmdd" content="20160318" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Project License</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2016-03-17</li>
+ <li id="publishDate" class="pull-right">Last Published: 2016-03-18</li>
</ul>
</div>
Modified: knox/site/mail-lists.html
URL: http://svn.apache.org/viewvc/knox/site/mail-lists.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/mail-lists.html (original)
+++ knox/site/mail-lists.html Fri Mar 18 13:23:31 2016
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2016-03-17
+ | Generated by Apache Maven Doxia at 2016-03-18
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20160317" />
+ <meta name="Date-Revision-yyyymmdd" content="20160318" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Project Mailing Lists</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2016-03-17</li>
+ <li id="publishDate" class="pull-right">Last Published: 2016-03-18</li>
</ul>
</div>
Modified: knox/site/project-info.html
URL: http://svn.apache.org/viewvc/knox/site/project-info.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/project-info.html (original)
+++ knox/site/project-info.html Fri Mar 18 13:23:31 2016
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2016-03-17
+ | Generated by Apache Maven Doxia at 2016-03-18
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20160317" />
+ <meta name="Date-Revision-yyyymmdd" content="20160318" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Project Information</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2016-03-17</li>
+ <li id="publishDate" class="pull-right">Last Published: 2016-03-18</li>
</ul>
</div>
Modified: knox/site/team-list.html
URL: http://svn.apache.org/viewvc/knox/site/team-list.html?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/site/team-list.html (original)
+++ knox/site/team-list.html Fri Mar 18 13:23:31 2016
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2016-03-17
+ | Generated by Apache Maven Doxia at 2016-03-18
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20160317" />
+ <meta name="Date-Revision-yyyymmdd" content="20160318" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Team list</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2016-03-17</li>
+ <li id="publishDate" class="pull-right">Last Published: 2016-03-18</li>
</ul>
</div>
Modified: knox/trunk/books/0.9.0/book.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.9.0/book.md?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/trunk/books/0.9.0/book.md (original)
+++ knox/trunk/books/0.9.0/book.md Fri Mar 18 13:23:31 2016
@@ -60,6 +60,7 @@
* #[Hive]
* #[Yarn]
* #[Storm]
+ * #[Common Service Config]
* #[Default Service HA support]
* #[UI Service Details]
* #[Limitations]
Modified: knox/trunk/books/0.9.0/book_service-details.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.9.0/book_service-details.md?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/trunk/books/0.9.0/book_service-details.md (original)
+++ knox/trunk/books/0.9.0/book_service-details.md Fri Mar 18 13:23:31 2016
@@ -82,6 +82,7 @@ Therefore each request via cURL will res
<<service_hive.md>>
<<service_yarn.md>>
<<service_storm.md>>
+<<service_config.md>>
<<service_default_ha.md>>
Modified: knox/trunk/books/0.9.0/config.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.9.0/config.md?rev=1735602&r1=1735601&r2=1735602&view=diff
==============================================================================
--- knox/trunk/books/0.9.0/config.md (original)
+++ knox/trunk/books/0.9.0/config.md Fri Mar 18 13:23:31 2016
@@ -128,6 +128,8 @@ gateway.keystore.type|Indicates the type
gateway.jdk.tls.ephemeralDHKeySize|jdk.tls.ephemeralDHKeySize, is defined to customize the
ephemeral DH key sizes. The minimum acceptable DH key size is 1024 bits, except for exportable
cipher suites or legacy mode (jdk.tls.ephemeralDHKeySize=legacy)|2048
gateway.threadpool.max|The maximum concurrent requests the server will process. The default
is 254. Connections beyond this will be queued.|254
gateway.httpclient.maxConnections|The maximum number of connections that a single httpclient
will maintain to a single host:port. The default is 32.|32
+gateway.httpclient.connectionTimeout|The amount of time to wait when attempting a connection.
The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds or minutes
respectively. The default timeout is system dependent. | System Dependent
+gateway.httpclient.socketTimeout|The amount of time to wait for data on a socket before aborting
the connection. The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds
or minutes respectively. The default timeout is system dependent but is likely to be indefinite.
| System Dependent
gateway.httpserver.requestBuffer|The size of the HTTP server request buffer. The default
is 16K.|16384
gateway.httpserver.requestHeaderBuffer|The size of the HTTP server request header buffer.
The default is 8K.|8192
gateway.httpserver.responseBuffer|The size of the HTTP server response buffer. The default
is 32K.|32768
Copied: knox/trunk/books/0.9.0/service_config.md (from r1735430, knox/trunk/books/0.9.0/service_default_ha.md)
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.9.0/service_config.md?p2=knox/trunk/books/0.9.0/service_config.md&p1=knox/trunk/books/0.9.0/service_default_ha.md&r1=1735430&r2=1735602&rev=1735602&view=diff
==============================================================================
--- knox/trunk/books/0.9.0/service_default_ha.md (original)
+++ knox/trunk/books/0.9.0/service_config.md Fri Mar 18 13:23:31 2016
@@ -15,87 +15,28 @@
limitations under the License.
--->
-### Default Service HA support ###
+### Common Service Config ###
-Knox provides connectivity based failover functionality for service calls that can be made
to more than one server
-instance in a cluster. To enable this functionality HaProvider configuration needs to be
enabled for the service and
-the service itself needs to be configured with more than one URL in the topology file.
+It is possible to override a few of the global configuration settings provided in gateway-site.xml
at the service level.
+These overrides are specified as name/value pairs within the \<service> elements of
a particular service.
+The overidden settings apply only to that service.
+
+The following table shows the common configuration settings available at the service level
via service level parameters.
+Individual services may support additional service level parameters.
+
+Property | Description | Default
+---------|-------------|---------
+httpclient.maxConnections|The maximum number of connections that a single httpclient will
maintain to a single host:port. The default is 32.|32
+httpclient.connectionTimeout|The amount of time to wait when attempting a connection. The
natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds or minutes respectively.
The default timeout is system dependent. | System Dependent
+httpclient.socketTimeout|The amount of time to wait for data on a socket before aborting
the connection. The natural unit is milliseconds but a 's' or 'm' suffix may be used for seconds
or minutes respectively. The default timeout is system dependent but is likely to be indefinite.
| System Dependent
-The default HA functionality works on a simple round robin algorithm, where the top of the
list of URLs is always used
-to route all of a service's REST calls until a connection error occurs. The top URL is then
put at the bottom of the
-list and the next URL is attempted. This goes on until the setting of 'maxFailoverAttempts'
is reached.
+The example below demonstrates how these service level parameters are used.
-At present the following services can use this default High Availability functionality and
have been tested for the
-same:
-
-* WEBHCAT
-* HBASE
-* OOZIE
-
-To enable HA functionality for a service in Knox the following configuration has to be added
to the topology file.
-
- <provider>
- <role>ha</role>
- <name>HaProvider</name>
- <enabled>true</enabled>
- <param>
- <name>{SERVICE}</name>
- <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
- </param>
- </provider>
-
-The role and name of the provider above must be as shown. The name in the 'param' section
i.e. `{SERVICE}` must match
-that of the service role name that is being configured for HA and the value in the 'param'
section is the configuration
-for that particular service in HA mode. For example, the value of `{SERVICE}` can be 'WEBHCAT',
'HBASE' or 'OOZIE'.
-
-To configure multiple services in HA mode, additional 'param' sections can be added.
-
-For example,
-
- <provider>
- <role>ha</role>
- <name>HaProvider</name>
- <enabled>true</enabled>
- <param>
- <name>OOZIE</name>
- <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
- </param>
- <param>
- <name>HBASE</name>
- <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
- </param>
+ <service>
+ <role>HIVE</role>
<param>
- <name>WEBHCAT</name>
- <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+ <name>httpclient.socketTimeout</name>
+ <value>180s</value>
</param>
- </provider>
-
-The various configuration parameters are described below:
-
-* maxFailoverAttempts -
-This is the maximum number of times a failover will be attempted. The failover strategy at
this time is very simplistic
-in that the next URL in the list of URLs provided for the service is used and the one that
failed is put at the bottom
-of the list. If the list is exhausted and the maximum number of attempts is not reached then
the first URL will be tried
-again.
-
-* failoverSleep -
-The amount of time in millis that the process will wait or sleep before attempting to failover.
-
-* enabled -
-Flag to turn the particular service on or off for HA.
-
-And for the service configuration itself the additional URLs should be added to the list.
-
- <service>
- <role>{SERVICE}</role>
- <url>http://host1:port1</url>
- <url>http://host2:port2</url>
</service>
-For example,
-
- <service>
- <role>OOZIE</role>
- <url>http://sandbox1:11000/oozie</url>
- <url>http://sandbox2:11000/oozie</url>
- </service>
|