knox-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kmin...@apache.org
Subject svn commit: r1692312 - in /knox: site/books/knox-0-7-0/user-guide.html trunk/books/0.7.0/book_service-details.md trunk/books/0.7.0/knox_cli.md trunk/books/0.7.0/service_service_test.md
Date Wed, 22 Jul 2015 18:20:28 GMT
Author: kminder
Date: Wed Jul 22 18:20:28 2015
New Revision: 1692312

URL: http://svn.apache.org/r1692312
Log:
KNOX-549: Test service connections through Knox with Knox CLI

Added:
    knox/trunk/books/0.7.0/service_service_test.md
Modified:
    knox/site/books/knox-0-7-0/user-guide.html
    knox/trunk/books/0.7.0/book_service-details.md
    knox/trunk/books/0.7.0/knox_cli.md

Modified: knox/site/books/knox-0-7-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/user-guide.html?rev=1692312&r1=1692311&r2=1692312&view=diff
==============================================================================
--- knox/site/books/knox-0-7-0/user-guide.html (original)
+++ knox/site/books/knox-0-7-0/user-guide.html Wed Jul 22 18:20:28 2015
@@ -707,6 +707,40 @@ ip-10-39-107-209.ec2.internal
       <td>Optional; Print extra debug info on failed authentication</td>
     </tr>
   </tbody>
+</table><h4><a id="Gateway+Service+Test"></a>Gateway Service Test</h4><h5><a
id="`bin/knoxcli.sh+service-test+[--cluster+c]+[--hostname+hostname]+[--port+port]+[--u+username]+[--p+password]+[--d]+[--help]`"></a><code>bin/knoxcli.sh
service-test [--cluster c] [--hostname hostname] [--port port] [--u username] [--p password]
[--d] [--help]</code></h5><p>This will test a topology configuration&rsquo;s
ability to connect to multiple hadoop services. Each service found in a topology will be tested
with multiple URLs. Results are printed to the console in JSON format..</p>
+<table>
+  <thead>
+    <tr>
+      <th>argument </th>
+      <th>description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td>--cluster </td>
+      <td>Required; name of cluster for which you want to test authentication</td>
+    </tr>
+    <tr>
+      <td>--hostname </td>
+      <td>Required; hostname of the cluster currently running on the machine</td>
+    </tr>
+    <tr>
+      <td>--port </td>
+      <td>Optional; port that the cluster is running on. If not supplied CLI will try
to read config files to find the port.</td>
+    </tr>
+    <tr>
+      <td>--u </td>
+      <td>Required; username to authorize against Hadoop services</td>
+    </tr>
+    <tr>
+      <td>--p </td>
+      <td>Required; password to match username</td>
+    </tr>
+    <tr>
+      <td>--d </td>
+      <td>Optional; Print extra debug info on failed authentication</td>
+    </tr>
+  </tbody>
 </table><h3><a id="Admin+API"></a>Admin API</h3><p>Access
to the administrator functions of Knox are provided by the Admin REST API.</p><h4><a
id="Admin+API+URL"></a>Admin API URL</h4><p>The URL mapping for the Knox
Admin API is simple:</p>
 <table>
   <tbody>
@@ -3526,7 +3560,201 @@ curl -ik -b ~/cookiejar.txt -c ~/cookiej
 
 curl -ik -b ~/cookiejar.txt -c ~/cookiejar.txt -u guest:guest-password -H &#39;x-csrf-token:{token-value}&#39;
-X POST \
  http://localhost:8744/api/v1/topology/{id}/kill/0
-</code></pre><h2><a id="Limitations"></a>Limitations</h2><h3><a
id="Secure+Oozie+POST/PUT+Request+Payload+Size+Restriction"></a>Secure Oozie POST/PUT
Request Payload Size Restriction</h3><p>With one exception there are no known
size limits for requests or responses payloads that pass through the gateway. The exception
involves POST or PUT request payload sizes for Oozie in a Kerberos secured Hadoop cluster.
In this one case there is currently a 4Kb payload size limit for the first request made to
the Hadoop cluster. This is a result of how the gateway negotiates a trust relationship between
itself and the cluster via SPNego. There is an undocumented configuration setting to modify
this limit&rsquo;s value if required. In the future this will be made more easily configuration
and at that time it will be documented.</p><h3><a id="Group+Membership+Propagation"></a>Group
Membership Propagation</h3><p>Groups that are acquired via Shiro Group Lookup
and/or Identity Assertion Group Princip
 al Mapping are not propagated to the Hadoop services. Therefore, groups used for Service
Level Authorization policy may not match those acquired within the cluster via GroupMappingServiceProvider
plugins.</p><h2><a id="Troubleshooting"></a>Troubleshooting</h2><h3><a
id="Finding+Logs"></a>Finding Logs</h3><p>When things aren&rsquo;t
working the first thing you need to do is examine the diagnostic logs. Depending upon how
you are running the gateway these diagnostic logs will be output to different locations.</p><h4><a
id="java+-jar+bin/gateway.jar"></a>java -jar bin/gateway.jar</h4><p>When
the gateway is run this way the diagnostic output is written directly to the console. If you
want to capture that output you will need to redirect the console output to a file using OS
specific techniques.</p>
+</code></pre><h3><a id="Service+Test+API"></a>Service Test
API</h3><p>The gateway supports a Service Test API that can be used to test Knox&rsquo;s
ability to connect to each of the different Hadoop services via a simeple HTTP GET request.
To be able to access this API one must add the following line into the topology for which
you wish to run the service test.</p>
+<pre><code>&lt;service&gt;
+    &lt;role&gt;SERVICE-TEST&lt;/role&gt;
+&lt;/service&gt;
+</code></pre><p>After adding the above to a topology, you can make a cURL
request with the following structure</p>
+<pre><code>curl -i -k &quot;https://{gateway-hostname}:{gateway-port}/gateway/path/{topology-name}/service-test?username=guest&amp;password=guest-password&quot;
+</code></pre><p>An alternate method of providing credentials:</p>
+<pre><code>curl -i -k -u guest:guest-password https://{gateway-hostname}:{gateway-port}/gateway/path/{topology-name}/service-test
+</code></pre><p>Below is an example response. The gateway is also capable
of returning XML if specified in the request&rsquo;s &ldquo;Accept&rdquo; HTTP
header.</p>
+<pre><code>{
+    &quot;serviceTestWrapper&quot;: {
+       &quot;Tests&quot;: {
+         &quot;ServiceTest&quot;: [
+          {
+          &quot;serviceName&quot;: &quot;WEBHDFS&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/webhdfs/v1/?op=LISTSTATUS&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;WEBHCAT&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/templeton/v1/status&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;WEBHCAT&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/templeton/v1/version&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;WEBHCAT&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/templeton/v1/version/hive&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;WEBHCAT&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/templeton/v1/version/hadoop&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;OOZIE&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/oozie/v1/admin/build-version&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;OOZIE&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/oozie/v1/admin/status&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;OOZIE&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/oozie/versions&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;WEBHBASE&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/hbase/version&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;WEBHBASE&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/hbase/version/cluster&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;WEBHBASE&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/hbase/status/cluster&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;WEBHBASE&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/hbase&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;RESOURCEMANAGER&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/resourcemanager/v1/{topology-name}/info&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;RESOURCEMANAGER&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/resourcemanager/v1/{topology-name}/metrics&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;RESOURCEMANAGER&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/resourcemanager/v1/{topology-name}/apps&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;FALCON&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/admin/stack&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;FALCON&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/admin/version&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;FALCON&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/metadata/lineage/serialize&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;FALCON&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/metadata/lineage/vertices/all&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;FALCON&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/metadata/lineage/edges/all&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;STORM&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/storm/api/v1/cluster/configuration&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;STORM&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/storm/api/v1/cluster/summary&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;STORM&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/storm/api/v1/supervisor/summary&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          },
+          {
+          &quot;serviceName&quot;: &quot;STORM&quot;,
+          &quot;requestURL&quot;: &quot;http://{gateway-host}:{gateway-port}/gateway/{topology-name}/storm/api/v1/topology/summary&quot;,
+          &quot;responseContent&quot;: &quot;Content-Length:0,Content-Type: application/json;charset=utf-8&quot;,
+          &quot;httpCode&quot;: 200,
+          &quot;message&quot;: &quot;Request sucessful.&quot;
+          }
+         ]
+       },
+       &quot;messages&quot;: {
+         &quot;message&quot;: [
+
+         ]
+       }
+    }
+}
+</code></pre><p>We can see that this service-test makes HTTP requests to
each of the services through Knox using the specified topology. The test will only make calls
to those services that have entries within the topology file.</p><h5><a id="Adding+and+Changing+test+URLs"></a>Adding
and Changing test URLs</h5><p>URLs for each service are stored in <code>{GATEWAY_HOME}/data/services/{service-name}/{service-version}/service.xml</code>.
Each <code>&lt;testURL&gt;</code> element represents a service resource
that will be tested if the service is set up in the topology. You can add or remove these
from teh service.xml files. Just note if you add URL&rsquo;s there is no guarantee in
the order they will be tested. All default URLs have been tested and work on various clusters.
If a new URL is added and doesn&rsquo;t respond in a way the user expects then it is up
to the user to determine whether the URL is correct or not.</p><h5><a id="Some+important+things+to+note:"></a>Some
important th
 ings to note:</h5>
+<ul>
+  <li>In the first cURL request, the quotes are necessary around the URL or else a
command line terminal will not include the <code>&amp;password</code> query
parameter in the request.</li>
+  <li>This API call does not require any credentials to receive a response from Knox,
but expect to receive 401 responses from each of the services if none are provided.</li>
+</ul><h2><a id="Limitations"></a>Limitations</h2><h3><a
id="Secure+Oozie+POST/PUT+Request+Payload+Size+Restriction"></a>Secure Oozie POST/PUT
Request Payload Size Restriction</h3><p>With one exception there are no known
size limits for requests or responses payloads that pass through the gateway. The exception
involves POST or PUT request payload sizes for Oozie in a Kerberos secured Hadoop cluster.
In this one case there is currently a 4Kb payload size limit for the first request made to
the Hadoop cluster. This is a result of how the gateway negotiates a trust relationship between
itself and the cluster via SPNego. There is an undocumented configuration setting to modify
this limit&rsquo;s value if required. In the future this will be made more easily configuration
and at that time it will be documented.</p><h3><a id="Group+Membership+Propagation"></a>Group
Membership Propagation</h3><p>Groups that are acquired via Shiro Group Lookup
and/or Identity Assertion Group Principal Mappi
 ng are not propagated to the Hadoop services. Therefore, groups used for Service Level Authorization
policy may not match those acquired within the cluster via GroupMappingServiceProvider plugins.</p><h2><a
id="Troubleshooting"></a>Troubleshooting</h2><h3><a id="Finding+Logs"></a>Finding
Logs</h3><p>When things aren&rsquo;t working the first thing you need to do
is examine the diagnostic logs. Depending upon how you are running the gateway these diagnostic
logs will be output to different locations.</p><h4><a id="java+-jar+bin/gateway.jar"></a>java
-jar bin/gateway.jar</h4><p>When the gateway is run this way the diagnostic output
is written directly to the console. If you want to capture that output you will need to redirect
the console output to a file using OS specific techniques.</p>
 <pre><code>java -jar bin/gateway.jar &gt; gateway.log
 </code></pre><h4><a id="bin/gateway.sh+start"></a>bin/gateway.sh
start</h4><p>When the gateway is run this way the diagnostic output is written
to /var/log/knox/knox.out and /var/log/knox/knox.err. Typically only knox.out will have content.</p><h3><a
id="Increasing+Logging"></a>Increasing Logging</h3><p>The <code>log4j.properties</code>
files <code>{GATEWAY_HOME}/conf</code> can be used to change the granularity of
the logging done by Knox. The Knox server must be restarted in order for these changes to
take effect. There are various useful loggers pre-populated but commented out.</p>
 <pre><code>log4j.logger.org.apache.hadoop.gateway=DEBUG # Use this logger to
increase the debugging of Apache Knox itself.

Modified: knox/trunk/books/0.7.0/book_service-details.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/book_service-details.md?rev=1692312&r1=1692311&r2=1692312&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/book_service-details.md (original)
+++ knox/trunk/books/0.7.0/book_service-details.md Wed Jul 22 18:20:28 2015
@@ -84,3 +84,213 @@ Therefore each request via cURL will res
 <<service_storm.md>>
 
 
+### Service Test API
+
+The gateway supports a Service Test API that can be used to test Knox's ability to connect
to each of the different Hadoop services via a simeple HTTP GET request. To be able to access
this API one must add the following line into the topology for which you wish to run the service
test.
+
+	<service>
+		<role>SERVICE-TEST</role>
+	</service>
+	
+After adding the above to a topology, you can make a cURL request with the following structure
+
+	curl -i -k "https://{gateway-hostname}:{gateway-port}/gateway/path/{topology-name}/service-test?username=guest&password=guest-password"
+
+An alternate method of providing credentials:
+	
+	curl -i -k -u guest:guest-password https://{gateway-hostname}:{gateway-port}/gateway/path/{topology-name}/service-test
+	
+Below is an example response. The gateway is also capable of returning XML if specified in
the request's "Accept" HTTP header.
+	
+	{
+	    "serviceTestWrapper": {
+		   "Tests": {
+			  "ServiceTest": [
+				 {
+					"serviceName": "WEBHDFS",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/webhdfs/v1/?op=LISTSTATUS",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHCAT",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/templeton/v1/status",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHCAT",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/templeton/v1/version",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHCAT",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/templeton/v1/version/hive",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHCAT",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/templeton/v1/version/hadoop",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "OOZIE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/oozie/v1/admin/build-version",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "OOZIE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/oozie/v1/admin/status",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "OOZIE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/oozie/versions",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHBASE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/hbase/version",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHBASE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/hbase/version/cluster",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHBASE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/hbase/status/cluster",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHBASE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/hbase",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "RESOURCEMANAGER",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/resourcemanager/v1/{topology-name}/info",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "RESOURCEMANAGER",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/resourcemanager/v1/{topology-name}/metrics",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "RESOURCEMANAGER",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/resourcemanager/v1/{topology-name}/apps",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "FALCON",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/admin/stack",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "FALCON",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/admin/version",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "FALCON",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/metadata/lineage/serialize",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "FALCON",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/metadata/lineage/vertices/all",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "FALCON",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/metadata/lineage/edges/all",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "STORM",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/storm/api/v1/cluster/configuration",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "STORM",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/storm/api/v1/cluster/summary",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "STORM",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/storm/api/v1/supervisor/summary",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "STORM",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/storm/api/v1/topology/summary",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 }
+			  ]
+		   },
+		   "messages": {
+			  "message": [
+
+			  ]
+		   }
+	    }
+	}
+
+	
+We can see that this service-test makes HTTP requests to each of the services through Knox
using the specified topology. The test will only make calls to those services that have entries
within the topology file.
+
+##### Adding and Changing test URLs
+
+URLs for each service are stored in `{GATEWAY_HOME}/data/services/{service-name}/{service-version}/service.xml`.
Each `<testURL>` element represents a service resource that will be tested if the service
is set up in the topology. You can add or remove these from teh service.xml files. Just note
if you add URL's there is no guarantee in the order they will be tested. All default URLs
have been tested and work on various clusters. If a new URL is added and doesn't respond in
a way the user expects then it is up to the user to determine whether the URL is correct or
not.
+
+##### Some important things to note:
+ - In the first cURL request, the quotes are necessary around the URL or else a command line
terminal will not include the `&password` query parameter in the request.
+ - This API call does not require any credentials to receive a response from Knox, but expect
to receive 401 responses from each of the services if none are provided.

Modified: knox/trunk/books/0.7.0/knox_cli.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/knox_cli.md?rev=1692312&r1=1692311&r2=1692312&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/knox_cli.md (original)
+++ knox/trunk/books/0.7.0/knox_cli.md Wed Jul 22 18:20:28 2015
@@ -110,4 +110,19 @@ This command will test a given topology'
 argument | description
 ---------|-----------
 \-\-cluster	| Required; name of cluster for which you want to test authentication
+\-\-d | Optional; Print extra debug info on failed authentication
+
+
+#### Gateway Service Test ####
+##### `bin/knoxcli.sh service-test [--cluster c] [--hostname hostname] [--port port] [--u
username] [--p password] [--d] [--help]` ####
+
+This will test a topology configuration's ability to connect to multiple hadoop services.
Each service found in a topology will be tested with multiple URLs. Results are printed to
the console in JSON format..
+
+argument | description
+---------|-----------
+\-\-cluster	| Required; name of cluster for which you want to test authentication
+\-\-hostname	| Required; hostname of the cluster currently running on the machine
+\-\-port	| Optional; port that the cluster is running on. If not supplied CLI will try to
read config files to find the port.
+\-\-u	| Required; username to authorize against Hadoop services
+\-\-p	| Required; password to match username
 \-\-d | Optional; Print extra debug info on failed authentication
\ No newline at end of file

Added: knox/trunk/books/0.7.0/service_service_test.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/service_service_test.md?rev=1692312&view=auto
==============================================================================
--- knox/trunk/books/0.7.0/service_service_test.md (added)
+++ knox/trunk/books/0.7.0/service_service_test.md Wed Jul 22 18:20:28 2015
@@ -0,0 +1,221 @@
+<!---
+   Licensed to the Apache Software Foundation (ASF) under one or more
+   contributor license agreements.  See the NOTICE file distributed with
+   this work for additional information regarding copyright ownership.
+   The ASF licenses this file to You under the Apache License, Version 2.0
+   (the "License"); you may not use this file except in compliance with
+   the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+--->
+
+### Service Test ###
+
+The gateway supports a Service Test API that can be used to test Knox's ability to connect
to each of the different Hadoop services via a simple HTTP GET request. To be able to access
this API one must add the following entry into the topology for which you wish to run the
service test.
+
+	<service>
+		<role>SERVICE-TEST</role>
+	</service>
+
+After adding the above to a topology, you can make a cURL request with the following structure
+
+	curl -i -k -u guest:guest-password https://{gateway-host}:{gateway-port}/{gateway-path}/{topology-name}/service-test
+
+Below is an example response. The gateway is also capable of returning XML if specified in
the request's "Accept" HTTP header.
+
+	{
+	    "serviceTestWrapper": {
+		   "Tests": {
+			  "ServiceTest": [
+				 {
+					"serviceName": "WEBHDFS",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/webhdfs/v1/?op=LISTSTATUS",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHCAT",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/templeton/v1/status",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHCAT",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/templeton/v1/version",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHCAT",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/templeton/v1/version/hive",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHCAT",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/templeton/v1/version/hadoop",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "OOZIE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/oozie/v1/admin/build-version",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "OOZIE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/oozie/v1/admin/status",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "OOZIE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/oozie/versions",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHBASE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/hbase/version",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHBASE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/hbase/version/cluster",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHBASE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/hbase/status/cluster",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "WEBHBASE",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/hbase",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "RESOURCEMANAGER",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/resourcemanager/v1/{topology-name}/info",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "RESOURCEMANAGER",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/resourcemanager/v1/{topology-name}/metrics",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "RESOURCEMANAGER",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/resourcemanager/v1/{topology-name}/apps",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "FALCON",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/admin/stack",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "FALCON",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/admin/version",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "FALCON",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/metadata/lineage/serialize",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "FALCON",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/metadata/lineage/vertices/all",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "FALCON",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/falcon/api/metadata/lineage/edges/all",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "STORM",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/storm/api/v1/cluster/configuration",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "STORM",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/storm/api/v1/cluster/summary",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "STORM",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/storm/api/v1/supervisor/summary",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 },
+				 {
+					"serviceName": "STORM",
+					"requestURL": "http://{gateway-host}:{gateway-port}/gateway/{topology-name}/storm/api/v1/topology/summary",
+					"responseContent": "Content-Length:0,Content-Type: application/json;charset=utf-8",
+					"httpCode": 200,
+					"message": "Request sucessful."
+				 }
+			  ]
+		   },
+		   "messages": {
+			  "message": [
+			  ]
+		   }
+	    }
+	}
+
+
+We can see that this service-test makes HTTP requests to each of the services through Knox
using the specified topology. The test will only make calls to those services that have entries
within the topology file.
+
+##### Adding and Changing test URLs
+
+URLs for each service are stored in `{GATEWAY_HOME}/data/services/{service-name}/{service-version}/service.xml`.
Each `<testURL>` element represents a service resource that will be tested if the service
is set up in the topology. You can add or remove these from teh service.xml files. Just note
if you add URL's there is no guarantee in the order they will be tested. All default URLs
have been tested and work on various clusters. If a new URL is added and doesn't respond in
a way the user expects then it is up to the user to determine whether the URL is correct or
not.
+
+##### Some important things to note:
+ - This API call does not require any credentials to receive a response from Knox, but expect
to receive 401 responses from each of the services if none are provided.



Mime
View raw message