knox-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From su...@apache.org
Subject svn commit: r1677170 - in /knox: site/ site/books/knox-0-4-0/ site/books/knox-0-5-0/ site/books/knox-0-6-0/ trunk/books/0.6.0/
Date Fri, 01 May 2015 15:54:03 GMT
Author: sumit
Date: Fri May  1 15:54:02 2015
New Revision: 1677170

URL: http://svn.apache.org/r1677170
Log:
KNOX-500 docs for Storm

Added:
    knox/trunk/books/0.6.0/service_storm.md
      - copied, changed from r1676779, knox/trunk/books/0.6.0/service_yarn.md
Modified:
    knox/site/books/knox-0-4-0/deployment-overview.png
    knox/site/books/knox-0-4-0/deployment-provider.png
    knox/site/books/knox-0-4-0/deployment-service.png
    knox/site/books/knox-0-4-0/runtime-overview.png
    knox/site/books/knox-0-4-0/runtime-request-processing.png
    knox/site/books/knox-0-5-0/deployment-overview.png
    knox/site/books/knox-0-5-0/deployment-provider.png
    knox/site/books/knox-0-5-0/deployment-service.png
    knox/site/books/knox-0-5-0/runtime-overview.png
    knox/site/books/knox-0-5-0/runtime-request-processing.png
    knox/site/books/knox-0-6-0/deployment-overview.png
    knox/site/books/knox-0-6-0/deployment-provider.png
    knox/site/books/knox-0-6-0/deployment-service.png
    knox/site/books/knox-0-6-0/runtime-overview.png
    knox/site/books/knox-0-6-0/runtime-request-processing.png
    knox/site/books/knox-0-6-0/user-guide.html
    knox/site/index.html
    knox/site/issue-tracking.html
    knox/site/license.html
    knox/site/mail-lists.html
    knox/site/project-info.html
    knox/site/team-list.html
    knox/trunk/books/0.6.0/book_getting-started.md
    knox/trunk/books/0.6.0/book_service-details.md

Modified: knox/site/books/knox-0-4-0/deployment-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/deployment-overview.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-4-0/deployment-provider.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/deployment-provider.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-4-0/deployment-service.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/deployment-service.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-4-0/runtime-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/runtime-overview.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-4-0/runtime-request-processing.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/runtime-request-processing.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-5-0/deployment-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/deployment-overview.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-5-0/deployment-provider.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/deployment-provider.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-5-0/deployment-service.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/deployment-service.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-5-0/runtime-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/runtime-overview.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-5-0/runtime-request-processing.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/runtime-request-processing.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-6-0/deployment-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/deployment-overview.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-6-0/deployment-provider.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/deployment-provider.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-6-0/deployment-service.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/deployment-service.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-6-0/runtime-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/runtime-overview.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-6-0/runtime-request-processing.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/runtime-request-processing.png?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
Binary files - no diff available.

Modified: knox/site/books/knox-0-6-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/user-guide.html?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
--- knox/site/books/knox-0-6-0/user-guide.html (original)
+++ knox/site/books/knox-0-6-0/user-guide.html Fri May  1 15:54:02 2015
@@ -279,6 +279,12 @@ curl -i -k -u guest:guest-password -X GE
       <td><img src="check.png"  alt="y"/> </td>
       <td><img src="check.png"  alt="y"/> </td>
     </tr>
+    <tr>
+      <td>Storm </td>
+      <td>0.9.3 </td>
+      <td><img src="check.png"  alt="y"/> </td>
+      <td><img src="error.png"  alt="n"/> </td>
+    </tr>
   </tbody>
 </table><h3><a id="More+Examples"></a>More Examples</h3><p>These
examples provide more detail about how to access various Apache Hadoop services via the Apache
Knox Gateway.</p>
 <ul>
@@ -288,6 +294,7 @@ curl -i -k -u guest:guest-password -X GE
   <li><a href="#HBase+Examples">HBase Examples</a></li>
   <li><a href="#Hive+Examples">Hive Examples</a></li>
   <li><a href="#Yarn+Examples">Yarn Examples</a></li>
+  <li><a href="#Storm+Examples">Storm Examples</a></li>
 </ul><h3><a id="Gateway+Samples"></a>Gateway Samples</h3><p>The
purpose of the samples within the {GATEWAY_HOME}/samples directory is to demonstrate the capabilities
of the Apache Knox Gateway to provide access to the numerous APIs that are available from
the service components of a Hadoop cluster.</p><p>Depending on exactly how your
Knox installation was done, there will be some number of steps required in order fully install
and configure the samples for use.</p><p>This section will help describe the assumptions
of the samples and the steps to get them to work in a couple of different deployment scenarios.</p><h4><a
id="Assumptions+of+the+Samples"></a>Assumptions of the Samples</h4><p>The
samples were initially written with the intent of working out of the box for the various Hadoop
demo environments that are deployed as a single node cluster inside of a VM. The following
assumptions were made from that context and should be understood in order to get the samples
to work in other 
 deployment scenarios:</p>
 <ul>
   <li>That there is a valid java JDK on the PATH for executing the samples</li>
@@ -3336,6 +3343,84 @@ The contents of the state-killed.json fi
 
 
 curl -ikv -u guest:guest-password -H Content-Type:application/json -X PUT -T state-killed.json
&#39;https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}/state&#39;
+</code></pre><h4><a id="Storm"></a>Storm</h4><p>Storm
is a distributed realtime computation system. Storm exposes REST APIs for UI functionality
that can be used for retrieving metrics data and configuration information as well as management
operations such as starting or stopping topologies.</p><p>The docs for this can
be found here</p><p><a href="https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md">https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md</a></p><p>To
enable this functionality, a topology file needs to have the following configuration:</p>
+<pre><code>&lt;service&gt;
+        &lt;role&gt;STORM&lt;/role&gt;
+        &lt;url&gt;http://&lt;hostname&gt;:&lt;port&gt;&lt;/url&gt;
+&lt;/service&gt;
+</code></pre><p>The default UI daemon port is 8744. If it is configured
to some other port, that configuration can be found in storm.yaml as the value for the property
&lsquo;ui.port&rsquo;.</p><p>In addition to the storm service configuration
above, a STORM-LOGVIEWER service must be configured if the log files are to be retrieved through
Knox. The value of the port for the logviewer can be found by the property &lsquo;logviewer.port&rsquo;
also in the file storm.yaml.</p>
+<pre><code>&lt;service&gt;
+        &lt;role&gt;STORM-LOGVIEWER&lt;/role&gt;
+        &lt;url&gt;http://&lt;hostname&gt;:&lt;port&gt;&lt;/url&gt;
+&lt;/service&gt;
+</code></pre><h4><a id="Storm+URL+Mapping"></a>Storm URL Mapping</h4><p>For
Storm URLs, the mapping of Knox Gateway accessible URLs to direct Storm URLs is the following.</p>
+<table>
+  <tbody>
+    <tr>
+      <td>Gateway </td>
+      <td><code>https://{gateway-host}:{gateway-port}/{gateway-path}/{cluster-name}/storm</code>
</td>
+    </tr>
+    <tr>
+      <td>Cluster </td>
+      <td><code>http://{storm-host}:{storm-port}</code> </td>
+    </tr>
+  </tbody>
+</table><p>For the log viewer the mapping is as follows</p>
+<table>
+  <tbody>
+    <tr>
+      <td>Gateway </td>
+      <td><code>https://{gateway-host}:{gateway-port}/{gateway-path}/{cluster-name}/storm/logviewer</code>
</td>
+    </tr>
+    <tr>
+      <td>Cluster </td>
+      <td><code>http://{storm-logviewer-host}:{storm-logviewer-port}</code>
</td>
+    </tr>
+  </tbody>
+</table><h4><a id="Storm+Examples+via+cURL"></a>Storm Examples via
cURL</h4><p>Some of the various calls that can be made and examples using curl
are listed below.</p>
+<pre><code># 0. Getting cluster configuration
+
+curl -ikv -u guest:guest-password -X GET &#39;https://localhost:8443/gateway/sandbox/storm/api/v1/cluster/configuration&#39;
+
+# 1. Getting cluster summary information
+
+curl -ikv -u guest:guest-password -X GET &#39;https://localhost:8443/gateway/sandbox/storm/api/v1/cluster/summary&#39;
+
+# 2. Getting supervisor summary information
+
+curl -ikv -u guest:guest-password -X GET &#39;https://localhost:8443/gateway/sandbox/storm/api/v1/supervisor/summary&#39;
+
+# 3. topologies summary information
+
+curl -ikv -u guest:guest-password -X GET &#39;https://localhost:8443/gateway/sandbox/storm/api/v1/topology/summary&#39;
+
+# 4. Getting specific topology information. Substitute {id} with the topology id.
+
+curl -ikv -u guest:guest-password -X GET &#39;https://localhost:8443/gateway/sandbox/storm/api/v1/topology/{id}&#39;
+
+# 5. To get component level information. Substitute {id} with the topology id and {component}
with the component id e.g. &#39;spout&#39;
+
+curl -ikv -u guest:guest-password -X GET &#39;https://localhost:8443/gateway/sandbox/storm/api/v1/topology/{id}/component/{component}&#39;
+</code></pre><p>The following POST operations all require a &lsquo;x-csrf-token&rsquo;
header along with other information that can be stored in a cookie file. In particular the
&lsquo;ring-session&rsquo; header and &lsquo;JSESSIONID&rsquo;.</p>
+<pre><code># 6. To activate a topology. Substitute {id} with the topology id
and {token-value} with the x-csrf-token value.
+
+curl -ik -b ~/cookiejar.txt -c ~/cookiejar.txt -u guest:guest-password -H &#39;x-csrf-token:{token-value}&#39;
-X POST \
+ http://localhost:8744/api/v1/topology/{id}/activate
+
+# 7. To de-activate a topology. Substitute {id} with the topology id and {token-value} with
the x-csrf-token value.
+
+curl -ik -b ~/cookiejar.txt -c ~/cookiejar.txt -u guest:guest-password -H &#39;x-csrf-token:{token-value}&#39;
-X POST \
+ http://localhost:8744/api/v1/topology/{id}/deactivate
+
+# 8. To rebalance a topology. Substitute {id} with the topology id and {token-value} with
the x-csrf-token value.
+
+curl -ik -b ~/cookiejar.txt -c ~/cookiejar.txt -u guest:guest-password -H &#39;x-csrf-token:{token-value}&#39;
-X POST \
+ http://localhost:8744/api/v1/topology/{id}/rebalance/0
+
+# 9. To kill a topology. Substitute {id} with the topology id and {token-value} with the
x-csrf-token value.
+
+curl -ik -b ~/cookiejar.txt -c ~/cookiejar.txt -u guest:guest-password -H &#39;x-csrf-token:{token-value}&#39;
-X POST \
+ http://localhost:8744/api/v1/topology/{id}/kill/0
 </code></pre><h2><a id="Limitations"></a>Limitations</h2><h3><a
id="Secure+Oozie+POST/PUT+Request+Payload+Size+Restriction"></a>Secure Oozie POST/PUT
Request Payload Size Restriction</h3><p>With one exception there are no known
size limits for requests or responses payloads that pass through the gateway. The exception
involves POST or PUT request payload sizes for Oozie in a Kerberos secured Hadoop cluster.
In this one case there is currently a 4Kb payload size limit for the first request made to
the Hadoop cluster. This is a result of how the gateway negotiates a trust relationship between
itself and the cluster via SPNego. There is an undocumented configuration setting to modify
this limit&rsquo;s value if required. In the future this will be made more easily configuration
and at that time it will be documented.</p><h3><a id="Group+Membership+Propagation"></a>Group
Membership Propagation</h3><p>Groups that are acquired via Shiro Group Lookup
and/or Identity Assertion Group Princip
 al Mapping are not propagated to the Hadoop services. Therefore, groups used for Service
Level Authorization policy may not match those acquired within the cluster via GroupMappingServiceProvider
plugins.</p><h2><a id="Troubleshooting"></a>Troubleshooting</h2><h3><a
id="Finding+Logs"></a>Finding Logs</h3><p>When things aren&rsquo;t
working the first thing you need to do is examine the diagnostic logs. Depending upon how
you are running the gateway these diagnostic logs will be output to different locations.</p><h4><a
id="java+-jar+bin/gateway.jar"></a>java -jar bin/gateway.jar</h4><p>When
the gateway is run this way the diagnostic output is written directly to the console. If you
want to capture that output you will need to redirect the console output to a file using OS
specific techniques.</p>
 <pre><code>java -jar bin/gateway.jar &gt; gateway.log
 </code></pre><h4><a id="bin/gateway.sh+start"></a>bin/gateway.sh
start</h4><p>When the gateway is run this way the diagnostic output is written
to /var/log/knox/knox.out and /var/log/knox/knox.err. Typically only knox.out will have content.</p><h3><a
id="Increasing+Logging"></a>Increasing Logging</h3><p>The <code>log4j.properties</code>
files <code>{GATEWAY_HOME}/conf</code> can be used to change the granularity of
the logging done by Knox. The Knox server must be restarted in order for these changes to
take effect. There are various useful loggers pre-populated but commented out.</p>

Modified: knox/site/index.html
URL: http://svn.apache.org/viewvc/knox/site/index.html?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
--- knox/site/index.html (original)
+++ knox/site/index.html Fri May  1 15:54:02 2015
@@ -1,5 +1,5 @@
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-29 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
       @import url("./css/site.css");
     </style>
     <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
-    <meta name="Date-Revision-yyyymmdd" content="20150429" />
+    <meta name="Date-Revision-yyyymmdd" content="20150501" />
     <meta http-equiv="Content-Language" content="en" />
                                                     
 <script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
                         <a href="https://cwiki.apache.org/confluence/display/KNOX/Index"
class="externalLink" title="Wiki">Wiki</a>
               
                     
-                &nbsp;| <span id="publishDate">Last Published: 2015-04-29</span>
+                &nbsp;| <span id="publishDate">Last Published: 2015-05-01</span>
               &nbsp;| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
             </div>
       <div class="clear">

Modified: knox/site/issue-tracking.html
URL: http://svn.apache.org/viewvc/knox/site/issue-tracking.html?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
--- knox/site/issue-tracking.html (original)
+++ knox/site/issue-tracking.html Fri May  1 15:54:02 2015
@@ -1,5 +1,5 @@
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-29 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
       @import url("./css/site.css");
     </style>
     <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
-    <meta name="Date-Revision-yyyymmdd" content="20150429" />
+    <meta name="Date-Revision-yyyymmdd" content="20150501" />
     <meta http-equiv="Content-Language" content="en" />
                                                     
 <script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
                         <a href="https://cwiki.apache.org/confluence/display/KNOX/Index"
class="externalLink" title="Wiki">Wiki</a>
               
                     
-                &nbsp;| <span id="publishDate">Last Published: 2015-04-29</span>
+                &nbsp;| <span id="publishDate">Last Published: 2015-05-01</span>
               &nbsp;| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
             </div>
       <div class="clear">

Modified: knox/site/license.html
URL: http://svn.apache.org/viewvc/knox/site/license.html?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
--- knox/site/license.html (original)
+++ knox/site/license.html Fri May  1 15:54:02 2015
@@ -1,5 +1,5 @@
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-29 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
       @import url("./css/site.css");
     </style>
     <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
-    <meta name="Date-Revision-yyyymmdd" content="20150429" />
+    <meta name="Date-Revision-yyyymmdd" content="20150501" />
     <meta http-equiv="Content-Language" content="en" />
                                                     
 <script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
                         <a href="https://cwiki.apache.org/confluence/display/KNOX/Index"
class="externalLink" title="Wiki">Wiki</a>
               
                     
-                &nbsp;| <span id="publishDate">Last Published: 2015-04-29</span>
+                &nbsp;| <span id="publishDate">Last Published: 2015-05-01</span>
               &nbsp;| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
             </div>
       <div class="clear">

Modified: knox/site/mail-lists.html
URL: http://svn.apache.org/viewvc/knox/site/mail-lists.html?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
--- knox/site/mail-lists.html (original)
+++ knox/site/mail-lists.html Fri May  1 15:54:02 2015
@@ -1,5 +1,5 @@
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-29 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
       @import url("./css/site.css");
     </style>
     <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
-    <meta name="Date-Revision-yyyymmdd" content="20150429" />
+    <meta name="Date-Revision-yyyymmdd" content="20150501" />
     <meta http-equiv="Content-Language" content="en" />
                                                     
 <script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
                         <a href="https://cwiki.apache.org/confluence/display/KNOX/Index"
class="externalLink" title="Wiki">Wiki</a>
               
                     
-                &nbsp;| <span id="publishDate">Last Published: 2015-04-29</span>
+                &nbsp;| <span id="publishDate">Last Published: 2015-05-01</span>
               &nbsp;| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
             </div>
       <div class="clear">

Modified: knox/site/project-info.html
URL: http://svn.apache.org/viewvc/knox/site/project-info.html?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
--- knox/site/project-info.html (original)
+++ knox/site/project-info.html Fri May  1 15:54:02 2015
@@ -1,5 +1,5 @@
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-29 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
       @import url("./css/site.css");
     </style>
     <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
-    <meta name="Date-Revision-yyyymmdd" content="20150429" />
+    <meta name="Date-Revision-yyyymmdd" content="20150501" />
     <meta http-equiv="Content-Language" content="en" />
                                                     
 <script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
                         <a href="https://cwiki.apache.org/confluence/display/KNOX/Index"
class="externalLink" title="Wiki">Wiki</a>
               
                     
-                &nbsp;| <span id="publishDate">Last Published: 2015-04-29</span>
+                &nbsp;| <span id="publishDate">Last Published: 2015-05-01</span>
               &nbsp;| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
             </div>
       <div class="clear">

Modified: knox/site/team-list.html
URL: http://svn.apache.org/viewvc/knox/site/team-list.html?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
--- knox/site/team-list.html (original)
+++ knox/site/team-list.html Fri May  1 15:54:02 2015
@@ -1,5 +1,5 @@
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-04-29 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
       @import url("./css/site.css");
     </style>
     <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
-    <meta name="Date-Revision-yyyymmdd" content="20150429" />
+    <meta name="Date-Revision-yyyymmdd" content="20150501" />
     <meta http-equiv="Content-Language" content="en" />
                                                     
 <script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
                         <a href="https://cwiki.apache.org/confluence/display/KNOX/Index"
class="externalLink" title="Wiki">Wiki</a>
               
                     
-                &nbsp;| <span id="publishDate">Last Published: 2015-04-29</span>
+                &nbsp;| <span id="publishDate">Last Published: 2015-05-01</span>
               &nbsp;| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
             </div>
       <div class="clear">

Modified: knox/trunk/books/0.6.0/book_getting-started.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/book_getting-started.md?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/book_getting-started.md (original)
+++ knox/trunk/books/0.6.0/book_getting-started.md Fri May  1 15:54:02 2015
@@ -80,6 +80,7 @@ This table enumerates the versions of va
 | Hive (via WebHCat)   | 0.13.0     | ![y]        | ![y]   |
 | Hive (via JDBC/ODBC) | 0.13.0     | ![y]        | ![y]   |
 | Yarn ResourceManager | 2.5.0      | ![y]        | ![y]   |
+| Storm                | 0.9.3      | ![y]        | ![n]   |
 
 
 ### More Examples ###
@@ -92,3 +93,4 @@ These examples provide more detail about
 * #[HBase Examples]
 * #[Hive Examples]
 * #[Yarn Examples]
+* #[Storm Examples]

Modified: knox/trunk/books/0.6.0/book_service-details.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/book_service-details.md?rev=1677170&r1=1677169&r2=1677170&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/book_service-details.md (original)
+++ knox/trunk/books/0.6.0/book_service-details.md Fri May  1 15:54:02 2015
@@ -80,5 +80,6 @@ Therefore each request via cURL will res
 <<service_hbase.md>>
 <<service_hive.md>>
 <<service_yarn.md>>
+<<service_storm.md>>
 
 

Copied: knox/trunk/books/0.6.0/service_storm.md (from r1676779, knox/trunk/books/0.6.0/service_yarn.md)
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/service_storm.md?p2=knox/trunk/books/0.6.0/service_storm.md&p1=knox/trunk/books/0.6.0/service_yarn.md&r1=1676779&r2=1677170&rev=1677170&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/service_yarn.md (original)
+++ knox/trunk/books/0.6.0/service_storm.md Fri May  1 15:54:02 2015
@@ -15,111 +15,100 @@
    limitations under the License.
 --->
 
-### Yarn ###
+#### Storm ###
 
-Knox provides gateway functionality for the REST APIs of the ResourceManager. The ResourceManager
REST API's allow the
-user to get information about the cluster - status on the cluster, metrics on the cluster,
scheduler information,
-information about nodes in the cluster, and information about applications on the cluster.
Also as of hadoop version
-2.5.0, the user can submit a new application as well as kill it (or get state) using the
'Writable' APIs.
+Storm is a distributed realtime computation system. Storm exposes REST APIs for UI functionality
that can be used for
+retrieving metrics data and configuration information as well as management operations such
as starting or stopping topologies.
 
 The docs for this can be found here
 
-http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html
+https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md
 
 To enable this functionality, a topology file needs to have the following configuration:
 
 
     <service>
-            <role>RESOURCEMANAGER</role>
-            <url>http://<hostname>:<port>/ws</url>
+            <role>STORM</role>
+            <url>http://<hostname>:<port></url>
     </service>
 
-The default resource manager http port is 8088. If it is configured to some other port, that
configuration can be
-found in yarn-site.xml under the property 'yarn.resourcemanager.webapp.address'.
+The default UI daemon port is 8744. If it is configured to some other port, that configuration
can be
+found in storm.yaml as the value for the property 'ui.port'.
 
-#### Yarn URL Mapping ####
+In addition to the storm service configuration above, a STORM-LOGVIEWER service must be configured
if the
+log files are to be retrieved through Knox. The value of the port for the logviewer can be
found by the property
+'logviewer.port' also in the file storm.yaml.
 
-For Yarn URLs, the mapping of Knox Gateway accessible URLs to direct Yarn URLs is the following.
+    <service>
+            <role>STORM-LOGVIEWER</role>
+            <url>http://<hostname>:<port></url>
+    </service>
+
+
+#### Storm URL Mapping ####
+
+For Storm URLs, the mapping of Knox Gateway accessible URLs to direct Storm URLs is the following.
+
+| ------- | -------------------------------------------------------------------------------------
|
+| Gateway | `https://{gateway-host}:{gateway-port}/{gateway-path}/{cluster-name}/storm` |
+| Cluster | `http://{storm-host}:{storm-port}`                                      |
+
+For the log viewer the mapping is as follows
 
 | ------- | -------------------------------------------------------------------------------------
|
-| Gateway | `https://{gateway-host}:{gateway-port}/{gateway-path}/{cluster-name}/resourcemanager`
|
-| Cluster | `http://{yarn-host}:{yarn-port}/ws}`                                      |
+| Gateway | `https://{gateway-host}:{gateway-port}/{gateway-path}/{cluster-name}/storm/logviewer`
|
+| Cluster | `http://{storm-logviewer-host}:{storm-logviewer-port}`                      
               |
 
 
-#### Yarn Examples via cURL
+#### Storm Examples via cURL
 
 Some of the various calls that can be made and examples using curl are listed below.
 
-    # 0. Getting cluster info
-    
-    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster'
-    
-    # 1. Getting cluster metrics
-    
-    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/metrics'
-    
-    To get the same information in an xml format
-    
-    curl -ikv -u guest:guest-password -H Accept:application/xml -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/metrics'
-    
-    # 2. Getting scheduler information
-    
-    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/scheduler'
-    
-    # 3. Getting all the applications listed and their information
-    
-    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps'
-    
-    # 4. Getting applications statistics
-    
-    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/appstatistics'
-    
-    Also query params can be used as below to filter the results
-    
-    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/appstatistics?states=accepted,running,finished&applicationTypes=mapreduce'
-    
-    # 5. To get a specific application (please note, replace the application id with a real
value)
-    
-    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}'
+    # 0. Getting cluster configuration
     
-    # 6. To get the attempts made for a particular application
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/storm/api/v1/cluster/configuration'
     
-    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}/appattempts'
+    # 1. Getting cluster summary information
     
-    # 7. To get information about the various nodes
-    
-    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/nodes'
-    
-    Also to get a specific node, use an id obtained in the response from above (the node
id is scrambled) and issue the following
-    
-    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/nodes/{node_id}'
-    
-    # 8. To create a new Application
-    
-    curl -ikv -u guest:guest-password -X POST 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/new-application'
-    
-    An application id is returned from the request above and this can be used to submit an
application.
-    
-    # 9. To submit an application, put together a request containing the application id received
in the above response (please refer to Yarn REST
-    API documentation).
-    
-    curl -ikv -u guest:guest-password -T request.json -H Content-Type:application/json -X
POST 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps'
-    
-    Here the request is saved in a file called request.json
-    
-    #10. To get application state
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/storm/api/v1/cluster/summary'
+
+    # 2. Getting supervisor summary information
     
-    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}/state'
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/storm/api/v1/supervisor/summary'
     
-    curl -ikv -u guest:guest-password -H Content-Type:application/json -X PUT -T state-killed.json
'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/application_1409008107556_0007/state'
+    # 3. topologies summary information
     
-    # 11. To kill an application that is running issue the below command with the application
id of the application that is to be killed.
-    The contents of the state-killed.json file are :
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/storm/api/v1/topology/summary'
     
-    {
-      "state":"KILLED"
-    }
+    # 4. Getting specific topology information. Substitute {id} with the topology id.
     
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/storm/api/v1/topology/{id}'
+
+    # 5. To get component level information. Substitute {id} with the topology id and {component}
with the component id e.g. 'spout'
     
-    curl -ikv -u guest:guest-password -H Content-Type:application/json -X PUT -T state-killed.json
'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}/state'
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/storm/api/v1/topology/{id}/component/{component}'
+
+
+The following POST operations all require a 'x-csrf-token' header along with other information
that can be stored in a cookie file.
+In particular the 'ring-session' header and 'JSESSIONID'.
+
+    # 6. To activate a topology. Substitute {id} with the topology id and {token-value} with
the x-csrf-token value.
+
+    curl -ik -b ~/cookiejar.txt -c ~/cookiejar.txt -u guest:guest-password -H 'x-csrf-token:{token-value}'
-X POST \
+     http://localhost:8744/api/v1/topology/{id}/activate
+
+    # 7. To de-activate a topology. Substitute {id} with the topology id and {token-value}
with the x-csrf-token value.
+
+    curl -ik -b ~/cookiejar.txt -c ~/cookiejar.txt -u guest:guest-password -H 'x-csrf-token:{token-value}'
-X POST \
+     http://localhost:8744/api/v1/topology/{id}/deactivate
+
+    # 8. To rebalance a topology. Substitute {id} with the topology id and {token-value}
with the x-csrf-token value.
+
+    curl -ik -b ~/cookiejar.txt -c ~/cookiejar.txt -u guest:guest-password -H 'x-csrf-token:{token-value}'
-X POST \
+     http://localhost:8744/api/v1/topology/{id}/rebalance/0
+
+    # 9. To kill a topology. Substitute {id} with the topology id and {token-value} with
the x-csrf-token value.
+
+    curl -ik -b ~/cookiejar.txt -c ~/cookiejar.txt -u guest:guest-password -H 'x-csrf-token:{token-value}'
-X POST \
+     http://localhost:8744/api/v1/topology/{id}/kill/0
 



Mime
View raw message