knox-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From su...@apache.org
Subject svn commit: r1677654 - in /knox: site/ site/books/knox-0-6-0/ trunk/books/0.6.0/ trunk/books/0.6.0/dev-guide/
Date Mon, 04 May 2015 18:52:17 GMT
Author: sumit
Date: Mon May  4 18:52:17 2015
New Revision: 1677654

URL: http://svn.apache.org/r1677654
Log:
KNOX-481 docs for Stacks and misc fixes

Modified:
    knox/site/books/knox-0-6-0/dev-guide.html
    knox/site/books/knox-0-6-0/user-guide.html
    knox/site/index.html
    knox/site/issue-tracking.html
    knox/site/license.html
    knox/site/mail-lists.html
    knox/site/project-info.html
    knox/site/team-list.html
    knox/trunk/books/0.6.0/book.md
    knox/trunk/books/0.6.0/book_service-details.md
    knox/trunk/books/0.6.0/dev-guide/book.md
    knox/trunk/books/0.6.0/service_storm.md

Modified: knox/site/books/knox-0-6-0/dev-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/dev-guide.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/books/knox-0-6-0/dev-guide.html (original)
+++ knox/site/books/knox-0-6-0/dev-guide.html Mon May  4 18:52:17 2015
@@ -144,6 +144,10 @@
       <td>The implementation of the Access service POC. </td>
     </tr>
     <tr>
+      <td>gateway-service-definitions </td>
+      <td>The implementation of the Service definition and rewrite files. </td>
+    </tr>
+    <tr>
       <td>gateway-service-hbase </td>
       <td>The implementation of the HBase service. </td>
     </tr>
@@ -160,10 +164,6 @@
       <td>The implementation of the Ticket Granting service POC. </td>
     </tr>
     <tr>
-      <td>gateway-service-webhcat </td>
-      <td>The implementation of the WebHCat service. </td>
-    </tr>
-    <tr>
       <td>gateway-service-webhdfs </td>
       <td>The implementation of the WebHdfs service. </td>
     </tr>
@@ -521,6 +521,94 @@ public interface ServiceDeploymentContri
     &lt;/dependencies&gt;
 
 &lt;/project&gt;
+</code></pre><h4><a id="Service+Definition+Files"></a>Service
Definition Files</h4><p>As of release 0.6.0, the gateway now also supports a declarative
way of plugging-in a new Service. A Service can be defined with a combination of two files,
these are:</p>
+<pre><code>service.xml
+rewrite.xml
+</code></pre><p>The rewrite.xml file contains the rewrite rules as defined
in other sections of this guide, and the service.xml file contains the various routes (paths)
to be provided by the Service and the rewrite rule bindings to those paths. This will be described
in further detail in this section.</p><p>While the service.xml file is absolutely
required, the rewrite.xml file in theory is optional (though it is highly unlikely that no
rewrite rules are needed).</p><p>To add a new service, simply add a service.xml
and rewrite.xml file in an appropriate directory (see <a href="#Service+Definition+Directory+Structure">Service
Definition Directory Structure</a>) in the module gateway-service-definitions to make
the new service part of the Knox build.</p><h5><a id="service.xml"></a>service.xml</h5><p>Below
is a sample of a very simple service.xml file, taking the same weather api example.</p>
+<pre><code class="xml">&lt;service role=&quot;WEATHER&quot; name=&quot;weather&quot;
version=&quot;0.1.0&quot;&gt;
+    &lt;routes&gt;
+        &lt;route path=&quot;/weather/**?**&quot;/&gt;
+    &lt;/routes&gt;
+&lt;/service&gt;
+
+</code></pre>
+<dl><dt><strong>service</strong></dt><dd>The root tag
is &lsquo;service&rsquo; that has the three required attributes: <em>role</em>,
<em>name</em> and <em>version</em>. These three values disambiguate
this service definition from others. To ensure the exact same service definition is being
used in a topology file, all values should be specified,</dd>
+</dl>
+<pre><code class="xml">&lt;topology&gt;
+    &lt;gateway&gt;
+        ....
+    &lt;/gateway&gt;
+    &lt;service&gt;
+        &lt;role&gt;WEATHER&lt;/role&gt;
+        &lt;name&gt;weather&lt;/name&gt;
+        &lt;version&gt;0.1.0&lt;/version&gt;
+        &lt;url&gt;http://api.openweathermap.org/data&lt;/url&gt;
+    &lt;/service&gt;
+    ....
+&lt;/topology&gt;
+</code></pre><p>If only <em>role</em> is specified in the topology
file (the only required element other than <em>url</em>) then the first service
definition of that role found will be used with the highest version of that role and name.
Similarly if only the <em>version</em> is omitted from the topology specification
of the service, the service definition of the highest version will be used. It is therefore
important to specify a version for a service if it is desired that a topology be locked down
to a specific version of a service.</p>
+<dl><dt><strong>routes</strong></dt><dd>Wrapper element
for one or more routes.</dd><dt><strong>route</strong></dt><dd>A
route specifies the <em>path</em> that the service is routing as well as any rewrite
bindings or policy bindings. Another child element that may be used here is a <em>dispatch</em>
element.</dd><dt><strong>rewrite</strong></dt><dd>A rewrite
rule or function that is to be applied to the path. A rewrite element contains a <em>apply</em>
attribute that references the rewrite function or rule by name. Along with the <em>apply</em>
attribute, a <em>to</em> attribute must be used. The <em>to</em> specifies
what part of the request or response to rewrite. The valid values for the <em>to</em>
attribute are:</dd>
+</dl>
+<ul>
+  <li>request.url</li>
+  <li>request.headers</li>
+  <li>request.cookies</li>
+  <li>request.body</li>
+  <li>response.headers</li>
+  <li>response.cookies</li>
+  <li>response.body</li>
+</ul><p>Below is an example of a snippet from the WebHDFS service definition</p>
+<pre><code class="xml">       &lt;route path=&quot;/webhdfs/v1/**?**&quot;&gt;
+            &lt;rewrite apply=&quot;WEBHDFS/webhdfs/inbound/namenode/file&quot;
to=&quot;request.url&quot;/&gt;
+            &lt;rewrite apply=&quot;WEBHDFS/webhdfs/outbound/namenode/headers&quot;
to=&quot;response.headers&quot;/&gt;
+        &lt;/route&gt;
+</code></pre>
+<dl><dt><strong>dispatch</strong></dt><dd>The dispatch
element can be used to plug-in a custom dispatch class. The interface for Dispatch can be
found in the module gateway-spi, org.apache.hadoop.gateway.dispatch.Dispatch.</dd>
+</dl><p>This element can be used at the service level (i.e. as a child of the
service tag) or at the route level. A dispatch specified at the route level takes precedence
over a dispatch specified at the service level. By default the dispatch used is org.apache.hadoop.gateway.dispatch.DefaultDispatch.</p><p>The
dispatch tag has four attributes that can be specified.</p><p><em>contributor-name</em>
: This attribute can be used to specify a deployment contributor to be invoked for a custom
dispatch.</p><p><em>classname</em> : This attribute can be used to
specify a custom dispatch class.</p><p><em>ha-contributor-name</em>
: This attribute can be used to specify a deployment contributor to be invoked for custom
HA dispatch functionality.</p><p><em>ha-classname</em> : This attribute
can be used to specify a custom dispatch class with HA functionality.</p><p>Only
one of contributor-name or classname should be specified and one of ha-contributor-name or
ha-classname should be specified.</
 p><p>If providing a custom dispatch, either a jar should be provided, see <a
href="#Class+Path">Class Path</a> or a <a href="#Maven+Module">Maven Module</a>
should be created.</p>
+<dl><dt><strong>policies</strong></dt><dd>This is a wrapper
tag for <em>policy</em> elements and can be a child of the <em>service</em>
tag or the <em>route</em> tag. Once again, just like with dispatch, the route
level policies defined override the ones at the service level.</dd>
+</dl><p>This element can contain one or more <em>policy</em> elements.
The order of the <em>policy</em> elements is important as that will be the order
of execution.</p>
+<dl><dt><strong>policy</strong></dt><dd>At this time
the policy element just has two attributes, <em>role</em> and <em>name</em>.
These are used to execute a deployment contributor by that role and name. Therefore new policies
must be added by using the deployment contributor mechanism.</dd>
+</dl><p>For example,</p>
+<pre><code class="xml">&lt;service role=&quot;FOO&quot; name=&quot;foo&quot;
version=&quot;1.0.0&quot;&gt;
+    &lt;policies&gt;
+        &lt;policy role=&quot;webappsec&quot;/&gt;
+        &lt;policy role=&quot;authentication&quot;/&gt;
+        &lt;policy role=&quot;rewrite&quot;/&gt;
+        &lt;policy role=&quot;identity-assertion&quot;/&gt;
+        &lt;policy role=&quot;authorization&quot;/&gt;
+    &lt;/policies&gt;
+    &lt;routes&gt;
+        &lt;route path=&quot;/foo/?**&quot;&gt;
+            &lt;rewrite apply=&quot;FOO/foo/inbound&quot; to=&quot;request.url&quot;/&gt;
+            &lt;policies&gt;
+                &lt;policy role=&quot;webappsec&quot;/&gt;
+                &lt;policy role=&quot;federation&quot;/&gt;
+                &lt;policy role=&quot;identity-assertion&quot;/&gt;
+                &lt;policy role=&quot;authorization&quot;/&gt;
+                &lt;policy role=&quot;rewrite&quot;/&gt;
+            &lt;/policies&gt;
+            &lt;dispatch contributor-name=&quot;http-client&quot; /&gt;
+        &lt;/route&gt;
+    &lt;/routes&gt;
+    &lt;dispatch contributor-name=&quot;custom-client&quot; ha-contributor-name=&quot;ha-client&quot;/&gt;
+&lt;/service&gt;
+</code></pre><h5><a id="rewrite.xml"></a>rewrite.xml</h5><p>The
rewrite.xml file that accompanies the service.xml file follows the same rules as described
in the section <a href="#Rewrite+Provider">Rewrite Provider</a>.</p><h4><a
id="Service+Definition+Directory+Structure"></a>Service Definition Directory Structure</h4><p>On
installation of the Knox gateway, the following directory structure can be found under ${GATEWAY_HOME}/data.
This is a mirror of the directories and files under the module gateway-service-definitions.</p>
+<pre><code>services
+    |______ service name
+                    |______ version
+                                |______service.xml
+                                |______rewrite.xml
+</code></pre><p>For example,</p>
+<pre><code>services
+    |______ webhdfs
+               |______ 2.4.0
+                         |______service.xml
+                         |______rewrite.xml
+</code></pre><p>To test out a new service, you can just add the appropriate
files (service.xml and rewrite.xml) in a directory under ${GATEWAY_HOME}/data/services. If
you want to make the service contribution to the Knox build, they files need to go in the
gateway-service-definitions module.</p><h4><a id="Service+Definition+Runtime+Behavior"></a>Service
Definition Runtime Behavior</h4><p>The runtime artifacts as well as the behavior
does not change whether the service is plugged in via the deployment descriptors or through
a service.xml file.</p><h4><a id="Custom+Dispatch+Dependency+Injection"></a>Custom
Dispatch Dependency Injection</h4><p>When writing a custom dispatch class, one
often needs configuration or gateway services. A lightweight dependency injection system is
used that can inject instances of classes or primitives available in the filter configuration&rsquo;s
init params or as a servlet context attribute.</p><p>Details of this can be found
in the module gateway-util-con
 figinjector and also an example use of it is in the class org.apache.hadoop.gateway.dispatch.DefaultDispatch.
Look at the following method for example:</p>
+<pre><code class="java"> @Configure
+   protected void setReplayBufferSize(@Default(&quot;8&quot;) int size) {
+      replayBufferSize = size;
+   }
 </code></pre><h3><a id="Providers"></a>Providers</h3>
 <pre><code class="java">public interface ProviderDeploymentContributor {
   String getRole();

Modified: knox/site/books/knox-0-6-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/user-guide.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/books/knox-0-6-0/user-guide.html (original)
+++ knox/site/books/knox-0-6-0/user-guide.html Mon May  4 18:52:17 2015
@@ -50,6 +50,7 @@
     <li><a href="#HBase">HBase</a></li>
     <li><a href="#Hive">Hive</a></li>
     <li><a href="#Yarn">Yarn</a></li>
+    <li><a href="#Storm">Storm</a></li>
   </ul></li>
   <li><a href="#Limitations">Limitations</a></li>
   <li><a href="#Troubleshooting">Troubleshooting</a></li>
@@ -2007,6 +2008,7 @@ dep/commons-codec-1.7.jar
   <li><a href="#HBase">HBase</a></li>
   <li><a href="#Hive">Hive</a></li>
   <li><a href="#Yarn">Yarn</a></li>
+  <li><a href="#Storm">Storm</a></li>
 </ul><h3><a id="Assumptions"></a>Assumptions</h3><p>This
document assumes a few things about your environment in order to simplify the examples.</p>
 <ul>
   <li>The JVM is executable as simply java.</li>
@@ -3343,7 +3345,7 @@ The contents of the state-killed.json fi
 
 
 curl -ikv -u guest:guest-password -H Content-Type:application/json -X PUT -T state-killed.json
&#39;https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}/state&#39;
-</code></pre><h4><a id="Storm"></a>Storm</h4><p>Storm
is a distributed realtime computation system. Storm exposes REST APIs for UI functionality
that can be used for retrieving metrics data and configuration information as well as management
operations such as starting or stopping topologies.</p><p>The docs for this can
be found here</p><p><a href="https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md">https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md</a></p><p>To
enable this functionality, a topology file needs to have the following configuration:</p>
+</code></pre><h3><a id="Storm"></a>Storm</h3><p>Storm
is a distributed realtime computation system. Storm exposes REST APIs for UI functionality
that can be used for retrieving metrics data and configuration information as well as management
operations such as starting or stopping topologies.</p><p>The docs for this can
be found here</p><p><a href="https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md">https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md</a></p><p>To
enable this functionality, a topology file needs to have the following configuration:</p>
 <pre><code>&lt;service&gt;
         &lt;role&gt;STORM&lt;/role&gt;
         &lt;url&gt;http://&lt;hostname&gt;:&lt;port&gt;&lt;/url&gt;
@@ -3377,7 +3379,7 @@ curl -ikv -u guest:guest-password -H Con
       <td><code>http://{storm-logviewer-host}:{storm-logviewer-port}</code>
</td>
     </tr>
   </tbody>
-</table><h4><a id="Storm+Examples+via+cURL"></a>Storm Examples via
cURL</h4><p>Some of the various calls that can be made and examples using curl
are listed below.</p>
+</table><h4><a id="Storm+Examples"></a>Storm Examples</h4><p>Some
of the various calls that can be made and examples using curl are listed below.</p>
 <pre><code># 0. Getting cluster configuration
 
 curl -ikv -u guest:guest-password -X GET &#39;https://localhost:8443/gateway/sandbox/storm/api/v1/cluster/configuration&#39;

Modified: knox/site/index.html
URL: http://svn.apache.org/viewvc/knox/site/index.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/index.html (original)
+++ knox/site/index.html Mon May  4 18:52:17 2015
@@ -1,5 +1,5 @@
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
       @import url("./css/site.css");
     </style>
     <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
-    <meta name="Date-Revision-yyyymmdd" content="20150501" />
+    <meta name="Date-Revision-yyyymmdd" content="20150504" />
     <meta http-equiv="Content-Language" content="en" />
                                                     
 <script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
                         <a href="https://cwiki.apache.org/confluence/display/KNOX/Index"
class="externalLink" title="Wiki">Wiki</a>
               
                     
-                &nbsp;| <span id="publishDate">Last Published: 2015-05-01</span>
+                &nbsp;| <span id="publishDate">Last Published: 2015-05-04</span>
               &nbsp;| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
             </div>
       <div class="clear">

Modified: knox/site/issue-tracking.html
URL: http://svn.apache.org/viewvc/knox/site/issue-tracking.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/issue-tracking.html (original)
+++ knox/site/issue-tracking.html Mon May  4 18:52:17 2015
@@ -1,5 +1,5 @@
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
       @import url("./css/site.css");
     </style>
     <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
-    <meta name="Date-Revision-yyyymmdd" content="20150501" />
+    <meta name="Date-Revision-yyyymmdd" content="20150504" />
     <meta http-equiv="Content-Language" content="en" />
                                                     
 <script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
                         <a href="https://cwiki.apache.org/confluence/display/KNOX/Index"
class="externalLink" title="Wiki">Wiki</a>
               
                     
-                &nbsp;| <span id="publishDate">Last Published: 2015-05-01</span>
+                &nbsp;| <span id="publishDate">Last Published: 2015-05-04</span>
               &nbsp;| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
             </div>
       <div class="clear">

Modified: knox/site/license.html
URL: http://svn.apache.org/viewvc/knox/site/license.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/license.html (original)
+++ knox/site/license.html Mon May  4 18:52:17 2015
@@ -1,5 +1,5 @@
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
       @import url("./css/site.css");
     </style>
     <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
-    <meta name="Date-Revision-yyyymmdd" content="20150501" />
+    <meta name="Date-Revision-yyyymmdd" content="20150504" />
     <meta http-equiv="Content-Language" content="en" />
                                                     
 <script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
                         <a href="https://cwiki.apache.org/confluence/display/KNOX/Index"
class="externalLink" title="Wiki">Wiki</a>
               
                     
-                &nbsp;| <span id="publishDate">Last Published: 2015-05-01</span>
+                &nbsp;| <span id="publishDate">Last Published: 2015-05-04</span>
               &nbsp;| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
             </div>
       <div class="clear">

Modified: knox/site/mail-lists.html
URL: http://svn.apache.org/viewvc/knox/site/mail-lists.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/mail-lists.html (original)
+++ knox/site/mail-lists.html Mon May  4 18:52:17 2015
@@ -1,5 +1,5 @@
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
       @import url("./css/site.css");
     </style>
     <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
-    <meta name="Date-Revision-yyyymmdd" content="20150501" />
+    <meta name="Date-Revision-yyyymmdd" content="20150504" />
     <meta http-equiv="Content-Language" content="en" />
                                                     
 <script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
                         <a href="https://cwiki.apache.org/confluence/display/KNOX/Index"
class="externalLink" title="Wiki">Wiki</a>
               
                     
-                &nbsp;| <span id="publishDate">Last Published: 2015-05-01</span>
+                &nbsp;| <span id="publishDate">Last Published: 2015-05-04</span>
               &nbsp;| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
             </div>
       <div class="clear">

Modified: knox/site/project-info.html
URL: http://svn.apache.org/viewvc/knox/site/project-info.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/project-info.html (original)
+++ knox/site/project-info.html Mon May  4 18:52:17 2015
@@ -1,5 +1,5 @@
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
       @import url("./css/site.css");
     </style>
     <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
-    <meta name="Date-Revision-yyyymmdd" content="20150501" />
+    <meta name="Date-Revision-yyyymmdd" content="20150504" />
     <meta http-equiv="Content-Language" content="en" />
                                                     
 <script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
                         <a href="https://cwiki.apache.org/confluence/display/KNOX/Index"
class="externalLink" title="Wiki">Wiki</a>
               
                     
-                &nbsp;| <span id="publishDate">Last Published: 2015-05-01</span>
+                &nbsp;| <span id="publishDate">Last Published: 2015-05-04</span>
               &nbsp;| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
             </div>
       <div class="clear">

Modified: knox/site/team-list.html
URL: http://svn.apache.org/viewvc/knox/site/team-list.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/team-list.html (original)
+++ knox/site/team-list.html Mon May  4 18:52:17 2015
@@ -1,5 +1,5 @@
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
       @import url("./css/site.css");
     </style>
     <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
-    <meta name="Date-Revision-yyyymmdd" content="20150501" />
+    <meta name="Date-Revision-yyyymmdd" content="20150504" />
     <meta http-equiv="Content-Language" content="en" />
                                                     
 <script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
                         <a href="https://cwiki.apache.org/confluence/display/KNOX/Index"
class="externalLink" title="Wiki">Wiki</a>
               
                     
-                &nbsp;| <span id="publishDate">Last Published: 2015-05-01</span>
+                &nbsp;| <span id="publishDate">Last Published: 2015-05-04</span>
               &nbsp;| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
             </div>
       <div class="clear">

Modified: knox/trunk/books/0.6.0/book.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/book.md?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/book.md (original)
+++ knox/trunk/books/0.6.0/book.md Mon May  4 18:52:17 2015
@@ -55,6 +55,7 @@
     * #[HBase]
     * #[Hive]
     * #[Yarn]
+    * #[Storm]
 * #[Limitations]
 * #[Troubleshooting]
 * #[Export Controls]

Modified: knox/trunk/books/0.6.0/book_service-details.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/book_service-details.md?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/book_service-details.md (original)
+++ knox/trunk/books/0.6.0/book_service-details.md Mon May  4 18:52:17 2015
@@ -35,6 +35,7 @@ These are the current Hadoop services wi
 * #[HBase]
 * #[Hive]
 * #[Yarn]
+* #[Storm]
 
 ### Assumptions
 

Modified: knox/trunk/books/0.6.0/dev-guide/book.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/dev-guide/book.md?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/dev-guide/book.md (original)
+++ knox/trunk/books/0.6.0/dev-guide/book.md Mon May  4 18:52:17 2015
@@ -116,11 +116,11 @@ This is also true of the other -release
 | gateway-provider-security-shiro                | Shiro authentiation integration.     
                    |
 | gateway-provider-security-webappsec            | Filters to prevent common webapp security
issues.         |
 | gateway-service-as                             | The implementation of the Access service
POC.             |
+| gateway-service-definitions                    | The implementation of the Service definition
and rewrite files. |
 | gateway-service-hbase                          | The implementation of the HBase service.
                 |
 | gateway-service-hive                           | The implementation of the Hive service.
                  |
 | gateway-service-oozie                          | The implementation of the Oozie service.
                 |
 | gateway-service-tgs                            | The implementation of the Ticket Granting
service POC.    |
-| gateway-service-webhcat                        | The implementation of the WebHCat service.
               |
 | gateway-service-webhdfs                        | The implementation of the WebHdfs service.
               |
 | gateway-server                                 | The implementation of the Knox gateway
server.            |
 | gateway-shell                                  | The implementation of the Knox Groovy
shell.              |
@@ -805,6 +805,202 @@ allRules.addRules( newRules );
 </project>
 ```
 
+#### Service Definition Files ####
+
+As of release 0.6.0, the gateway now also supports a declarative way of plugging-in a new
Service. A Service can be defined with a
+combination of two files, these are:
+
+    service.xml
+    rewrite.xml
+
+The rewrite.xml file contains the rewrite rules as defined in other sections of this guide,
and the service.xml file contains
+the various routes (paths) to be provided by the Service and the rewrite rule bindings to
those paths. This will be described in further detail
+in this section.
+
+While the service.xml file is absolutely required, the rewrite.xml file in theory is optional
(though it is highly unlikely that no rewrite rules
+are needed).
+
+To add a new service, simply add a service.xml and rewrite.xml file in an appropriate directory
(see #[Service Definition Directory Structure])
+in the module gateway-service-definitions to make the new service part of the Knox build.
+
+##### service.xml #####
+
+Below is a sample of a very simple service.xml file, taking the same weather api example.
+
+```xml
+<service role="WEATHER" name="weather" version="0.1.0">
+    <routes>
+        <route path="/weather/**?**"/>
+    </routes>
+</service>
+
+```
+
+**service**
+: The root tag is 'service' that has the three required attributes: *role*, *name* and *version*.
These three values disambiguate this
+service definition from others. To ensure the exact same service definition is being used
in a topology file, all values should be specified,
+
+```xml
+<topology>
+    <gateway>
+        ....
+    </gateway>
+    <service>
+        <role>WEATHER</role>
+        <name>weather</name>
+        <version>0.1.0</version>
+        <url>http://api.openweathermap.org/data</url>
+    </service>
+    ....
+</topology>
+```
+
+If only *role* is specified in the topology file (the only required element other than *url*)
then the first service definition of that role found
+will be used with the highest version of that role and name. Similarly if only the *version*
is omitted from the topology specification of the service,
+the service definition of the highest version will be used. It is therefore important to
specify a version for a service if it is desired that
+a topology be locked down to a specific version of a service.
+
+
+**routes**
+: Wrapper element for one or more routes.
+
+**route**
+: A route specifies the *path* that the service is routing as well as any rewrite bindings
or policy bindings. Another child element that may be used here
+is a *dispatch* element.
+
+**rewrite**
+: A rewrite rule or function that is to be applied to the path. A rewrite element contains
a *apply* attribute that references the rewrite function or rule
+by name. Along with the *apply* attribute, a *to* attribute must be used. The *to* specifies
what part of the request or response to rewrite.
+The valid values for the *to* attribute are:
+
+- request.url
+- request.headers
+- request.cookies
+- request.body
+- response.headers
+- response.cookies
+- response.body
+
+Below is an example of a snippet from the WebHDFS service definition
+
+```xml
+       <route path="/webhdfs/v1/**?**">
+            <rewrite apply="WEBHDFS/webhdfs/inbound/namenode/file" to="request.url"/>
+            <rewrite apply="WEBHDFS/webhdfs/outbound/namenode/headers" to="response.headers"/>
+        </route>
+```
+
+**dispatch**
+: The dispatch element can be used to plug-in a custom dispatch class. The interface for
Dispatch can be found in the module
+gateway-spi, org.apache.hadoop.gateway.dispatch.Dispatch.
+
+This element can be used at the service level (i.e. as a child of the service tag) or at
the route level. A dispatch specified
+at the route level takes precedence over a dispatch specified at the service level. By default
the dispatch used is
+org.apache.hadoop.gateway.dispatch.DefaultDispatch.
+
+The dispatch tag has four attributes that can be specified.
+
+*contributor-name* : This attribute can be used to specify a deployment contributor to be
invoked for a custom dispatch.
+
+*classname* : This attribute can be used to specify a custom dispatch class.
+
+*ha-contributor-name* : This attribute can be used to specify a deployment contributor to
be invoked for custom HA dispatch functionality.
+
+*ha-classname* : This attribute can be used to specify a custom dispatch class with HA functionality.
+
+Only one of contributor-name or classname should be specified and one of ha-contributor-name
or ha-classname should be specified.
+
+If providing a custom dispatch, either a jar should be provided, see #[Class Path] or a #[Maven
Module] should be created.
+
+**policies**
+: This is a wrapper tag for *policy* elements and can be a child of the *service* tag or
the *route* tag. Once again, just like with
+dispatch, the route level policies defined override the ones at the service level.
+
+This element can contain one or more *policy* elements. The order of the *policy* elements
is important as that will be the
+order of execution.
+
+**policy**
+: At this time the policy element just has two attributes, *role* and *name*. These are used
to execute a deployment contributor
+by that role and name. Therefore new policies must be added by using the deployment contributor
mechanism.
+
+For example,
+
+```xml
+<service role="FOO" name="foo" version="1.0.0">
+    <policies>
+        <policy role="webappsec"/>
+        <policy role="authentication"/>
+        <policy role="rewrite"/>
+        <policy role="identity-assertion"/>
+        <policy role="authorization"/>
+    </policies>
+    <routes>
+        <route path="/foo/?**">
+            <rewrite apply="FOO/foo/inbound" to="request.url"/>
+            <policies>
+                <policy role="webappsec"/>
+                <policy role="federation"/>
+                <policy role="identity-assertion"/>
+                <policy role="authorization"/>
+                <policy role="rewrite"/>
+            </policies>
+            <dispatch contributor-name="http-client" />
+        </route>
+    </routes>
+    <dispatch contributor-name="custom-client" ha-contributor-name="ha-client"/>
+</service>
+```
+
+##### rewrite.xml #####
+
+The rewrite.xml file that accompanies the service.xml file follows the same rules as described
in the section #[Rewrite Provider].
+
+#### Service Definition Directory Structure ####
+
+On installation of the Knox gateway, the following directory structure can be found under
${GATEWAY_HOME}/data. This
+is a mirror of the directories and files under the module gateway-service-definitions.
+
+    services
+        |______ service name
+                        |______ version
+                                    |______service.xml
+                                    |______rewrite.xml
+
+
+For example,
+
+    services
+        |______ webhdfs
+                   |______ 2.4.0
+                             |______service.xml
+                             |______rewrite.xml
+
+
+To test out a new service, you can just add the appropriate files (service.xml and rewrite.xml)
in a directory under
+${GATEWAY_HOME}/data/services. If you want to make the service contribution to the Knox build,
they files need to go
+in the gateway-service-definitions module.
+
+#### Service Definition Runtime Behavior ####
+
+The runtime artifacts as well as the behavior does not change whether the service is plugged
in via the deployment
+descriptors or through a service.xml file.
+
+#### Custom Dispatch Dependency Injection ####
+
+When writing a custom dispatch class, one often needs configuration or gateway services.
A lightweight dependency injection
+system is used that can inject instances of classes or primitives available in the filter
configuration's init params or
+as a servlet context attribute.
+
+Details of this can be found in the module gateway-util-configinjector and also an example
use of it is in the class
+org.apache.hadoop.gateway.dispatch.DefaultDispatch. Look at the following method for example:
+
+```java
+ @Configure
+   protected void setReplayBufferSize(@Default("8") int size) {
+      replayBufferSize = size;
+   }
+```
+
 ### Providers ###
 
 ```java

Modified: knox/trunk/books/0.6.0/service_storm.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/service_storm.md?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/service_storm.md (original)
+++ knox/trunk/books/0.6.0/service_storm.md Mon May  4 18:52:17 2015
@@ -15,7 +15,7 @@
    limitations under the License.
 --->
 
-#### Storm ###
+### Storm ###
 
 Storm is a distributed realtime computation system. Storm exposes REST APIs for UI functionality
that can be used for
 retrieving metrics data and configuration information as well as management operations such
as starting or stopping topologies.
@@ -60,7 +60,7 @@ For the log viewer the mapping is as fol
 | Cluster | `http://{storm-logviewer-host}:{storm-logviewer-port}`                      
               |
 
 
-#### Storm Examples via cURL
+#### Storm Examples
 
 Some of the various calls that can be made and examples using curl are listed below.
 



Mime
View raw message