knox-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From m...@apache.org
Subject svn commit: r1809165 [12/12] - in /knox: site/books/knox-0-14-0/ trunk/books/0.14.0/ trunk/books/0.14.0/dev-guide/
Date Thu, 21 Sep 2017 13:57:58 GMT
Added: knox/trunk/books/0.14.0/service_yarn.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.14.0/service_yarn.md?rev=1809165&view=auto
==============================================================================
--- knox/trunk/books/0.14.0/service_yarn.md (added)
+++ knox/trunk/books/0.14.0/service_yarn.md Thu Sep 21 13:57:57 2017
@@ -0,0 +1,124 @@
+<!---
+   Licensed to the Apache Software Foundation (ASF) under one or more
+   contributor license agreements.  See the NOTICE file distributed with
+   this work for additional information regarding copyright ownership.
+   The ASF licenses this file to You under the Apache License, Version 2.0
+   (the "License"); you may not use this file except in compliance with
+   the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+--->
+
+### Yarn ###
+
+Knox provides gateway functionality for the REST APIs of the ResourceManager. The ResourceManager
REST APIs allow the
+user to get information about the cluster - status on the cluster, metrics on the cluster,
scheduler information,
+information about nodes in the cluster, and information about applications on the cluster.
Also as of Hadoop version
+2.5.0, the user can submit a new application as well as kill it (or get state) using the
'Writable' APIs.
+
+The docs for this can be found here
+
+http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html
+
+To enable this functionality, a topology file needs to have the following configuration:
+
+    <service>
+        <role>RESOURCEMANAGER</role>
+        <url>http://<hostname>:<port>/ws</url>
+    </service>
+
+The default resource manager http port is 8088. If it is configured to some other port, that
configuration can be
+found in `yarn-site.xml` under the property `yarn.resourcemanager.webapp.address`.
+
+#### Yarn URL Mapping ####
+
+For Yarn URLs, the mapping of Knox Gateway accessible URLs to direct Yarn URLs is the following.
+
+| ------- | -------------------------------------------------------------------------------------
|
+| Gateway | `https://{gateway-host}:{gateway-port}/{gateway-path}/{cluster-name}/resourcemanager`
|
+| Cluster | `http://{yarn-host}:{yarn-port}/ws}`                                      |
+
+
+#### Yarn Examples via cURL
+
+Some of the various calls that can be made and examples using curl are listed below.
+
+    # 0. Getting cluster info
+    
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster'
+    
+    # 1. Getting cluster metrics
+    
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/metrics'
+    
+    To get the same information in an xml format
+    
+    curl -ikv -u guest:guest-password -H Accept:application/xml -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/metrics'
+    
+    # 2. Getting scheduler information
+    
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/scheduler'
+    
+    # 3. Getting all the applications listed and their information
+    
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps'
+    
+    # 4. Getting applications statistics
+    
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/appstatistics'
+    
+    Also query params can be used as below to filter the results
+    
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/appstatistics?states=accepted,running,finished&applicationTypes=mapreduce'
+    
+    # 5. To get a specific application (please note, replace the application id with a real
value)
+    
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}'
+    
+    # 6. To get the attempts made for a particular application
+    
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}/appattempts'
+    
+    # 7. To get information about the various nodes
+    
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/nodes'
+    
+    Also to get a specific node, use an id obtained in the response from above (the node
id is scrambled) and issue the following
+    
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/nodes/{node_id}'
+    
+    # 8. To create a new Application
+    
+    curl -ikv -u guest:guest-password -X POST 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/new-application'
+    
+    An application id is returned from the request above and this can be used to submit an
application.
+    
+    # 9. To submit an application, put together a request containing the application id received
in the above response (please refer to Yarn REST
+    API documentation).
+    
+    curl -ikv -u guest:guest-password -T request.json -H Content-Type:application/json -X
POST 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps'
+    
+    Here the request is saved in a file called request.json
+    
+    #10. To get application state
+    
+    curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}/state'
+    
+    curl -ikv -u guest:guest-password -H Content-Type:application/json -X PUT -T state-killed.json
'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/application_1409008107556_0007/state'
+    
+    # 11. To kill an application that is running issue the below command with the application
id of the application that is to be killed.
+    The contents of the state-killed.json file are :
+    
+    {
+      "state":"KILLED"
+    }
+    
+    
+    curl -ikv -u guest:guest-password -H Content-Type:application/json -X PUT -T state-killed.json
'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}/state'
+

Added: knox/trunk/books/0.14.0/websocket-support.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.14.0/websocket-support.md?rev=1809165&view=auto
==============================================================================
--- knox/trunk/books/0.14.0/websocket-support.md (added)
+++ knox/trunk/books/0.14.0/websocket-support.md Thu Sep 21 13:57:57 2017
@@ -0,0 +1,76 @@
+<!--
+   Licensed to the Apache Software Foundation (ASF) under one or more
+   contributor license agreements.  See the NOTICE file distributed with
+   this work for additional information regarding copyright ownership.
+   The ASF licenses this file to You under the Apache License, Version 2.0
+   (the "License"); you may not use this file except in compliance with
+   the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+-->
+
+## Websocket Support ##
+
+### Introduction
+
+Websocket is a communication protocol that allows full duplex communication over a single
TCP connection.
+Knox provides out-of-the-box support for websocket protocol, currently only text messages
are supported.
+
+### Configuration ###
+
+By default websocket functionality is disabled, it can be easily enabled by changing the
'gateway.websocket.feature.enabled' property to 'true' in <KNOX-HOME>/conf/gateway-site.xml
file.  
+
+      <property>
+          <name>gateway.websocket.feature.enabled</name>
+          <value>true</value>
+          <description>Enable/Disable websocket feature.</description>
+      </property>
+
+Service and rewrite rules need to changed accordingly to match the appropriate websocket
context.
+
+### Example ###
+
+In the following sample configuration we assume that the backend websocket URL is ws://myhost:9999/ws.
And 'gateway.websocket.feature.enabled' property is set to 'true' as shown above.
+
+#### rewrite ####
+
+Example code snippet from <KNOX-HOME>/data/services/{myservice}/{version}/rewrite.xml
where myservice = websocket and version = 0.6.0
+
+      <rules>
+        <rule dir="IN" name="WEBSOCKET/ws/inbound" pattern="*://*:*/**/ws">
+          <rewrite template="{$serviceUrl[WEBSOCKET]}/ws"/>
+        </rule>
+      </rules>
+
+#### service ####
+
+Example code snippet from <KNOX-HOME>/data/services/{myservice}/{version}/service.xml
where myservice = websocket and version = 0.6.0
+
+      <service role="WEBSOCKET" name="websocket" version="0.6.0">
+        <policies>
+              <policy role="webappsec"/>
+              <policy role="authentication" name="Anonymous"/>
+              <policy role="rewrite"/>
+              <policy role="authorization"/>
+        </policies>
+        <routes>
+          <route path="/ws">
+              <rewrite apply="WEBSOCKET/ws/inbound" to="request.url"/>
+          </route>
+        </routes>
+      </service>
+
+#### topology ####
+
+Finally, update the topology file at <KNOX-HOME>/conf/{topology}.xml  with the backend
service url
+
+      <service>
+          <role>WEBSOCKET</role>
+          <url>ws://myhost:9999/ws</url>
+      </service>

Added: knox/trunk/books/0.14.0/x-forwarded-headers.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.14.0/x-forwarded-headers.md?rev=1809165&view=auto
==============================================================================
--- knox/trunk/books/0.14.0/x-forwarded-headers.md (added)
+++ knox/trunk/books/0.14.0/x-forwarded-headers.md Thu Sep 21 13:57:57 2017
@@ -0,0 +1,76 @@
+<!---
+   Licensed to the Apache Software Foundation (ASF) under one or more
+   contributor license agreements.  See the NOTICE file distributed with
+   this work for additional information regarding copyright ownership.
+   The ASF licenses this file to You under the Apache License, Version 2.0
+   (the "License"); you may not use this file except in compliance with
+   the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+--->
+
+### X-Forwarded-* Headers Support ###
+Out-of-the-box Knox provides support for some X-Forwarded-* headers through the use of a
Servlet Filter. Specifically the
+headers handled/populated by Knox are:
+
+* X-Forwarded-For
+* X-Forwarded-Proto
+* X-Forwarded-Port
+* X-Forwarded-Host
+* X-Forwarded-Server
+* X-Forwarded-Context
+
+If this functionality can be turned off by a configuration setting in the file gateway-site.xml
and redeploying the
+necessary topology/topologies.
+
+The setting is (under the 'configuration' tag) :
+
+    <property>
+        <name>gateway.xforwarded.enabled</name>
+        <value>false</value>
+    </property>
+
+If this setting is absent, the default behavior is that the X-Forwarded-* header support
is on or in other words,
+'gateway.xforwarded.enabled' is set to 'true' by default.
+
+
+#### Header population ####
+
+The following are the various rules for population of these headers:
+
+##### X-Forwarded-For #####
+
+This header represents a list of client IP addresses. If the header is already present Knox
adds a comma separated value
+to the list. The value added is the client's IP address as Knox sees it. This value is added
to the end of the list.
+
+##### X-Forwarded-Proto #####
+
+The protocol used in the client request. If this header is passed into Knox its value is
maintained, otherwise Knox will
+populate the header with the value 'https' if the request is a secure one or 'http' otherwise.
+
+##### X-Forwarded-Port #####
+
+The port used in the client request. If this header is passed into Knox its value is maintained,
otherwise Knox will
+populate the header with the value of the port that the request was made coming into Knox.
+
+##### X-Forwarded-Host #####
+
+Represents the original host requested by the client in the Host HTTP request header. The
value passed into Knox is maintained
+by Knox. If no value is present, Knox populates the header with the value of the HTTP Host
header.
+
+##### X-Forwarded-Server #####
+
+The hostname of the server Knox is running on.
+
+##### X-Forwarded-Context #####
+
+This header value contains the context path of the request to Knox.
+
+
+



Mime
View raw message