knox-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lmc...@apache.org
Subject [1/2] git commit: Updated to include start instructions with the master secret and descriptions of the security artifacts and related management.
Date Sat, 16 Mar 2013 14:39:11 GMT
Updated to include start instructions with the master secret and descriptions of the security
artifacts and related management.

Project: http://git-wip-us.apache.org/repos/asf/incubator-knox/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-knox/commit/b7f3cb4b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-knox/tree/b7f3cb4b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-knox/diff/b7f3cb4b

Branch: refs/heads/master
Commit: b7f3cb4b4d6c3c22902bf485eed787fdf84b5ad8
Parents: 282261f
Author: Larry McCay <lmccay@hortonworks.com>
Authored: Fri Mar 15 17:35:56 2013 -0400
Committer: Larry McCay <lmccay@hortonworks.com>
Committed: Fri Mar 15 17:35:56 2013 -0400

----------------------------------------------------------------------
 gateway-release/readme.md |   81 +++++++++++++++++++++++++++++++++++++++-
 1 files changed, 80 insertions(+), 1 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-knox/blob/b7f3cb4b/gateway-release/readme.md
----------------------------------------------------------------------
diff --git a/gateway-release/readme.md b/gateway-release/readme.md
index 9432630..5532768 100644
--- a/gateway-release/readme.md
+++ b/gateway-release/readme.md
@@ -142,7 +142,11 @@ diagnostic enough.
 
 4. Start the Gateway server
      java -jar bin/gateway-server-0.2.0-SNAPSHOT.jar
-   Take note of the port identified in the logging output as you will need this for accessing
the gateway.
+   a. Take note of the port identified in the logging output as you will need this for accessing
the gateway.
+   b. The server will prompt you for the master secret (password). This secret is used to
secure artifacts used 
+      to secure artifacts used by the gateway server for things like SSL, credential/password
aliasing. This secret
+      will have to be entered at startup unless you choose to persist it. Remember this secret
and keep it safe. It
+      represents the keys to the kingdom. See the Persisting the Master section for more
information.
 
 5. Configure the Gateway with the topology of your Hadoop cluster
    a. Edit the file {GATEWAY_HOME}/deployments/sample.xml
@@ -184,6 +188,81 @@ diagnostic enough.
    http://hadoop.apache.org/docs/r1.0.4/webhdfs.html
      and
    http://people.apache.org/~thejas/templeton_doc_v1/
+   
+------------------------------------------------------------------------------
+##Persisting the Master
+------------------------------------------------------------------------------
+The master secret is required to start the server. This secret is used to access secured
artifacts by the gateway 
+instance. Keystore, trust stores and credential stores are all protected with the master
secret.
+
+You may persist the master secret by supplying the *-persist-master* switch at startup. This
will result in a
+warning indicating that persisting the secret is less secure than providing it at startup.
We do make some provisions in
+order to protect the persisted password. 
+
+It is encrypted with AES 128 bit encryption and where possible the file permissions are set
to only be accessable by 
+the user that the gateway is running as. 
+
+After persisting the secret, ensure that the file at config/security/master has the appropriate
permissions set for your 
+environment. This is probably the most important layer of defense for master secret. Do not
assume that the encryption if
+sufficient protection.
+
+A specific user should be created to run the gateway this will protect a persisted master
file.
+
+------------------------------------------------------------------------------
+##Management of Security Artifacts
+------------------------------------------------------------------------------
+There are a number of artifacts that are used by the gateway in ensuring the security of
wire level communications, 
+access to protected resources and the encryption of sensitive data. These artifacts can be
managed from outside of
+the gateway instances or generated and populated by the gateway instance itself.
+
+The following is a description of how this is coordinated with both standalone (development,
demo, etc) gateway 
+instances and instances as part of a cluster of gateways in mind.
+
+Upon start of the gateway server we:
+
+1. Look for an identity store at conf/security/keystores/gateway.jks. The identity store
contains the certificate 
+   and private key used to represent the identity of the server for SSL connections and signtature
creation.
+	a. If there is no identity store we create one and generate a self-signed certificate for
use in standalone/demo 
+   	   mode. The certificate is stored with an alias of gateway-identity.
+   	b. If there is an identity store found than we ensure that it can be loaded using the
provided master secret and
+   	   that there is an alias with called gateway-identity.
+2. Look for a credential store at conf/security/keystores/__gateway-credentials.jceks. This
credential store is used
+   to store secrets/passwords that are used by the gateway. For instance, this is where the
passphrase for accessing
+   the gateway-identity certificate is kept.
+   a. If there is no credential store found then we create one and populate it with a generated
passphrase for the alias
+      gateway-identity-passphrase. This is coordinated with the population of the self-signed
cert into the identity-store.
+   b. If a credential store is found then we ensure that it can be loaded using the provided
master secret and that the 
+      expected aliases have been populated with secrets.
+      
+Upon deployment of a Hadoop cluster topology within the gateway we:
+
+1. Look for a credential store for the topology. For instance, we have a sample topology
that gets deployed out of the box.
+   We look for conf/security/keystores/sample-credentials.jceks. This topology specific credential
store is used for storing
+   secrets/passwords that are used for encrypting sensitive data with topology specific keys.
+   a. If no credential store is found for the topology being deployed then one is created
for it. Population of the aliases
+      is delegated to the configured providers within the system that will require the use
of a secret for a particular
+      task. They may programmatically set the value of the secret or choose to have the value
for the specified alias
+      generated through the AliasService..
+   b. If a credential store is found then we ensure that it can be loaded with the provided
master secret and the confgured
+      providers have the opportunity to ensure that the aliases are populated and if not
to populate them.
+ 
+ By leveraging the algorithm described above we can provide a window of opportunity for management
of these artifacts in a
+ number of ways.
+ 
+ 1. Using a single gateway instance as a master instance the artifacts can be generated or
placed into the expected location
+    and then replicated across all of the slave instances before startup.
+ 2. Using an NFS mount as a central location for the artifacts would provide a single source
of truth without the need to 
+    replicate them over the network. Of course, NFS mounts have their own challenges.
+
+Summary of Secrets to be Managed:
+
+1. Master secret - the same for all gateway instances in a cluster of gateways
+2. All security related artifacts are protected with the master secret
+3. Secrets used by the gateway itself are stored within the gateway credential store and
are the same across all gateway instances in the cluster of gateways
+4. Secrets used by providers within cluster topologies are stored in topology specific credential
stores and are the same for the same topology across the cluster of gateway instances. However,
they are specific to the topology - so secrets for one hadoop cluster are different from those
of another. This allows for failover from one gateway instance to another even when encryption
is being used while not allowing the compromise of one encryption key to expose the data for
all clusters.
+
+NOTE: the SSL certificate will need special consideration depending on the type of certificate.
Wildcard certs may be able to be shared across all gateway instances in a cluster. When certs
are dedicated to specific machines the gateway identity store will not be able to be blindly
replicated as hostname verification problems will ensue. Obviously, truststores will need
to be taken into account as well.
+
 
 ------------------------------------------------------------------------------
 ##Mapping Gateway URLs to Hadoop cluster URLs


Mime
View raw message