fluo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [fluo-muchos] keith-ratcliffe commented on a change in pull request #274: Updates to enable launch of EC2 instances via user-defined templates
Date Wed, 04 Sep 2019 22:12:28 GMT
keith-ratcliffe commented on a change in pull request #274: Updates to enable launch of EC2
instances via user-defined templates
URL: https://github.com/apache/fluo-muchos/pull/274#discussion_r320997801
 
 

 ##########
 File path: conf/templates/README.md
 ##########
 @@ -0,0 +1,188 @@
+## Muchos EC2 Cluster Templates
+
+Cluster templates are intended to provide greater flexibility, if needed,
+with respect to instance type selection and launch configuration for
+your EC2 hosts. For example, cluster templates may be ideal for use
+cases that require distinct, per-host launch configurations, and
+for use cases that require hosts to have persistent, EBS-backed data
+volumes (rather than ephemeral volumes, the muchos default for EC2
+clusters)
+
+If you are already familiar with muchos and with the basics of EC2
+launch requests, then creating your own launch templates will be simple
+and straightforward.
+
+Please follow the guidance provided here to ensure compatibility between
+your custom templates and muchos automation
+
+## Configuration
+
+### Select a cluster template in *muchos.props*
+```
+[ec2]
+...
+cluster_template = example
+...
+```
+The configured value must match the name of a subdirectory under
+`conf/templates`
+
+```
+~$ ls -1a fluo-muchos/conf/templates/example
+.
+..
+devices
+metrics.json
+namenode.json
+resourcemanager.json
+zookeeper.json
+worker.json
+```
+The subdirectory will contain one or more user-defined EC2 launch
+templates (*\*.json*) for your various host types, and it will
+include a *devices* file specifying the desired mount points for all
+data volumes (excluding root volumes, as they are mounted
+automatically)
+
+### Defining EC2 launch templates and device mounts for your hosts
+
+#### Launch Templates: *{service-name}.json* files
+
+Each JSON file represents a standard EC2 launch request, and each file
+name must match one of the predefined muchos service names, as
+defined in the **nodes** section of *muchos.props*. E.g.,
+```
+... 
+[nodes]
+leader1 = namenode,resourcemanager,accumulomaster
+leader2 = metrics,zookeeper
+worker1 = worker
+worker2 = worker
+worker3 = worker
+worker4 = worker
+```
+In template mode, the first service listed for a given host denotes the
+template to be selected for its launch.
+
+Based on the example given above:
+* **leader1** selects **namenode.json**
 
 Review comment:
   > `leader1` also list a `resourcemanager` in the example, so what happens if `namenode.json`
and `resourcemanager.json ` exists?
   
   In that case, `resourcemanager.json` is simply ignored, since `namenode` is the first service
listed for that host. Good point though...I realize now that the example I provided should
have omitted all but `namenode.json`, `metrics.json`, and `worker.json`, since those are the
only relevant files, based on the given the `nodes` config
   
   In retrospect, perhaps it would be more intuitive to have **{hostname}.json** files instead,
at for non-worker node types, and then for the worker template (which presumably would be
common to _all_ worker nodes) either require "worker.json" to exist, or have the user explicitly
identify the "worker_template_name" in muchos.props

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message