helix-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ka...@apache.org
Subject [10/24] [HELIX-348] Simplify website layout
Date Thu, 02 Jan 2014 21:52:55 GMT
http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/markdown/tutorial_admin.md
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/markdown/tutorial_admin.md b/website/0.6.1-incubating/src/site/markdown/tutorial_admin.md
new file mode 100644
index 0000000..a06b868
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/markdown/tutorial_admin.md
@@ -0,0 +1,166 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Helix Tutorial: Admin Operations
+--------------------------------
+
+Helix provides interfaces for the operator to administer the cluster.  For convenience, there is a command line interface as well as a REST interface.
+
+### Helix Admin Operations
+
+First, make sure you get to the command-line tool, or include it in your shell PATH.
+
+```
+cd helix/helix-core/target/helix-core-pkg/bin
+```
+
+Get help
+
+```
+./helix-admin.sh --help
+```
+
+All other commands have this form:
+
+```
+./helix-admin.sh --zkSvr <ZookeeperServerAddress (Required)> <command> <parameters>
+```
+
+### Commands
+
+Add a new cluster
+
+```
+--addCluster <clusterName>
+```
+
+Add a new Instance to a cluster
+
+```
+--addNode <clusterName> <InstanceAddress (host:port)>
+```
+
+Add a State model to a cluster
+
+```
+--addStateModelDef <clusterName> <filename>
+```
+
+Add a resource to a cluster
+
+```
+--addResource <clusterName> <resourceName> <partitionNum> <stateModelRef> <mode (AUTO_REBALANCE|AUTO|CUSTOMIZED)>
+```
+
+Upload an IdealState (Partition to Node Mapping)
+
+```
+--addIdealState <clusterName> <resourceName> <filename>
+```
+
+Delete a cluster
+
+```
+--dropCluster <clusterName>
+```
+
+Delete a resource (drop an existing resource from a cluster)
+
+```
+--dropResource <clusterName> <resourceName>
+```
+
+Drop an existing instance from a cluster
+
+```
+--dropNode <clusterName> <InstanceAddress (host:port)>
+```
+
+Enable/disable the entire cluster. This will pause the controller, which means no transitions will be triggered, but the existing nodes in the cluster continue to function without any management by the controller.
+
+```
+--enableCluster <clusterName> <true/false>
+```
+
+Enable/disable an instance. This is useful to take a node out of the cluster for maintenance/upgrade.
+
+```
+--enableInstance <clusterName> <InstanceName> <true/false>
+```
+
+Enable/disable a partition
+
+```
+--enablePartition <clusterName> <instanceName> <resourceName> <partitionName> <true/false>
+```
+
+Query information about a cluster
+
+```
+--listClusterInfo <clusterName>
+```
+
+List existing clusters (remember, Helix can manage multiple clusters)
+
+```
+--listClusters
+```
+
+Query info of a single instance in a cluster
+
+```
+--listInstanceInfo <clusterName> <InstanceName>
+```
+
+List instances in a cluster
+
+```
+--listInstances <clusterName>
+```
+
+Query information about a partition
+
+```
+--listPartitionInfo <clusterName> <resourceName> <partitionName>
+```
+
+Query information about a resource
+
+```
+--listResourceInfo <clusterName> <resourceName>
+```
+
+List resources hosted in a cluster
+
+```
+--listResources <clusterName>
+```
+
+Query information about a state model in a cluster
+
+```
+--listStateModel <clusterName> <stateModelName>
+```
+
+Query information about state models in a cluster
+
+```
+--listStateModels <clusterName>
+```
+

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/markdown/tutorial_controller.md
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/markdown/tutorial_controller.md b/website/0.6.1-incubating/src/site/markdown/tutorial_controller.md
new file mode 100644
index 0000000..012d99a
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/markdown/tutorial_controller.md
@@ -0,0 +1,89 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Helix Tutorial: Controller
+--------------------------
+
+Next, let\'s implement the controller.  This is the brain of the cluster.  Helix makes sure there is exactly one active controller running the cluster.
+
+### Start a Connection
+
+The Helix manager requires the following parameters:
+
+* clusterName: A logical name to represent the group of nodes
+* instanceName: A logical name of the process creating the manager instance. Generally this is host:port
+* instanceType: Type of the process. This can be one of the following types, in this case use CONTROLLER:
+    * CONTROLLER: Process that controls the cluster, any number of controllers can be started but only one will be active at any given time
+    * PARTICIPANT: Process that performs the actual task in the distributed system
+    * SPECTATOR: Process that observes the changes in the cluster
+    * ADMIN: To carry out system admin actions
+* zkConnectString: Connection string to ZooKeeper. This is of the form host1:port1,host2:port2,host3:port3
+
+```
+manager = HelixManagerFactory.getZKHelixManager(clusterName,
+                                                instanceName,
+                                                instanceType,
+                                                zkConnectString);
+```
+
+### Controller Code
+
+The Controller needs to know about all changes in the cluster. Helix takes care of this with the default implementation.
+If you need additional functionality, see GenericHelixController on how to configure the pipeline.
+
+```
+manager = HelixManagerFactory.getZKHelixManager(clusterName,
+                                                instanceName,
+                                                InstanceType.CONTROLLER,
+                                                zkConnectString);
+manager.connect();
+GenericHelixController controller = new GenericHelixController();
+manager.addConfigChangeListener(controller);
+manager.addLiveInstanceChangeListener(controller);
+manager.addIdealStateChangeListener(controller);
+manager.addExternalViewChangeListener(controller);
+manager.addControllerListener(controller);
+```
+The snippet above shows how the controller is started. You can also start the controller using command line interface.
+
+```
+cd helix/helix-core/target/helix-core-pkg/bin
+./run-helix-controller.sh --zkSvr <Zookeeper ServerAddress (Required)>  --cluster <Cluster name (Required)>
+```
+
+### Controller Deployment Modes
+
+Helix provides multiple options to deploy the controller.
+
+#### STANDALONE
+
+The Controller can be started as a separate process to manage a cluster. This is the recommended approach. However, since one controller can be a single point of failure, multiple controller processes are required for reliability.  Even if multiple controllers are running, only one will be actively managing the cluster at any time and is decided by a leader-election process. If the leader fails, another leader will take over managing the cluster.
+
+Even though we recommend this method of deployment, it has the drawback of having to manage an additional service for each cluster. See the Controller as a Service option.
+
+#### EMBEDDED
+
+If setting up a separate controller process is not viable, then it is possible to embed the controller as a library in each of the participants.
+
+#### CONTROLLER AS A SERVICE
+
+One of the cool features we added in Helix was to use a set of controllers to manage a large number of clusters.
+
+For example if you have X clusters to be managed, instead of deploying X*3 (3 controllers for fault tolerance) controllers for each cluster, one can deploy just 3 controllers.  Each controller can manage X/3 clusters.  If any controller fails, the remaining two will manage X/2 clusters.
+

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/markdown/tutorial_health.md
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/markdown/tutorial_health.md b/website/0.6.1-incubating/src/site/markdown/tutorial_health.md
new file mode 100644
index 0000000..96a7726
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/markdown/tutorial_health.md
@@ -0,0 +1,43 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Helix Tutorial: Customizing Health Checks
+-----------------------------------------
+
+In this chapter, we\'ll learn how to customize the health check, based on metrics of your distributed system.
+
+### Health Checks
+
+Note: _this in currently in development mode, not yet ready for production_
+
+Helix provides the ability for each node in the system to report health metrics on a periodic basis.
+
+Helix supports multiple ways to aggregate these metrics:
+
+* SUM
+* AVG
+* EXPONENTIAL DECAY
+* WINDOW
+
+Helix persists the aggregated value only.
+
+Applications can define a threshold on the aggregate values according to the SLAs, and when the SLA is violated Helix will fire an alert.
+Currently Helix only fires an alert, but in a future release we plan to use these metrics to either mark the node dead or load balance the partitions.
+This feature will be valuable for distributed systems that support multi-tenancy and have a large variation in work load patterns.  In addition, this can be used to detect skewed partitions (hotspots) and rebalance the cluster.
+

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/markdown/tutorial_messaging.md
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/markdown/tutorial_messaging.md b/website/0.6.1-incubating/src/site/markdown/tutorial_messaging.md
new file mode 100644
index 0000000..589f321
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/markdown/tutorial_messaging.md
@@ -0,0 +1,68 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Helix Tutorial: Messaging
+-------------------------
+
+In this chapter, we\'ll learn about messaging, a convenient feature in Helix for sending messages between nodes of a cluster.  This is an interesting feature that is quite useful in practice. It is common that nodes in a distributed system require a mechanism to interact with each other.
+
+### Example: Bootstrapping a Replica
+
+Consider a search system  where the index replica starts up and it does not have an index. A typical solution is to get the index from a common location, or to copy the index from another replica.
+
+Helix provides a messaging API for intra-cluster communication between nodes in the system.  This API provides a mechanism to specify the message recipient in terms of resource, partition, and state rather than specifying hostnames.  Helix ensures that the message is delivered to all of the required recipients. In this particular use case, the instance can specify the recipient criteria as all replicas of the desired partition to bootstrap.
+Since Helix is aware of the global state of the system, it can send the message to the appropriate nodes. Once the nodes respond, Helix provides the bootstrapping replica with all the responses.
+
+This is a very generic API and can also be used to schedule various periodic tasks in the cluster, such as data backups, log cleanup, etc.
+System Admins can also perform ad-hoc tasks, such as on-demand backups or a system command (such as rm -rf ;) across all nodes of the cluster
+
+```
+ClusterMessagingService messagingService = manager.getMessagingService();
+
+// Construct the Message
+Message requestBackupUriRequest = new Message(
+    MessageType.USER_DEFINE_MSG, UUID.randomUUID().toString());
+requestBackupUriRequest
+    .setMsgSubType(BootstrapProcess.REQUEST_BOOTSTRAP_URL);
+requestBackupUriRequest.setMsgState(MessageState.NEW);
+
+// Set the Recipient criteria: all nodes that satisfy the criteria will receive the message
+Criteria recipientCriteria = new Criteria();
+recipientCriteria.setInstanceName("%");
+recipientCriteria.setRecipientInstanceType(InstanceType.PARTICIPANT);
+recipientCriteria.setResource("MyDB");
+recipientCriteria.setPartition("");
+
+// Should be processed only by process(es) that are active at the time of sending the message
+// This means if the recipient is restarted after message is sent, it will not be processe.
+recipientCriteria.setSessionSpecific(true);
+
+// wait for 30 seconds
+int timeout = 30000;
+
+// the handler that will be invoked when any recipient responds to the message.
+BootstrapReplyHandler responseHandler = new BootstrapReplyHandler();
+
+// this will return only after all recipients respond or after timeout
+int sentMessageCount = messagingService.sendAndWait(recipientCriteria,
+    requestBackupUriRequest, responseHandler, timeout);
+```
+
+See HelixManager.DefaultMessagingService in the [Javadocs](http://helix.incubator.apache.org/javadocs/0.6.1-incubating/reference/org/apache/helix/messaging/DefaultMessagingService.html) for more information.
+

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/markdown/tutorial_participant.md
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/markdown/tutorial_participant.md b/website/0.6.1-incubating/src/site/markdown/tutorial_participant.md
new file mode 100644
index 0000000..384dfda
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/markdown/tutorial_participant.md
@@ -0,0 +1,100 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Helix Tutorial: Participant
+---------------------------
+
+In this chapter, we\'ll learn how to implement a __Participant__, which is a primary functional component of a distributed system.
+
+
+### Start a Connection
+
+The Helix manager is a common component that connects each system component with the controller.
+
+It requires the following parameters:
+
+* clusterName: A logical name to represent the group of nodes
+* instanceName: A logical name of the process creating the manager instance. Generally this is host:port
+* instanceType: Type of the process. This can be one of the following types, in this case, use PARTICIPANT
+    * CONTROLLER: Process that controls the cluster, any number of controllers can be started but only one will be active at any given time
+    * PARTICIPANT: Process that performs the actual task in the distributed system
+    * SPECTATOR: Process that observes the changes in the cluster
+    * ADMIN: To carry out system admin actions
+* zkConnectString: Connection string to ZooKeeper. This is of the form host1:port1,host2:port2,host3:port3
+
+After the Helix manager instance is created, the only thing that needs to be registered is the state model factory.
+The methods of the state model will be called when controller sends transitions to the participant.  In this example, we'll use the OnlineOffline factory.  Other options include:
+
+* MasterSlaveStateModelFactory
+* LeaderStandbyStateModelFactory
+* BootstrapHandler
+
+
+```
+manager = HelixManagerFactory.getZKHelixManager(clusterName,
+                                                instanceName,
+                                                InstanceType.PARTICIPANT,
+                                                zkConnectString);
+StateMachineEngine stateMach = manager.getStateMachineEngine();
+
+//create a stateModelFactory that returns a statemodel object for each partition.
+stateModelFactory = new OnlineOfflineStateModelFactory();
+stateMach.registerStateModelFactory(stateModelType, stateModelFactory);
+manager.connect();
+```
+
+### Example State Model Factory
+
+Helix doesn\'t know what it means to change from OFFLINE\-\-\>ONLINE or ONLINE\-\-\>OFFLINE.  The following code snippet shows where you insert your system logic for these two state transitions.
+
+```
+public class OnlineOfflineStateModelFactory extends
+    StateModelFactory<StateModel> {
+  @Override
+  public StateModel createNewStateModel(String stateUnitKey) {
+    OnlineOfflineStateModel stateModel = new OnlineOfflineStateModel();
+    return stateModel;
+  }
+  @StateModelInfo(states = "{'OFFLINE','ONLINE'}", initialState = "OFFINE")
+  public static class OnlineOfflineStateModel extends StateModel {
+    @Transition(from = "OFFLINE", to = "ONLINE")
+    public void onBecomeOnlineFromOffline(Message message,
+        NotificationContext context) {
+      System.out.println("OnlineOfflineStateModel.onBecomeOnlineFromOffline()");
+
+      ////////////////////////////////////////////////////////////////////////////////////////////////
+      // Application logic to handle transition                                                     //
+      // For example, you might start a service, run initialization, etc                            //
+      ////////////////////////////////////////////////////////////////////////////////////////////////
+    }
+
+    @Transition(from = "ONLINE", to = "OFFLINE")
+    public void onBecomeOfflineFromOnline(Message message,
+        NotificationContext context) {
+      System.out.println("OnlineOfflineStateModel.onBecomeOfflineFromOnline()");
+
+      ////////////////////////////////////////////////////////////////////////////////////////////////
+      // Application logic to handle transition                                                     //
+      // For example, you might shutdown a service, log this event, or change monitoring settings   //
+      ////////////////////////////////////////////////////////////////////////////////////////////////
+    }
+  }
+}
+```
+

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/markdown/tutorial_propstore.md
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/markdown/tutorial_propstore.md b/website/0.6.1-incubating/src/site/markdown/tutorial_propstore.md
new file mode 100644
index 0000000..50f4fab
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/markdown/tutorial_propstore.md
@@ -0,0 +1,31 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Helix Tutorial: Application Property Store
+------------------------------------------
+
+In this chapter, we\'ll learn how to use the application property store.
+
+### Property Store
+
+It is common that an application needs support for distributed, shared data structures.  Helix uses ZooKeeper to store the application data and hence provides notifications when the data changes.
+
+While you could use ZooKeeper directly, Helix supports caching the data with a write-through cache. This is far more efficient than reading from ZooKeeper for every access.
+
+See [HelixManager.getHelixPropertyStore](http://helix.incubator.apache.org/javadocs/0.6.1-incubating/reference/org/apache/helix/store/package-summary.html) for details.

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/markdown/tutorial_rebalance.md
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/markdown/tutorial_rebalance.md b/website/0.6.1-incubating/src/site/markdown/tutorial_rebalance.md
new file mode 100644
index 0000000..0f2da4d
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/markdown/tutorial_rebalance.md
@@ -0,0 +1,169 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Helix Tutorial: Rebalancing Algorithms
+--------------------------------------
+
+The placement of partitions in a distributed system is essential for the reliability and scalability of the system.  For example, when a node fails, it is important that the partitions hosted on that node are reallocated evenly among the remaining nodes. Consistent hashing is one such algorithm that can satisfy this guarantee.  Helix provides a variant of consistent hashing based on the RUSH algorithm.
+
+This means given a number of partitions, replicas and number of nodes, Helix does the automatic assignment of partition to nodes such that:
+
+* Each node has the same number of partitions
+* Replicas of the same partition do not stay on the same node
+* When a node fails, the partitions will be equally distributed among the remaining nodes
+* When new nodes are added, the number of partitions moved will be minimized along with satisfying the above criteria
+
+Helix employs a rebalancing algorithm to compute the _ideal state_ of the system.  When the _current state_ differs from the _ideal state_, Helix uses it as the target state of the system and computes the appropriate transitions needed to bring it to the _ideal state_.
+
+Helix makes it easy to perform this operation, while giving you control over the algorithm.  In this section, we\'ll see how to implement the desired behavior.
+
+Helix has three options for rebalancing, in increasing order of customization by the system builder:
+
+* AUTO_REBALANCE
+* AUTO
+* CUSTOMIZED
+
+```
+          |AUTO REBALANCE|   AUTO     |   CUSTOMIZED  |
+          ---------------------------------------------
+ LOCATION | HELIX        |  APP       |     APP       |
+          ---------------------------------------------
+    STATE | HELIX        |  HELIX     |     APP       |
+          ---------------------------------------------
+```
+
+
+### AUTO_REBALANCE
+
+When the ideal state mode is set to AUTO_REBALANCE, Helix controls both the location of the replica along with the state. This option is useful for applications where creation of a replica is not expensive.
+
+For example, consider this system that uses a MasterSlave state model, with 3 partitions and 2 replicas in the ideal state.
+
+```
+{
+  "id" : "MyResource",
+  "simpleFields" : {
+    "IDEAL_STATE_MODE" : "AUTO_REBALANCE",
+    "NUM_PARTITIONS" : "3",
+    "REPLICAS" : "2",
+    "STATE_MODEL_DEF_REF" : "MasterSlave",
+  }
+  "listFields" : {
+    "MyResource_0" : [],
+    "MyResource_1" : [],
+    "MyResource_2" : []
+  },
+  "mapFields" : {
+  }
+}
+```
+
+If there are 3 nodes in the cluster, then Helix will balance the masters and slaves equally.  The ideal state is therefore:
+
+```
+{
+  "id" : "MyResource",
+  "simpleFields" : {
+    "NUM_PARTITIONS" : "3",
+    "REPLICAS" : "2",
+    "STATE_MODEL_DEF_REF" : "MasterSlave",
+  },
+  "mapFields" : {
+    "MyResource_0" : {
+      "N1" : "MASTER",
+      "N2" : "SLAVE",
+    },
+    "MyResource_1" : {
+      "N2" : "MASTER",
+      "N3" : "SLAVE",
+    },
+    "MyResource_2" : {
+      "N3" : "MASTER",
+      "N1" : "SLAVE",
+    }
+  }
+}
+```
+
+Another typical example is evenly distributing a group of tasks among the currently healthy processes. For example, if there are 60 tasks and 4 nodes, Helix assigns 15 tasks to each node.
+When one node fails, Helix redistributes its 15 tasks to the remaining 3 nodes, resulting in a balanced 20 tasks per node. Similarly, if a node is added, Helix re-allocates 3 tasks from each of the 4 nodes to the 5th node, resulting in a balanced distribution of 12 tasks per node.
+
+### AUTO
+
+When the application needs to control the placement of the replicas, use the AUTO ideal state mode.
+
+Example: In the ideal state below, the partition \'MyResource_0\' is constrained to be placed only on node1 or node2.  The choice of _state_ is still controlled by Helix.  That means MyResource_0.MASTER could be on node1 and MyResource_0.SLAVE on node2, or vice-versa but neither would be placed on node3.
+
+```
+{
+  "id" : "MyResource",
+  "simpleFields" : {
+    "IDEAL_STATE_MODE" : "AUTO",
+    "NUM_PARTITIONS" : "3",
+    "REPLICAS" : "2",
+    "STATE_MODEL_DEF_REF" : "MasterSlave",
+  }
+  "listFields" : {
+    "MyResource_0" : [node1, node2],
+    "MyResource_1" : [node2, node3],
+    "MyResource_2" : [node3, node1]
+  },
+  "mapFields" : {
+  }
+}
+```
+
+The MasterSlave state model requires that a partition has exactly one MASTER at all times, and the other replicas should be SLAVEs.  In this simple example with 2 replicas per partition, there would be one MASTER and one SLAVE.  Upon failover, a SLAVE has to assume mastership, and a new SLAVE will be generated.
+
+In this mode when node1 fails, unlike in AUTO_REBALANCE mode, the partition is _not_ moved from node1 to node3. Instead, Helix will decide to change the state of MyResource_0 on node2 from SLAVE to MASTER, based on the system constraints.
+
+### CUSTOMIZED
+
+Finally, Helix offers a third mode called CUSTOMIZED, in which the application controls the placement _and_ state of each replica. The application needs to implement a callback interface that Helix invokes when the cluster state changes.
+Within this callback, the application can recompute the ideal state. Helix will then issue appropriate transitions such that the _IdealState_ and _CurrentState_ converge.
+
+Here\'s an example, again with 3 partitions, 2 replicas per partition, and the MasterSlave state model:
+
+```
+{
+  "id" : "MyResource",
+  "simpleFields" : {
+      "IDEAL_STATE_MODE" : "CUSTOM",
+    "NUM_PARTITIONS" : "3",
+    "REPLICAS" : "2",
+    "STATE_MODEL_DEF_REF" : "MasterSlave",
+  },
+  "mapFields" : {
+    "MyResource_0" : {
+      "N1" : "MASTER",
+      "N2" : "SLAVE",
+    },
+    "MyResource_1" : {
+      "N2" : "MASTER",
+      "N3" : "SLAVE",
+    },
+    "MyResource_2" : {
+      "N3" : "MASTER",
+      "N1" : "SLAVE",
+    }
+  }
+}
+```
+
+Suppose the current state of the system is 'MyResource_0' \-\> {N1:MASTER, N2:SLAVE} and the application changes the ideal state to 'MyResource_0' \-\> {N1:SLAVE,N2:MASTER}. While the application decides which node is MASTER and which is SLAVE, Helix will not blindly issue MASTER \-\-\> SLAVE to N1 and SLAVE \-\-\> MASTER to N2 in parallel, since that might result in a transient state where both N1 and N2 are masters, which violates the MasterSlave constraint that there is exactly one MASTER at a time.  Helix will first issue MASTER \-\-\> SLAVE to N1 and after it is completed, it will issue SLAVE \-\-\> MASTER to N2.

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/markdown/tutorial_spectator.md
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/markdown/tutorial_spectator.md b/website/0.6.1-incubating/src/site/markdown/tutorial_spectator.md
new file mode 100644
index 0000000..881bddb
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/markdown/tutorial_spectator.md
@@ -0,0 +1,73 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Helix Tutorial: Spectator
+-------------------------
+
+Next, we\'ll learn how to implement a __spectator__.  Typically, a spectator needs to react to changes within the distributed system.  Examples: a client that needs to know where to send a request, a topic consumer in a consumer group.  The spectator is automatically informed of changes in the _external state_ of the cluster, but it does not have to add any code to keep track of other components in the system.
+
+### Start a Connection
+
+Same as for a participant, The Helix manager is the common component that connects each system component with the cluster.
+
+It requires the following parameters:
+
+* clusterName: A logical name to represent the group of nodes
+* instanceName: A logical name of the process creating the manager instance. Generally this is host:port
+* instanceType: Type of the process. This can be one of the following types, in this case, use SPECTATOR:
+    * CONTROLLER: Process that controls the cluster, any number of controllers can be started but only one will be active at any given time
+    * PARTICIPANT: Process that performs the actual task in the distributed system
+    * SPECTATOR: Process that observes the changes in the cluster
+    * ADMIN: To carry out system admin actions
+* zkConnectString: Connection string to ZooKeeper. This is of the form host1:port1,host2:port2,host3:port3
+
+After the Helix manager instance is created, the only thing that needs to be registered is the listener.  When the ExternalView changes, the listener is notified.
+
+A spectator observes the cluster and is notified when the state of the system changes. Helix consolidates the state of entire cluster in one Znode called ExternalView.
+Helix provides a default implementation RoutingTableProvider that caches the cluster state and updates it when there is a change in the cluster.
+
+```
+manager = HelixManagerFactory.getZKHelixManager(clusterName,
+                                                instanceName,
+                                                InstanceType.SPECTATOR,
+                                                zkConnectString);
+manager.connect();
+RoutingTableProvider routingTableProvider = new RoutingTableProvider();
+manager.addExternalViewChangeListener(routingTableProvider);
+```
+
+### Spectator Code
+
+In the following code snippet, the application sends the request to a valid instance by interrogating the external view.  Suppose the desired resource for this request is in the partition myDB_1.
+
+```
+// instances = routingTableProvider.getInstances(, "PARTITION_NAME", "PARTITION_STATE");
+instances = routingTableProvider.getInstances("myDB", "myDB_1", "ONLINE");
+
+////////////////////////////////////////////////////////////////////////////////////////////////
+// Application-specific code to send a request to one of the instances                        //
+////////////////////////////////////////////////////////////////////////////////////////////////
+
+theInstance = instances.get(0);  // should choose an instance and throw an exception if none are available
+result = theInstance.sendRequest(yourApplicationRequest, responseObject);
+
+```
+
+When the external view changes, the application needs to react by sending requests to a different instance.
+

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/markdown/tutorial_state.md
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/markdown/tutorial_state.md b/website/0.6.1-incubating/src/site/markdown/tutorial_state.md
new file mode 100644
index 0000000..7a98c09
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/markdown/tutorial_state.md
@@ -0,0 +1,65 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Helix Tutorial: State Machine Configuration
+-------------------------------------------
+
+In this chapter, we\'ll learn about the state models provided by Helix, and how to create your own custom state model.
+
+### State Models
+
+Helix comes with 3 default state models that are commonly used.  It is possible to have multiple state models in a cluster.
+Every resource that is added should be configured to use a single state model that will govern its _ideal state_.
+
+#### MASTER-SLAVE
+
+* Has 3 states: OFFLINE, SLAVE, MASTER
+* Maximum # of masters: 1
+* Slaves are based on the replication factor. The replication factor can be specified while adding the resource
+
+#### ONLINE-OFFLINE
+
+Has 2 states: OFFLINE and ONLINE.  This simple state model is a good starting point for most applications.
+
+#### LEADER-STANDBY
+
+1 Leader and multiple stand-bys.  The idea is that exactly one leader accomplishes a designated task, the stand-bys are ready to take over if the leader fails.
+
+### Constraints
+
+In addition to the state machine configuration, one can specify the constraints of states and transitions.
+
+For example, one can say:
+
+* MASTER:1
+
+to indicate that the maximum number of replicas in MASTER state at any time is 1
+
+* OFFLINE-SLAVE:5
+
+to indicate that the maximum number of OFFLINE-SLAVE transitions that can happen concurrently in the system is 5 in this example.
+
+#### State Priority
+
+Helix uses a greedy approach to satisfy the state constraints. For example, if the state machine configuration says it needs 1 MASTER and 2 SLAVES, but only 1 node is active, Helix must promote it to MASTER. This behavior is achieved by providing the state priority list as MASTER,SLAVE.
+
+#### State Transition Priority
+
+Helix tries to fire as many transitions as possible in parallel to reach the stable state without violating constraints. By default, Helix simply sorts the transitions alphabetically and fires as many as it can without violating the constraints. You can control this by overriding the priority order.
+

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/markdown/tutorial_throttling.md
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/markdown/tutorial_throttling.md b/website/0.6.1-incubating/src/site/markdown/tutorial_throttling.md
new file mode 100644
index 0000000..0760d74
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/markdown/tutorial_throttling.md
@@ -0,0 +1,35 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Helix Tutorial: Throttling
+--------------------------
+
+In this chapter, we\'ll learn how to control the parallel execution of cluster tasks.  Only a centralized cluster manager with global knowledge (i.e. Helix) is capable of coordinating this decision.
+
+### Throttling
+
+Since all state changes in the system are triggered through transitions, Helix can control the number of transitions that can happen in parallel. Some of the transitions may be lightweight, but some might involve moving data, which is quite expensive from a network and IOPS perspective.
+
+Helix allows applications to set a threshold on transitions. The threshold can be set at multiple scopes:
+
+* MessageType e.g STATE_TRANSITION
+* TransitionType e.g SLAVE-MASTER
+* Resource e.g database
+* Node i.e per-node maximum transitions in parallel
+

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/resources/.htaccess
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/resources/.htaccess b/website/0.6.1-incubating/src/site/resources/.htaccess
new file mode 100644
index 0000000..d5c7bf3
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/resources/.htaccess
@@ -0,0 +1,20 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+Redirect /download.html /download.cgi

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/resources/download.cgi
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/resources/download.cgi b/website/0.6.1-incubating/src/site/resources/download.cgi
new file mode 100644
index 0000000..f9a0e30
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/resources/download.cgi
@@ -0,0 +1,22 @@
+#!/bin/sh
+# Just call the standard mirrors.cgi script. It will use download.html
+# as the input template.
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+exec /www/www.apache.org/dyn/mirrors/mirrors.cgi $*

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/resources/images/PFS-Generic.png
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/resources/images/PFS-Generic.png b/website/0.6.1-incubating/src/site/resources/images/PFS-Generic.png
new file mode 100644
index 0000000..7eea3a0
Binary files /dev/null and b/website/0.6.1-incubating/src/site/resources/images/PFS-Generic.png differ

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/resources/images/RSYNC_BASED_PFS.png
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/resources/images/RSYNC_BASED_PFS.png b/website/0.6.1-incubating/src/site/resources/images/RSYNC_BASED_PFS.png
new file mode 100644
index 0000000..0cc55ae
Binary files /dev/null and b/website/0.6.1-incubating/src/site/resources/images/RSYNC_BASED_PFS.png differ

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/site.xml
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/site.xml b/website/0.6.1-incubating/src/site/site.xml
new file mode 100644
index 0000000..857dd01
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/site.xml
@@ -0,0 +1,140 @@
+<?xml version="1.0" encoding="ISO-8859-1"?>
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+-->
+<project name="Apache Helix 0.6.1-incubating">
+  <bannerLeft>
+    <src>../images/helix-logo.jpg</src>
+    <href>http://helix.incubator.apache.org/</href>
+  </bannerLeft>
+  <bannerRight>
+    <src>../images/feather_small.gif</src>
+    <href>http://www.apache.org/</href>
+  </bannerRight>
+  <version position="none"/>
+
+  <publishDate position="right"/>
+
+  <skin>
+    <groupId>lt.velykis.maven.skins</groupId>
+    <artifactId>reflow-maven-skin</artifactId>
+    <version>1.0.0</version>
+  </skin>
+
+  <body>
+
+    <head>
+      <script type="text/javascript">
+
+        var _gaq = _gaq || [];
+        _gaq.push(['_setAccount', 'UA-3211522-12']);
+        _gaq.push(['_trackPageview']);
+
+        (function() {
+        var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+        ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+        var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+        })();
+
+      </script>
+
+    </head>
+
+    <breadcrumbs position="left">
+      <item name="Apache Helix" href="http://helix.incubator.apache.org/"/>
+      <item name="Release 0.6.1-incubating" href="http://helix.incubator.apache.org/0.6.1-incubating-docs/"/>
+    </breadcrumbs>
+
+    <links>
+      <item name="Helix 0.6.1-incubating" href="./index.html"/>
+    </links>
+
+    <menu name="Get Helix">
+      <item name="Download" href="./download.html"/>
+      <item name="Building" href="./Building.html"/>
+      <item name="Release Notes" href="./releasenotes/release-0.6.1-incubating.html"/>
+    </menu>
+
+    <menu name="Hands-On">
+      <item name="Quick Start" href="./Quickstart.html"/>
+      <item name="Tutorial" href="./Tutorial.html"/>
+      <item name="Javadocs" href="http://helix.incubator.apache.org/javadocs/0.6.1-incubating"/>
+    </menu>
+
+    <menu name="Recipes">
+      <item name="Distributed lock manager" href="./recipes/lock_manager.html"/>
+      <item name="Rabbit MQ consumer group" href="./recipes/rabbitmq_consumer_group.html"/>
+      <item name="Rsync replicated file store" href="./recipes/rsync_replicated_file_store.html"/>
+      <item name="Service discovery" href="./recipes/service_discovery.html"/>
+      <item name="Distributed task DAG execution" href="./recipes/task_dag_execution.html"/>
+    </menu>
+<!--
+    <menu ref="reports" inherit="bottom"/>
+    <menu ref="modules" inherit="bottom"/>
+
+
+    <menu name="ASF">
+      <item name="How Apache Works" href="http://www.apache.org/foundation/how-it-works.html"/>
+      <item name="Foundation" href="http://www.apache.org/foundation/"/>
+      <item name="Sponsoring Apache" href="http://www.apache.org/foundation/sponsorship.html"/>
+      <item name="Thanks" href="http://www.apache.org/foundation/thanks.html"/>
+    </menu>
+-->
+    <footer>
+      <div class="row span16"><div>Apache Helix, Apache, the Apache feather logo, and the Apache Helix project logos are trademarks of The Apache Software Foundation.
+        All other marks mentioned may be trademarks or registered trademarks of their respective owners.</div>
+        <a href="${project.url}/privacy-policy.html">Privacy Policy</a>
+      </div>
+    </footer>
+
+
+  </body>
+
+  <custom>
+    <reflowSkin>
+      <theme>default</theme>
+      <highlightJs>false</highlightJs>
+      <brand>
+        <name>Apache Helix</name>
+        <href>http://helix.incubator.apache.org</href>
+      </brand>
+      <slogan>A cluster management framework for partitioned and replicated distributed resources</slogan>
+      <bottomNav>
+        <column>Get Helix</column>
+        <column>Hands-On</column>
+        <column>Recipes</column>
+      </bottomNav>
+      <pages>
+        <index>
+          <sections>
+            <columns>3</columns>
+          </sections>
+        </index>
+      </pages>
+    </reflowSkin>
+    <!--fluidoSkin>
+      <topBarEnabled>true</topBarEnabled>
+      <sideBarEnabled>true</sideBarEnabled>
+      <googleSearch></googleSearch>
+      <twitter>
+        <user>ApacheHelix</user>
+        <showUser>true</showUser>
+        <showFollowers>false</showFollowers>
+      </twitter>
+    </fluidoSkin-->
+  </custom>
+
+</project>

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/site/xdoc/download.xml.vm
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/site/xdoc/download.xml.vm b/website/0.6.1-incubating/src/site/xdoc/download.xml.vm
new file mode 100644
index 0000000..2fc59c3
--- /dev/null
+++ b/website/0.6.1-incubating/src/site/xdoc/download.xml.vm
@@ -0,0 +1,145 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+
+-->
+#set( $releaseName = "0.6.1-incubating" )
+#set( $releaseDate = "05/29/2013" )
+<document xmlns="http://maven.apache.org/XDOC/2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+          xsi:schemaLocation="http://maven.apache.org/XDOC/2.0 http://maven.apache.org/xsd/xdoc-2.0.xsd">
+
+  <properties>
+    <title>Apache Helix Downloads</title>
+    <author email="dev@helix.incubator.apache.org">Apache Helix Documentation Team</author>
+  </properties>
+
+  <body>
+    <div class="toc_container">
+      <macro name="toc">
+        <param name="class" value="toc"/>
+      </macro>
+    </div>
+
+    <section name="Apache Helix Downloads">
+      <p>Apache Helix artifacts are distributed in source and binary form under the terms of the
+        <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.
+        See the included <tt>LICENSE</tt> and <tt>NOTICE</tt> files included in each artifact for additional license
+        information.
+      </p>
+      <p>Use the links below to download a source distribution of Apache Helix.
+      It is good practice to <a href="#Verifying_Releases">verify the integrity</a> of the distribution files.</p>
+    </section>
+
+    <section name="Release">
+      <p>Release date: ${releaseDate} </p>
+      <p><a href="releasenotes/release-${releaseName}.html">${releaseName} Release notes</a></p>
+      <subsection name="${releaseName} Sources">
+        <table>
+          <thead>
+            <tr>
+              <th>Artifact</th>
+              <th>Signatures</th>
+            </tr>
+          </thead>
+          <tbody>
+            <tr>
+              <td>
+                <a href="http://archive.apache.org/dist/incubator/helix/${releaseName}/src/helix-${releaseName}-src.zip">helix-${releaseName}-src.zip</a>
+              </td>
+              <td>
+                <a href="http://archive.apache.org/dist/incubator/helix/${releaseName}/src/helix-${releaseName}-src.zip.asc">asc</a>
+                <a href="http://archive.apache.org/dist/incubator/helix/${releaseName}/src/helix-${releaseName}-src.zip.md5">md5</a>
+                <a href="http://archive.apache.org/dist/incubator/helix/${releaseName}/src/helix-${releaseName}-src.zip.sha1">sha1</a>
+              </td>
+            </tr>
+          </tbody>
+        </table>
+      </subsection>
+      <subsection name="${releaseName} Binaries">
+        <table>
+          <thead>
+            <tr>
+              <th>Artifact</th>
+              <th>Signatures</th>
+            </tr>
+          </thead>
+          <tbody>
+            <tr>
+              <td>
+                <a href="http://archive.apache.org/dist/incubator/helix/${releaseName}/binaries/helix-core-${releaseName}-pkg.tar">helix-core-${releaseName}-pkg.tar</a>
+              </td>
+              <td>
+                <a href="http://archive.apache.org/dist/incubator/helix/${releaseName}/binaries/helix-core-${releaseName}-pkg.tar.asc">asc</a>
+                <a href="http://archive.apache.org/dist/incubator/helix/${releaseName}/binaries/helix-core-${releaseName}-pkg.tar.md5">md5</a>
+                <a href="http://archive.apache.org/dist/incubator/helix/${releaseName}/binaries/helix-core-${releaseName}-pkg.tar.sha1">sha1</a>
+              </td>
+            </tr>
+            <tr>
+              <td>
+                <a href="http://archive.apache.org/dist/incubator/helix/${releaseName}/binaries/helix-admin-webapp-${releaseName}-pkg.tar">helix-admin-webapp-${releaseName}-pkg.tar</a>
+              </td>
+              <td>
+                <a href="http://archive.apache.org/dist/incubator/helix/${releaseName}/binaries/helix-admin-webapp-${releaseName}-pkg.tar.asc">asc</a>
+                <a href="http://archive.apache.org/dist/incubator/helix/${releaseName}/binaries/helix-admin-webapp-${releaseName}-pkg.tar.md5">md5</a>
+                <a href="http://archive.apache.org/dist/incubator/helix/${releaseName}/binaries/helix-admin-webapp-${releaseName}-pkg.tar.sha1">sha1</a>
+              </td>
+            </tr>
+          </tbody>
+        </table>
+      </subsection>
+    </section>
+
+<!--    <section name="Older Releases">
+    </section>-->
+
+    <section name="Verifying Releases">
+      <p>We strongly recommend you verify the integrity of the downloaded files with both PGP and MD5.</p>
+
+      <p>The PGP signatures can be verified using <a href="http://www.pgpi.org/">PGP</a> or
+      <a href="http://www.gnupg.org/">GPG</a>.
+      First download the <a href="http://archive.apache.org/dist/incubator/helix/KEYS">KEYS</a> as well as the
+      <tt>*.asc</tt> signature file for the particular distribution. Make sure you get these files from the main
+      distribution directory, rather than from a mirror. Then verify the signatures using one of the following sets of
+      commands:
+
+        <source>$ pgp -ka KEYS
+$ pgp helix-*.zip.asc</source>
+
+        <source>$ gpg --import KEYS
+$ gpg --verify helix-*.zip.asc</source>
+       </p>
+    <p>Alternatively, you can verify the MD5 signature on the files. A Unix/Linux program called
+      <code>md5</code> or
+      <code>md5sum</code> is included in most distributions.  It is also available as part of
+      <a href="http://www.gnu.org/software/textutils/textutils.html">GNU Textutils</a>.
+      Windows users can get binary md5 programs from these (and likely other) places:
+      <ul>
+        <li>
+          <a href="http://www.md5summer.org/">http://www.md5summer.org/</a>
+        </li>
+        <li>
+          <a href="http://www.fourmilab.ch/md5/">http://www.fourmilab.ch/md5/</a>
+        </li>
+        <li>
+          <a href="http://www.pc-tools.net/win32/md5sums/">http://www.pc-tools.net/win32/md5sums/</a>
+        </li>
+      </ul>
+    </p>
+    </section>
+  </body>
+</document>

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.1-incubating/src/test/conf/testng.xml
----------------------------------------------------------------------
diff --git a/website/0.6.1-incubating/src/test/conf/testng.xml b/website/0.6.1-incubating/src/test/conf/testng.xml
new file mode 100644
index 0000000..58f0803
--- /dev/null
+++ b/website/0.6.1-incubating/src/test/conf/testng.xml
@@ -0,0 +1,27 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
+<suite name="Suite" parallel="none">
+  <test name="Test" preserve-order="false">
+    <packages>
+      <package name="org.apache.helix"/>
+    </packages>
+  </test>
+</suite>

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.2-incubating/pom.xml
----------------------------------------------------------------------
diff --git a/website/0.6.2-incubating/pom.xml b/website/0.6.2-incubating/pom.xml
new file mode 100644
index 0000000..785147b
--- /dev/null
+++ b/website/0.6.2-incubating/pom.xml
@@ -0,0 +1,51 @@
+<?xml version="1.0" encoding="UTF-8" ?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+
+  <parent>
+    <groupId>org.apache.helix</groupId>
+    <artifactId>website</artifactId>
+    <version>0.7.1-incubating-SNAPSHOT</version>
+  </parent>
+
+  <artifactId>0.6.2-incubating-docs</artifactId>
+  <packaging>bundle</packaging>
+  <name>Apache Helix :: Website :: 0.6.2-incubating</name>
+
+  <properties>
+  </properties>
+
+  <dependencies>
+    <dependency>
+      <groupId>org.testng</groupId>
+      <artifactId>testng</artifactId>
+      <version>6.0.1</version>
+    </dependency>
+  </dependencies>
+  <build>
+    <pluginManagement>
+      <plugins>
+      </plugins>
+    </pluginManagement>
+    <plugins>
+    </plugins>
+  </build>
+</project>

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.2-incubating/src/site/apt/privacy-policy.apt
----------------------------------------------------------------------
diff --git a/website/0.6.2-incubating/src/site/apt/privacy-policy.apt b/website/0.6.2-incubating/src/site/apt/privacy-policy.apt
new file mode 100644
index 0000000..ada9363
--- /dev/null
+++ b/website/0.6.2-incubating/src/site/apt/privacy-policy.apt
@@ -0,0 +1,52 @@
+ ----
+ Privacy Policy
+ -----
+ Olivier Lamy
+ -----
+ 2013-02-04
+ -----
+
+~~ Licensed to the Apache Software Foundation (ASF) under one
+~~ or more contributor license agreements.  See the NOTICE file
+~~ distributed with this work for additional information
+~~ regarding copyright ownership.  The ASF licenses this file
+~~ to you under the Apache License, Version 2.0 (the
+~~ "License"); you may not use this file except in compliance
+~~ with the License.  You may obtain a copy of the License at
+~~
+~~   http://www.apache.org/licenses/LICENSE-2.0
+~~
+~~ Unless required by applicable law or agreed to in writing,
+~~ software distributed under the License is distributed on an
+~~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+~~ KIND, either express or implied.  See the License for the
+~~ specific language governing permissions and limitations
+~~ under the License.
+
+Privacy Policy
+
+  Information about your use of this website is collected using server access logs and a tracking cookie. The 
+  collected information consists of the following:
+
+  [[1]] The IP address from which you access the website;
+  
+  [[2]] The type of browser and operating system you use to access our site;
+  
+  [[3]] The date and time you access our site;
+  
+  [[4]] The pages you visit; and
+  
+  [[5]] The addresses of pages from where you followed a link to our site.
+
+  []
+
+  Part of this information is gathered using a tracking cookie set by the 
+  {{{http://www.google.com/analytics/}Google Analytics}} service and handled by Google as described in their 
+  {{{http://www.google.com/privacy.html}privacy policy}}. See your browser documentation for instructions on how to 
+  disable the cookie if you prefer not to share this data with Google.
+
+  We use the gathered information to help us make our site more useful to visitors and to better understand how and 
+  when our site is used. We do not track or collect personally identifiable information or associate gathered data 
+  with any personally identifying information from other sources.
+
+  By using this website, you consent to the collection of this data in the manner and for the purpose described above.

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.2-incubating/src/site/apt/releasenotes/release-0.6.2-incubating.apt
----------------------------------------------------------------------
diff --git a/website/0.6.2-incubating/src/site/apt/releasenotes/release-0.6.2-incubating.apt b/website/0.6.2-incubating/src/site/apt/releasenotes/release-0.6.2-incubating.apt
new file mode 100644
index 0000000..51afc62
--- /dev/null
+++ b/website/0.6.2-incubating/src/site/apt/releasenotes/release-0.6.2-incubating.apt
@@ -0,0 +1,181 @@
+ -----
+ Release Notes for Apache Helix 0.6.2-incubating
+ -----
+
+~~ Licensed to the Apache Software Foundation (ASF) under one                      
+~~ or more contributor license agreements.  See the NOTICE file                    
+~~ distributed with this work for additional information                           
+~~ regarding copyright ownership.  The ASF licenses this file                      
+~~ to you under the Apache License, Version 2.0 (the                               
+~~ "License"); you may not use this file except in compliance                      
+~~ with the License.  You may obtain a copy of the License at                      
+~~                                                                                 
+~~   http://www.apache.org/licenses/LICENSE-2.0                                    
+~~                                                                                 
+~~ Unless required by applicable law or agreed to in writing,                      
+~~ software distributed under the License is distributed on an                     
+~~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY                          
+~~ KIND, either express or implied.  See the License for the                       
+~~ specific language governing permissions and limitations                         
+~~ under the License.
+
+~~ NOTE: For help with the syntax of this file, see:
+~~ http://maven.apache.org/guides/mini/guide-apt-format.html
+
+Release Notes for Apache Helix 0.6.2-incubating
+
+  The Apache Helix team would like to announce the release of Apache Helix 0.6.2-incubating
+
+  This is the third release under the Apache umbrella.
+
+  Helix is a generic cluster management framework used for the automatic management of partitioned, replicated and distributed resources hosted on a cluster of nodes. Helix provides the following features:
+
+  * Automatic assignment of resource/partition to nodes
+
+  * Node failure detection and recovery
+
+  * Dynamic addition of Resources
+
+  * Dynamic addition of nodes to the cluster
+
+  * Pluggable distributed state machine to manage the state of a resource via state transitions
+
+  * Automatic load balancing and throttling of transitions
+
+  []
+
+* Changes
+
+** Sub-task
+
+  * [HELIX-28] - ZkHelixManager.handleNewSession() can happen when a liveinstance already exists
+
+  * [HELIX-85] - Remove mock service module
+
+  * [HELIX-106] - Remove all string constants in the code
+
+  * [HELIX-107] - Add support to set custom objects into ZNRecord
+
+  * [HELIX-124] - race condition in ZkHelixManager.handleNewSession()
+
+  * [HELIX-165] - Add dependency for Guava libraries
+
+  * [HELIX-169] - Take care of consecutive handleNewSession() and session expiry during handleNewSession() 
+
+  * [HELIX-170] - HelixManager#isLeader() should compare both instanceName and sessionId 
+
+  * [HELIX-195] - Race condition between FINALIZE callbacks and Zk Callbacks
+
+  * [HELIX-207] - Add javadocs to classes and public methods in the top-level package
+
+  * [HELIX-208] - Add javadocs to classes and public methods in the model package
+
+  * [HELIX-277] - FULL_AUTO rebalancer should not prefer nodes that are just coming up
+
+** Bug
+
+  * [HELIX-7] - Tune test parameters to fix random test failures
+
+  * [HELIX-87] - Bad repository links in website
+
+  * [HELIX-117] - backward incompatibility problem in accessing zkPath vis HelixWebAdmin
+
+  * [HELIX-118] - PropertyStore -> HelixPropertyStore backwards incompatible location
+
+  * [HELIX-119] - HelixManager serializer no longer needs ByteArraySerializer for /PROPERTYSTORE
+
+  * [HELIX-129] - ZKDumper should use byte[] instead of String to read/write file/zk
+
+  * [HELIX-131] - Connection timeout not set while connecting to zookeeper via zkHelixAdmin
+
+  * [HELIX-133] - Cluster-admin command parsing does not work with removeConfig
+
+  * [HELIX-140] - In ClusterSetup.java, the removeConfig is wrong wired to getConfig
+
+  * [HELIX-141] - Autorebalance does not work reliably and fails when replica>1
+
+  * [HELIX-144] - Need to validate StateModelDefinition when adding new StateModelDefinition to Cluster
+
+  * [HELIX-147] - Fix typo in Idealstate property max_partitions_per_instance
+
+  * [HELIX-148] - Current preferred placement for auto rebalace is suboptimal for n > p
+
+  * [HELIX-150] - Auto rebalance might not evenly distribute states across nodes
+
+  * [HELIX-151] - Auto rebalance doesn't assign some replicas when other nodes could make room
+
+  * [HELIX-153] - Auto rebalance tester uses the returned map fields, but production uses only list fields
+
+  * [HELIX-155] - PropertyKey.instances() is wrongly wired to CONFIG type instead of INSTANCES type
+
+  * [HELIX-197] - state model leak
+
+  * [HELIX-199] - ZNRecord should not publish rawPayload unless it exists
+
+  * [HELIX-216] - Allow HelixAdmin addResource to accept the old rebalancing types
+
+  * [HELIX-221] - Can't find default error->dropped transition method using name convention
+
+  * [HELIX-257] - Upgrade Restlet to 2.1.4 - due security flaw
+
+  * [HELIX-258] - Upgrade Apache Camel due to CVE-2013-4330
+
+  * [HELIX-264] - fix zkclient#close() bug
+
+  * [HELIX-279] - Apply gc handling fixes to main ZKHelixManager class
+
+  * [HELIX-280] - Full auto rebalancer should check for resource tag first
+
+  * [HELIX-288] - helix-core uses an old version of guava
+
+  * [HELIX-299] - Some files in 0.6.2 are missing license headers
+
+** Improvement
+
+  * [HELIX-20] - AUTO-REBALANCE helix controller should re-assign disabled partitions on a node to other available nodes
+
+  * [HELIX-70] - Make Helix OSGi ready
+
+  * [HELIX-149] - Allow clients to pass in preferred placement strategies
+
+  * [HELIX-198] - Unify helix code style
+
+  * [HELIX-218] - Add a reviewboard submission script
+
+  * [HELIX-284] - Support participant auto join in YAML cluster setup
+
+** New Feature
+
+  * [HELIX-215] - Allow setting up the cluster with a YAML file
+
+** Task
+
+  * [HELIX-95] - Tracker for 0.6.2 release
+
+  * [HELIX-154] - Auto rebalance algorithm should not depend on state
+
+  * [HELIX-166] - Rename modes to auto, semi-auto, and custom
+
+  * [HELIX-173] - Move rebalancing strategies to separate classes that implement the Rebalancer interface
+
+  * [HELIX-188] - Add admin command line / REST API documentations
+
+  * [HELIX-194] - ZNRecord has too many constructors
+
+  * [HELIX-205] - Have user-defined rebalancers use RebalanceMode.USER_DEFINED
+
+  * [HELIX-210] - Add support to set data with expect version in BaseDataAccessor
+
+  * [HELIX-217] - Remove mock service module
+
+  * [HELIX-273] - Rebalancer interface should remain unchanged in 0.6.2
+
+  * [HELIX-274] - Verify FULL_AUTO tagged node behavior
+
+  * [HELIX-285] - add integration test util's
+
+  []
+
+  Cheers,
+  --
+  The Apache Helix Team

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.2-incubating/src/site/markdown/Building.md
----------------------------------------------------------------------
diff --git a/website/0.6.2-incubating/src/site/markdown/Building.md b/website/0.6.2-incubating/src/site/markdown/Building.md
new file mode 100644
index 0000000..fd16376
--- /dev/null
+++ b/website/0.6.2-incubating/src/site/markdown/Building.md
@@ -0,0 +1,42 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Build Instructions
+------------------
+
+### From Source
+
+Requirements: JDK 1.6+, Maven 2.0.8+
+
+```
+git clone https://git-wip-us.apache.org/repos/asf/incubator-helix.git
+cd incubator-helix
+git checkout tags/helix-0.6.2-incubating
+mvn install package -DskipTests
+```
+
+### Maven Dependency
+
+```
+<dependency>
+  <groupId>org.apache.helix</groupId>
+  <artifactId>helix-core</artifactId>
+  <version>0.6.2-incubating</version>
+</dependency>
+```

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/439125ae/website/0.6.2-incubating/src/site/markdown/Features.md
----------------------------------------------------------------------
diff --git a/website/0.6.2-incubating/src/site/markdown/Features.md b/website/0.6.2-incubating/src/site/markdown/Features.md
new file mode 100644
index 0000000..ba9d0e7
--- /dev/null
+++ b/website/0.6.2-incubating/src/site/markdown/Features.md
@@ -0,0 +1,313 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+<head>
+  <title>Features</title>
+</head>
+
+Features
+----------------------------
+
+
+### CONFIGURING IDEALSTATE
+
+
+Read concepts page for definition of Idealstate.
+
+The placement of partitions in a DDS is very critical for reliability and scalability of the system. 
+For example, when a node fails, it is important that the partitions hosted on that node are reallocated evenly among the remaining nodes. Consistent hashing is one such algorithm that can guarantee this.
+Helix by default comes with a variant of consistent hashing based of the RUSH algorithm. 
+
+This means given a number of partitions, replicas and number of nodes Helix does the automatic assignment of partition to nodes such that
+
+* Each node has the same number of partitions and replicas of the same partition do not stay on the same node.
+* When a node fails, the partitions will be equally distributed among the remaining nodes
+* When new nodes are added, the number of partitions moved will be minimized along with satisfying the above two criteria.
+
+
+Helix provides multiple ways to control the placement and state of a replica. 
+
+```
+
+            |AUTO REBALANCE|   AUTO     |   CUSTOM  |       
+            -----------------------------------------
+   LOCATION | HELIX        |  APP       |  APP      |
+            -----------------------------------------
+      STATE | HELIX        |  HELIX     |  APP      |
+            -----------------------------------------
+```
+
+#### HELIX EXECUTION MODE 
+
+
+Idealstate is defined as the state of the DDS when all nodes are up and running and healthy. 
+Helix uses this as the target state of the system and computes the appropriate transitions needed in the system to bring it to a stable state. 
+
+Helix supports 3 different execution modes which allows application to explicitly control the placement and state of the replica.
+
+##### AUTO_REBALANCE
+
+When the idealstate mode is set to AUTO_REBALANCE, Helix controls both the location of the replica along with the state. This option is useful for applications where creation of a replica is not expensive. Example
+
+```
+{
+  "id" : "MyResource",
+  "simpleFields" : {
+    "IDEAL_STATE_MODE" : "AUTO_REBALANCE",
+    "NUM_PARTITIONS" : "3",
+    "REPLICAS" : "2",
+    "STATE_MODEL_DEF_REF" : "MasterSlave",
+  }
+  "listFields" : {
+    "MyResource_0" : [],
+    "MyResource_1" : [],
+    "MyResource_2" : []
+  },
+  "mapFields" : {
+  }
+}
+```
+
+If there are 3 nodes in the cluster, then Helix will internally compute the ideal state as 
+
+```
+{
+  "id" : "MyResource",
+  "simpleFields" : {
+    "NUM_PARTITIONS" : "3",
+    "REPLICAS" : "2",
+    "STATE_MODEL_DEF_REF" : "MasterSlave",
+  },
+  "mapFields" : {
+    "MyResource_0" : {
+      "N1" : "MASTER",
+      "N2" : "SLAVE",
+    },
+    "MyResource_1" : {
+      "N2" : "MASTER",
+      "N3" : "SLAVE",
+    },
+    "MyResource_2" : {
+      "N3" : "MASTER",
+      "N1" : "SLAVE",
+    }
+  }
+}
+```
+
+Another typical example is evenly distributing a group of tasks among the currently alive processes. For example, if there are 60 tasks and 4 nodes, Helix assigns 15 tasks to each node. 
+When one node fails Helix redistributes its 15 tasks to the remaining 3 nodes. Similarly, if a node is added, Helix re-allocates 3 tasks from each of the 4 nodes to the 5th node. 
+
+#### AUTO
+
+When the idealstate mode is set to AUTO, Helix only controls STATE of the replicas where as the location of the partition is controlled by application. Example: The below idealstate indicates thats 'MyResource_0' must be only on node1 and node2.  But gives the control of assigning the STATE to Helix.
+
+```
+{
+  "id" : "MyResource",
+  "simpleFields" : {
+    "IDEAL_STATE_MODE" : "AUTO",
+    "NUM_PARTITIONS" : "3",
+    "REPLICAS" : "2",
+    "STATE_MODEL_DEF_REF" : "MasterSlave",
+  }
+  "listFields" : {
+    "MyResource_0" : [node1, node2],
+    "MyResource_1" : [node2, node3],
+    "MyResource_2" : [node3, node1]
+  },
+  "mapFields" : {
+  }
+}
+```
+In this mode when node1 fails, unlike in AUTO-REBALANCE mode the partition is not moved from node1 to others nodes in the cluster. Instead, Helix will decide to change the state of MyResource_0 in N2 based on the system constraints. For example, if a system constraint specified that there should be 1 Master and if the Master failed, then node2 will be made the new master. 
+
+#### CUSTOM
+
+Helix offers a third mode called CUSTOM, in which application can completely control the placement and state of each replica. Applications will have to implement an interface that Helix will invoke when the cluster state changes. 
+Within this callback, the application can recompute the idealstate. Helix will then issue appropriate transitions such that Idealstate and Currentstate converges.
+
+```
+{
+  "id" : "MyResource",
+  "simpleFields" : {
+      "IDEAL_STATE_MODE" : "CUSTOM",
+    "NUM_PARTITIONS" : "3",
+    "REPLICAS" : "2",
+    "STATE_MODEL_DEF_REF" : "MasterSlave",
+  },
+  "mapFields" : {
+    "MyResource_0" : {
+      "N1" : "MASTER",
+      "N2" : "SLAVE",
+    },
+    "MyResource_1" : {
+      "N2" : "MASTER",
+      "N3" : "SLAVE",
+    },
+    "MyResource_2" : {
+      "N3" : "MASTER",
+      "N1" : "SLAVE",
+    }
+  }
+}
+```
+
+For example, the current state of the system might be 'MyResource_0' -> {N1:MASTER,N2:SLAVE} and the application changes the ideal state to 'MyResource_0' -> {N1:SLAVE,N2:MASTER}. Helix will not blindly issue MASTER-->SLAVE to N1 and SLAVE-->MASTER to N2 in parallel since it might result in a transient state where both N1 and N2 are masters.
+Helix will first issue MASTER-->SLAVE to N1 and after its completed it will issue SLAVE-->MASTER to N2. 
+ 
+
+### State Machine Configuration
+
+Helix comes with 3 default state models that are most commonly used. Its possible to have multiple state models in a cluster. 
+Every resource that is added should have a reference to the state model. 
+
+* MASTER-SLAVE: Has 3 states OFFLINE,SLAVE,MASTER. Max masters is 1. Slaves will be based on the replication factor. Replication factor can be specified while adding the resource
+* ONLINE-OFFLINE: Has 2 states OFFLINE and ONLINE. Very simple state model and most applications start off with this state model.
+* LEADER-STANDBY:1 Leader and many stand bys. In general the standby's are idle.
+
+Apart from providing the state machine configuration, one can specify the constraints of states and transitions.
+
+For example one can say
+Master:1. Max number of replicas in Master state at any time is 1.
+OFFLINE-SLAVE:5 Max number of Offline-Slave transitions that can happen concurrently in the system
+
+STATE PRIORITY
+Helix uses greedy approach to satisfy the state constraints. For example if the state machine configuration says it needs 1 master and 2 slaves but only 1 node is active, Helix must promote it to master. This behavior is achieved by providing the state priority list as MASTER,SLAVE.
+
+STATE TRANSITION PRIORITY
+Helix tries to fire as many transitions as possible in parallel to reach the stable state without violating constraints. By default Helix simply sorts the transitions alphabetically and fires as many as it can without violating the constraints. 
+One can control this by overriding the priority order.
+ 
+### Config management
+
+Helix allows applications to store application specific properties. The configuration can have different scopes.
+
+* Cluster
+* Node specific
+* Resource specific
+* Partition specific
+
+Helix also provides notifications when any configs are changed. This allows applications to support dynamic configuration changes.
+
+See HelixManager.getConfigAccessor for more info
+
+### Intra cluster messaging api
+
+This is an interesting feature which is quite useful in practice. Often times, nodes in DDS requires a mechanism to interact with each other. One such requirement is a process of bootstrapping a replica.
+
+Consider a search system use case where the index replica starts up and it does not have an index. One of the commonly used solutions is to get the index from a common location or to copy the index from another replica.
+Helix provides a messaging api, that can be used to talk to other nodes in the system. The value added that Helix provides here is, message recipient can be specified in terms of resource, 
+partition, state and Helix ensures that the message is delivered to all of the required recipients. In this particular use case, the instance can specify the recipient criteria as all replicas of P1. 
+Since Helix is aware of the global state of the system, it can send the message to appropriate nodes. Once the nodes respond Helix provides the bootstrapping replica with all the responses.
+
+This is a very generic api and can also be used to schedule various periodic tasks in the cluster like data backups etc. 
+System Admins can also perform adhoc tasks like on demand backup or execute a system command(like rm -rf ;-)) across all nodes.
+
+```
+      ClusterMessagingService messagingService = manager.getMessagingService();
+      //CONSTRUCT THE MESSAGE
+      Message requestBackupUriRequest = new Message(
+          MessageType.USER_DEFINE_MSG, UUID.randomUUID().toString());
+      requestBackupUriRequest
+          .setMsgSubType(BootstrapProcess.REQUEST_BOOTSTRAP_URL);
+      requestBackupUriRequest.setMsgState(MessageState.NEW);
+      //SET THE RECIPIENT CRITERIA, All nodes that satisfy the criteria will receive the message
+      Criteria recipientCriteria = new Criteria();
+      recipientCriteria.setInstanceName("%");
+      recipientCriteria.setRecipientInstanceType(InstanceType.PARTICIPANT);
+      recipientCriteria.setResource("MyDB");
+      recipientCriteria.setPartition("");
+      //Should be processed only the process that is active at the time of sending the message. 
+      //This means if the recipient is restarted after message is sent, it will not be processed.
+      recipientCriteria.setSessionSpecific(true);
+      // wait for 30 seconds
+      int timeout = 30000;
+      //The handler that will be invoked when any recipient responds to the message.
+      BootstrapReplyHandler responseHandler = new BootstrapReplyHandler();
+      //This will return only after all recipients respond or after timeout.
+      int sentMessageCount = messagingService.sendAndWait(recipientCriteria,
+          requestBackupUriRequest, responseHandler, timeout);
+```
+
+See HelixManager.getMessagingService for more info.
+
+
+### Application specific property storage
+
+There are several usecases where applications needs support for distributed data structures. Helix uses Zookeeper to store the application data and hence provides notifications when the data changes. 
+One value add Helix provides is the ability to specify cache the data and also write through cache. This is more efficient than reading from ZK every time.
+
+See HelixManager.getHelixPropertyStore
+
+### Throttling
+
+Since all state changes in the system are triggered through transitions, Helix can control the number of transitions that can happen in parallel. Some of the transitions may be light weight but some might involve moving data around which is quite expensive.
+Helix allows applications to set threshold on transitions. The threshold can be set at the multiple scopes.
+
+* MessageType e.g STATE_TRANSITION
+* TransitionType e.g SLAVE-MASTER
+* Resource e.g database
+* Node i.e per node max transitions in parallel.
+
+See HelixManager.getHelixAdmin.addMessageConstraint() 
+
+### Health monitoring and alerting
+
+This in currently in development mode, not yet productionized.
+
+Helix provides ability for each node in the system to report health metrics on a periodic basis. 
+Helix supports multiple ways to aggregate these metrics like simple SUM, AVG, EXPONENTIAL DECAY, WINDOW. Helix will only persist the aggregated value.
+Applications can define threshold on the aggregate values according to the SLA's and when the SLA is violated Helix will fire an alert. 
+Currently Helix only fires an alert but eventually we plan to use this metrics to either mark the node dead or load balance the partitions. 
+This feature will be valuable in for distributed systems that support multi-tenancy and have huge variation in work load patterns. Another place this can be used is to detect skewed partitions and rebalance the cluster.
+
+This feature is not yet stable and do not recommend to be used in production.
+
+
+### Controller deployment modes
+
+Read Architecture wiki for more details on the Role of a controller. In simple words, it basically controls the participants in the cluster by issuing transitions.
+
+Helix provides multiple options to deploy the controller.
+
+#### STANDALONE
+
+Controller can be started as a separate process to manage a cluster. This is the recommended approach. How ever since one controller can be a single point of failure, multiple controller processes are required for reliability.
+Even if multiple controllers are running only one will be actively managing the cluster at any time and is decided by a leader election process. If the leader fails, another leader will resume managing the cluster.
+
+Even though we recommend this method of deployment, it has the drawback of having to manage an additional service for each cluster. See Controller As a Service option.
+
+#### EMBEDDED
+
+If setting up a separate controller process is not viable, then it is possible to embed the controller as a library in each of the participant. 
+
+#### CONTROLLER AS A SERVICE
+
+One of the cool feature we added in helix was use a set of controllers to manage a large number of clusters. 
+For example if you have X clusters to be managed, instead of deploying X*3(3 controllers for fault tolerance) controllers for each cluster, one can deploy only 3 controllers. Each controller can manage X/3 clusters. 
+If any controller fails the remaining two will manage X/2 clusters. At LinkedIn, we always deploy controllers in this mode. 
+
+
+
+
+
+
+
+ 


Mime
View raw message