stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Isuru Haththotuwa <isu...@apache.org>
Subject [Discuss] High Level Flow for Composite Application Deployment
Date Wed, 03 Sep 2014 06:44:22 GMT
This is to discuss $subject. Sorry for the long mail.

In the current single subscription based mechanism, SM creates the
Subscription and sends the relevant information (cluster id, host name,
payload, etc.) to CC. This is not feasible in the Composite App scenario.
There are several reasons for this:

   1. Composite App structure is a complex nested object. If we parse this
   at SM and we send over to CC via an API call, we would need to recursively
   traverse the tree structure and convert to the stub object. Again in CC, it
   will need to convert to the Topology model.
   2. If we send the same information from SM to CC via a new event, again
   from the SM we need to convert it to the event model and from CC we need to
   convert it to the Topology model (because SM does not publish to the
   Topology).

Therefore, in both scenarios mentioned above, there are two expensive
conversions happening.

To overcome this, I came up with the following changes.

   1. Composite App definition will be directly parsed at CC, and the
   topology will be updated from there.
   2. CC will send an event (Application Created) to notify the listeners.
   This will have the relevant Subscription related information (cluster id,
   hostname, etc. -  what is currently being generated at SM).
   3. The payload information will be published to the meta data service.
   4. SM will build a information model using the information sent in
   Application Created event and the information queried from the meta data
   service to expose to outside over the rest API.


In Composite App implementation, mainly would need two flows; deploying a
Group definition(s) and deploying Composite App definition.

1.



Main steps:

   - send the Group Definition from rest API to CC
   - validate definition and persist in CC side

Validation here includes verifying whether a all the defined cartridges and
sub groups are already deployed.

​ 2.


​
​Main steps:

   - send the Group Definition from rest API to CC
   - CC traverses and parses the app definition
   - CC generates a unique key for the app
   - Application Created event sent from CC. This will have all the
   Subscription related information
   - Update meta data service with relevant payload data. These data should
   be accessible by providing the relevant rest URL. Authentication is
   performed using the generated app specific key.

By using the meta data service, minimum amount of information should be
sent by payload (the meta data service endpoint, application key, etc.).
Other information can be queries using the meta data service.
After the Application Created event is sent by CC, Autoscaler will respond
to this event as follows:


​

Main steps:

   - Autoscaler acts upon Application Created event;
          i. Traverse the tree, create Application Monitors and Group
   Monitors (hierarchical). It will find the order in which clusters should be
   created and members spawned (according to the dependency ordering defined
   in the Group Definition).
          ii. create cluster monitor for the particular cluster(s)
   - Autoscaler asks CC to spawn the cluster with minimum no of instances.
   - AS will wait for Cluster Created event to mark that cluster as
   created. When a particular cluster has minimum number of instances
   activate, notify relevant parent Monitor and mark Cluster Status as ACTIVE
   - When a Group's all dependent clusters and sub groups are active,
   notify relevant parent Group Monitor and mark Group Status as ACTIVE
   - When an application's all groups and clusters are active, The App
   Monitor will be notified, and the App Status would be marked as ACTIVE

The App Monitor, Group Monitor and the Cluster Monitors forms a hierarchy.
Therefore, it should be possible to register an Observer for each Monitor
so that interested parties can be notified. In this case, parent Monitors
should be notified by the child monitors about the changes happening (state
changes, etc.). This will be useful in performing Group based scaling and
error handling (When a member of a Cluster goes down, how to handle the
termination of dependent members and re-spawn, etc.).

This is a considerable amount of change from what we have in Stratos
currently. Please share your feedback and questions.

-- 
Thanks and Regards,

Isuru H.
+94 716 358 048* <http://wso2.com/>*


* <http://wso2.com/>*

Mime
View raw message