stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Eppel (meppel)" <>
Subject RE: [Discuss] High Level Flow for Composite Application Deployment
Date Wed, 03 Sep 2014 20:03:09 GMT
Hi Isuru,


I generally agree with flow to avoid multiple, expensive conversions . I have a couple of
questions, some are just to clarify the flow and data model, some of them we need to answer
as we go deeper into the design / implementation:

Will we maintain backwards compatibility for single cartridge subscriptions ?

Any parsing of the composite application is done at the CC, Rest API will just “forward”
the composite application to CC ?

ApplicationCreatedEvent has all the subscription related information which provides the data
to spin up the VMs. All the grouping related information (like dependencies) will be retrieved
by autoscaler from topology model ?

Before spinning up any VMs the dependencies conditions need to be  satisfied,  I would think
that the application monitor will calculate the dependencies (since we are using a hierarchical
model) which can then be queried by the autoscaler, WDYT ?

For group scaling will the scaling information (e.g group scaling policy) be part of subscription
information ( Application Created event) or some other means ?

When a group scales, multiple instances of the group (definition / type) can exist (group
instance), will we have a group monitor for each group instance or one for each group (type)
only ?

When we scale groups, subscription information (for cartridge subscriptions within the group)
won’t change for the new group instance – will it be necessary to create a new Cluster
, ClusterMonitor, etc or can we think of a way to just add on to the existing model instances
(e.g. adding more VM instances in a cluster to scale up by group) ?



From: [] On Behalf Of Isuru Haththotuwa
Sent: Tuesday, September 02, 2014 11:44 PM
To: dev
Subject: [Discuss] High Level Flow for Composite Application Deployment

This is to discuss $subject. Sorry for the long mail.

In the current single subscription based mechanism, SM creates the Subscription and sends
the relevant information (cluster id, host name, payload, etc.) to CC. This is not feasible
in the Composite App scenario. There are several reasons for this:

  1.  Composite App structure is a complex nested object. If we parse this at SM and we send
over to CC via an API call, we would need to recursively traverse the tree structure and convert
to the stub object. Again in CC, it will need to convert to the Topology model.
  2.  If we send the same information from SM to CC via a new event, again from the SM we
need to convert it to the event model and from CC we need to convert it to the Topology model
(because SM does not publish to the Topology).

Therefore, in both scenarios mentioned above, there are two expensive conversions happening.

To overcome this, I came up with the following changes.

  1.  Composite App definition will be directly parsed at CC, and the topology will be updated
from there.
  2.  CC will send an event (Application Created) to notify the listeners. This will have
the relevant Subscription related information (cluster id, hostname, etc. -  what is currently
being generated at SM).
  3.  The payload information will be published to the meta data service.
  4.  SM will build a information model using the information sent in Application Created
event and the information queried from the meta data service to expose to outside over the
rest API.

In Composite App implementation, mainly would need two flows; deploying a Group definition(s)
and deploying Composite App definition.


Main steps:

  *   send the Group Definition from rest API to CC
  *   validate definition and persist in CC side

Validation here includes verifying whether a all the defined cartridges and sub groups are
already deployed.

​ 2.

​Main steps:

  *   send the Group Definition from rest API to CC
  *   CC traverses and parses the app definition
  *   CC generates a unique key for the app
  *   Application Created event sent from CC. This will have all the Subscription related
  *   Update meta data service with relevant payload data. These data should be accessible
by providing the relevant rest URL. Authentication is performed using the generated app specific

By using the meta data service, minimum amount of information should be sent by payload (the
meta data service endpoint, application key, etc.). Other information can be queries using
the meta data service.
After the Application Created event is sent by CC, Autoscaler will respond to this event as

Main steps:

  *   Autoscaler acts upon Application Created event;
       i. Traverse the tree, create Application Monitors and Group Monitors (hierarchical).
It will find the order in which clusters should be created and members spawned (according
to the dependency ordering defined in the Group Definition).
       ii. create cluster monitor for the particular cluster(s)
  *   Autoscaler asks CC to spawn the cluster with minimum no of instances.
  *   AS will wait for Cluster Created event to mark that cluster as created. When a particular
cluster has minimum number of instances activate, notify relevant parent Monitor and mark
Cluster Status as ACTIVE
  *   When a Group's all dependent clusters and sub groups are active, notify relevant parent
Group Monitor and mark Group Status as ACTIVE
  *   When an application's all groups and clusters are active, The App Monitor will be notified,
and the App Status would be marked as ACTIVE

The App Monitor, Group Monitor and the Cluster Monitors forms a hierarchy. Therefore, it should
be possible to register an Observer for each Monitor so that interested parties can be notified.
In this case, parent Monitors should be notified by the child monitors about the changes happening
(state changes, etc.). This will be useful in performing Group based scaling and error handling
(When a member of a Cluster goes down, how to handle the termination of dependent members
and re-spawn, etc.).

This is a considerable amount of change from what we have in Stratos currently. Please share
your feedback and questions.

Thanks and Regards,

Isuru H.
+94 716 358 048

View raw message