stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sameera Jayasoma <sameera.madus...@gmail.com>
Subject Re: Stratos Milestone plan
Date Fri, 05 Jul 2013 16:07:30 GMT
Hi Lakmal

IMHO we need to consider the option of having more coarse-grained
components. If you look at the existing components and feature, there are
too fine-grained. This is true even in the previous Stratos and Carbon code
bases. Lets not make the same mistake again here. If we go ahead with
fine-grained components then it will become a maintenance night-mare in the
future.

So I would suggest few logical components and few logical features(i.e
Installable Units).

Thanks,
Sameera.


On Tue, Jul 2, 2013 at 9:24 AM, Lakmal Warusawithana <lakmal@wso2.com>wrote:

> Hi all,
>
> I will start separate threads to discuss each item in detail. What I can
> see milestone one should be, changing code to apache and re-factor the code
> to new structure. After discuss in detail will prioritize and then finalize
> the next milestone items.
>
> Current Stratos we use osgi model and will continue it. We will simplify
> folder structure as below.
>
>
> /
> -Components
>        -- all osgi components
>  - Features (logically bundle components together)
>        -- adc
>        -- load balancer
>        -- auto scaller
>        -- manager
>        -- cloud controller
>        -- cli
>        -- cartridge agent
> - Products (bundle several features and make a product)
> - Service-stubs
>        -- all service stubs
> - Tools
>        -- tools for create cartridges
>
>
> Please share your thoughts.
>
>
> thanks
>
>
> On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana <
> dkasunw@gmail.com> wrote:
>
>> +1 for idea of load balance within static members. It allows to have
>> more openings and be more flexible.
>>
>>
>> Thank You,
>> Dharshana.
>>
>>
>> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana <sanjeewa190@gmail.com>wrote:
>>
>>>
>>>
>>>
>>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando <
>>> nirmal070125@gmail.com> wrote:
>>>
>>>> Hi Sanjeewa,
>>>>
>>>>
>>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana <
>>>> sanjeewa190@gmail.com> wrote:
>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana <lakmal@wso2.com
>>>>> > wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> I hope all commiters / folks are in the mailing list now. Here I
have
>>>>>> listed below, some identified improvements, features that like to
have in
>>>>>> Stratos.
>>>>>>
>>>>>> Any other improvements/features are mostly welcome and will add those
>>>>>> to the bellow list, then we can discuss and then will create milestones
>>>>>> plan for releases. ( We will commit current code based soon after
we got
>>>>>> the git repository available)
>>>>>>
>>>>>> Before that if some one want to get quick overview of current
>>>>>> release/architecture of Stratos please see [1] for my reason blog
post or
>>>>>> more detail [2] with current product documentation, which will migrate
to
>>>>>> Apache soon. Also we will organize some hangout sessions to explain
current
>>>>>> architecture and code based, and it may helpful other commierts to
come
>>>>>> up-to speed quickly.
>>>>>>
>>>>>> Here the list;
>>>>>>
>>>>>>    - Re-factor current code to Apache
>>>>>>
>>>>>> First we will move current code based to apache and then start
>>>>>> re-factoring to apache and make a release without adding any other
>>>>>> improvement or feature.
>>>>>>
>>>>>>
>>>>>>    - IaaS Independent cartridge for Linux.
>>>>>>
>>>>>> With the current architecture when you create a cartridge its depend
>>>>>> on the IaaS. For example If you create a cartridge in EC2, we have
to make
>>>>>> an  AMI  cartridge. Likewise you have to create cartridges (say PHP
>>>>>> cartridge) for each and every IaaSs for support the IaaSs. As a solution
we
>>>>>> can introduce LXC based cartridges (which IaaS independent cartridges)
and
>>>>>> can be consider LXC is the default Stratos runtime container for
Linux
>>>>>> cartridges. Also this will provide more scallability to the PaaS
(I will
>>>>>> start separate mail thread with more detail on this)
>>>>>>
>>>>>>
>>>>>>    - Support Windows based cartridges.
>>>>>>
>>>>>> If IaaS support windows VM then we can have windows based cartridges.
>>>>>> It will lead to have .net cartridges
>>>>>>
>>>>>>
>>>>>>    - Expand auto-scaling algorithm
>>>>>>
>>>>>> Currently auto scaling decision take based on looking at length of
>>>>>> the http/https request queue. We have to expand this to consider
CPU usage,
>>>>>> memory utilization ..etc of the cartridges.
>>>>>>
>>>>>> Before we introduce que length based decision making mechanism we
had
>>>>> load average based decision making mechanism. Each server need to expose
>>>>> some API(some service) to get load average of given server. Load
>>>>> calculation can be different from one server to other. Load balancer
will
>>>>> do web service call to those end points and update load average values
of
>>>>> member list periodically.
>>>>>
>>>>
>>>> We will separate out the load balancer logic and the auto-scaling logic
>>>> soon. This would require some code level changes. But the vision is there.
>>>>
>>>> In brief, we should build APIs in Auto-scaler, so that the external
>>>> components could provide/insert information that matters the decision of
>>>> scaling up/down, depending on the algorithm used.
>>>>
>>>>  Normally that service returns integer value(let say 0 to 10) and we
>>>>> can get some idea about load from that. When it comes to traffic route
we
>>>>> will give high priority to server with minimum load and so on. So that
type
>>>>> of implementation would be ideal for this case as well. Synapse member
>>>>> selection algorithm can be configurable and we can add new algorithm
for
>>>>> this if need.
>>>>>
>>>>
>>>> This is an extension point, one could leverage.
>>>>
>>>>>
>>>>>>    - TCP level load balance.
>>>>>>
>>>>>> Current we only doing http/https based load balancing but need to
>>>>>> expand this to do TCP level load balancing.
>>>>>>
>>>>>> This would be again nice feature IMO. Also i think we need
>>>>> considerable amount of synapse transport level improvements to implement
>>>>> this. WDYT?
>>>>>
>>>>
>>>> We may require to write a TCP load balance endpoint, if one is not
>>>> already available (I couldn't find any reference).
>>>>
>>>>>
>>>>>>    - Health monitoring
>>>>>>
>>>>>> Current release we haven't much in the cartridge heath monitoring
but
>>>>>> its very important and need to have in the future release.
>>>>>>
>>>>>> If we implemented above mentioned load average calculation mechanism
>>>>> then we can do this easily.
>>>>>
>>>>
>>>> Yes, health monitor would ideally consider the load average of the
>>>> Cartridge instances, memory consumption etc. and transit the state
>>>> according to a pre-defined curriculum, IMO.
>>>>
>>>>  We can call that end point and get whatever we need. Also it would be
>>>>> ideal if we can have some simple user interface for load balancer to
view
>>>>> active services and subscribed nodes and their status. Or else we can
use
>>>>> cluster message and set status of server as member attribute.
>>>>>
>>>>
>>>>> Also i would like to suggest static load balance endpoint support for
>>>>> load balancer(as a suggestion). Let say some user have fixed set of back
>>>>> end servers and need to route traffic to them through load balancer.
That
>>>>> would be useful feature and we can consider it when we design milestone
>>>>> plan.
>>>>>
>>>>
>>>> I'm afraid this won't be a use-case when you consider Apache Stratos.
>>>> Apache Stratos is a fully dynamic environment i.e. all the clusters get
>>>> build, as and when required. We don't have a concept call static endpoints
>>>> in the context of Stratos.
>>>>
>>> Yes i can understand. Thought it would be more useful if someone try to
>>> load balance within static members.
>>>
>>>>
>>>> Sample configuration would be as follows(It tells everything about
>>>>> implementation). Sometimes back me and azeez had offline discussion about
>>>>> this.
>>>>>
>>>>>    TestService {
>>>>>     hosts       server.test.com;
>>>>>     servers   {
>>>>>             server1 {
>>>>>                 ip = 192.0.0.1;
>>>>>                 http = 80;
>>>>>                 https = 443;
>>>>>             }
>>>>>
>>>>>             server2 {
>>>>>                 ip = 192.0.0.2;
>>>>>                 http = 80;
>>>>>                 https = 443;
>>>>>             }
>>>>>       }
>>>>>     }
>>>>>
>>>>>>
>>>>>>    - Improving ADC (Artifact Distribution Coordinator)
>>>>>>
>>>>>> Current release we only support git based artifact distribution and
>>>>>> need to provide more scalable deployment options
>>>>>>
>>>>>>
>>>>>>    - Light weight cartridge agent
>>>>>>
>>>>>> Current load balancer used tribes based clustering. To support tribes
>>>>>> clustering we have to use java based cartridge agent and it will
put some
>>>>>> unnecessary heavy load to non java cartridges like PHP. If we can
moved to
>>>>>> Hazelcast Clustering we can have python based light weight cartridge
agent
>>>>>> across all type of cartridges.
>>>>>>
>>>>>>
>>>>>>    - Adding more and more cartridges
>>>>>>
>>>>>> We can have more and more cartridges like Python, Ruby, Node.js,
>>>>>> Mongo etc.
>>>>>>
>>>>>>
>>>>>> thanks
>>>>>>
>>>>>> [1]
>>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html
>>>>>> [2]
>>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Lakmal Warusawithana
>>>>>> Software Architect; WSO2 Inc.
>>>>>> Mobile : +94714289692
>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *Sanjeewa Malalgoda
>>>>> **B.Sc. Engineering(Hons)
>>>>> Dip. in Com.Sc.
>>>>> AMIESL , MIACSIT, CCNA
>>>>>
>>>>> *
>>>>> Mobile +94713068779
>>>>>
>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards,
>>>> Nirmal
>>>>
>>>> C.S.Nirmal J. Fernando
>>>> Senior Software Engineer,
>>>> WSO2 Inc.
>>>>
>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>
>>>
>>>
>>> Thanks,
>>> --
>>>
>>> *Sanjeewa Malalgoda**
>>>
>>> *
>>> Mobile +94713068779
>>>
>>> http://sanjeewamalalgoda.blogspot.com/
>>>
>>
>>
>>
>> --
>> ...................................................
>> Dharshana Kasun Warusavitharana
>>
>>
>>
>>
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>


-- 
Sameera Jayasoma

blog: http://sameera.adahas.org
twitter: https://twitter.com/sameerajayasoma
flickr: http://www.flickr.com/photos/sameera-jayasoma/

Mime
View raw message