stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Sandaruwan <lahi...@wso2.com>
Subject Re: Stratos Milestone plan
Date Fri, 05 Jul 2013 16:17:06 GMT
Hi Sameera,

+1. I think this is our plan after we are done with re-factoring. We have
all the components in the base folder as per current plan. Coarse-grained
components is a must in this case.

Thanks.


On Fri, Jul 5, 2013 at 9:37 PM, Sameera Jayasoma <sameera.madushan@gmail.com
> wrote:

> Hi Lakmal
>
> IMHO we need to consider the option of having more coarse-grained
> components. If you look at the existing components and feature, there are
> too fine-grained. This is true even in the previous Stratos and Carbon code
> bases. Lets not make the same mistake again here. If we go ahead with
> fine-grained components then it will become a maintenance night-mare in the
> future.
>
> So I would suggest few logical components and few logical features(i.e
> Installable Units).
>
> Thanks,
> Sameera.
>
>
> On Tue, Jul 2, 2013 at 9:24 AM, Lakmal Warusawithana <lakmal@wso2.com>wrote:
>
>> Hi all,
>>
>> I will start separate threads to discuss each item in detail. What I can
>> see milestone one should be, changing code to apache and re-factor the code
>> to new structure. After discuss in detail will prioritize and then finalize
>> the next milestone items.
>>
>> Current Stratos we use osgi model and will continue it. We will simplify
>> folder structure as below.
>>
>>
>> /
>> -Components
>>        -- all osgi components
>>  - Features (logically bundle components together)
>>        -- adc
>>        -- load balancer
>>        -- auto scaller
>>        -- manager
>>        -- cloud controller
>>        -- cli
>>        -- cartridge agent
>> - Products (bundle several features and make a product)
>> - Service-stubs
>>        -- all service stubs
>> - Tools
>>        -- tools for create cartridges
>>
>>
>> Please share your thoughts.
>>
>>
>> thanks
>>
>>
>> On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana <
>> dkasunw@gmail.com> wrote:
>>
>>> +1 for idea of load balance within static members. It allows to have
>>> more openings and be more flexible.
>>>
>>>
>>> Thank You,
>>> Dharshana.
>>>
>>>
>>> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana <sanjeewa190@gmail.com
>>> > wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando <
>>>> nirmal070125@gmail.com> wrote:
>>>>
>>>>> Hi Sanjeewa,
>>>>>
>>>>>
>>>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana <
>>>>> sanjeewa190@gmail.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana <
>>>>>> lakmal@wso2.com> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I hope all commiters / folks are in the mailing list now. Here
I
>>>>>>> have listed below, some identified improvements, features that
like to have
>>>>>>> in Stratos.
>>>>>>>
>>>>>>> Any other improvements/features are mostly welcome and will add
>>>>>>> those to the bellow list, then we can discuss and then will create
>>>>>>> milestones plan for releases. ( We will commit current code based
soon
>>>>>>> after we got the git repository available)
>>>>>>>
>>>>>>> Before that if some one want to get quick overview of current
>>>>>>> release/architecture of Stratos please see [1] for my reason
blog post or
>>>>>>> more detail [2] with current product documentation, which will
migrate to
>>>>>>> Apache soon. Also we will organize some hangout sessions to explain
current
>>>>>>> architecture and code based, and it may helpful other commierts
to come
>>>>>>> up-to speed quickly.
>>>>>>>
>>>>>>> Here the list;
>>>>>>>
>>>>>>>    - Re-factor current code to Apache
>>>>>>>
>>>>>>> First we will move current code based to apache and then start
>>>>>>> re-factoring to apache and make a release without adding any
other
>>>>>>> improvement or feature.
>>>>>>>
>>>>>>>
>>>>>>>    - IaaS Independent cartridge for Linux.
>>>>>>>
>>>>>>> With the current architecture when you create a cartridge its
depend
>>>>>>> on the IaaS. For example If you create a cartridge in EC2, we
have to make
>>>>>>> an  AMI  cartridge. Likewise you have to create cartridges (say
PHP
>>>>>>> cartridge) for each and every IaaSs for support the IaaSs. As
a solution we
>>>>>>> can introduce LXC based cartridges (which IaaS independent cartridges)
and
>>>>>>> can be consider LXC is the default Stratos runtime container
for Linux
>>>>>>> cartridges. Also this will provide more scallability to the PaaS
(I will
>>>>>>> start separate mail thread with more detail on this)
>>>>>>>
>>>>>>>
>>>>>>>    - Support Windows based cartridges.
>>>>>>>
>>>>>>> If IaaS support windows VM then we can have windows based
>>>>>>> cartridges. It will lead to have .net cartridges
>>>>>>>
>>>>>>>
>>>>>>>    - Expand auto-scaling algorithm
>>>>>>>
>>>>>>> Currently auto scaling decision take based on looking at length
of
>>>>>>> the http/https request queue. We have to expand this to consider
CPU usage,
>>>>>>> memory utilization ..etc of the cartridges.
>>>>>>>
>>>>>>> Before we introduce que length based decision making mechanism
we
>>>>>> had load average based decision making mechanism. Each server need
to
>>>>>> expose some API(some service) to get load average of given server.
Load
>>>>>> calculation can be different from one server to other. Load balancer
will
>>>>>> do web service call to those end points and update load average values
of
>>>>>> member list periodically.
>>>>>>
>>>>>
>>>>> We will separate out the load balancer logic and the auto-scaling
>>>>> logic soon. This would require some code level changes. But the vision
is
>>>>> there.
>>>>>
>>>>> In brief, we should build APIs in Auto-scaler, so that the external
>>>>> components could provide/insert information that matters the decision
of
>>>>> scaling up/down, depending on the algorithm used.
>>>>>
>>>>>  Normally that service returns integer value(let say 0 to 10) and we
>>>>>> can get some idea about load from that. When it comes to traffic
route we
>>>>>> will give high priority to server with minimum load and so on. So
that type
>>>>>> of implementation would be ideal for this case as well. Synapse member
>>>>>> selection algorithm can be configurable and we can add new algorithm
for
>>>>>> this if need.
>>>>>>
>>>>>
>>>>> This is an extension point, one could leverage.
>>>>>
>>>>>>
>>>>>>>    - TCP level load balance.
>>>>>>>
>>>>>>> Current we only doing http/https based load balancing but need
to
>>>>>>> expand this to do TCP level load balancing.
>>>>>>>
>>>>>>> This would be again nice feature IMO. Also i think we need
>>>>>> considerable amount of synapse transport level improvements to implement
>>>>>> this. WDYT?
>>>>>>
>>>>>
>>>>> We may require to write a TCP load balance endpoint, if one is not
>>>>> already available (I couldn't find any reference).
>>>>>
>>>>>>
>>>>>>>    - Health monitoring
>>>>>>>
>>>>>>> Current release we haven't much in the cartridge heath monitoring
>>>>>>> but its very important and need to have in the future release.
>>>>>>>
>>>>>>> If we implemented above mentioned load average calculation mechanism
>>>>>> then we can do this easily.
>>>>>>
>>>>>
>>>>> Yes, health monitor would ideally consider the load average of the
>>>>> Cartridge instances, memory consumption etc. and transit the state
>>>>> according to a pre-defined curriculum, IMO.
>>>>>
>>>>>  We can call that end point and get whatever we need. Also it would be
>>>>>> ideal if we can have some simple user interface for load balancer
to view
>>>>>> active services and subscribed nodes and their status. Or else we
can use
>>>>>> cluster message and set status of server as member attribute.
>>>>>>
>>>>>
>>>>>> Also i would like to suggest static load balance endpoint support
for
>>>>>> load balancer(as a suggestion). Let say some user have fixed set
of back
>>>>>> end servers and need to route traffic to them through load balancer.
That
>>>>>> would be useful feature and we can consider it when we design milestone
>>>>>> plan.
>>>>>>
>>>>>
>>>>> I'm afraid this won't be a use-case when you consider Apache Stratos.
>>>>> Apache Stratos is a fully dynamic environment i.e. all the clusters get
>>>>> build, as and when required. We don't have a concept call static endpoints
>>>>> in the context of Stratos.
>>>>>
>>>> Yes i can understand. Thought it would be more useful if someone try to
>>>> load balance within static members.
>>>>
>>>>>
>>>>> Sample configuration would be as follows(It tells everything about
>>>>>> implementation). Sometimes back me and azeez had offline discussion
about
>>>>>> this.
>>>>>>
>>>>>>    TestService {
>>>>>>     hosts       server.test.com;
>>>>>>     servers   {
>>>>>>             server1 {
>>>>>>                 ip = 192.0.0.1;
>>>>>>                 http = 80;
>>>>>>                 https = 443;
>>>>>>             }
>>>>>>
>>>>>>             server2 {
>>>>>>                 ip = 192.0.0.2;
>>>>>>                 http = 80;
>>>>>>                 https = 443;
>>>>>>             }
>>>>>>       }
>>>>>>     }
>>>>>>
>>>>>>>
>>>>>>>    - Improving ADC (Artifact Distribution Coordinator)
>>>>>>>
>>>>>>> Current release we only support git based artifact distribution
and
>>>>>>> need to provide more scalable deployment options
>>>>>>>
>>>>>>>
>>>>>>>    - Light weight cartridge agent
>>>>>>>
>>>>>>> Current load balancer used tribes based clustering. To support
>>>>>>> tribes clustering we have to use java based cartridge agent and
it will put
>>>>>>> some unnecessary heavy load to non java cartridges like PHP.
If we can
>>>>>>> moved to Hazelcast Clustering we can have python based light
weight
>>>>>>> cartridge agent across all type of cartridges.
>>>>>>>
>>>>>>>
>>>>>>>    - Adding more and more cartridges
>>>>>>>
>>>>>>> We can have more and more cartridges like Python, Ruby, Node.js,
>>>>>>> Mongo etc.
>>>>>>>
>>>>>>>
>>>>>>> thanks
>>>>>>>
>>>>>>> [1]
>>>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html
>>>>>>> [2]
>>>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Lakmal Warusawithana
>>>>>>> Software Architect; WSO2 Inc.
>>>>>>> Mobile : +94714289692
>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> *Sanjeewa Malalgoda
>>>>>> **B.Sc. Engineering(Hons)
>>>>>> Dip. in Com.Sc.
>>>>>> AMIESL , MIACSIT, CCNA
>>>>>>
>>>>>> *
>>>>>> Mobile +94713068779
>>>>>>
>>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Nirmal
>>>>>
>>>>> C.S.Nirmal J. Fernando
>>>>> Senior Software Engineer,
>>>>> WSO2 Inc.
>>>>>
>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>
>>>>
>>>>
>>>> Thanks,
>>>> --
>>>>
>>>> *Sanjeewa Malalgoda**
>>>>
>>>> *
>>>> Mobile +94713068779
>>>>
>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> ...................................................
>>> Dharshana Kasun Warusavitharana
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Software Architect; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>
>
> --
> Sameera Jayasoma
>
> blog: http://sameera.adahas.org
> twitter: https://twitter.com/sameerajayasoma
> flickr: http://www.flickr.com/photos/sameera-jayasoma/
>



-- 
--
Lahiru Sandaruwan
Software Engineer,
Platform Technologies,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware

email: lahirus@wso2.com cell: (+94) 773 325 954
blog: http://lahiruwrites.blogspot.com/
twitter: http://twitter.com/lahirus
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146

Mime
View raw message