stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shariq Muhammed <sha...@wso2.com>
Subject Re: Stratos Milestone plan
Date Sun, 07 Jul 2013 12:20:17 GMT
Would it also make sense to have a /dependencies project where we can merge
the libs coming from carbon kernel, there we can group stuff user-mgt,
registry, and kernel (carbon.core, carbon.utils etc). One advantage is it
would be much easier for developers to have one or two coarse-grained libs
instead of the many jars in the classpath, just a thought .. !


On Sat, Jul 6, 2013 at 9:26 AM, Lakmal Warusawithana <lakmal@wso2.com>wrote:

> Hi Sameera,
>
> Yes have to create more coarse-grained components. Will do after the
> initial refactor completed.
>
> thanks
>
>
> On Fri, Jul 5, 2013 at 9:37 PM, Sameera Jayasoma <
> sameera.madushan@gmail.com> wrote:
>
>> Hi Lakmal
>>
>> IMHO we need to consider the option of having more coarse-grained
>> components. If you look at the existing components and feature, there are
>> too fine-grained. This is true even in the previous Stratos and Carbon code
>> bases. Lets not make the same mistake again here. If we go ahead with
>> fine-grained components then it will become a maintenance night-mare in the
>> future.
>>
>> So I would suggest few logical components and few logical features(i.e
>> Installable Units).
>>
>> Thanks,
>> Sameera.
>>
>>
>> On Tue, Jul 2, 2013 at 9:24 AM, Lakmal Warusawithana <lakmal@wso2.com>wrote:
>>
>>> Hi all,
>>>
>>> I will start separate threads to discuss each item in detail. What I can
>>> see milestone one should be, changing code to apache and re-factor the code
>>> to new structure. After discuss in detail will prioritize and then finalize
>>> the next milestone items.
>>>
>>> Current Stratos we use osgi model and will continue it. We will simplify
>>> folder structure as below.
>>>
>>>
>>> /
>>> -Components
>>>        -- all osgi components
>>>  - Features (logically bundle components together)
>>>        -- adc
>>>        -- load balancer
>>>        -- auto scaller
>>>        -- manager
>>>        -- cloud controller
>>>        -- cli
>>>        -- cartridge agent
>>> - Products (bundle several features and make a product)
>>> - Service-stubs
>>>        -- all service stubs
>>> - Tools
>>>        -- tools for create cartridges
>>>
>>>
>>> Please share your thoughts.
>>>
>>>
>>> thanks
>>>
>>>
>>> On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana <
>>> dkasunw@gmail.com> wrote:
>>>
>>>> +1 for idea of load balance within static members. It allows to have
>>>> more openings and be more flexible.
>>>>
>>>>
>>>> Thank You,
>>>> Dharshana.
>>>>
>>>>
>>>> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana <
>>>> sanjeewa190@gmail.com> wrote:
>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando <
>>>>> nirmal070125@gmail.com> wrote:
>>>>>
>>>>>> Hi Sanjeewa,
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana <
>>>>>> sanjeewa190@gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana <
>>>>>>> lakmal@wso2.com> wrote:
>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> I hope all commiters / folks are in the mailing list now.
Here I
>>>>>>>> have listed below, some identified improvements, features
that like to have
>>>>>>>> in Stratos.
>>>>>>>>
>>>>>>>> Any other improvements/features are mostly welcome and will
add
>>>>>>>> those to the bellow list, then we can discuss and then will
create
>>>>>>>> milestones plan for releases. ( We will commit current code
based soon
>>>>>>>> after we got the git repository available)
>>>>>>>>
>>>>>>>> Before that if some one want to get quick overview of current
>>>>>>>> release/architecture of Stratos please see [1] for my reason
blog post or
>>>>>>>> more detail [2] with current product documentation, which
will migrate to
>>>>>>>> Apache soon. Also we will organize some hangout sessions
to explain current
>>>>>>>> architecture and code based, and it may helpful other commierts
to come
>>>>>>>> up-to speed quickly.
>>>>>>>>
>>>>>>>> Here the list;
>>>>>>>>
>>>>>>>>    - Re-factor current code to Apache
>>>>>>>>
>>>>>>>> First we will move current code based to apache and then
start
>>>>>>>> re-factoring to apache and make a release without adding
any other
>>>>>>>> improvement or feature.
>>>>>>>>
>>>>>>>>
>>>>>>>>    - IaaS Independent cartridge for Linux.
>>>>>>>>
>>>>>>>> With the current architecture when you create a cartridge
its
>>>>>>>> depend on the IaaS. For example If you create a cartridge
in EC2, we have
>>>>>>>> to make an  AMI  cartridge. Likewise you have to create cartridges
(say PHP
>>>>>>>> cartridge) for each and every IaaSs for support the IaaSs.
As a solution we
>>>>>>>> can introduce LXC based cartridges (which IaaS independent
cartridges) and
>>>>>>>> can be consider LXC is the default Stratos runtime container
for Linux
>>>>>>>> cartridges. Also this will provide more scallability to the
PaaS (I will
>>>>>>>> start separate mail thread with more detail on this)
>>>>>>>>
>>>>>>>>
>>>>>>>>    - Support Windows based cartridges.
>>>>>>>>
>>>>>>>> If IaaS support windows VM then we can have windows based
>>>>>>>> cartridges. It will lead to have .net cartridges
>>>>>>>>
>>>>>>>>
>>>>>>>>    - Expand auto-scaling algorithm
>>>>>>>>
>>>>>>>> Currently auto scaling decision take based on looking at
length of
>>>>>>>> the http/https request queue. We have to expand this to consider
CPU usage,
>>>>>>>> memory utilization ..etc of the cartridges.
>>>>>>>>
>>>>>>>> Before we introduce que length based decision making mechanism
we
>>>>>>> had load average based decision making mechanism. Each server
need to
>>>>>>> expose some API(some service) to get load average of given server.
Load
>>>>>>> calculation can be different from one server to other. Load balancer
will
>>>>>>> do web service call to those end points and update load average
values of
>>>>>>> member list periodically.
>>>>>>>
>>>>>>
>>>>>> We will separate out the load balancer logic and the auto-scaling
>>>>>> logic soon. This would require some code level changes. But the vision
is
>>>>>> there.
>>>>>>
>>>>>> In brief, we should build APIs in Auto-scaler, so that the external
>>>>>> components could provide/insert information that matters the decision
of
>>>>>> scaling up/down, depending on the algorithm used.
>>>>>>
>>>>>>  Normally that service returns integer value(let say 0 to 10) and
we
>>>>>>> can get some idea about load from that. When it comes to traffic
route we
>>>>>>> will give high priority to server with minimum load and so on.
So that type
>>>>>>> of implementation would be ideal for this case as well. Synapse
member
>>>>>>> selection algorithm can be configurable and we can add new algorithm
for
>>>>>>> this if need.
>>>>>>>
>>>>>>
>>>>>> This is an extension point, one could leverage.
>>>>>>
>>>>>>>
>>>>>>>>    - TCP level load balance.
>>>>>>>>
>>>>>>>> Current we only doing http/https based load balancing but
need to
>>>>>>>> expand this to do TCP level load balancing.
>>>>>>>>
>>>>>>>> This would be again nice feature IMO. Also i think we need
>>>>>>> considerable amount of synapse transport level improvements to
implement
>>>>>>> this. WDYT?
>>>>>>>
>>>>>>
>>>>>> We may require to write a TCP load balance endpoint, if one is not
>>>>>> already available (I couldn't find any reference).
>>>>>>
>>>>>>>
>>>>>>>>    - Health monitoring
>>>>>>>>
>>>>>>>> Current release we haven't much in the cartridge heath monitoring
>>>>>>>> but its very important and need to have in the future release.
>>>>>>>>
>>>>>>>> If we implemented above mentioned load average calculation
>>>>>>> mechanism then we can do this easily.
>>>>>>>
>>>>>>
>>>>>> Yes, health monitor would ideally consider the load average of the
>>>>>> Cartridge instances, memory consumption etc. and transit the state
>>>>>> according to a pre-defined curriculum, IMO.
>>>>>>
>>>>>>  We can call that end point and get whatever we need. Also it would
>>>>>>> be ideal if we can have some simple user interface for load balancer
to
>>>>>>> view active services and subscribed nodes and their status. Or
else we can
>>>>>>> use cluster message and set status of server as member attribute.
>>>>>>>
>>>>>>
>>>>>>> Also i would like to suggest static load balance endpoint support
>>>>>>> for load balancer(as a suggestion). Let say some user have fixed
set of
>>>>>>> back end servers and need to route traffic to them through load
balancer.
>>>>>>> That would be useful feature and we can consider it when we design
>>>>>>> milestone plan.
>>>>>>>
>>>>>>
>>>>>> I'm afraid this won't be a use-case when you consider Apache Stratos.
>>>>>> Apache Stratos is a fully dynamic environment i.e. all the clusters
get
>>>>>> build, as and when required. We don't have a concept call static
endpoints
>>>>>> in the context of Stratos.
>>>>>>
>>>>> Yes i can understand. Thought it would be more useful if someone try
>>>>> to load balance within static members.
>>>>>
>>>>>>
>>>>>> Sample configuration would be as follows(It tells everything about
>>>>>>> implementation). Sometimes back me and azeez had offline discussion
about
>>>>>>> this.
>>>>>>>
>>>>>>>    TestService {
>>>>>>>     hosts       server.test.com;
>>>>>>>     servers   {
>>>>>>>             server1 {
>>>>>>>                 ip = 192.0.0.1;
>>>>>>>                 http = 80;
>>>>>>>                 https = 443;
>>>>>>>             }
>>>>>>>
>>>>>>>             server2 {
>>>>>>>                 ip = 192.0.0.2;
>>>>>>>                 http = 80;
>>>>>>>                 https = 443;
>>>>>>>             }
>>>>>>>       }
>>>>>>>     }
>>>>>>>
>>>>>>>>
>>>>>>>>    - Improving ADC (Artifact Distribution Coordinator)
>>>>>>>>
>>>>>>>> Current release we only support git based artifact distribution
and
>>>>>>>> need to provide more scalable deployment options
>>>>>>>>
>>>>>>>>
>>>>>>>>    - Light weight cartridge agent
>>>>>>>>
>>>>>>>> Current load balancer used tribes based clustering. To support
>>>>>>>> tribes clustering we have to use java based cartridge agent
and it will put
>>>>>>>> some unnecessary heavy load to non java cartridges like PHP.
If we can
>>>>>>>> moved to Hazelcast Clustering we can have python based light
weight
>>>>>>>> cartridge agent across all type of cartridges.
>>>>>>>>
>>>>>>>>
>>>>>>>>    - Adding more and more cartridges
>>>>>>>>
>>>>>>>> We can have more and more cartridges like Python, Ruby, Node.js,
>>>>>>>> Mongo etc.
>>>>>>>>
>>>>>>>>
>>>>>>>> thanks
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html
>>>>>>>> [2]
>>>>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Lakmal Warusawithana
>>>>>>>> Software Architect; WSO2 Inc.
>>>>>>>> Mobile : +94714289692
>>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> *Sanjeewa Malalgoda
>>>>>>> **B.Sc. Engineering(Hons)
>>>>>>> Dip. in Com.Sc.
>>>>>>> AMIESL , MIACSIT, CCNA
>>>>>>>
>>>>>>> *
>>>>>>> Mobile +94713068779
>>>>>>>
>>>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards,
>>>>>> Nirmal
>>>>>>
>>>>>> C.S.Nirmal J. Fernando
>>>>>> Senior Software Engineer,
>>>>>> WSO2 Inc.
>>>>>>
>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>
>>>>>
>>>>>
>>>>> Thanks,
>>>>> --
>>>>>
>>>>> *Sanjeewa Malalgoda**
>>>>>
>>>>> *
>>>>> Mobile +94713068779
>>>>>
>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> ...................................................
>>>> Dharshana Kasun Warusavitharana
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Lakmal Warusawithana
>>> Software Architect; WSO2 Inc.
>>> Mobile : +94714289692
>>> Blog : http://lakmalsview.blogspot.com/
>>>
>>>
>>
>>
>> --
>> Sameera Jayasoma
>>
>> blog: http://sameera.adahas.org
>> twitter: https://twitter.com/sameerajayasoma
>> flickr: http://www.flickr.com/photos/sameera-jayasoma/
>>
>
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>


-- 
Thanks,
Shariq.
Phone: +94 777 202 225

Mime
View raw message