stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Isuru Haththotuwa <isu...@wso2.com>
Subject Re: Stratos Milestone plan
Date Sun, 07 Jul 2013 12:30:11 GMT
On Sun, Jul 7, 2013 at 5:50 PM, Shariq Muhammed <shariq@wso2.com> wrote:

> Would it also make sense to have a /dependencies project where we can
> merge the libs coming from carbon kernel, there we can group stuff user-
> mgt, registry, and kernel (carbon.core, carbon.utils etc). One advantage
> is it would be much easier for developers to have one or two coarse-grained
> libs instead of the many jars in the classpath, just a thought .. !
>

+1. We can make dependencies as well as the components more coarse grained
to avoid unwanted complexity.

>
>
> On Sat, Jul 6, 2013 at 9:26 AM, Lakmal Warusawithana <lakmal@wso2.com>wrote:
>
>> Hi Sameera,
>>
>> Yes have to create more coarse-grained components. Will do after the
>> initial refactor completed.
>>
>> thanks
>>
>>
>> On Fri, Jul 5, 2013 at 9:37 PM, Sameera Jayasoma <
>> sameera.madushan@gmail.com> wrote:
>>
>>> Hi Lakmal
>>>
>>> IMHO we need to consider the option of having more coarse-grained
>>> components. If you look at the existing components and feature, there are
>>> too fine-grained. This is true even in the previous Stratos and Carbon code
>>> bases. Lets not make the same mistake again here. If we go ahead with
>>> fine-grained components then it will become a maintenance night-mare in the
>>> future.
>>>
>>> So I would suggest few logical components and few logical features(i.e
>>> Installable Units).
>>>
>>> Thanks,
>>> Sameera.
>>>
>>>
>>> On Tue, Jul 2, 2013 at 9:24 AM, Lakmal Warusawithana <lakmal@wso2.com>wrote:
>>>
>>>> Hi all,
>>>>
>>>> I will start separate threads to discuss each item in detail. What I
>>>> can see milestone one should be, changing code to apache and re-factor the
>>>> code to new structure. After discuss in detail will prioritize and then
>>>> finalize the next milestone items.
>>>>
>>>> Current Stratos we use osgi model and will continue it. We will
>>>> simplify folder structure as below.
>>>>
>>>>
>>>> /
>>>> -Components
>>>>        -- all osgi components
>>>>  - Features (logically bundle components together)
>>>>        -- adc
>>>>        -- load balancer
>>>>        -- auto scaller
>>>>        -- manager
>>>>        -- cloud controller
>>>>        -- cli
>>>>        -- cartridge agent
>>>> - Products (bundle several features and make a product)
>>>> - Service-stubs
>>>>        -- all service stubs
>>>> - Tools
>>>>        -- tools for create cartridges
>>>>
>>>>
>>>> Please share your thoughts.
>>>>
>>>>
>>>> thanks
>>>>
>>>>
>>>> On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana <
>>>> dkasunw@gmail.com> wrote:
>>>>
>>>>> +1 for idea of load balance within static members. It allows to have
>>>>> more openings and be more flexible.
>>>>>
>>>>>
>>>>> Thank You,
>>>>> Dharshana.
>>>>>
>>>>>
>>>>> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana <
>>>>> sanjeewa190@gmail.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando <
>>>>>> nirmal070125@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Sanjeewa,
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana <
>>>>>>> sanjeewa190@gmail.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana <
>>>>>>>> lakmal@wso2.com> wrote:
>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> I hope all commiters / folks are in the mailing list
now. Here I
>>>>>>>>> have listed below, some identified improvements, features
that like to have
>>>>>>>>> in Stratos.
>>>>>>>>>
>>>>>>>>> Any other improvements/features are mostly welcome and
will add
>>>>>>>>> those to the bellow list, then we can discuss and then
will create
>>>>>>>>> milestones plan for releases. ( We will commit current
code based soon
>>>>>>>>> after we got the git repository available)
>>>>>>>>>
>>>>>>>>> Before that if some one want to get quick overview of
current
>>>>>>>>> release/architecture of Stratos please see [1] for my
reason blog post or
>>>>>>>>> more detail [2] with current product documentation, which
will migrate to
>>>>>>>>> Apache soon. Also we will organize some hangout sessions
to explain current
>>>>>>>>> architecture and code based, and it may helpful other
commierts to come
>>>>>>>>> up-to speed quickly.
>>>>>>>>>
>>>>>>>>> Here the list;
>>>>>>>>>
>>>>>>>>>    - Re-factor current code to Apache
>>>>>>>>>
>>>>>>>>> First we will move current code based to apache and then
start
>>>>>>>>> re-factoring to apache and make a release without adding
any other
>>>>>>>>> improvement or feature.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>    - IaaS Independent cartridge for Linux.
>>>>>>>>>
>>>>>>>>> With the current architecture when you create a cartridge
its
>>>>>>>>> depend on the IaaS. For example If you create a cartridge
in EC2, we have
>>>>>>>>> to make an  AMI  cartridge. Likewise you have to create
cartridges (say PHP
>>>>>>>>> cartridge) for each and every IaaSs for support the IaaSs.
As a solution we
>>>>>>>>> can introduce LXC based cartridges (which IaaS independent
cartridges) and
>>>>>>>>> can be consider LXC is the default Stratos runtime container
for Linux
>>>>>>>>> cartridges. Also this will provide more scallability
to the PaaS (I will
>>>>>>>>> start separate mail thread with more detail on this)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>    - Support Windows based cartridges.
>>>>>>>>>
>>>>>>>>> If IaaS support windows VM then we can have windows based
>>>>>>>>> cartridges. It will lead to have .net cartridges
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>    - Expand auto-scaling algorithm
>>>>>>>>>
>>>>>>>>> Currently auto scaling decision take based on looking
at length of
>>>>>>>>> the http/https request queue. We have to expand this
to consider CPU usage,
>>>>>>>>> memory utilization ..etc of the cartridges.
>>>>>>>>>
>>>>>>>>> Before we introduce que length based decision making
mechanism we
>>>>>>>> had load average based decision making mechanism. Each server
need to
>>>>>>>> expose some API(some service) to get load average of given
server. Load
>>>>>>>> calculation can be different from one server to other. Load
balancer will
>>>>>>>> do web service call to those end points and update load average
values of
>>>>>>>> member list periodically.
>>>>>>>>
>>>>>>>
>>>>>>> We will separate out the load balancer logic and the auto-scaling
>>>>>>> logic soon. This would require some code level changes. But the
vision is
>>>>>>> there.
>>>>>>>
>>>>>>> In brief, we should build APIs in Auto-scaler, so that the external
>>>>>>> components could provide/insert information that matters the
decision of
>>>>>>> scaling up/down, depending on the algorithm used.
>>>>>>>
>>>>>>>  Normally that service returns integer value(let say 0 to 10)
and
>>>>>>>> we can get some idea about load from that. When it comes
to traffic route
>>>>>>>> we will give high priority to server with minimum load and
so on. So that
>>>>>>>> type of implementation would be ideal for this case as well.
Synapse member
>>>>>>>> selection algorithm can be configurable and we can add new
algorithm for
>>>>>>>> this if need.
>>>>>>>>
>>>>>>>
>>>>>>> This is an extension point, one could leverage.
>>>>>>>
>>>>>>>>
>>>>>>>>>    - TCP level load balance.
>>>>>>>>>
>>>>>>>>> Current we only doing http/https based load balancing
but need to
>>>>>>>>> expand this to do TCP level load balancing.
>>>>>>>>>
>>>>>>>>> This would be again nice feature IMO. Also i think we
need
>>>>>>>> considerable amount of synapse transport level improvements
to implement
>>>>>>>> this. WDYT?
>>>>>>>>
>>>>>>>
>>>>>>> We may require to write a TCP load balance endpoint, if one is
not
>>>>>>> already available (I couldn't find any reference).
>>>>>>>
>>>>>>>>
>>>>>>>>>    - Health monitoring
>>>>>>>>>
>>>>>>>>> Current release we haven't much in the cartridge heath
monitoring
>>>>>>>>> but its very important and need to have in the future
release.
>>>>>>>>>
>>>>>>>>> If we implemented above mentioned load average calculation
>>>>>>>> mechanism then we can do this easily.
>>>>>>>>
>>>>>>>
>>>>>>> Yes, health monitor would ideally consider the load average of
the
>>>>>>> Cartridge instances, memory consumption etc. and transit the
state
>>>>>>> according to a pre-defined curriculum, IMO.
>>>>>>>
>>>>>>>  We can call that end point and get whatever we need. Also it
would
>>>>>>>> be ideal if we can have some simple user interface for load
balancer to
>>>>>>>> view active services and subscribed nodes and their status.
Or else we can
>>>>>>>> use cluster message and set status of server as member attribute.
>>>>>>>>
>>>>>>>
>>>>>>>> Also i would like to suggest static load balance endpoint
support
>>>>>>>> for load balancer(as a suggestion). Let say some user have
fixed set of
>>>>>>>> back end servers and need to route traffic to them through
load balancer.
>>>>>>>> That would be useful feature and we can consider it when
we design
>>>>>>>> milestone plan.
>>>>>>>>
>>>>>>>
>>>>>>> I'm afraid this won't be a use-case when you consider Apache
>>>>>>> Stratos. Apache Stratos is a fully dynamic environment i.e. all
the
>>>>>>> clusters get build, as and when required. We don't have a concept
call
>>>>>>> static endpoints in the context of Stratos.
>>>>>>>
>>>>>> Yes i can understand. Thought it would be more useful if someone
try
>>>>>> to load balance within static members.
>>>>>>
>>>>>>>
>>>>>>> Sample configuration would be as follows(It tells everything
about
>>>>>>>> implementation). Sometimes back me and azeez had offline
discussion about
>>>>>>>> this.
>>>>>>>>
>>>>>>>>    TestService {
>>>>>>>>     hosts       server.test.com;
>>>>>>>>     servers   {
>>>>>>>>             server1 {
>>>>>>>>                 ip = 192.0.0.1;
>>>>>>>>                 http = 80;
>>>>>>>>                 https = 443;
>>>>>>>>             }
>>>>>>>>
>>>>>>>>             server2 {
>>>>>>>>                 ip = 192.0.0.2;
>>>>>>>>                 http = 80;
>>>>>>>>                 https = 443;
>>>>>>>>             }
>>>>>>>>       }
>>>>>>>>     }
>>>>>>>>
>>>>>>>>>
>>>>>>>>>    - Improving ADC (Artifact Distribution Coordinator)
>>>>>>>>>
>>>>>>>>> Current release we only support git based artifact distribution
>>>>>>>>> and need to provide more scalable deployment options
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>    - Light weight cartridge agent
>>>>>>>>>
>>>>>>>>> Current load balancer used tribes based clustering. To
support
>>>>>>>>> tribes clustering we have to use java based cartridge
agent and it will put
>>>>>>>>> some unnecessary heavy load to non java cartridges like
PHP. If we can
>>>>>>>>> moved to Hazelcast Clustering we can have python based
light weight
>>>>>>>>> cartridge agent across all type of cartridges.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>    - Adding more and more cartridges
>>>>>>>>>
>>>>>>>>> We can have more and more cartridges like Python, Ruby,
Node.js,
>>>>>>>>> Mongo etc.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> thanks
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html
>>>>>>>>> [2]
>>>>>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Lakmal Warusawithana
>>>>>>>>> Software Architect; WSO2 Inc.
>>>>>>>>> Mobile : +94714289692
>>>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> *Sanjeewa Malalgoda
>>>>>>>> **B.Sc. Engineering(Hons)
>>>>>>>> Dip. in Com.Sc.
>>>>>>>> AMIESL , MIACSIT, CCNA
>>>>>>>>
>>>>>>>> *
>>>>>>>> Mobile +94713068779
>>>>>>>>
>>>>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards,
>>>>>>> Nirmal
>>>>>>>
>>>>>>> C.S.Nirmal J. Fernando
>>>>>>> Senior Software Engineer,
>>>>>>> WSO2 Inc.
>>>>>>>
>>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> --
>>>>>>
>>>>>> *Sanjeewa Malalgoda**
>>>>>>
>>>>>> *
>>>>>> Mobile +94713068779
>>>>>>
>>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> ...................................................
>>>>> Dharshana Kasun Warusavitharana
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Lakmal Warusawithana
>>>> Software Architect; WSO2 Inc.
>>>> Mobile : +94714289692
>>>> Blog : http://lakmalsview.blogspot.com/
>>>>
>>>>
>>>
>>>
>>> --
>>> Sameera Jayasoma
>>>
>>> blog: http://sameera.adahas.org
>>> twitter: https://twitter.com/sameerajayasoma
>>> flickr: http://www.flickr.com/photos/sameera-jayasoma/
>>>
>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Software Architect; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>
>
> --
> Thanks,
> Shariq.
> Phone: +94 777 202 225
>



-- 
Thanks and Regards,

Isuru H.

Mime
View raw message