stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lakmal Warusawithana <lak...@wso2.com>
Subject Re: Stratos Milestone plan
Date Sun, 07 Jul 2013 12:36:15 GMT
Yeah that will make life easier.


On Sun, Jul 7, 2013 at 6:00 PM, Isuru Haththotuwa <isuruh@wso2.com> wrote:

>
> On Sun, Jul 7, 2013 at 5:50 PM, Shariq Muhammed <shariq@wso2.com> wrote:
>
>> Would it also make sense to have a /dependencies project where we can
>> merge the libs coming from carbon kernel, there we can group stuff user-
>> mgt, registry, and kernel (carbon.core, carbon.utils etc). One advantage
>> is it would be much easier for developers to have one or two coarse-grained
>> libs instead of the many jars in the classpath, just a thought .. !
>>
>
> +1. We can make dependencies as well as the components more coarse grained
> to avoid unwanted complexity.
>
>>
>>
>> On Sat, Jul 6, 2013 at 9:26 AM, Lakmal Warusawithana <lakmal@wso2.com>wrote:
>>
>>> Hi Sameera,
>>>
>>> Yes have to create more coarse-grained components. Will do after the
>>> initial refactor completed.
>>>
>>> thanks
>>>
>>>
>>> On Fri, Jul 5, 2013 at 9:37 PM, Sameera Jayasoma <
>>> sameera.madushan@gmail.com> wrote:
>>>
>>>> Hi Lakmal
>>>>
>>>> IMHO we need to consider the option of having more coarse-grained
>>>> components. If you look at the existing components and feature, there are
>>>> too fine-grained. This is true even in the previous Stratos and Carbon code
>>>> bases. Lets not make the same mistake again here. If we go ahead with
>>>> fine-grained components then it will become a maintenance night-mare in the
>>>> future.
>>>>
>>>> So I would suggest few logical components and few logical features(i.e
>>>> Installable Units).
>>>>
>>>> Thanks,
>>>> Sameera.
>>>>
>>>>
>>>> On Tue, Jul 2, 2013 at 9:24 AM, Lakmal Warusawithana <lakmal@wso2.com>wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I will start separate threads to discuss each item in detail. What I
>>>>> can see milestone one should be, changing code to apache and re-factor
the
>>>>> code to new structure. After discuss in detail will prioritize and then
>>>>> finalize the next milestone items.
>>>>>
>>>>> Current Stratos we use osgi model and will continue it. We will
>>>>> simplify folder structure as below.
>>>>>
>>>>>
>>>>> /
>>>>> -Components
>>>>>        -- all osgi components
>>>>>  - Features (logically bundle components together)
>>>>>        -- adc
>>>>>        -- load balancer
>>>>>        -- auto scaller
>>>>>        -- manager
>>>>>        -- cloud controller
>>>>>        -- cli
>>>>>        -- cartridge agent
>>>>> - Products (bundle several features and make a product)
>>>>> - Service-stubs
>>>>>        -- all service stubs
>>>>> - Tools
>>>>>        -- tools for create cartridges
>>>>>
>>>>>
>>>>> Please share your thoughts.
>>>>>
>>>>>
>>>>> thanks
>>>>>
>>>>>
>>>>> On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana <
>>>>> dkasunw@gmail.com> wrote:
>>>>>
>>>>>> +1 for idea of load balance within static members. It allows to have
>>>>>> more openings and be more flexible.
>>>>>>
>>>>>>
>>>>>> Thank You,
>>>>>> Dharshana.
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana <
>>>>>> sanjeewa190@gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando <
>>>>>>> nirmal070125@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Sanjeewa,
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana <
>>>>>>>> sanjeewa190@gmail.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana
<
>>>>>>>>> lakmal@wso2.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi All,
>>>>>>>>>>
>>>>>>>>>> I hope all commiters / folks are in the mailing list
now. Here I
>>>>>>>>>> have listed below, some identified improvements,
features that like to have
>>>>>>>>>> in Stratos.
>>>>>>>>>>
>>>>>>>>>> Any other improvements/features are mostly welcome
and will add
>>>>>>>>>> those to the bellow list, then we can discuss and
then will create
>>>>>>>>>> milestones plan for releases. ( We will commit current
code based soon
>>>>>>>>>> after we got the git repository available)
>>>>>>>>>>
>>>>>>>>>> Before that if some one want to get quick overview
of current
>>>>>>>>>> release/architecture of Stratos please see [1] for
my reason blog post or
>>>>>>>>>> more detail [2] with current product documentation,
which will migrate to
>>>>>>>>>> Apache soon. Also we will organize some hangout sessions
to explain current
>>>>>>>>>> architecture and code based, and it may helpful other
commierts to come
>>>>>>>>>> up-to speed quickly.
>>>>>>>>>>
>>>>>>>>>> Here the list;
>>>>>>>>>>
>>>>>>>>>>    - Re-factor current code to Apache
>>>>>>>>>>
>>>>>>>>>> First we will move current code based to apache and
then start
>>>>>>>>>> re-factoring to apache and make a release without
adding any other
>>>>>>>>>> improvement or feature.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    - IaaS Independent cartridge for Linux.
>>>>>>>>>>
>>>>>>>>>> With the current architecture when you create a cartridge
its
>>>>>>>>>> depend on the IaaS. For example If you create a cartridge
in EC2, we have
>>>>>>>>>> to make an  AMI  cartridge. Likewise you have to
create cartridges (say PHP
>>>>>>>>>> cartridge) for each and every IaaSs for support the
IaaSs. As a solution we
>>>>>>>>>> can introduce LXC based cartridges (which IaaS independent
cartridges) and
>>>>>>>>>> can be consider LXC is the default Stratos runtime
container for Linux
>>>>>>>>>> cartridges. Also this will provide more scallability
to the PaaS (I will
>>>>>>>>>> start separate mail thread with more detail on this)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    - Support Windows based cartridges.
>>>>>>>>>>
>>>>>>>>>> If IaaS support windows VM then we can have windows
based
>>>>>>>>>> cartridges. It will lead to have .net cartridges
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    - Expand auto-scaling algorithm
>>>>>>>>>>
>>>>>>>>>> Currently auto scaling decision take based on looking
at length
>>>>>>>>>> of the http/https request queue. We have to expand
this to consider CPU
>>>>>>>>>> usage, memory utilization ..etc of the cartridges.
>>>>>>>>>>
>>>>>>>>>> Before we introduce que length based decision making
mechanism we
>>>>>>>>> had load average based decision making mechanism. Each
server need to
>>>>>>>>> expose some API(some service) to get load average of
given server. Load
>>>>>>>>> calculation can be different from one server to other.
Load balancer will
>>>>>>>>> do web service call to those end points and update load
average values of
>>>>>>>>> member list periodically.
>>>>>>>>>
>>>>>>>>
>>>>>>>> We will separate out the load balancer logic and the auto-scaling
>>>>>>>> logic soon. This would require some code level changes. But
the vision is
>>>>>>>> there.
>>>>>>>>
>>>>>>>> In brief, we should build APIs in Auto-scaler, so that the
external
>>>>>>>> components could provide/insert information that matters
the decision of
>>>>>>>> scaling up/down, depending on the algorithm used.
>>>>>>>>
>>>>>>>>  Normally that service returns integer value(let say 0 to
10) and
>>>>>>>>> we can get some idea about load from that. When it comes
to traffic route
>>>>>>>>> we will give high priority to server with minimum load
and so on. So that
>>>>>>>>> type of implementation would be ideal for this case as
well. Synapse member
>>>>>>>>> selection algorithm can be configurable and we can add
new algorithm for
>>>>>>>>> this if need.
>>>>>>>>>
>>>>>>>>
>>>>>>>> This is an extension point, one could leverage.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>    - TCP level load balance.
>>>>>>>>>>
>>>>>>>>>> Current we only doing http/https based load balancing
but need to
>>>>>>>>>> expand this to do TCP level load balancing.
>>>>>>>>>>
>>>>>>>>>> This would be again nice feature IMO. Also i think
we need
>>>>>>>>> considerable amount of synapse transport level improvements
to implement
>>>>>>>>> this. WDYT?
>>>>>>>>>
>>>>>>>>
>>>>>>>> We may require to write a TCP load balance endpoint, if one
is not
>>>>>>>> already available (I couldn't find any reference).
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>    - Health monitoring
>>>>>>>>>>
>>>>>>>>>> Current release we haven't much in the cartridge
heath monitoring
>>>>>>>>>> but its very important and need to have in the future
release.
>>>>>>>>>>
>>>>>>>>>> If we implemented above mentioned load average calculation
>>>>>>>>> mechanism then we can do this easily.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Yes, health monitor would ideally consider the load average
of the
>>>>>>>> Cartridge instances, memory consumption etc. and transit
the state
>>>>>>>> according to a pre-defined curriculum, IMO.
>>>>>>>>
>>>>>>>>  We can call that end point and get whatever we need. Also
it would
>>>>>>>>> be ideal if we can have some simple user interface for
load balancer to
>>>>>>>>> view active services and subscribed nodes and their status.
Or else we can
>>>>>>>>> use cluster message and set status of server as member
attribute.
>>>>>>>>>
>>>>>>>>
>>>>>>>>> Also i would like to suggest static load balance endpoint
support
>>>>>>>>> for load balancer(as a suggestion). Let say some user
have fixed set of
>>>>>>>>> back end servers and need to route traffic to them through
load balancer.
>>>>>>>>> That would be useful feature and we can consider it when
we design
>>>>>>>>> milestone plan.
>>>>>>>>>
>>>>>>>>
>>>>>>>> I'm afraid this won't be a use-case when you consider Apache
>>>>>>>> Stratos. Apache Stratos is a fully dynamic environment i.e.
all the
>>>>>>>> clusters get build, as and when required. We don't have a
concept call
>>>>>>>> static endpoints in the context of Stratos.
>>>>>>>>
>>>>>>> Yes i can understand. Thought it would be more useful if someone
try
>>>>>>> to load balance within static members.
>>>>>>>
>>>>>>>>
>>>>>>>> Sample configuration would be as follows(It tells everything
about
>>>>>>>>> implementation). Sometimes back me and azeez had offline
discussion about
>>>>>>>>> this.
>>>>>>>>>
>>>>>>>>>    TestService {
>>>>>>>>>     hosts       server.test.com;
>>>>>>>>>     servers   {
>>>>>>>>>             server1 {
>>>>>>>>>                 ip = 192.0.0.1;
>>>>>>>>>                 http = 80;
>>>>>>>>>                 https = 443;
>>>>>>>>>             }
>>>>>>>>>
>>>>>>>>>             server2 {
>>>>>>>>>                 ip = 192.0.0.2;
>>>>>>>>>                 http = 80;
>>>>>>>>>                 https = 443;
>>>>>>>>>             }
>>>>>>>>>       }
>>>>>>>>>     }
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    - Improving ADC (Artifact Distribution Coordinator)
>>>>>>>>>>
>>>>>>>>>> Current release we only support git based artifact
distribution
>>>>>>>>>> and need to provide more scalable deployment options
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    - Light weight cartridge agent
>>>>>>>>>>
>>>>>>>>>> Current load balancer used tribes based clustering.
To support
>>>>>>>>>> tribes clustering we have to use java based cartridge
agent and it will put
>>>>>>>>>> some unnecessary heavy load to non java cartridges
like PHP. If we can
>>>>>>>>>> moved to Hazelcast Clustering we can have python
based light weight
>>>>>>>>>> cartridge agent across all type of cartridges.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    - Adding more and more cartridges
>>>>>>>>>>
>>>>>>>>>> We can have more and more cartridges like Python,
Ruby, Node.js,
>>>>>>>>>> Mongo etc.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> thanks
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html
>>>>>>>>>> [2]
>>>>>>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Lakmal Warusawithana
>>>>>>>>>> Software Architect; WSO2 Inc.
>>>>>>>>>> Mobile : +94714289692
>>>>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> *Sanjeewa Malalgoda
>>>>>>>>> **B.Sc. Engineering(Hons)
>>>>>>>>> Dip. in Com.Sc.
>>>>>>>>> AMIESL , MIACSIT, CCNA
>>>>>>>>>
>>>>>>>>> *
>>>>>>>>> Mobile +94713068779
>>>>>>>>>
>>>>>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best Regards,
>>>>>>>> Nirmal
>>>>>>>>
>>>>>>>> C.S.Nirmal J. Fernando
>>>>>>>> Senior Software Engineer,
>>>>>>>> WSO2 Inc.
>>>>>>>>
>>>>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> --
>>>>>>>
>>>>>>> *Sanjeewa Malalgoda**
>>>>>>>
>>>>>>> *
>>>>>>> Mobile +94713068779
>>>>>>>
>>>>>>> http://sanjeewamalalgoda.blogspot.com/
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> ...................................................
>>>>>> Dharshana Kasun Warusavitharana
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Lakmal Warusawithana
>>>>> Software Architect; WSO2 Inc.
>>>>> Mobile : +94714289692
>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Sameera Jayasoma
>>>>
>>>> blog: http://sameera.adahas.org
>>>> twitter: https://twitter.com/sameerajayasoma
>>>> flickr: http://www.flickr.com/photos/sameera-jayasoma/
>>>>
>>>
>>>
>>>
>>> --
>>> Lakmal Warusawithana
>>> Software Architect; WSO2 Inc.
>>> Mobile : +94714289692
>>> Blog : http://lakmalsview.blogspot.com/
>>>
>>>
>>
>>
>> --
>> Thanks,
>> Shariq.
>> Phone: +94 777 202 225
>>
>
>
>
> --
> Thanks and Regards,
>
> Isuru H.
>
>
>


-- 
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692
Blog : http://lakmalsview.blogspot.com/

Mime
View raw message