stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Imesh Gunaratne <im...@apache.org>
Subject Re: AWS Load Balancing Support for Stratos - Weekly Update
Date Sun, 09 Aug 2015 17:38:16 GMT
Hi Swapnil,

Great! Nice to hear that. Is there a way to find the total number of
requests being processed (requests in flight) within a given time window?
If so we need to find this for each cluster.

Thanks

On Sat, Aug 8, 2015 at 4:13 PM, Swapnil Patil <swapnil.rp15@gmail.com>
wrote:

> Sorry, forgot to mention that it is mentioned that 'max' is preferred for SurgeQueueLength.
> We should decide which to use - max or average.
>
> Thanks and Regards,
> Swapnil
>
> On Sat, Aug 8, 2015 at 10:36 AM, Swapnil Patil <swapnil.rp15@gmail.com>
> wrote:
>
>> Hi Imesh,
>>
>> I was able to write a sample code to retrieve the total number of
>> requests handled by load balancer. I have checked in the code to my github
>> repository of AWS API client [1].
>>
>> As per [2], 'SurgeQueueLength' means "The total number of requests that
>> are pending submission to a registered instance". I hope this is what we
>> should use for finding in flight request count. Please correct me if I am
>> wrong here.
>>
>> If this is correct then we need to decide how we will retrieve this. So
>> this retrieval is based on the start time, end time and interval. The
>> interval is specified in seconds and needs to be multiple of 60. It returns
>> data points over all the intervals, one for each interval. Each data point
>> is sum/average of all requests in that interval.
>>
>> I think we can use average of surge queue length over an interval and for
>> simplicity we can have only one interval. Now we need to decide what should
>> be this interval.
>>
>> Any suggestions on this?
>>
>> [1] https://github.com/swaprp15/AWS_ELB_Client
>> [2]
>> http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-cloudwatch-metrics.html
>>
>> Thanks and Regards,
>> Swapnil
>>
>> On Tue, Aug 4, 2015 at 10:48 AM, Swapnil Patil <swapnil.rp15@gmail.com>
>> wrote:
>>
>>> Hi Imesh,
>>>
>>> Thanks for the reference. I will go through it and check how we can use
>>> it in our extension and get back.
>>>
>>> Thanks and Regards,
>>> Swapnil
>>> On Aug 4, 2015 9:17 AM, "Imesh Gunaratne" <imesh@apache.org> wrote:
>>>
>>>> Great! Thanks for the update Swapnil! Regarding finding request count,
>>>> please have a look at [1] and [2]. According to [1] there is a way to find
>>>> the request count via cloud watch.
>>>>
>>>> [1]
>>>> http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-cloudwatch-metrics.html
>>>> [2]
>>>> http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/access-log-collection.html
>>>>
>>>> Thanks
>>>>
>>>> On Mon, Aug 3, 2015 at 10:49 PM, Swapnil Patil <swapnil.rp15@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> Last week I worked on following things.
>>>>>
>>>>>    1. Adding logic for creating security group for load balancers in
>>>>>    particular region. Adding inbound rules to these security groups based
on
>>>>>    the port mappings of a member of the cluster.
>>>>>    2. Making code changes suggested in the code review meeting.
>>>>>    3. Continued on the documentation.
>>>>>
>>>>> I have committed my latest changes at my GitHub repo [1]
>>>>>
>>>>> I still have not got any response for question on finding outstanding
>>>>> requests for an aws instance. Forum thread - [2]
>>>>>
>>>>> Next week I am planning following tasks.
>>>>>
>>>>>    1. Adding domain mappings for a cluster to include the DNS name of
>>>>>    load balancer. Need to add an API method to add domain mapping using
>>>>>    cluster id.
>>>>>    2. Getting documentation published on wiki and adding some more
>>>>>    content.
>>>>>
>>>>> [1] https://github.com/swaprp15/stratos
>>>>> [2] https://forums.aws.amazon.com/thread.jspa?threadID=200006
>>>>>
>>>>> Thanks and Regards,
>>>>> Swapnil
>>>>>
>>>>> On Mon, Jul 27, 2015 at 5:36 AM, Swapnil Patil <swapnil.rp15@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> Last week I worked on following things.
>>>>>>
>>>>>> 1. Tested the extension for scale up and scale down.
>>>>>> 2. Tested the extension for changing the port mappings of a cartridge.
>>>>>> 3. We had a hangout session for a demo and discussion of issues.
>>>>>> 4. Made code changes for using prefix and sequence number for load
>>>>>> balancer for uniqueness.
>>>>>> 5. Extracted port mappings from a member instead of service.
>>>>>> 6. Corrected the shutdown hook.
>>>>>> 7. Tested above code changes.
>>>>>> 8. I have started writing document to include how to install, how
to
>>>>>> use, aws terminologies used and algorithm used to configure load
balancers.
>>>>>>
>>>>>> I have committed all the latest changes in github repo [1].
>>>>>>
>>>>>> Next week I am planning following tasks.
>>>>>> 1. Adding domain mappings for a cluster to include the DNS name of
>>>>>> load balancer.
>>>>>> 2. Creating a security group and using it for load balancer.
>>>>>> 3. Continue writing documentation.
>>>>>> 4. Find a way to get in flight request count. I have posted a
>>>>>> question for the same on aws forum. [2]
>>>>>>
>>>>>> [1] https://github.com/swaprp15/stratos
>>>>>> [2] https://forums.aws.amazon.com/thread.jspa?threadID=200006
>>>>>>
>>>>>> Thanks and Regards,
>>>>>> Swapnil
>>>>>>
>>>>>>
>>>>>> On Tue, Jul 21, 2015 at 3:56 PM, Swapnil Patil <
>>>>>> swapnil.rp15@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Imesh,
>>>>>>>
>>>>>>> Sure. We will have a demo.
>>>>>>>
>>>>>>> Thanks and Regards,
>>>>>>> Swapnil
>>>>>>>
>>>>>>> On Tue, Jul 21, 2015 at 3:35 PM, Imesh Gunaratne <imesh@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Swapnil,
>>>>>>>>
>>>>>>>> Great! Will have it at 2 PM IST. It would be great if you
could
>>>>>>>> demo what you have completed so far at this meeting.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> On Tue, Jul 21, 2015 at 5:40 PM, Swapnil Patil <
>>>>>>>> swapnil.rp15@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Imesh,
>>>>>>>>>
>>>>>>>>> Apologies for the delay.
>>>>>>>>>
>>>>>>>>> Any time tomorrow would be fine for me.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Swapnil
>>>>>>>>> On Jul 21, 2015 9:54 AM, "Imesh Gunaratne" <imesh@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Swapnil,
>>>>>>>>>>
>>>>>>>>>> Thanks for the update. Shall we have a Google Hangout
this week?
>>>>>>>>>> Please let us know your availability.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 20, 2015 at 11:47 AM, Swapnil Patil <
>>>>>>>>>> swapnil.rp15@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi All,
>>>>>>>>>>>
>>>>>>>>>>> In last week I worked on following tasks.
>>>>>>>>>>>
>>>>>>>>>>> 1. Tested and modified extension to make sure
it works properly
>>>>>>>>>>> when a simple application is deployed and undeployed.
>>>>>>>>>>> 2. I made changes to automatically find out aws
region in which
>>>>>>>>>>> the load balancer should be created. This is
also useful while setting
>>>>>>>>>>> endpoint of each AWS API call.
>>>>>>>>>>> 3. I have published screencast on YouTube. Here
is the link -
>>>>>>>>>>> https://youtu.be/ZNl8JVB7xeU.
>>>>>>>>>>> 4. I have committed these changes to GitHub repo.
URL -
>>>>>>>>>>> https://github.com/swaprp15/stratos
>>>>>>>>>>>
>>>>>>>>>>> I am planning next steps as follows.
>>>>>>>>>>>
>>>>>>>>>>> 1. Testing scale up and scale down.
>>>>>>>>>>> 2. Testing the scenario when port mappings of
the service are
>>>>>>>>>>> changed.
>>>>>>>>>>> 3. Creating a security group for load balancer.
AWS does use
>>>>>>>>>>> default security group for load balancers created.
But then we need to
>>>>>>>>>>> manually configure it to allow required ports.
So it would be better if
>>>>>>>>>>> extension automatically create and use it.
>>>>>>>>>>> 4. Implementing logic for getting in flight request
count.
>>>>>>>>>>>
>>>>>>>>>>> I would appreciate any thoughts on next steps.
>>>>>>>>>>>
>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>> Swapnil
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jul 16, 2015 at 6:33 PM, Swapnil Patil
<
>>>>>>>>>>> swapnil.rp15@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Akila,
>>>>>>>>>>>>
>>>>>>>>>>>> I updated the issue with latest status and
github repo. But I
>>>>>>>>>>>> am not able to find a way to assign this
to myself.
>>>>>>>>>>>>
>>>>>>>>>>>> Could you please let me know how to do it?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>> Swapnil
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jul 16, 2015 at 3:53 PM, Swapnil
Patil <
>>>>>>>>>>>> swapnil.rp15@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Akila,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sure. Will do.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>>> Swapnil
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jul 16, 2015 at 2:02 PM, Akila
Ravihansa Perera <
>>>>>>>>>>>>> ravihansa@wso2.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Swapnil,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Could you update the JIRA with latest
status of the project
>>>>>>>>>>>>>> at [1]. Please put relevant docs
links, GitHub repos and assign this task
>>>>>>>>>>>>>> to yourself.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/STRATOS-500
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 29, 2015 at 4:48 PM,
Swapnil Patil <
>>>>>>>>>>>>>> swapnil.rp15@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Udara,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Okay. I will go through it and
see if we can use it instead
>>>>>>>>>>>>>>> of aws sdk.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks for the link.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Will send update on this soon.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Swapnil
>>>>>>>>>>>>>>> On Jun 29, 2015 3:37 PM, "Udara
Liyanage" <udara@wso2.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Swapnil,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> As you have mentioned during
the call, you are using aws
>>>>>>>>>>>>>>>> Java SDK to talk to AWS IaaS.
However it is better to use JClouds if
>>>>>>>>>>>>>>>> possible. JClouds is the
library which we use to talk to IaaS layer. It is
>>>>>>>>>>>>>>>> better if you can use it
rather than adding another dependency. Please
>>>>>>>>>>>>>>>> review the Jclouds lab [1]
regarding was elb.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>> https://github.com/jclouds/jclouds-labs-aws/blob/master/aws-elb/pom.xml
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Jun 29, 2015 at 12:13
PM, Swapnil Patil <
>>>>>>>>>>>>>>>> swapnil.rp15@gmail.com>
wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi Imesh,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ok. 2 PM today is fine
for me.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>>>>>>> Swapnil
>>>>>>>>>>>>>>>>> On Jun 29, 2015 11:31
AM, "Imesh Gunaratne" <
>>>>>>>>>>>>>>>>> imesh@apache.org>
wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Swapnil,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Shall we have a Hangout
today to go through the current
>>>>>>>>>>>>>>>>>> progress? I have
sent you a meeting request to have this today at 2:00 PM
>>>>>>>>>>>>>>>>>> IST. Let me know
your availability.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Jun 29, 2015
at 11:14 AM, Swapnil Patil <
>>>>>>>>>>>>>>>>>> swapnil.rp15@gmail.com>
wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Last week I worked
on testing load balancer extension
>>>>>>>>>>>>>>>>>>> with ec2 cartridges.
Till now I was able to spawn ec2 cartridge and have
>>>>>>>>>>>>>>>>>>> Stratos receive
these events successfully.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I tested aws
extension on this. While deploying a new
>>>>>>>>>>>>>>>>>>> application,
configure method of extension was called. But there were few
>>>>>>>>>>>>>>>>>>> errors related
to my code changes.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> In this week
I will continue to test the extension and
>>>>>>>>>>>>>>>>>>> make necessary
changes in order to act on topology events properly and
>>>>>>>>>>>>>>>>>>> create and configure
aws load balancers.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>>>>>>>>> Swapnil
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, Jun 23,
2015 at 9:11 AM, Swapnil Patil <
>>>>>>>>>>>>>>>>>>> swapnil.rp15@gmail.com>
wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi Imesh,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks for
the feedback.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Okay. I had
tried it earlier but couldn't start
>>>>>>>>>>>>>>>>>>>> Stratos.
I guess may be it was low memory error or may be the zip I was
>>>>>>>>>>>>>>>>>>>> using was
defective. I will try again on single EC2 instance with correct
>>>>>>>>>>>>>>>>>>>> zip. And
if I get memory issues I will try manual configuration with MB,
>>>>>>>>>>>>>>>>>>>> CEP on different
EC2 instances.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks and
Regards,
>>>>>>>>>>>>>>>>>>>> Swapnil
>>>>>>>>>>>>>>>>>>>> On Jun 23,
2015 2:08 PM, "Imesh Gunaratne" <
>>>>>>>>>>>>>>>>>>>> imesh@apache.org>
wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks
for the update Swapnil!
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Mon,
Jun 22, 2015 at 12:46 PM, Swapnil Patil <
>>>>>>>>>>>>>>>>>>>>> swapnil.rp15@gmail.com>
wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 3.
To test the other part (creating AWS load
>>>>>>>>>>>>>>>>>>>>>> balancers).
I need to create cartridges in EC2. For that I need to find a
>>>>>>>>>>>>>>>>>>>>>> way
to access my local machine from ec2 instance. I searched for a way to
>>>>>>>>>>>>>>>>>>>>>> do
this. It seems I need to do port forwarding and firewall changes in my
>>>>>>>>>>>>>>>>>>>>>> router
to be able to access my computer from an EC2 instance as per [3]. I
>>>>>>>>>>>>>>>>>>>>>> will
look into this in this week.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
It would be easier to run ActiveMQ and Stratos on
>>>>>>>>>>>>>>>>>>>>> EC2.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> Imesh
Gunaratne
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Senior
Technical Lead, WSO2
>>>>>>>>>>>>>>>>>>>>> Committer
& PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Senior Technical
Lead, WSO2
>>>>>>>>>>>>>>>>>> Committer & PMC
Member, Apache Stratos
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Udara Liyanage
>>>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>>>> WSO2, Inc.: http://wso2.com
>>>>>>>>>>>>>>>> lean. enterprise. middleware
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> web: http://udaraliyanage.wordpress.com
>>>>>>>>>>>>>>>> phone: +94 71 443 6897
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Akila Ravihansa Perera
>>>>>>>>>>>>>> Software Engineer, WSO2
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>
>>>>>>>>>> Senior Technical Lead, WSO2
>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Imesh Gunaratne
>>>>>>>>
>>>>>>>> Senior Technical Lead, WSO2
>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Senior Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>
>


-- 
Imesh Gunaratne

Senior Technical Lead, WSO2
Committer & PMC Member, Apache Stratos

Mime
View raw message