stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Swapnil Patil <swapnil.r...@gmail.com>
Subject Re: AWS Load Balancing Support for Stratos - Weekly Update
Date Sat, 08 Aug 2015 10:36:40 GMT
Hi Imesh,

I was able to write a sample code to retrieve the total number of requests
handled by load balancer. I have checked in the code to my github
repository of AWS API client [1].

As per [2], 'SurgeQueueLength' means "The total number of requests that are
pending submission to a registered instance". I hope this is what we should
use for finding in flight request count. Please correct me if I am wrong
here.

If this is correct then we need to decide how we will retrieve this. So
this retrieval is based on the start time, end time and interval. The
interval is specified in seconds and needs to be multiple of 60. It returns
data points over all the intervals, one for each interval. Each data point
is sum/average of all requests in that interval.

I think we can use average of surge queue length over an interval and for
simplicity we can have only one interval. Now we need to decide what should
be this interval.

Any suggestions on this?

[1] https://github.com/swaprp15/AWS_ELB_Client
[2]
http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-cloudwatch-metrics.html

Thanks and Regards,
Swapnil

On Tue, Aug 4, 2015 at 10:48 AM, Swapnil Patil <swapnil.rp15@gmail.com>
wrote:

> Hi Imesh,
>
> Thanks for the reference. I will go through it and check how we can use it
> in our extension and get back.
>
> Thanks and Regards,
> Swapnil
> On Aug 4, 2015 9:17 AM, "Imesh Gunaratne" <imesh@apache.org> wrote:
>
>> Great! Thanks for the update Swapnil! Regarding finding request count,
>> please have a look at [1] and [2]. According to [1] there is a way to find
>> the request count via cloud watch.
>>
>> [1]
>> http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-cloudwatch-metrics.html
>> [2]
>> http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/access-log-collection.html
>>
>> Thanks
>>
>> On Mon, Aug 3, 2015 at 10:49 PM, Swapnil Patil <swapnil.rp15@gmail.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> Last week I worked on following things.
>>>
>>>    1. Adding logic for creating security group for load balancers in
>>>    particular region. Adding inbound rules to these security groups based on
>>>    the port mappings of a member of the cluster.
>>>    2. Making code changes suggested in the code review meeting.
>>>    3. Continued on the documentation.
>>>
>>> I have committed my latest changes at my GitHub repo [1]
>>>
>>> I still have not got any response for question on finding outstanding
>>> requests for an aws instance. Forum thread - [2]
>>>
>>> Next week I am planning following tasks.
>>>
>>>    1. Adding domain mappings for a cluster to include the DNS name of
>>>    load balancer. Need to add an API method to add domain mapping using
>>>    cluster id.
>>>    2. Getting documentation published on wiki and adding some more
>>>    content.
>>>
>>> [1] https://github.com/swaprp15/stratos
>>> [2] https://forums.aws.amazon.com/thread.jspa?threadID=200006
>>>
>>> Thanks and Regards,
>>> Swapnil
>>>
>>> On Mon, Jul 27, 2015 at 5:36 AM, Swapnil Patil <swapnil.rp15@gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> Last week I worked on following things.
>>>>
>>>> 1. Tested the extension for scale up and scale down.
>>>> 2. Tested the extension for changing the port mappings of a cartridge.
>>>> 3. We had a hangout session for a demo and discussion of issues.
>>>> 4. Made code changes for using prefix and sequence number for load
>>>> balancer for uniqueness.
>>>> 5. Extracted port mappings from a member instead of service.
>>>> 6. Corrected the shutdown hook.
>>>> 7. Tested above code changes.
>>>> 8. I have started writing document to include how to install, how to
>>>> use, aws terminologies used and algorithm used to configure load balancers.
>>>>
>>>> I have committed all the latest changes in github repo [1].
>>>>
>>>> Next week I am planning following tasks.
>>>> 1. Adding domain mappings for a cluster to include the DNS name of load
>>>> balancer.
>>>> 2. Creating a security group and using it for load balancer.
>>>> 3. Continue writing documentation.
>>>> 4. Find a way to get in flight request count. I have posted a question
>>>> for the same on aws forum. [2]
>>>>
>>>> [1] https://github.com/swaprp15/stratos
>>>> [2] https://forums.aws.amazon.com/thread.jspa?threadID=200006
>>>>
>>>> Thanks and Regards,
>>>> Swapnil
>>>>
>>>>
>>>> On Tue, Jul 21, 2015 at 3:56 PM, Swapnil Patil <swapnil.rp15@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Imesh,
>>>>>
>>>>> Sure. We will have a demo.
>>>>>
>>>>> Thanks and Regards,
>>>>> Swapnil
>>>>>
>>>>> On Tue, Jul 21, 2015 at 3:35 PM, Imesh Gunaratne <imesh@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hi Swapnil,
>>>>>>
>>>>>> Great! Will have it at 2 PM IST. It would be great if you could demo
>>>>>> what you have completed so far at this meeting.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> On Tue, Jul 21, 2015 at 5:40 PM, Swapnil Patil <
>>>>>> swapnil.rp15@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Imesh,
>>>>>>>
>>>>>>> Apologies for the delay.
>>>>>>>
>>>>>>> Any time tomorrow would be fine for me.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Swapnil
>>>>>>> On Jul 21, 2015 9:54 AM, "Imesh Gunaratne" <imesh@apache.org>
wrote:
>>>>>>>
>>>>>>>> Hi Swapnil,
>>>>>>>>
>>>>>>>> Thanks for the update. Shall we have a Google Hangout this
week?
>>>>>>>> Please let us know your availability.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> On Mon, Jul 20, 2015 at 11:47 AM, Swapnil Patil <
>>>>>>>> swapnil.rp15@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> In last week I worked on following tasks.
>>>>>>>>>
>>>>>>>>> 1. Tested and modified extension to make sure it works
properly
>>>>>>>>> when a simple application is deployed and undeployed.
>>>>>>>>> 2. I made changes to automatically find out aws region
in which
>>>>>>>>> the load balancer should be created. This is also useful
while setting
>>>>>>>>> endpoint of each AWS API call.
>>>>>>>>> 3. I have published screencast on YouTube. Here is the
link -
>>>>>>>>> https://youtu.be/ZNl8JVB7xeU.
>>>>>>>>> 4. I have committed these changes to GitHub repo. URL
-
>>>>>>>>> https://github.com/swaprp15/stratos
>>>>>>>>>
>>>>>>>>> I am planning next steps as follows.
>>>>>>>>>
>>>>>>>>> 1. Testing scale up and scale down.
>>>>>>>>> 2. Testing the scenario when port mappings of the service
are
>>>>>>>>> changed.
>>>>>>>>> 3. Creating a security group for load balancer. AWS does
use
>>>>>>>>> default security group for load balancers created. But
then we need to
>>>>>>>>> manually configure it to allow required ports. So it
would be better if
>>>>>>>>> extension automatically create and use it.
>>>>>>>>> 4. Implementing logic for getting in flight request count.
>>>>>>>>>
>>>>>>>>> I would appreciate any thoughts on next steps.
>>>>>>>>>
>>>>>>>>> Thanks and Regards,
>>>>>>>>> Swapnil
>>>>>>>>>
>>>>>>>>> On Thu, Jul 16, 2015 at 6:33 PM, Swapnil Patil <
>>>>>>>>> swapnil.rp15@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Akila,
>>>>>>>>>>
>>>>>>>>>> I updated the issue with latest status and github
repo. But I am
>>>>>>>>>> not able to find a way to assign this to myself.
>>>>>>>>>>
>>>>>>>>>> Could you please let me know how to do it?
>>>>>>>>>>
>>>>>>>>>> Thanks and Regards,
>>>>>>>>>> Swapnil
>>>>>>>>>>
>>>>>>>>>> On Thu, Jul 16, 2015 at 3:53 PM, Swapnil Patil <
>>>>>>>>>> swapnil.rp15@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Akila,
>>>>>>>>>>>
>>>>>>>>>>> Sure. Will do.
>>>>>>>>>>>
>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>> Swapnil
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jul 16, 2015 at 2:02 PM, Akila Ravihansa
Perera <
>>>>>>>>>>> ravihansa@wso2.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Swapnil,
>>>>>>>>>>>>
>>>>>>>>>>>> Could you update the JIRA with latest status
of the project at
>>>>>>>>>>>> [1]. Please put relevant docs links, GitHub
repos and assign this task to
>>>>>>>>>>>> yourself.
>>>>>>>>>>>>
>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/STRATOS-500
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jun 29, 2015 at 4:48 PM, Swapnil
Patil <
>>>>>>>>>>>> swapnil.rp15@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Udara,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Okay. I will go through it and see if
we can use it instead of
>>>>>>>>>>>>> aws sdk.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for the link.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Will send update on this soon.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Swapnil
>>>>>>>>>>>>> On Jun 29, 2015 3:37 PM, "Udara Liyanage"
<udara@wso2.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Swapnil,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As you have mentioned during the
call, you are using aws Java
>>>>>>>>>>>>>> SDK to talk to AWS IaaS. However
it is better to use JClouds if possible.
>>>>>>>>>>>>>> JClouds is the library which we use
to talk to IaaS layer. It is better if
>>>>>>>>>>>>>> you can use it rather than adding
another dependency. Please review the
>>>>>>>>>>>>>> Jclouds lab [1] regarding was elb.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>> https://github.com/jclouds/jclouds-labs-aws/blob/master/aws-elb/pom.xml
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 29, 2015 at 12:13 PM,
Swapnil Patil <
>>>>>>>>>>>>>> swapnil.rp15@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Imesh,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ok. 2 PM today is fine for me.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>>>>> Swapnil
>>>>>>>>>>>>>>> On Jun 29, 2015 11:31 AM, "Imesh
Gunaratne" <
>>>>>>>>>>>>>>> imesh@apache.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Swapnil,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Shall we have a Hangout today
to go through the current
>>>>>>>>>>>>>>>> progress? I have sent you
a meeting request to have this today at 2:00 PM
>>>>>>>>>>>>>>>> IST. Let me know your availability.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Jun 29, 2015 at 11:14
AM, Swapnil Patil <
>>>>>>>>>>>>>>>> swapnil.rp15@gmail.com>
wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Last week I worked on
testing load balancer extension with
>>>>>>>>>>>>>>>>> ec2 cartridges. Till
now I was able to spawn ec2 cartridge and have Stratos
>>>>>>>>>>>>>>>>> receive these events
successfully.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I tested aws extension
on this. While deploying a new
>>>>>>>>>>>>>>>>> application, configure
method of extension was called. But there were few
>>>>>>>>>>>>>>>>> errors related to my
code changes.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> In this week I will continue
to test the extension and
>>>>>>>>>>>>>>>>> make necessary changes
in order to act on topology events properly and
>>>>>>>>>>>>>>>>> create and configure
aws load balancers.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>>>>>>> Swapnil
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Jun 23, 2015
at 9:11 AM, Swapnil Patil <
>>>>>>>>>>>>>>>>> swapnil.rp15@gmail.com>
wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Imesh,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks for the feedback.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Okay. I had tried
it earlier but couldn't start Stratos.
>>>>>>>>>>>>>>>>>> I guess may be it
was low memory error or may be the zip I was using was
>>>>>>>>>>>>>>>>>> defective. I will
try again on single EC2 instance with correct zip. And if
>>>>>>>>>>>>>>>>>> I get memory issues
I will try manual configuration with MB, CEP on
>>>>>>>>>>>>>>>>>> different EC2 instances.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>>>>>>>>> Swapnil
>>>>>>>>>>>>>>>>>> On Jun 23, 2015 2:08
PM, "Imesh Gunaratne" <
>>>>>>>>>>>>>>>>>> imesh@apache.org>
wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks for the
update Swapnil!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Jun 22,
2015 at 12:46 PM, Swapnil Patil <
>>>>>>>>>>>>>>>>>>> swapnil.rp15@gmail.com>
wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 3. To test
the other part (creating AWS load
>>>>>>>>>>>>>>>>>>>> balancers).
I need to create cartridges in EC2. For that I need to find a
>>>>>>>>>>>>>>>>>>>> way to access
my local machine from ec2 instance. I searched for a way to
>>>>>>>>>>>>>>>>>>>> do this.
It seems I need to do port forwarding and firewall changes in my
>>>>>>>>>>>>>>>>>>>> router to
be able to access my computer from an EC2 instance as per [3]. I
>>>>>>>>>>>>>>>>>>>> will look
into this in this week.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> It
would be easier to run ActiveMQ and Stratos on
>>>>>>>>>>>>>>>>>>> EC2.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Senior Technical
Lead, WSO2
>>>>>>>>>>>>>>>>>>> Committer &
PMC Member, Apache Stratos
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Senior Technical Lead, WSO2
>>>>>>>>>>>>>>>> Committer & PMC Member,
Apache Stratos
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Udara Liyanage
>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>> WSO2, Inc.: http://wso2.com
>>>>>>>>>>>>>> lean. enterprise. middleware
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> web: http://udaraliyanage.wordpress.com
>>>>>>>>>>>>>> phone: +94 71 443 6897
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Akila Ravihansa Perera
>>>>>>>>>>>> Software Engineer, WSO2
>>>>>>>>>>>>
>>>>>>>>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Imesh Gunaratne
>>>>>>>>
>>>>>>>> Senior Technical Lead, WSO2
>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Imesh Gunaratne
>>>>>>
>>>>>> Senior Technical Lead, WSO2
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Senior Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>

Mime
View raw message