jclouds-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Everett Toews <everett.to...@RACKSPACE.COM>
Subject Re: Security group listing on nova slow
Date Mon, 17 Feb 2014 21:45:33 GMT
abayer, this is SecurityGroupExtension related. Do you have some time to look into it?


On Feb 17, 2014, at 1:10 PM, Simon Hildrew <simon.hildrew@theguardian.com<mailto:simon.hildrew@theguardian.com>>

On 3 February 2014 21:01, Simon Hildrew <simon.hildrew@theguardian.com<mailto:simon.hildrew@theguardian.com>>
On 3 February 2014 20:51, Andrew Phillips <andrewp@apache.org<mailto:andrewp@apache.org>>
Making a call to listSecurityGroupsInLocation and passing in the single
zone I'm interested in has made no difference - it is still making the
underlying call to the API many many times before the jclouds API call
returns. It is always exactly 92 calls to the underlying API

Does this include things like authentication calls, or is this the "straight" call for security
groups? If so, is there any obvious variation in parameters or other request characteristics?
Or is it literally always the same call?

There are two other calls to start with - one to keystone to auth and then another to get
the extensions in order to locate the security group endpoint. The following 92 calls are
always identical and get an identical response each time (as observed via both tcpdump and
today using the jclouds.wire logging). It's really odd behaviour!

Hopefully, one of the OpenStack experts on the list can shed more light on where these might
be coming from.

That would be great - I'm happy to post some of my logs or carry out any experiments that
might help shed more light. The problem seems to be worse in some region / tenants than others
so I'm going to try the same experiment in some other places tomorrow and see if there is
a pattern.

I've been continuing to try and shed more light on this, but have been super busy with other
things over the last fortnight.

I think I may have worked out what the root cause of the behaviour I'm seeing is, although
I see no simple solution and would love some advice from the OpenStack experts.

The initial list happens in one go. But in the NovaSecurityGroupToSecurityGroup class they
make calls to transform each openstack SecurityGroupRule into a JClouds IpPermission object.
This is done in the SecurityGroupRuleToIpPermission class, which takes a Predicate<AtomicReference<ZoneAndName>>
and the supplied predicate is called for every transformation that occurs in which the source
is another security group instead of a CIDR. The implementation of the predicate is FindSecurityGroupWithNameAndReturnTrue
- which makes an API request for the list of security groups in a zone and then returns true
if the supplied reference is in the newly obtained list. The API call is made regardless of
whether it is the same zone or not.

In the particular zone I'm querying, there are a large number of rules (60) that reference
other security groups (all in the same zone). This doesn't account for the now 111 calls for
exactly the same request - but it does account for more than half of them. I think there must
be another transformation that also uses the predicate (I'd carry on looking but it's past
time for me to go home).

Part of me wonders whether I should use the NovaApi directly and do the translation myself.

As always, any help or thoughts gratefully received.


Please consider the environment before printing this email.
Visit theguardian.com<http://theguardian.com>

On your mobile, download the Guardian iPhone app theguardian.com/iphone<http://theguardian.com/iphone>
and our iPad edition theguardian.com/iPad<http://theguardian.com/iPad>
Save up to 57% by subscribing to the Guardian and Observer - choose the papers you want and
get full digital access.
Visit subscribe.theguardian.com<http://subscribe.theguardian.com>

This e-mail and all attachments are confidential and may also
be privileged. If you are not the named recipient, please notify
the sender and delete the e-mail and all attachments immediately.
Do not disclose the contents to another person. You may not use
the information for any purpose, or store, or copy, it in any way.

Guardian News & Media Limited is not liable for any computer
viruses or other material transmitted with or as part of this
e-mail. You should employ virus checking software.

Guardian News & Media Limited

A member of Guardian Media Group plc
Registered Office
PO Box 68164
Kings Place
90 York Way

Registered in England Number 908396


View raw message