jclouds-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Gaul (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (JCLOUDS-490) Security group listing on nova slow
Date Fri, 23 Jan 2015 22:07:34 GMT

     [ https://issues.apache.org/jira/browse/JCLOUDS-490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Andrew Gaul updated JCLOUDS-490:
    Labels: openstack-nova  (was: )

> Security group listing on nova slow
> -----------------------------------
>                 Key: JCLOUDS-490
>                 URL: https://issues.apache.org/jira/browse/JCLOUDS-490
>             Project: jclouds
>          Issue Type: Bug
>          Components: jclouds-compute
>    Affects Versions: 1.7.0, 1.7.1
>         Environment: Linux, Scala, Openstack
>            Reporter: Simon Hildrew
>              Labels: openstack-nova
> Calls to retrieve the security groups in an Openstack tenant using the jClouds ComputeService
API is inefficient and very slow.
> The following (scala) code is an example of a slow call:
> val context = ContextBuilder.newBuilder("openstack-nova")
>     .endpoint(origin.endpoint)
>     .credentials(s"${origin.tenant}:${origin.user}", origin.secret)
>     .build(classOf[ComputeServiceContext])
>   val compute = context.getComputeService
>   val securityGroupExtension = compute.getSecurityGroupExtension.get
>   val secGroups = securityGroupExtension.listSecurityGroups
>   val result = secGroups.map ( fromJCloud(_, existingGroups) )
> Most of this is very quick, but when debugging I the call to listSecurityGroups is slow.
In one example run this took 6 minutes and 17 seconds to retrieve 32 security groups. During
this example call, 93 separate requests are made to the Openstack API server. The first is
a call to GET /v2/<id>/extensions and the following 92 are calls to GET /v2/<id>/os-security-groups.
These 92 calls all get an identical response from the API and the calls are made with about
4 seconds delay. After all these calls the method returns with the correct response.
> One of the causes (but this doesn't seem to account for all the the excess calls I'm
seeing) is that the NovaSecurityGroupToSecurityGroup class makes calls to transform each openstack
SecurityGroupRule into a JClouds IpPermission object. This is done in the SecurityGroupRuleToIpPermission
class, which takes a Predicate<AtomicReference<ZoneAndName>> and the supplied
predicate is called for every transformation that occurs in which the source is another security
group instead of a CIDR. The implementation of the predicate is FindSecurityGroupWithNameAndReturnTrue
- which makes an API request for the list of security groups in a zone and then returns true
if the supplied reference is in the newly obtained list. The API call is made regardless of
whether it is the same zone or not.
> In the particular zone I'm querying, there are a large number of rules (60) that reference
other security groups (all in the same zone). As I said, this doesn't account for all the
calls for exactly the same request - but it does account for more than half of them. I think
there must be another transformation that also uses the predicate. 
> I note that the predicate actually modifies the object, which makes it trickier to remove
without changing the behaviour.
> As a work around I am now using the NovaApi directly and can report that it takes a few
seconds, instead of the 6 minutes.

This message was sent by Atlassian JIRA

View raw message