libcloud-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Russell Alexander Spitzer (JIRA)" <j...@apache.org>
Subject [jira] [Created] (LIBCLOUD-527) GCE Libcloud Driver list_nodes and list_volumes Scaling Improvement
Date Fri, 07 Mar 2014 16:46:43 GMT
Russell Alexander Spitzer created LIBCLOUD-527:
--------------------------------------------------

             Summary: GCE Libcloud Driver list_nodes and list_volumes Scaling Improvement
                 Key: LIBCLOUD-527
                 URL: https://issues.apache.org/jira/browse/LIBCLOUD-527
             Project: Libcloud
          Issue Type: Improvement
    Affects Versions: 0.14.0-beta3
            Reporter: Russell Alexander Spitzer


The current code for the GCE Driver has two significant bottle necks in the list_nodes and
list_volumes code. In both cases the slowdown can be traced to the response to object methods
_to_node and _to_storage_volume. These methods are called in serial from both list_node and
list_volume methods and each call to one of these methods requires an additional http request.
For list_nodes the price is doubled since each _to_node request requires a _to_storage_volume
request.

Basically, list_nodes and list_volumes require 1 + 2 * Nodes and 1 + Volumes requests to complete.
I've found that at large numbers of nodes this is a considerable pain point.

I've been working on a branch at
https://github.com/RussellSpitzer/libcloud/tree/GCE_perf_to_node
that I would be happy to contribute upstream if there is any desire for it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message