libcloud-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Russell Alexander Spitzer (JIRA)" <>
Subject [jira] [Commented] (LIBCLOUD-527) GCE Libcloud Driver list_nodes and list_volumes Scaling Improvement
Date Fri, 07 Mar 2014 19:23:45 GMT


Russell Alexander Spitzer commented on LIBCLOUD-527:

In _to_node:2973 -2977
        extra['boot_disk'] = None
        for disk in extra['disks']:
            if disk.get('boot') and disk.get('type') == 'PERSISTENT':
                bd = self._get_components_from_path(disk['source'])
                extra['boot_disk'] = self.ex_get_volume(bd['name'], bd['zone']) # Requires
2 requests

in _to_storage_volume:3108
        extra['zone'] = self.ex_get_zone(volume['zone'])# Requires 1 Request

> GCE Libcloud Driver list_nodes and list_volumes Scaling Improvement
> -------------------------------------------------------------------
>                 Key: LIBCLOUD-527
>                 URL:
>             Project: Libcloud
>          Issue Type: Improvement
>    Affects Versions: 0.14.0-beta3
>            Reporter: Russell Alexander Spitzer
> The current code for the GCE Driver has two significant bottle necks in the list_nodes
and list_volumes code. In both cases the slowdown can be traced to the response to object
methods _to_node and _to_storage_volume. These methods are called in serial from both list_node
and list_volume methods and each call to one of these methods requires an additional http
request. For list_nodes the price is doubled since each _to_node request requires a _to_storage_volume
> Basically, list_nodes and list_volumes require 1 + 2 * Nodes and 1 + Volumes requests
to complete. I've found that at large numbers of nodes this is a considerable pain point.
> I've been working on a branch at
> that I would be happy to contribute upstream if there is any desire for it.

This message was sent by Atlassian JIRA

View raw message