libcloud-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "charles walker (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LIBCLOUD-991) Can not retrieve more than 500VM on Google cloud
Date Mon, 09 Apr 2018 12:16:01 GMT

    [ https://issues.apache.org/jira/browse/LIBCLOUD-991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430435#comment-16430435
] 

charles walker commented on LIBCLOUD-991:
-----------------------------------------

I think first i need to spend some time understanding the logic because it seems I miss something.
Talking with another DEV in my company about that he was surprise and told me it was already
fix and send me a commit: [https://github.com/ansible/ansible/pull/18554/files]

I realize we were not talking about the same thing since it is a commit in Ansible but seems
like to the same topic. After some more reading on the code it seems the 500 limitation is
well know and in fact should maybe taken care on customer side (with pagination support) like
Ansible done but i m still wondering if it would be nicer to have it directly integrated on
libcloud...

Maybe with a new method using GCE_LIST ([https://github.com/apache/libcloud/blob/trunk/libcloud/compute/drivers/gce.py#L209)] ?
I still thinking about it and wondering if this Jira ticket is the best place to gather feedback
on this topic 

> Can not retrieve more than 500VM on Google cloud
> ------------------------------------------------
>
>                 Key: LIBCLOUD-991
>                 URL: https://issues.apache.org/jira/browse/LIBCLOUD-991
>             Project: Libcloud
>          Issue Type: Bug
>          Components: Compute
>         Environment: Google cloud project with more than 500VMs (python 3 and last libcloud
version AVL on pip)
>            Reporter: charles walker
>            Priority: Minor
>
> Today I notice that some of our VMs were not retrieve by our internal toolings. After
some investigation it seems that libcloud "list_nodes()" method on google can only retrieve
max 500VMs:
>  
> [root@stackviewer-5cbcd8695-x964t /]# grep -o Vm response.log | wc -l
> 500
>  
> when we have in reality around 750 VMs. So i had a quick look and my first guess is that
it comes from the call made by liblcoud using this REST API:
> [https://cloud.google.com/compute/docs/reference/rest/v1/instances/aggregatedList]
> with an optional argument "maxResults" that has a default value of 500 (which seems also
to be the max value BTW). I think i m on the good lead due to this 500 limit mentioned in
the doc and also observe on my side.
> I will have a deeper look but just quickly open this ticket in case other people already
saw this limit
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message