From notifications-return-40000-apmail-jclouds-notifications-archive=jclouds.apache.org@jclouds.apache.org Wed Jan 25 17:13:34 2017 Return-Path: X-Original-To: apmail-jclouds-notifications-archive@www.apache.org Delivered-To: apmail-jclouds-notifications-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BD4BD1900C for ; Wed, 25 Jan 2017 17:13:34 +0000 (UTC) Received: (qmail 60336 invoked by uid 500); 25 Jan 2017 17:13:34 -0000 Delivered-To: apmail-jclouds-notifications-archive@jclouds.apache.org Received: (qmail 60311 invoked by uid 500); 25 Jan 2017 17:13:34 -0000 Mailing-List: contact notifications-help@jclouds.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@jclouds.apache.org Delivered-To: mailing list notifications@jclouds.apache.org Received: (qmail 60301 invoked by uid 99); 25 Jan 2017 17:13:34 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 25 Jan 2017 17:13:34 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 413B1185D86 for ; Wed, 25 Jan 2017 17:13:34 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.199 X-Spam-Level: X-Spam-Status: No, score=-1.199 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id xL3CbgyT9kjW for ; Wed, 25 Jan 2017 17:13:31 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 2A3635F2F1 for ; Wed, 25 Jan 2017 17:13:31 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 93EFFE040F for ; Wed, 25 Jan 2017 17:13:27 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id D211125295 for ; Wed, 25 Jan 2017 17:13:26 +0000 (UTC) Date: Wed, 25 Jan 2017 17:13:26 +0000 (UTC) From: "Ignasi Barrera (JIRA)" To: notifications@jclouds.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (JCLOUDS-1229) Azure ARM - No retry is the rate limit is exceeded MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/JCLOUDS-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15838160#comment-15838160 ] Ignasi Barrera commented on JCLOUDS-1229: ----------------------------------------- [~tbouron] could you confirm that adding the rate limit module to the context mitigates or fixes your issue? > Azure ARM - No retry is the rate limit is exceeded > -------------------------------------------------- > > Key: JCLOUDS-1229 > URL: https://issues.apache.org/jira/browse/JCLOUDS-1229 > Project: jclouds > Issue Type: Bug > Components: jclouds-compute, jclouds-labs > Affects Versions: 2.0.0 > Reporter: Thomas Bouron > Labels: azurecompute-arm > > Trying to provision a VM in Azure ARM. My deployment failed because of the rate limit exceeded with the stacktrace below. > I was expecting jclouds to perform a back-off retry as it does for other clouds but does not seem to be the case (See {{AzureRateLimitRetryHandler}} - is that wired in correctly, and does it correctly handle 429 response code)? > There were 390 API calls made in 2mins 22 secs before getting this exception. > {noformat} > 2017-01-25 11:57:04,702 DEBUG 106 o.j.r.i.InvokeHttpMethod [nager-XBhutgvo-4] >> invoking resourcegroup:list > 2017-01-25 11:57:04,702 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService [nager-XBhutgvo-4] Sending request 300674855: GET https://management.azure.com/subscriptions/341751b0-xxxx-45ce-9498-xxxxxxxxxxxx/resourcegroups?api-version=2015-01-01 HTTP/1.1 > 2017-01-25 11:57:04,729 DEBUG 106 o.j.i.ContentMetadataCodec$DefaultContentMetadataCodec [nager-XBhutgvo-4] Invalid Expires header (-1); should be in RFC-1123 format; treating as already expired: Error parsing data at 0 > 2017-01-25 11:57:04,729 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService [nager-XBhutgvo-4] Receiving response 300674855: HTTP/1.1 200 OK > 2017-01-25 11:57:04,732 DEBUG 106 o.j.r.i.InvokeHttpMethod [nager-XBhutgvo-4] >> invoking storageaccount:list > 2017-01-25 11:57:04,733 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService [nager-XBhutgvo-4] Sending request -641560959: GET https://management.azure.com/subscriptions/341751b0-xxxx-45ce-9498-xxxxxxxxxxxx/resourcegroups/aleks-test-res/providers/Microsoft.Storage/storageAccou > 2017-01-25 11:57:04,763 DEBUG 143 o.a.b.u.r.Repeater [nager-XBhutgvo-6] Repeater (repeated failure; excluding stacktrace): java.lang.AssertionError: service.isUp expected equals true but found false > 2017-01-25 11:57:05,264 DEBUG 143 o.a.b.u.r.Repeater [nager-XBhutgvo-6] Repeater (repeated failure; excluding stacktrace): java.lang.AssertionError: service.isUp expected equals true but found false > 2017-01-25 11:57:05,764 DEBUG 143 o.a.b.u.r.Repeater [nager-XBhutgvo-6] Repeater (repeated failure; excluding stacktrace): java.lang.AssertionError: service.isUp expected equals true but found false > 2017-01-25 11:57:05,938 DEBUG 106 o.j.i.ContentMetadataCodec$DefaultContentMetadataCodec [nager-XBhutgvo-4] Invalid Expires header (-1); should be in RFC-1123 format; treating as already expired: Error parsing data at 0 > 2017-01-25 11:57:05,938 DEBUG 106 o.j.h.o.OkHttpCommandExecutorService [nager-XBhutgvo-4] Receiving response -641560959: HTTP/1.1 429 > 2017-01-25 11:57:05,940 WARN 126 o.a.b.l.j.JcloudsLocation [nager-XBhutgvo-4] Error loading available images to report (following original error matching template which will be rethrown): org.jclouds.azurecompute.arm.exceptions.AzureComputeRateLimitExceededException: HTT > {x-ms-ratelimit-remaining-subscription-reads=[14947]} > org.jclouds.azurecompute.arm.exceptions.AzureComputeRateLimitExceededException: HTTP/1.1 429 > {x-ms-ratelimit-remaining-subscription-reads=[14947]} > at org.jclouds.azurecompute.arm.handlers.AzureComputeErrorHandler.handleError(AzureComputeErrorHandler.java:73) > at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:65) > at org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:140) > at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:109) > at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90) > at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73) > at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44) > at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117) > at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87) > at com.sun.proxy.$Proxy174.list(Unknown Source) > at org.jclouds.azurecompute.arm.compute.AzureComputeServiceAdapter.listImages(AzureComputeServiceAdapter.java:231) > at org.jclouds.compute.config.ComputeServiceAdapterContextModule$2.get(ComputeServiceAdapterContextModule.java:121) > at org.jclouds.compute.config.ComputeServiceAdapterContextModule$2.get(ComputeServiceAdapterContextModule.java:118) > at org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier$SetAndThrowAuthorizationExceptionSupplierBackedLoader.load(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:75) > at org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier$SetAndThrowAuthorizationExceptionSupplierBackedLoader.load(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:57) > at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527) > at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319) > at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282) > at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197) > at com.google.common.cache.LocalCache.get(LocalCache.java:3937) > at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941) > at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824) > at org.jclouds.rest.suppliers.MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.get(MemoizedRetryOnTimeOutButNotOnAuthorizationExceptionSupplier.java:150) > at org.jclouds.compute.suppliers.ImageCacheSupplier.get(ImageCacheSupplier.java:106) > at org.jclouds.compute.suppliers.ImageCacheSupplier.get(ImageCacheSupplier.java:58) > at org.jclouds.compute.internal.BaseComputeService.listImages(BaseComputeService.java:384) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.8.0_111-internal] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:1.8.0_111-internal] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_111-internal] > at java.lang.reflect.Method.invoke(Method.java:498)[:1.8.0_111-internal] > at com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:37)[69:com.google.inject:3.0.0] > at com.sun.proxy.$Proxy155.listImages(Unknown Source)[:] > at org.apache.brooklyn.location.jclouds.JcloudsLocation.logAvailableTemplates(JcloudsLocation.java:1581)[126:org.apache.brooklyn.locations-jclouds:0.11.0.20170123_2005] > at org.apache.brooklyn.location.jclouds.JcloudsLocation.buildTemplate(JcloudsLocation.java:1432)[126:org.apache.brooklyn.locations-jclouds:0.11.0.20170123_2005] > at org.apache.brooklyn.location.jclouds.JcloudsLocation.obtainOnce(JcloudsLocation.java:676)[126:org.apache.brooklyn.locations-jclouds:0.11.0.20170123_2005] > at org.apache.brooklyn.location.jclouds.JcloudsLocation.obtain(JcloudsLocation.java:606)[126:org.apache.brooklyn.locations-jclouds:0.11.0.20170123_2005] > at org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ObtainLocationTask.call(MachineLifecycleEffectorTasks.java:443)[131:org.apache.brooklyn.software-base:0.11.0.20170123_2005] > at org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ObtainLocationTask.call(MachineLifecycleEffectorTasks.java:433)[131:org.apache.brooklyn.software-base:0.11.0.20170123_2005] > at org.apache.brooklyn.util.core.task.Tasks.withBlockingDetails(Tasks.java:106)[120:org.apache.brooklyn.core:0.11.0.20170123_2005] > at org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ProvisionMachineTask.call(MachineLifecycleEffectorTasks.java:414)[131:org.apache.brooklyn.software-base:0.11.0.20170123_2005] > at org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ProvisionMachineTask.call(MachineLifecycleEffectorTasks.java:390)[131:org.apache.brooklyn.software-base:0.11.0.20170123_2005] > at org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:359)[120:org.apache.brooklyn.core:0.11.0.20170123_2005] > at org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:522)[120:org.apache.brooklyn.core:0.11.0.20170123_2005] > at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_111-internal] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_111-internal] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_111-internal] > at java.lang.Thread.run(Thread.java:745)[:1.8.0_111-internal] > Caused by: org.jclouds.http.HttpResponseException: command: GET https://management.azure.com/subscriptions/341751b0-xxxx-45ce-9498-xxxxxxxxxxxx/resourcegroups/aleks-test-res/providers/Microsoft.Storage/storageAccounts HTTP/1.1 failed with response: HTTP/1.1 429 ; content: > at org.jclouds.azurecompute.arm.handlers.AzureComputeErrorHandler.handleError(AzureComputeErrorHandler.java:45) > ... 46 more > 2017-01-25 11:57:05,949 ERROR 126 o.a.b.l.j.JcloudsLocation [nager-XBhutgvo-4] Failed to start VM for azurecompute-arm:southeastasia:https://management.azure.com/subscriptions/341751b0-xxxx-45ce-9498-xxxxxxxxxxxx@MongoDBServerImpl{id=k9ah7ty4jo, hostname=null, port=null} > {x-ms-ratelimit-remaining-subscription-reads=[14756]} > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)