libcloud-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clemens Wolff (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LIBCLOUD-1037) Azure Storage driver crashes when used with Azurite
Date Wed, 27 Feb 2019 20:18:00 GMT

     [ https://issues.apache.org/jira/browse/LIBCLOUD-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Clemens Wolff updated LIBCLOUD-1037:
------------------------------------
    Description: 
There's a popular emulator for Azure Storage called [Azurite|https://github.com/Azure/Azurite].
I tried to configure libcloud to talk to Azurite like so:

```
 from libcloud.storage.types import Provider
 from libcloud.storage.providers import get_driver
 \# the secret and account name are defined in the azurite documentation
 key = 'devstoreaccount1'
 secret = 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=='

 \# run azurite via `docker run -e executable=blob -d -t -p 10000:10000 -v arafato/azurite`
 host = 'localhost:10000'

AzureBlobs = get_driver(Provider.AZURE_BLOBS)
 driver = AzureBlobs(key, secret, host=host, secure=False)
 driver.create_container('somecontainer')
 ```

Running this script crashes: [Python stacktrace|https://user-images.githubusercontent.com/1086421/53447380-fa862c00-39e2-11e9-916a-97fd45be9de8.png],
[Azurite stacktrace|https://user-images.githubusercontent.com/1086421/53447528-4933c600-39e3-11e9-9723-3648f9823ef4.png].

The root cause is that the AzureBlobsStorageDriver currently assumes that the storage accounts
are always going to be identified via the hostname (e.g. someaccount.blob.core.windows.net)
but Azurite expects the accounts to be identified via a path prefix (e.g. /someaccount).

Fixing this will require a change in the driver to add a prefix to all action URLs if a custom
host is set. I believe it's valuable to make this change since it would enable the Azure Blobs
driver in libcloud to be used more easily for automated integration testing where developers
don't necessarily want to talk to a live Azure Storage account. I have a sample patch for
this issue at [CatalystCode/libcloud@1037_fix-azurite|https://github.com/apache/libcloud/compare/trunk...CatalystCode:1037_fix-azurite].

I suspect that also the new [Blob Storage on IoT edge|https://docs.microsoft.com/en-us/azure/iot-edge/how-to-store-data-blob]
will see the same kinds of errors since it also identifies accounts by URL prefix instead
of hostname.

  was:
There's a popular emulator for Azure Storage called [Azurite|https://github.com/Azure/Azurite].
I tried to configure libcloud to talk to Azurite like so:

```
 from libcloud.storage.types import Provider
 from libcloud.storage.providers import get_driver
 \# the secret and account name are defined in the azurite documentation
 key = 'devstoreaccount1'
 secret = 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=='

 \# run azurite via `docker run -e executable=blob -d -t -p 10000:10000 -v arafato/azurite`
 host = 'localhost:10000'

AzureBlobs = get_driver(Provider.AZURE_BLOBS)
 driver = AzureBlobs(key, secret, host=host, secure=False)
 driver.create_container('somecontainer')
 ```

Running this script crashes: [Python stacktrace|https://user-images.githubusercontent.com/1086421/53447380-fa862c00-39e2-11e9-916a-97fd45be9de8.png],
[Azurite stacktrace|https://user-images.githubusercontent.com/1086421/53447528-4933c600-39e3-11e9-9723-3648f9823ef4.png].

The root cause is that the AzureBlobsStorageDriver currently assumes that the storage accounts
are always going to be identified via the hostname (e.g. someaccount.blob.core.windows.net)
but Azurite expects the accounts to be identified via a path prefix (e.g. /someaccount).

Fixing this will require a change in the driver to add a prefix to all action URLs if a custom
host is set. I believe it's valuable to make this change since it would enable the Azure Blobs
driver in libcloud to be used more easily for automated integration testing where developers
don't necessarily want to talk to a live Azure Storage account. I have a sample patch for
this issue at [CatalystCode/libcloud@1037_fix-azurite|https://github.com/apache/libcloud/compare/trunk...CatalystCode:1037_fix-azurite].

I suspect that also the new [Blob Storage on IoT edge](https://docs.microsoft.com/en-us/azure/iot-edge/how-to-store-data-blob)
service will see the same kinds of errors since it also identifies accounts by URL prefix
instead of hostname.


> Azure Storage driver crashes when used with Azurite
> ---------------------------------------------------
>
>                 Key: LIBCLOUD-1037
>                 URL: https://issues.apache.org/jira/browse/LIBCLOUD-1037
>             Project: Libcloud
>          Issue Type: Bug
>            Reporter: Clemens Wolff
>            Priority: Major
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> There's a popular emulator for Azure Storage called [Azurite|https://github.com/Azure/Azurite].
I tried to configure libcloud to talk to Azurite like so:
> ```
>  from libcloud.storage.types import Provider
>  from libcloud.storage.providers import get_driver
>  \# the secret and account name are defined in the azurite documentation
>  key = 'devstoreaccount1'
>  secret = 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=='
>  \# run azurite via `docker run -e executable=blob -d -t -p 10000:10000 -v arafato/azurite`
>  host = 'localhost:10000'
> AzureBlobs = get_driver(Provider.AZURE_BLOBS)
>  driver = AzureBlobs(key, secret, host=host, secure=False)
>  driver.create_container('somecontainer')
>  ```
> Running this script crashes: [Python stacktrace|https://user-images.githubusercontent.com/1086421/53447380-fa862c00-39e2-11e9-916a-97fd45be9de8.png],
[Azurite stacktrace|https://user-images.githubusercontent.com/1086421/53447528-4933c600-39e3-11e9-9723-3648f9823ef4.png].
> The root cause is that the AzureBlobsStorageDriver currently assumes that the storage
accounts are always going to be identified via the hostname (e.g. someaccount.blob.core.windows.net)
but Azurite expects the accounts to be identified via a path prefix (e.g. /someaccount).
> Fixing this will require a change in the driver to add a prefix to all action URLs if
a custom host is set. I believe it's valuable to make this change since it would enable the
Azure Blobs driver in libcloud to be used more easily for automated integration testing where
developers don't necessarily want to talk to a live Azure Storage account. I have a sample
patch for this issue at [CatalystCode/libcloud@1037_fix-azurite|https://github.com/apache/libcloud/compare/trunk...CatalystCode:1037_fix-azurite].
> I suspect that also the new [Blob Storage on IoT edge|https://docs.microsoft.com/en-us/azure/iot-edge/how-to-store-data-blob]
will see the same kinds of errors since it also identifies accounts by URL prefix instead
of hostname.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message