jclouds-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Timur Alperovich (JIRA)" <j...@apache.org>
Subject [jira] [Created] (JCLOUDS-955) jclouds strips off the delimiter from common prefixes (at least in S3)
Date Thu, 02 Jul 2015 23:54:04 GMT
Timur Alperovich created JCLOUDS-955:

             Summary: jclouds strips off the delimiter from common prefixes (at least in S3)
                 Key: JCLOUDS-955
                 URL: https://issues.apache.org/jira/browse/JCLOUDS-955
             Project: jclouds
          Issue Type: Bug
            Reporter: Timur Alperovich

I noticed that jclouds strips off the delimiter when returning results that include common
prefixes. For example, if blobs "test/foo" and "test/bar" are in the container, listing the
container non-recursively (i.e. setting the delimiter to "/"), will return "test", whereas
the providers commonly return "test/".

AWS S3\[[#1]\], OpenStack Swift\[#2\], Google Cloud Storage\[[#3]\], and Azure\[[#4]\] all
include the delimiter string (or character, in the case of Swift) in the results.

This may not be a valid issue if this was always the intent of jclouds to elide the delimiter
parameter. In which case, could someone elaborate about the reasons for that here? If this
is the expected behavior, I would be happy to amend the prefix changes to _not_ include the
delimiter in the Local Blob Store listings. Otherwise, I'm also happy to expose the results
as-is in the other providers.

{anchor:1} 1. From [AWS S3 List  documentation|http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGET.html]:
"In effect, CommonPrefixes lists keys that act like subdirectories in the directory specified
by Prefix. For example, if prefix is notes/ and delimiter is a slash \(/\), in notes/summer/july,
the common prefix is notes/summer/."

{anchor:2} 2. Swift does not explicitly document this, however, using the swift client, one
can observe the behavior where specifying "\-" as a delimiter results in the following json
(the container has keys "tmp-foo" and "tmp-bar"): \{"subdir": "tmp-"\}.

{anchor:3} 3. From the [GCS API|https://cloud.google.com/storage/docs/json_api/v1/objects/list]:
"Objects whose names, aside from the prefix, contain delimiter will have their name, truncated
after the delimiter, returned in prefixes."

{anchor:4} 4. From the [Azure Blobs API|https://msdn.microsoft.com/en-us/library/azure/hh452233.aspx]:
"The value of the BlobPrefix element is substring+delimiter, where substring is the common
substring that begins one or more blob names, and delimiter is the value of the delimiter

This message was sent by Atlassian JIRA

View raw message