jclouds-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (JCLOUDS-1008) Google storage provider cannot retrieve "foo/bar"
Date Tue, 20 Oct 2015 22:25:27 GMT

    [ https://issues.apache.org/jira/browse/JCLOUDS-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14965884#comment-14965884

ASF subversion and git services commented on JCLOUDS-1008:

Commit 94b3cba6c254aaf8d67e633d2fe46b3c4c30a2c2 in jclouds's branch refs/heads/master from
[ https://git-wip-us.apache.org/repos/asf?p=jclouds.git;h=94b3cba ]

JCLOUDS-1008: Add @Encoded annotation.

Certain providers (e.g. Google Cloud Storage) place tokens that should
be encoded in the request path (e.g. GET
http://<host>/b/<bucket>/o/<object>) and expect them to be
percent-encoded. In the above example a GET request for "foo/bar"
should be translated to http://<host>/b/<bucket>/o/foo%2Fbar.
Currently, there is no way to express this in jclouds, as the entire
request path is encoded exactly once and there is no control over
whether a request parameter should be handled specially. In the
example above, "/" are not encoded in the path and the URL is
submitted as "http://<host>/b/<bucket>/o/foo/bar", which may be wrong.

This patch extends the annotation processor to support @Encoded for
the individual parameters of the request. However, this means that the
entire path is _NOT_ URL encoded. The caller *must* make sure that the
appropriate parameters are encoded -- ones that are marked with the
@Encoded annotation. Parameters not marked with the @Encoded
annotation are URI encoded prior to being added to the path. This
means that "/" characters will also be URI encoded in this case (i.e.
"foo/bar" is turned into "foo%2Fbar").

For the Google Storage provider, we will annotate the parameters that
are going to be pre-encoded (object names) and ensure the provider
encodes them prior to calling the API (separate patch in

> Google storage provider cannot retrieve "foo/bar"
> -------------------------------------------------
>                 Key: JCLOUDS-1008
>                 URL: https://issues.apache.org/jira/browse/JCLOUDS-1008
>             Project: jclouds
>          Issue Type: Bug
>          Components: jclouds-core, jclouds-labs-google
>    Affects Versions: 1.9.0, 1.9.1, 2.0.0
>            Reporter: Timur Alperovich
> The Google cloud storage provider can PUT, but not GET a blob containing "/" in its name.
When attempting to retrieve "foo/bar", _null_ is always returned.
> The reason for this behavior is twofold. First, Google expects "/" in the path parameters
to be percent-encoded. Secondly, jclouds-core does not have a way to express that a specific
path parameter should be encoded prior to being added to the path.
> Currently, the rest processor does the following for the endpoint: assemble all of the
tokens together and call {code}urlEncode(){code} on the result. When it does so, it also skips
encoding any "/" characters. If an encoded token is passed to the processor, it ends up being
encoded twice, as the "%" character is encoded as %25.
> jclouds-core should add {code}@Encoded{code} parameter annotation, which would encode
each parameter individually (including any "/" characters) before assembling the URL. In this
case, the consumers of the API must make sure that they do not need the entire path to be
> I plan to submit a PR to add the annotation and to use it in the google-cloud-storage
provider. When the annotation is not used, endpoints will be processed as is.

This message was sent by Atlassian JIRA

View raw message