nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Wing <jvw...@gmail.com>
Subject Re: Url encoding with S3 processors
Date Sun, 10 Sep 2017 17:31:38 GMT
Joe,

The good news is that you can use expression language directly in
FetchS3Object's Object Key property to decode your keys.  For example,

${filename:urlDecode()}

However, I'm not sure that could be done without you specifying that the
key should be decoded. As I understand it, the following two keys:

file with space.txt
file+with+space.txt

are both valid, but distinctly separate keys. If we decoded by default, we
could create confusion with unexpected key changes.

Thanks,

James

On Thu, Sep 7, 2017 at 6:59 AM, Joe Gresock <jgresock@gmail.com> wrote:

> Has anyone run into any URL encoding issues with the AWS nifi processors in
> version 1.3.0?  My flow has GetSQS and then FetchS3Object, along with the
> AWS feature of automated SQS events upon S3 file upload.  Some of my S3
> objects apparently have characters in their object keys that need to be URL
> encoded (e.g., %28 for open paren and + for space), and they come across
> encoded in the SQS message.  However, FetchS3Object apparently doesn't like
> this encoded URL, and it complains that the key does not exist.
>
> I suppose I could add an UpdateAttributes processor to translate certain
> urls in the keys, but this seems like a workaround.
>
>
> --
> I know what it is to be in need, and I know what it is to have plenty.  I
> have learned the secret of being content in any and every situation,
> whether well fed or hungry, whether living in plenty or in want.  I can do
> all this through him who gives me strength.    *-Philippians 4:12-13*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message