nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chakrader Dewaragatla <Chakrader.Dewaraga...@lifelock.com>
Subject Re: FetchS3Object processor
Date Thu, 08 Oct 2015 18:32:39 GMT
Thanks Aldrin. GenerateFlowFile helped to trigger.

Thanks,
-Chakri

From: Aldrin Piri <aldrinpiri@gmail.com<mailto:aldrinpiri@gmail.com>>
Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Date: Thursday, October 8, 2015 at 11:22 AM
To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Subject: Re: FetchS3Object processor

Chakri,

The FetchS3Object processor does not act as a source directly and is instead driven by incoming
FlowFiles.  The use case was that one could utilize Expression Language to interact with buckets
and paths in a dynamic fashion.  Alternatively, if you are looking to just grab a specific
file, you can use GenerateFlowFile to "trigger" the FetchS3Object with its static configuration.

On Thu, Oct 8, 2015 at 1:45 PM, Chakrader Dewaragatla <Chakrader.Dewaragatla@lifelock.com<mailto:Chakrader.Dewaragatla@lifelock.com>>
wrote:
Thanks Joe.
I have S3fetch and putfile processors setup, files are not fetching despite files exist on
s3. Am I missing anything ? Does S3fetch processor need any additional processor to feed data
?

Thanks,
-Chakri


From: Joe Skora <jskora@gmail.com<mailto:jskora@gmail.com>>
Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Date: Thursday, October 8, 2015 at 3:38 AM
To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Subject: Re: FetchS3Object processor

Chakri,

The Amazon docs explain here<http://docs.aws.amazon.com/AmazonS3/latest/UG/FolderOperations.html>
that S3 doesn't support pathes within a bucket, but you can embed path-like naming into the
"Object Key" parameter to FetchS3Object.

For your example, you would use bucket="s3://<bucket>" and key="<path>/<objectfile>"
to incorporate the path as you store the object.

I hope that helps.

Regards,
Joe

On Thu, Oct 8, 2015 at 2:39 AM, Chakrader Dewaragatla <Chakrader.Dewaragatla@lifelock.com<mailto:Chakrader.Dewaragatla@lifelock.com>>
wrote:
Nifi users  - FetchS3Object processor do not have option to specify Object file path, how
do you set that ?

Eg : s3://<bucket>/<path>/<objectfile>

Thanks,
-Chakri
________________________________
The information contained in this transmission may contain privileged and confidential information.
It is intended only for the use of the person(s) named above. If you are not the intended
recipient, you are hereby notified that any review, dissemination, distribution or duplication
of this communication is strictly prohibited. If you are not the intended recipient, please
contact the sender by reply email and destroy all copies of the original message.
________________________________

________________________________
The information contained in this transmission may contain privileged and confidential information.
It is intended only for the use of the person(s) named above. If you are not the intended
recipient, you are hereby notified that any review, dissemination, distribution or duplication
of this communication is strictly prohibited. If you are not the intended recipient, please
contact the sender by reply email and destroy all copies of the original message.
________________________________

________________________________
The information contained in this transmission may contain privileged and confidential information.
It is intended only for the use of the person(s) named above. If you are not the intended
recipient, you are hereby notified that any review, dissemination, distribution or duplication
of this communication is strictly prohibited. If you are not the intended recipient, please
contact the sender by reply email and destroy all copies of the original message.
________________________________

Mime
View raw message