nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajesh Biswas" <rajesh.bis...@bridgera.com>
Subject RE: Suggestion required HTTP and S3
Date Thu, 20 Sep 2018 14:12:12 GMT
Bryan,
I mean the file extension is lost.

Thanks and Regards,
Rajesh Biswas | +91 9886433461 | www.bridgera.com


-----Original Message-----
From: Bryan Bende [mailto:bbende@gmail.com] 
Sent: Thursday, September 20, 2018 7:08 PM
To: dev@nifi.apache.org
Subject: Re: Suggestion required HTTP and S3

Hello,

I'm not sure what you mean by "upload the actual image file" ?

A flow file is made up of attributes and content, and the content is
just bytes which can be anything.

If you make a POST to an end-point provided by ListenHTTP or
HandleHttpRequest, and you send the image in the body of the POST,
then the image will be the content of the flow file.

I haven't used any S3 processors so I can't explain that behavior.

-Bryan

On Thu, Sep 20, 2018 at 8:00 AM Rajesh Biswas
<rajesh.biswas@bridgera.com> wrote:
>
> Hello Bryan,
> We started implanting the requirement based on your suggestion, but we faced below issues:
>
> 1. Can we upload the actual image file instead of the flow-file that nifi creates?
> (It is more like, can we convert the flow-file into its original format. i.e. jpg, jpeg,png
etc)
> 2. Every time I try to make a upload request to S3 bucket, its making 2 requests simultaneously,
 i.e 2 request for 1 image.
>
> Thanks and Regards,
> Rajesh Biswas | +91 9886433461 | www.bridgera.com
>
>
> -----Original Message-----
> From: Bryan Bende [mailto:bbende@gmail.com]
> Sent: Tuesday, September 11, 2018 7:34 PM
> To: dev@nifi.apache.org
> Subject: Re: Suggestion required HTTP and S3
>
> I think the existing processors such as HandleHttpRequest can be used.
> The body of the POST will become the flow file content, and the
> headers will become flow file attributes.
>
> After HandleHttpRequest you can use RouteOnAttribute to make a
> decision based on one of the headers (flow file attributes).
>
> If it makes your criteria then you send it to a PutS3Object processor
> to write the flow file contents to S3.
>
> The success relationship of PutS3Object will have the flow file with
> new attributes added for s3.bucket, s3.key, etc, which may be enough
> for you to construct the URL.
>
> You could then use ReplaceText to overwrite the content of the flow
> with some dynamic expression like ${host}/${s3.bucket}/${s3.key} (or
> whatever the URL is).
>
> Then send to PublishKafka.
>
> Thanks,
>
> Bryan
>
>
> On Tue, Sep 11, 2018 at 9:55 AM, Rajesh Biswas
> <rajesh.biswas@bridgera.com> wrote:
> > Dear Experts,
> >
> > I need your suggestion to design a requirement for my current project.
> >
> > I would like to listen for HTTP post request thru NiFi which will have image
> > file attached.
> >
> > I need to 1st extract the HTTP header information (to understand the type of
> > request), extract the image file and store the image file to S3 storage.
> >
> > Would you please suggest if I require to write any custom processor or
> > existing Processors are sufficient.
> >
> > Also I would like to pass the file URL in S3 to a Kafka queue, would you
> > please suggest what is the best way to get the S3 URL.
> >
> >
> >
> > Thanks and Regards,
> >
> > Rajesh Biswas | +91 9886433461 |  <http://www.bridgera.com/>
> > www.bridgera.com
> >
> >
> >
> >
> >
> > ---
> > This email has been checked for viruses by Avast antivirus software.
> > https://www.avast.com/antivirus
>


Mime
View raw message