nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: Suggestion required HTTP and S3
Date Thu, 20 Sep 2018 14:22:36 GMT
It depends which processor you are using...

With ListenHttp you can send "filename" as a header.

With HandleHttpRequest all the headers get added as attributes
starting with "http.headers.<header-name>" so if you sent "filename"
as a header you'd have "http.headers.filename" and if you needed to
move it to the filename attribute you could just use an
UpdateAttribute with filename = ${http.headers.filename}
On Thu, Sep 20, 2018 at 10:12 AM Rajesh Biswas
<rajesh.biswas@bridgera.com> wrote:
>
> Bryan,
> I mean the file extension is lost.
>
> Thanks and Regards,
> Rajesh Biswas | +91 9886433461 | www.bridgera.com
>
>
> -----Original Message-----
> From: Bryan Bende [mailto:bbende@gmail.com]
> Sent: Thursday, September 20, 2018 7:08 PM
> To: dev@nifi.apache.org
> Subject: Re: Suggestion required HTTP and S3
>
> Hello,
>
> I'm not sure what you mean by "upload the actual image file" ?
>
> A flow file is made up of attributes and content, and the content is
> just bytes which can be anything.
>
> If you make a POST to an end-point provided by ListenHTTP or
> HandleHttpRequest, and you send the image in the body of the POST,
> then the image will be the content of the flow file.
>
> I haven't used any S3 processors so I can't explain that behavior.
>
> -Bryan
>
> On Thu, Sep 20, 2018 at 8:00 AM Rajesh Biswas
> <rajesh.biswas@bridgera.com> wrote:
> >
> > Hello Bryan,
> > We started implanting the requirement based on your suggestion, but we faced below
issues:
> >
> > 1. Can we upload the actual image file instead of the flow-file that nifi creates?
> > (It is more like, can we convert the flow-file into its original format. i.e. jpg,
jpeg,png etc)
> > 2. Every time I try to make a upload request to S3 bucket, its making 2 requests
simultaneously,  i.e 2 request for 1 image.
> >
> > Thanks and Regards,
> > Rajesh Biswas | +91 9886433461 | www.bridgera.com
> >
> >
> > -----Original Message-----
> > From: Bryan Bende [mailto:bbende@gmail.com]
> > Sent: Tuesday, September 11, 2018 7:34 PM
> > To: dev@nifi.apache.org
> > Subject: Re: Suggestion required HTTP and S3
> >
> > I think the existing processors such as HandleHttpRequest can be used.
> > The body of the POST will become the flow file content, and the
> > headers will become flow file attributes.
> >
> > After HandleHttpRequest you can use RouteOnAttribute to make a
> > decision based on one of the headers (flow file attributes).
> >
> > If it makes your criteria then you send it to a PutS3Object processor
> > to write the flow file contents to S3.
> >
> > The success relationship of PutS3Object will have the flow file with
> > new attributes added for s3.bucket, s3.key, etc, which may be enough
> > for you to construct the URL.
> >
> > You could then use ReplaceText to overwrite the content of the flow
> > with some dynamic expression like ${host}/${s3.bucket}/${s3.key} (or
> > whatever the URL is).
> >
> > Then send to PublishKafka.
> >
> > Thanks,
> >
> > Bryan
> >
> >
> > On Tue, Sep 11, 2018 at 9:55 AM, Rajesh Biswas
> > <rajesh.biswas@bridgera.com> wrote:
> > > Dear Experts,
> > >
> > > I need your suggestion to design a requirement for my current project.
> > >
> > > I would like to listen for HTTP post request thru NiFi which will have image
> > > file attached.
> > >
> > > I need to 1st extract the HTTP header information (to understand the type of
> > > request), extract the image file and store the image file to S3 storage.
> > >
> > > Would you please suggest if I require to write any custom processor or
> > > existing Processors are sufficient.
> > >
> > > Also I would like to pass the file URL in S3 to a Kafka queue, would you
> > > please suggest what is the best way to get the S3 URL.
> > >
> > >
> > >
> > > Thanks and Regards,
> > >
> > > Rajesh Biswas | +91 9886433461 |  <http://www.bridgera.com/>
> > > www.bridgera.com
> > >
> > >
> > >
> > >
> > >
> > > ---
> > > This email has been checked for viruses by Avast antivirus software.
> > > https://www.avast.com/antivirus
> >
>

Mime
View raw message