nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Serff <andrew.se...@acesinc.net>
Subject Re: File Upload to a ListenHTTP Processor
Date Mon, 08 Feb 2016 20:46:10 GMT
Thanks for the suggestions everyone, but they don’t seem to work.  A couple of problems:

  1.  using --data(-binary) posts the data in the request body. This works for small files,
but if I post something larger (Mbs) like an mp3 file for example, the content gets truncated
for some reason when it gets into NiFi. Also according to the form spec, multipart/form-data
should be used for larger file due to performance: https://www.w3.org/TR/html401/interact/forms.html#h-17.13.4
  2.  Looking at what gets added to the file, the content type of the form data is set to
an octet-stream.

To me, it looks like the ListenHTTP processor just doesn’t handle multipart requests.  All
it does it take the Request input stream and writes it to the content of the FlowFile.  I
guess maybe I should submit an issue against it?

Andrew

From: Jeroen Jacobs <jeroen.jacobs@headincloud.be<mailto:jeroen.jacobs@headincloud.be>>
Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Date: Friday, February 5, 2016 at 5:44 PM
To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Subject: Re: File Upload to a ListenHTTP Processor


Hi,


Something like this should do the trick indeed:



curl -X POST -d @maven-4.0.0.xsd http://localhost:8080/ingest --header "Content-Type:text/xml"

However, I'm not sure it will set the file extension correctly. Since the MIME type is set
text/xml, it might give the file an .xml extension.


Kind regards,

Jeroen.


________________________________
From: Andy LoPresto <alopresto.apache@gmail.com<mailto:alopresto.apache@gmail.com>>
Sent: Saturday, February 6, 2016 12:45 AM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>; Joe Percivall
Subject: Re: File Upload to a ListenHTTP Processor

Andrew,

You may also want to change the content type you are providing. I understand you may not know
the format a priori, but “form-data” is probably not what you want. Look at `-H "Content-Type:
text/xml”` or `-H “Content-Type: application/octet-stream”` for arbitrary binary data.


Andy LoPresto
alopresto.apache@gmail.com<mailto:alopresto.apache@gmail.com>
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Feb 5, 2016, at 3:34 PM, Joe Percivall <joepercivall@yahoo.com<mailto:joepercivall@yahoo.com>>
wrote:

Hello Andrew,

I believe I was running into something similar before and my problem was actually with the
curl command itself. When I added "--data-binary" to the command it worked.

Relevant SO question: http://stackoverflow.com/questions/9134003/binary-data-posting-with-curl

Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall<http://linkedin.com/in/Percivall>
e: joepercivall@yahoo.com<mailto:joepercivall@yahoo.com>



On Friday, February 5, 2016 5:30 PM, Andrew Serff <andrew.serff@acesinc.net<mailto:andrew.serff@acesinc.net>>
wrote:



Hello, I’m new to NiFi, and I’m just trying out my options for ingest into a data flow
we are trying to set up.  We want to expose both the capability to SFTP a file into a dropbox
or perform an HTTP File Upload. So I’ve set up both a GetFile and ListenHTTP processors.
 After both processors, I just add a success relationship to a PutFile processor to write
the file to an archive directory.  The dropbox works as expected, however the ListenHTTP processor
isn’t working quite as I’d expect it to, so I’m hoping someone can point out what I’m
doing wrong.

As a simple case, I just set up a ListenHTTP processor and have it listening on port 8080
at the “ingest” context.  So I can hit it at http://localhost:8080/ingest. No security
or anything yet. Then from the command line, I’m trying to upload file like so:

curl -i -X POST -H "Content-Type: multipart/form-data" -H "filename: maven-4.0.0.xsd" -F "data=@maven-4.0.0.xsd<mailto:data=@maven-4.0.0.xsd>"
http://localhost:8080/ingest

The file is uploaded and written to the directory, however it is wrapped with the form encoding.
 So the beginning of the file now looks like:

--------------------------78b35889e5299cc2
Content-Disposition: form-data; name="data"; filename="maven-4.0.0.xsd"
Content-Type: application/octet-stream

<?xml version="1.0"?>
<!-- =================== DO NOT EDIT THIS FILE ====================         -->
<!-- Generated by Modello 1.8.3,                                            -->
<!-- any modifications will be overwritten.                                 -->
<!-- ==============================================================         -->
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" xmlns="http://maven.apache.org/POM/4.0.0"
targetNamespace="http://maven.apache.org/POM/4.0.0">
 <xs:element name="project" type="Model">
   <xs:annotation>


Obviously the file is no longer a valid xsd file nor the same as what was sent from the source
system. I used an xsd file just for testing, but we could have any type of file (binary, text,
what have you…). I have also tried this same upload using a Java client and I get the same
result.

Can anyone let me know how we can get this to work?
Thanks
Andrew

Mime
View raw message