nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: Error log in ListenTCP
Date Tue, 09 Oct 2018 15:33:11 GMT
ListenTCP only uses the Character Set property for writing the
demarcator to the output stream of the flow file, it takes the String
value of Character Set and calls value.getBytes(charset).

For the actual data it is never converted from bytes to characters, so
it takes byte[] from the incoming stream and writes byte[] directly to
the flow file output stream.
On Tue, Oct 9, 2018 at 11:17 AM Rajesh Biswas
<rajesh.biswas@bridgera.com> wrote:
>
> Hello Bryan,
>
> We need to read binary stream data thru TCP.
> As the default chatacter encoding is UTF-8 for ListenTCP, the binary data is getting
altered after passing through the processors.
> Would you please suggest any other alternative?
> Below is the sample data:
> "00010001990010000015B00100019900101001"
> The processor group we are using
> ListenTCP -> PutFile
>
> Configuration for ListenTCP
> Local Network InterfaceNo value set  No Value
> Port 9111
> Receive Buffer Size 65507 B
> Max Size of Message Queue 10000
> Max Size of Socket Buffer 1 MB
> Character Set UTF-8
> Max Batch Size 1
> Batching Message Delimiter \n
> Max Number of TCP Connections 2
> SSL Context ServiceNo value set
> Client Auth NONE
>
>
> Thank you very much:
>
>
> Thanks and Regards,
> Rajesh Biswas | +91 9886433461 | www.bridgera.com
>
> -----Original Message-----
> From: Bryan Bende [mailto:bbende@gmail.com]
> Sent: Tuesday, October 9, 2018 1:20 AM
> To: dev@nifi.apache.org
> Subject: Re: Error log in ListenTCP
>
> ListenTCP uses new-line characters to determine logical message boundaries,
> so the data itself does not have to be characters, but there has to be
> new-lines indicating the end of a logical piece of data.
>
> Imagine streaming the content of several images over the same connection,
> the client would have to send something like the following... bytes of
> image 1, new line, bytes of image 2, new line, bytes of image 3, new line,
> etc.
>
>
> On Mon, Oct 8, 2018 at 1:52 PM Rajesh Biswas <rajesh.biswas@bridgera.com>
> wrote:
>
> > Hello Team,
> >
> > I am facing one issue to get binary data into Nifi.
> >
> > As of now I am using ListenTCP processor, when I am sending character
> > string data I am able to see the data in flow file.
> >
> > But when I am trying to send Binary data, I am not able to see any data
> > thru Data province.
> >
> > Would you please guide me:
> >
> > 1.       Which log files to see if there are any error (I checked
> > nifi-app.log and nifi-user.log, did not see anything)
> >
> > 2.       Which processor to use to read binary data thru TCP protocol (I
> > am not able to change Character Set other than UTF-8)
> >
> >
> >
> > Would you please help me with your experience?
> >
> >
> >
> > Thanks and Regards,
> >
> > *Rajesh Biswas* | +91 9886433461 | www.bridgera.com
> >
> >
> >
> >
> > <https://www.avast.com/en-in/recommend?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient&utm_term=default3&tag=8220120d-9b09-4fc1-8af8-da3fc2cdea29>
I’m
> > protected online with Avast Free Antivirus. Get it here — it’s free
> > forever.
> > <https://www.avast.com/en-in/recommend?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient&utm_term=default3&tag=8220120d-9b09-4fc1-8af8-da3fc2cdea29>
> > <#m_-3931440512116331171_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> >
>
>
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>

Mime
View raw message