nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: PutTCP connector not cleaning up dangling connections
Date Wed, 20 Sep 2017 19:36:15 GMT
For what it is worth, I can't reproduce this on my Mac laptop either...

Setup GenerateFlowFile with same sample text your provided, followed
by PutTCP with the \r\n delimiter, this sends to ListenTCP in the same
NiFi instance.

If I look at the content of the flow files coming out of ListenTCP,
and I put the content viewer in Hex mode, I see:

0x00000000 0D 40 43 4F 4D 4D 41 4E 44 0D 0D .@COMMAND..

The 0D 40 43 4F 4D 4D 41 4E 44 0D 0D lines up with \r@COMMAND\r  which
would make sense since ListenTCP read-off the \r\n at the end.


On Wed, Sep 20, 2017 at 1:20 PM, Bryan Bende <bbende@gmail.com> wrote:
> I'll keep looking at this, but just to summarize the changes that were
> made in case something jumps out to someone else...
>
> The changes were to avoid reading the entire content of the flow file
> into memory.
>
> Previous Behavior:
> - Create a ByteArrayOuputStream
> - Use an InputStreamCallback to read entire bytes of flow file, and
> write them to above output stream
> - Get value of the outgoing delimiter property
> - If delimiter is not null, then get the bytes of the delimiter using
> the charset configured in the processor, and write the delimiter bytes
> to the above output stream
> - Get byte array from the ByteArrayOutputStream and pass it to the
> channel sender, which in turns calls write on a SocketChannel
>
> New behavior:
> - Get the direct SockeOutputStream from the sender
> - Get the InputStream of the flow file
> - Call IOUtils.copy to copy the flow file input stream to the socket
> output stream in chunks
> - Perform exact same delimiter logic as above
>
>
>
>
> On Wed, Sep 20, 2017 at 1:13 PM, Davy De Waele <ddewaele@gmail.com> wrote:
>> Correct about the delimiter.
>> Think character set is (was) automatically filled in with utf-8. (Never
>> noticed a default option)
>>
>> On Wed, 20 Sep 2017 at 19:11, Bryan Bende <bbende@gmail.com> wrote:
>>>
>>> Thanks for the info and for digging into this..
>>>
>>> Just to confirm, in your PutTCP configuration, you are saying that you
>>> have "Outgoing Message Delimiter" set to \r\n  ?
>>>
>>> And do you have the "Character Set" property set to the default of "UTF-8"
>>> ?
>>>
>>> On Wed, Sep 20, 2017 at 12:59 PM, ddewaele <ddewaele@gmail.com> wrote:
>>> > I've put up a git repo that I use to test the PutTCP behavior.
>>> >
>>> >
>>> > (https://github.com/ddewaele/com.ixortalk.tcp/compare/dev_puttcp_1.4?expand=1)
>>> >
>>> > What I've done.
>>> >
>>> > - Built Nifi 1.4.0-SNAPSHOT + your PR and used that as my base nifi
>>> > installation
>>> > - Noticed weird characters in the tcp responses
>>> > - Switched to 1.3.0. No more weird characters
>>> > - Isolated the PutTCP processor by creating a custom PutTCP (see github
>>> > repo
>>> > below) to experiment (instead of the built-in PutTCP)
>>> > - Repo has 2 branches. master contains an "older" PutTCP processor from
>>> > the
>>> > Nifi code base. With that one I don't have the weird characters (even on
>>> > a
>>> > nifi 1.4.0-SNAPSHOT + your PR).
>>> > - If I use the dev_puttcp_1.4 branch on my nifi I get the weird
>>> > characters.
>>> >
>>> >
>>> > Some info on my setup :
>>> >
>>> > I'm sending commands to PutTCP using a GenerateFlowFile processor.
>>> > payload = ${literal('\r')}@COMMAND${literal('\r')}
>>> > PutTCP adds a delimier (\r\n)
>>> >
>>> > Regarding provenance. PutTCP adds the delimiter to the payload before
>>> > sending it to the TCP socket. That  delimiter is not in the flowfile
>>> > that is
>>> > put in provenance I think.
>>> >
>>> > Cannot reproduce the issue on my mac with my mock tcp server. But on the
>>> > actual environment (Linux CentOS) with the Moxa devices, there is a
>>> > clear
>>> > difference in behaviour. Will see if I can tunnel my through the devices
>>> >
>>> > All the info I have right now ....
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/

Mime
View raw message