nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Payne <marka...@hotmail.com>
Subject Re: [Data Flow] File content not read completely
Date Thu, 15 Feb 2018 13:39:15 GMT
Valencia,

The SplitText processor does not change the ‘filename’ attribute of the FlowFile. So you
will end up with multiple FlowFiles having the same name. PutFile may well be overwriting
the same file many times - or failing to to write the files do to filename conflicts. You
can resolve this, if it’s your problem, by adding an UpdateAttribute to your flow just before
PutFile and changing the filename to something unique like ${UUID()} or ${filename}.${nextInt()}

Hope this helps!

-Mark

Sent from my iPhone

On Feb 15, 2018, at 4:59 AM, Valencia Serrao <vserrao@us.ibm.com<mailto:vserrao@us.ibm.com>>
wrote:


Hi All,

I've started hands-on with Nifi. Basic flows I was able to do without any
issues. But currently I've tried adding more steps to the flow.

Flow intent: Get a local file, split the text on new line, extract text based on regex, Put
matched/unmatched data on respective kafka topics and
finally write the kafka contents on the local targets set in PutFile.
Current Flow steps: GetFile, SplitText, ExtractText, PutKafka -( 2 of them,one for matched
and unmatched), and 2 PutFiles components.

The issue I'm facing is that - after the flow execution I see only one entry in each of the
2 PutFile targets and rest of the content is not written to them even if the criteria is matched.
I feel its not looping through the whole file or something like that. But I had read that
Nifi flow is executed for all contents in source files. Maybe I've missed some config somewhere.

It would be really helpful if anyone could help on this issue.

Regards,
Valencia
Mime
View raw message