nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Clarke <matt.clarke....@gmail.com>
Subject Re: splitText output appears to be getting dropped
Date Fri, 19 Feb 2016 02:00:02 GMT
Jeff,
      It appears you files are being dropped because your are
auto-terminating the failure relationship on your putFile processor. When
the splitText processor splits the file by lines every new file has the
same filename as the original it came from. My guess is the first file is
being worked to disk and all others are failing because a file of the same
name already exists in target dir. Try adding an UpdateAttribute processor
after the splitText to rename all the files. Easiest way is to append the
files uuid to its filename.  I also do not recommend auto-terminating
failure relationships except in rare cases.

Matt
On Feb 18, 2016 8:36 PM, "Jeff Lord" <jeffrey.lord@gmail.com> wrote:

> I have a pretty simple flow where I query for a list of ids using
> executeProcess and than pass that list along to splitText where I am trying
> to split on each line to than dynamically build a url further down the line
> using updateAttribute and so on.
>
> executeProcess -> splitText -> putFile
>
> For some reason I am only getting one file written with one line.
> I would expect something more like 100 files each with one line.
> Using the provenance reporter it appears that some of my items are being
> dropped.
>
> Time02/18/2016 17:13:46.145 PST
> Event DurationNo value set
> Lineage Duration00:00:12.187
> TypeDROP
> FlowFile Uuid7fa42367-490d-4b54-a32f-d062a885474a
> File Size14 bytes
> Component Id3b37a828-ba2c-4047-ba7a-578fd0684ce6
> Component NamePutFile
> Component TypePutFile
> DetailsAuto-Terminated by failure Relationship
>
> Any ideas on what I need to change here?
>
> Thanks in advance,
>
> Jeff
>

Mime
View raw message