nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Taft <a...@adamtaft.com>
Subject Re: Keep Files
Date Mon, 16 Nov 2015 01:00:09 GMT
Also, as a potential work-around, it's possible to use GetFile with
"delete" mode and then somewhere in your flow, use PutFile to place the
file back down into a "complete" directory.  i.e. something like:

/path/incoming  <- use GetFile to pick up files here
/path/complete  <- use PutFile to place files here after processing

As a variation of the above, if you need the files consistently in the same
directory, you could configure GetFile to only pick up certain file
patterns.  In this way, you could rename a file after it has been processed:

/path/incoming  <- use GetFile to pick up files named $filename.new
/path/incoming  <- rename file (using UpdateAttribute) to
$filename.complete and use PutFile to place files here after rename

Hope that gives you some possible alternatives.

Adam



On Sat, Nov 14, 2015 at 10:49 AM, Mark Petronic <markpetronic@gmail.com>
wrote:

> Keep, yes, There is a parameter to configure that. Read once. No. But there
> is a set of processors in the works to address that. ListFile and
> FetchFile. ListFile will return the list of files that have changed since
> the last time the files were read - it is stateful. FetchFile can then take
> a list and fetch them, and I would assume it would have a parameter for
> keep=<yes|no> like GetFile. Not sure of the status of the changes - have
> not checked recently but see:
> https://issues.apache.org/jira/browse/NIFI-631
>
> Mark
>
> On Fri, Nov 13, 2015 at 8:55 AM, plj <plj@mitre.org> wrote:
>
> > Is there a way for GetFile to not delete a file but only read it once?  I
> > have a directory with files in it.  I only want the new files that are
> > added
> > to the to be processed.  It seems that if I set GetFile to not delete the
> > files, the same files get read over and over.
> >
> >
> > thoughts?
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/Keep-Files-tp4864.html
> > Sent from the Apache NiFi Developer List mailing list archive at
> > Nabble.com.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message