nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomislav Novosel <to.novo...@gmail.com>
Subject Re: Minimum file age
Date Mon, 28 Jan 2019 12:08:13 GMT
Thank you for replies and suggestions.

Yeah, it is a little disturbing lack of feature, but it's ok. For now, as I
have control over remote server, I use dotted file and remove dot (".")
after file is finished.
Ignore dotted file property set to true.

Cheers,
Tom

On Mon, 28 Jan 2019 at 11:09, <Josef.Zahner1@swisscom.com> wrote:

> Hi Tom
>
>
>
> I suggest to use a Wait Processor (to delay the fetch) together with
> DetectDuplicate Processor. In that way you will fetch the file only once
> and after it has been written completely (as long as you know how long it
> takes in max. to finish writing). I know it’s not nice but that’s how we do
> it for the moment… I’m waiting for this feature as well :-(.
>
>
>
> Cheers Josef
>
>
>
>
>
> *From: *Arpad Boda <aboda@hortonworks.com>
> *Reply-To: *"users@nifi.apache.org" <users@nifi.apache.org>
> *Date: *Monday, 28 January 2019 at 10:17
> *To: *"users@nifi.apache.org" <users@nifi.apache.org>
> *Subject: *Re: Minimum file age
>
>
>
> Hi,
>
>
>
> It’s on the way: https://issues.apache.org/jira/browse/NIFI-5977 :)
>
>
>
> Regards,
>
> Arpad
>
>
>
> *From: *Tomislav Novosel <to.novosel@gmail.com>
> *Reply-To: *"users@nifi.apache.org" <users@nifi.apache.org>
> *Date: *Monday, 28 January 2019 at 09:19
> *To: *"users@nifi.apache.org" <users@nifi.apache.org>
> *Subject: *Minimum file age
>
>
>
> Hi all,
>
>
>
> I'm having issue with SFTPList processor in Nifi. When reading files from
> folder where another process writes files, it lists the same file multiple
> time and ingests file multiple times because modification date of file
> changes rapidly as another process writes to it.
>
>
>
> It appears that Nifi lists faster than external process writes, so before
> the end of writing (conversion of file from one format to another), Nifi
> lists file multiple time and then creates duplicates.
>
>
>
> There is no property of Minimum file age like in ListFile processor.
>
>
>
> How can I resolve this to wait for a moment when the file is converted
> completely and then to list file and pass it to FetchSFTP processor?
>
>
>
> Thanks in advance,
>
> Tom.
>

Mime
View raw message