nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Keeney <nextves...@gmail.com>
Subject Re: ListFile, FetchFile PutSFTP Some files not transferred, No errors
Date Wed, 03 May 2017 00:45:43 GMT
Thanks. This gives me a lot to work with. I will have additional questions
but i'm going to try and figure all this out.

On Tue, May 2, 2017 at 12:00 PM James Wing <jvwing@gmail.com> wrote:

> Jim,
>
> Your description of different success by user makes me think of file
> permissions.  Does NiFi have equal access to those files?  As far as
> general troubleshooting, I recommend the following:
>
> 1.)  Copy/pasting your ListFile and FetchFile separately so you can
> wrestle with their properties, clear the state as necessary, stop, start,
> etc. without disrupting your regular flow.
>
> 2.)  Try to clarify if the problem is listing or fetching the unprocessed
> files, by running the processors selectively and monitoring all of the
> output relationships.  Are you handling the various failure relationships
> for FetchFile?
>
> 3.)  Also, you might try using GenerateFlowFile -> FetchFile to mock up a
> list entry for one of the unprocessed files and directly test FetchFile.
> FetchFile might provide more helpful feedback through its various failure
> relationships, including permissions.
>
> 4.)  There's always interactive debugging, if that's something you're
> comfortable with.
>
>
> Thanks,
>
> James
>
> On Tue, May 2, 2017 at 6:00 AM, James Keeney <nextvestor@gmail.com> wrote:
>
>> I have setup a processor chain with ListFile, FetchFile and then PutSFTP.
>>
>> Users FTP to the server that has NIFI and then NIFI is used to transfer
>> the files to our production servers using PutSFTP.
>>
>> When one user performs the transfer everything works, but when another
>> does it some of the files are not transferred.
>>
>> I have checked and all of the files made it onto the server with the
>> initial FTP transfer. I setup the PutSFTP so a reject or failure would
>> route the file to a directory using putfile. No files end up in that
>> directory so PutSFTP appears to be working correctly. So, It appears that
>> the breakdown occurs in the ListFile, FetchFile process. How would I go
>> about debugging this? No errors are thrown that I can see.
>>
>> Jim K.
>> --
>> Jim Keeney
>> President, FitterWeb
>> E: jim@fitterweb.com
>> M: 703-568-5887 <(703)%20568-5887>
>>
>> *FitterWeb Consulting*
>> *Are you lean and agile enough? *
>>
>
> --
Jim Keeney
President, FitterWeb
E: jim@fitterweb.com
M: 703-568-5887

*FitterWeb Consulting*
*Are you lean and agile enough? *

Mime
View raw message