nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <ozhurakou...@hortonworks.com>
Subject Re: How does GetSFTP treat files being loaded?
Date Fri, 03 Jun 2016 15:10:42 GMT
Huagen

Just to clarify. There isn’t really an SFTP server. SFTP is just an FTP layer over SSH -
allowing you to get FTP-like experience when accessing file system over SSH.
Now, tis your question.
I am assuming you are implying that some other system is uploading a large file. In any event
a typical and well established pattern for upload/download is to use temporary name until
upload/download has finished and then rename. This way the consuming system doesn’t see
“work-in-progress” until it is finished. In other words  the consuming system will never
see that file until its fully uploaded. 

Hope this helps.
Cheers
Oleg

> On Jun 3, 2016, at 10:59 AM, Huagen peng <huagen.peng@gmail.com> wrote:
> 
> Hi,
> 
> I need to get files from a SFTP server and then remove the files afterward.  GetSFTP
seems to be the processor to use.  If a user uploads a large file, say 20G, to the server
and the GetSFTP processor happens to be running in the middle of the uploading, what is the
expected behavior.  Does the processor pick up the file? If so, can the processor get the
entire file and then safely delete it? Does anybody has experience on that?
> 
> Huagen

Mime
View raw message