spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christophe PRÉAUD (JIRA) <>
Subject [jira] [Created] (SPARK-4764) Ensure that files are fetched atomically
Date Fri, 05 Dec 2014 14:22:13 GMT
Christophe PRÉAUD created SPARK-4764:

             Summary: Ensure that files are fetched atomically
                 Key: SPARK-4764
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 1.1.0
            Reporter: Christophe PRÉAUD
            Priority: Minor

It does not seem necessary in the doFetchFile method to first download the file into a local
directory then move it (it may cause a copy instead of a rename).
tempFile (where the file is downloaded) should be created in the same directory than targetFile,
so that the move from tempFile to targetFile is always atomic

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message