hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13082) RawLocalFileSystem does not fail when moving file to a non-existing directory
Date Thu, 05 May 2016 19:19:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272899#comment-15272899

Chris Nauroth commented on HADOOP-13082:

bq. I do not really understand this. "rename-creates-dest-dirs" are set correctly now in either
in localfs.xml or rawlocal.xml. I can't find contract-test-options.

I think the issue may be that the failure is in {{FileSystemContractBaseTest}}.  This is a
"contract" test suite, but it predates the larger contract test framework that Steve wrote,
so it doesn't respect the contract options like rename-creates-dest-dirs.

Short-term, I think skipping via a subclass override makes sense.  Longer term, perhaps we
need to look at consolidating all test suites named "Contract" back into the larger contract
test framework.  There is likely some redundancy between the test cases, and eliminating the
redundancy might improve test execution times.

As far as this JIRA, I agree with resolving it as Later.  I don't anticipate being able to
change this behavior of {{RawLocalFileSystem}} any time soon because of the downstream ecosystem

Echoing Steve's comment, thank you very much for digging into this.  Semantic differences
across the different file systems are tricky issues.

> RawLocalFileSystem does not fail when moving file to a non-existing directory
> -----------------------------------------------------------------------------
>                 Key: HADOOP-13082
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13082
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.23.0
>            Reporter: Andras Bokor
>            Assignee: Andras Bokor
> FileSystemContractBaseTest#testRenameFileMoveToNonExistentDirectory: creates a file then
move it to a non-existing directory. It should fail but it will not (with RawLocalFileSystem)
because in RawLocalFileSystem#rename(Path, Path) method we have a fallback behavior that accomplishes
the rename by a full copy. The full copy will create the new directory and copy the file there.
> I see two possible solutions here:
> # Remove the fallback full copy behavior
> # Before full cp we should check whether the parent directory exists or not. If not return
false an do not do the full copy.
> The fallback logic was added by [HADOOP-9805|https://issues.apache.org/jira/browse/HADOOP-9805].

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message