hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-16878) FileUtil.copy() to throw IOException if the source and destination are the same
Date Tue, 01 Jun 2021 10:37:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-16878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17354995#comment-17354995
] 

Steve Loughran commented on HADOOP-16878:
-----------------------------------------

I think we should roll back for now as its too late for a fix. It's not a critical feature,
just something which surfaced in our QE workflows which proved a PITA to track down.

bq. This doesn't seems to handle the case where the nameservice of both source & target
filesystem is same, but they point to different clusters. 

Tell me more? You saying we can have >1 path which resolve to different places?

> FileUtil.copy() to throw IOException if the source and destination are the same
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-16878
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16878
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 3.3.0
>            Reporter: Gabor Bota
>            Assignee: Gabor Bota
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.4.0
>
>         Attachments: hdfsTest.patch
>
>          Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> We encountered an error during a test in our QE when the file destination and source
path were the same. This happened during an ADLS test, and there were no meaningful error
messages, so it was hard to find the root cause of the failure.
> The error we saw was that file size has changed during the copy operation. The new file
creation in the destination - which is the same as the source - creates a file and sets the
file length to zero. After this, getting the source file will fail because the sile size changed
during the operation.
> I propose a solution to at least log in error level in the {{FileUtil}} if the source
and destination of the copy operation is the same, so debugging issues like this will be easier
in the future.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message