hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sunlisheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15893) fs.TrashPolicyDefault: can't create trash directory and race condition
Date Thu, 29 Nov 2018 09:36:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16702927#comment-16702927
] 

sunlisheng commented on HADOOP-15893:
-------------------------------------

I add a patch.

> fs.TrashPolicyDefault: can't create trash directory and race condition
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-15893
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15893
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: common
>            Reporter: sunlisheng
>            Priority: Major
>         Attachments: HADOOP-15893.001.patch, HADOOP-15893.002.patch
>
>
> there is race condition in method moveToTrash class TrashPolicyDefault
> try {
>  if (!fs.mkdirs(baseTrashPath, PERMISSION))
> { // create current LOG.warn("Can't create(mkdir) trash directory: " + baseTrashPath);
return false; }
> } catch (FileAlreadyExistsException e) {
>  // find the path which is not a directory, and modify baseTrashPath
>  // & trashPath, then mkdirs
>  Path existsFilePath = baseTrashPath;
>  while (!fs.exists(existsFilePath))
> { existsFilePath = existsFilePath.getParent(); }
> {color:#FF0000}// case{color}
> {color:#FF0000}  other thread deletes existsFilePath here ,the results doesn't  meet
expectation{color}
> {color:#FF0000} for example{color}
> {color:#FF0000}   there is /user/u_sunlisheng/.Trash/Current/user/u_sunlisheng/b{color}
> {color:#FF0000}   when delete /user/u_sunlisheng/b/a. if existsFilePath is deleted,
the result is /user/u_sunlisheng/.Trash/Current/user/u_sunlisheng+timstamp/b/a{color}
> {color:#FF0000}  so  when existsFilePath is deleted, don't modify baseTrashPath. 
  {color}
> baseTrashPath = new Path(baseTrashPath.toString().replace(
>  existsFilePath.toString(), existsFilePath.toString() + Time.now())
>  );
> trashPath = new Path(baseTrashPath, trashPath.getName());
>  // retry, ignore current failure
>  --i;
>  continue;
>  } catch (IOException e)
> { LOG.warn("Can't create trash directory: " + baseTrashPath, e); cause = e; break; }
> After catch FileAlreadyExistsException, the  same name file is deleted.
> So don't modify baseTrashPath when existsFilePath is deleted.
> But this case show hardly in untitest.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message