jclouds-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (JCLOUDS-1137) FilesystemStorageStrategyImpl throws NoSuchFileException when listing files
Date Wed, 10 Aug 2016 22:10:20 GMT

    [ https://issues.apache.org/jira/browse/JCLOUDS-1137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15416112#comment-15416112
] 

ASF subversion and git services commented on JCLOUDS-1137:
----------------------------------------------------------

Commit 4874a1eb185870eef88731080ae8cf9ceebc3be2 in jclouds's branch refs/heads/gsoc2016-ivan
from [~gaul]
[ https://git-wip-us.apache.org/repos/asf?p=jclouds.git;h=4874a1e ]

JCLOUDS-1137: Handle TOCTOU during blobMetadata

A similar issue exists when getting a blob payload when a caller
simultaneously removes the blob.


> FilesystemStorageStrategyImpl throws NoSuchFileException when listing files
> ---------------------------------------------------------------------------
>
>                 Key: JCLOUDS-1137
>                 URL: https://issues.apache.org/jira/browse/JCLOUDS-1137
>             Project: jclouds
>          Issue Type: Bug
>          Components: jclouds-core
>    Affects Versions: 1.9.2
>            Reporter: Francois Rigault
>            Assignee: Andrew Gaul
>              Labels: filesystem
>             Fix For: 2.0.0
>
>
> When listing files in a local blobstore, we are sometimes hitting the following exception:
> {noformat}
> java.nio.file.NoSuchFileException: /opt/data/./ggg/fff_local_db.db-wal
>         at com.google.common.base.Throwables.propagate(Throwables.java:160)
>         at org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.getBlob(FilesystemStorageStrategyImpl.java:373)
>         at org.jclouds.blobstore.config.LocalBlobStore.loadBlob(LocalBlobStore.java:414)
> ...
> Caused by: java.nio.file.NoSuchFileException: /opt/data/./ggg/fff_local_db.db-wal
>         at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
>         at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
>         at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
>         at sun.nio.fs.UnixPath.openForAttributeAccess(UnixPath.java:787)
>         at sun.nio.fs.LinuxUserDefinedFileAttributeView.list(LinuxUserDefinedFileAttributeView.java:100)
>         at org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.getBlob(FilesystemStorageStrategyImpl.java:333)
> {noformat}
> There is some kind of race condition when a file is being removed in the middle of a
getBlob: to reproduce, one can put a breakpoint in FilesystemStorageStrategyImpl before the
call to UserDefinedFileAttributeView.list, and remove the current file from the file system
when hitting the breakpoint.
> The workaround we are using is to provide our own version of FilesystemStorageStrategyImpl
that patches the getBlob method in this fashion:
> {code:java}
>     @Override
>     public Blob getBlob(String container, String key) {
>         try {
>             return super.getBlob(container, key);
>         } catch (RuntimeException t) {
>             if (t.getCause() instanceof NoSuchFileException) {
>                 log.warn("file `" + key + "' has been removed during the getBlob", t);
>                 return newBlob("deleted." + key + "." + System.currentTimeMillis());
>             } else {
>                 throw t;
>             }
>         }
>     }
> {code}
> This ensures a blob is returned. Returning null is not enough as LocalBlobStore.list
would thrown an exception "blob is not present although it was in the list of container".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message