flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (Jira)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-10954) Hardlink from files of previous local stored state might cross devices
Date Wed, 03 Jun 2020 10:54:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-10954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17124855#comment-17124855

Stephan Ewen commented on FLINK-10954:

I am wondering if we even need to handle multiple directories in the RocksDB state backend.

If there are multiple directories on different file systems, then we cannot use them together,
because we cannot hardlink across file system boundaries.

If the different directories are on the same file system, then why do we need different directories
at all? Then the effect should be the same if we use one or more directories, we still have
the same storage capacity etc.

Having multiple directories on the same file system is only relevant for the batch algorithms,
because there every directory gets a different async thread for spilling / pre-fetching.

So, could we simplify this by saying that RocksDB randomly chooses one directory from the
temp.dirs at the beginning, and then all the code assumes a single local root directory? This
sounds like it would lead to the simplest implementation and not be worse.

What do you think, [~yunta] 

> Hardlink from files of previous local stored state might cross devices
> ----------------------------------------------------------------------
>                 Key: FLINK-10954
>                 URL: https://issues.apache.org/jira/browse/FLINK-10954
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / State Backends
>    Affects Versions: 1.6.2
>            Reporter: Yun Tang
>            Assignee: Yun Tang
>            Priority: Critical
>             Fix For: 1.12.0
> Currently, local recovery's base directories is initialized from '{{io.tmp.dirs}}' if
parameter '{{taskmanager.state.local.root-dirs}}' is not set. For Yarn environment, the tmp
dirs is replaced by its '{{LOCAL_DIRS}}', which might consist of directories from different
devices, such as /dump/1/nm-local-dir, /dump/2/nm-local-dir. The local directory for RocksDB
is initialized from IOManager's spillingDirectories, which might located in different device
from local recovery's folder. However, hard-link between different devices is not allowed,
it will throw exception below:
> {code:java}
> java.nio.file.FileSystemException: target -> souce: Invalid cross-device link
> {code}

This message was sent by Atlassian Jira

View raw message