flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Lin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-10601) Make user home dir consistent with Flink default filesystem
Date Thu, 18 Oct 2018 12:08:00 GMT
Paul Lin created FLINK-10601:

             Summary: Make user home dir consistent with Flink default filesystem
                 Key: FLINK-10601
                 URL: https://issues.apache.org/jira/browse/FLINK-10601
             Project: Flink
          Issue Type: Improvement
          Components: YARN
    Affects Versions: 1.6.1
            Reporter: Paul Lin

Currently, Flink client uses the raw default Hadoop filesystem to upload local files, and
this could be problematic when using a non-default filesystem for HA or checkpointing in delegation
tokens scenarios. The jobmanager only has the delegation tokens for the default FS, so it
gets authentication errors when trying to connect a non-default filesystem.

So I propose to replace the default FS property in yarn configuration with the Flink filesystem
property `fs.default-scheme` on the client side (AbstractYarnClusterDescriptor), to avoid
this problem and also make the client behavior more configurable.

This message was sent by Atlassian JIRA

View raw message