flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Li (Jira)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-22534) Set delegation token's service name as credential alias
Date Tue, 11 May 2021 07:46:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-22534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342384#comment-17342384

Rui Li commented on FLINK-22534:

Hi [~zuston], let me try to understand this. The issue here is some inconsistency regarding
the DT alias we use. When generating DT for HDFS and HBase, we (or hadoop/hbase code) use
service name as DT alias. But for tokens in current UGI (as well as when retrieving tokens
in HadoopModule), we use token identifier as the alias. Further more, token identifiers may
not even be unique, such as in the case of HDFS HA mode. Therefore we should always use service
name as alias. Is this correct?

> Set delegation token's service name as credential alias
> -------------------------------------------------------
>                 Key: FLINK-22534
>                 URL: https://issues.apache.org/jira/browse/FLINK-22534
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / Hadoop Compatibility
>            Reporter: Junfan Zhang
>            Assignee: Junfan Zhang
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: debug2.PNG
> h4. What
> Set the Hadoop delegation token's service name as credential alias.
> h4. Why
> In current implementation, Flink will use delegation token's service name or identifer
as credential alias, refer to Flink code [HadoopModule|https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-runtime/src/main/java/org/apache/flink/runtime/security/modules/HadoopModule.java#L101]
and [Yarn Utils|https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-yarn/src/main/java/org/apache/flink/yarn/Utils.java#L209].
> Firstly, I think we could use the same way to set credential alias, like delegation token's
service name. It will be more clear.
> Secondly, when fetching HDFS delegation token and then inject all tokens to current UserGroupInformation
in Hadoop HDFS HA mode, it will cause the problem of overwriting the different delegation
tokens with the same identifier, [refer to code here|https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-yarn/src/main/java/org/apache/flink/yarn/Utils.java#L209].
> h5. When does the same identifier delegation tokens appear?
> When in HDFS HA mode, Hadoop HA delegation tokens will have the same identifier(Refer
to HDFS-9276), but its' service name is different. So we can use service name as alias.
> The following figure from HDFS-9276 can show that the identifier of HA delegation token
is the same.
>   !debug2.PNG!

This message was sent by Atlassian Jira

View raw message