hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-13155) Implement TokenRenewer to renew and cancel delegation tokens in KMS
Date Sat, 28 May 2016 00:24:12 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Xiao Chen updated HADOOP-13155:
    Attachment: HADOOP-13155.04.patch

Had an offline review with [~yzhangal], and patch 4 addressing his comments:

* {{KMSTokenRenewer}}, use its own logger
* Added more logs when {{KMSTokenRenewer}} found the keyProvider is not a DTExt instance
* Regarding the template usage when creating delegation tokens:
** The way of creating a new {{Token<AbstractDelegationTokenIdentifier>}} for {{DelegationTokenAuthenticatedURL$Token#setDelegationToken}}
seems verbose. 
Since we're accepting a generic type, I think this is the safe way to go. Casting may end
up throwing exceptions. I refactored KMSCP with a {{generateDelegationToken}} method to do
this for both the renew and cancel.
** Also, constructing the Token<ADTI> using the 4 parameters seems non-optimal
However, I don't feel changing its copy constructor to accepting Token<?> is a good
idea... IIUC the template class Token is supposed to only accept {{T}}. For this reason, I
didn't change anything. Feel free to comment if you think otherwise.

One thing Yongjun also brought up is the move of {{dfs.encryption.key.provider.uri}} from
{{HdfsClientConfigKeys}} to {{CommonConfigurationKeys}}.
- The reason of this move is that the renewer is in common (and kms), hence we need the util
method to create provider in common, hence the need of reading that config from common. 
- I left the name dfs.xxx for compatibility, but it's a bit weird to have a dfs.* in common
configurations. Not sure what's the best way of handling this.. [~andrew.wang], do you have
any advice on it? Thanks!

> Implement TokenRenewer to renew and cancel delegation tokens in KMS
> -------------------------------------------------------------------
>                 Key: HADOOP-13155
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13155
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Xiao Chen
>            Assignee: Xiao Chen
>         Attachments: HADOOP-13155.01.patch, HADOOP-13155.02.patch, HADOOP-13155.03.patch,
HADOOP-13155.04.patch, HADOOP-13155.pre.patch
> Service DelegationToken (DT) renewal is done in Yarn by {{org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer}},
where it calls {{Token#renew}} and uses ServiceLoader to find the renewer class ([code|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java#L382]),
and invokes the renew method from it.
> We seem to miss the token renewer class in KMS / HttpFSFileSystem, and hence Yarn defaults
to {{TrivialRenewer}} for DT of such kinds, resulting in the token not being renewed.
> As a side note, {{HttpFSFileSystem}} does have a {{renewDelegationToken}} API, but I
don't see it invoked in hadoop code base. KMS does not have any renew hook.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message