hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13155) Implement TokenRenewer in KMS and HttpFS
Date Mon, 16 May 2016 17:21:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15284861#comment-15284861

Xiao Chen commented on HADOOP-13155:

Hm, seems I can't edit the description. Sorry for message flooding, but here's more details:

I found this problem when debugging an issue with KMS DT renewal.

Token's code snippet to get renewer is like this:
  private static ServiceLoader<TokenRenewer> renewers =

  private synchronized TokenRenewer getRenewer() throws IOException {
    if (renewer != null) {
      return renewer;
    renewer = TRIVIAL_RENEWER;
    synchronized (renewers) {
      for (TokenRenewer canidate : renewers) {
        if (canidate.handleKind(this.kind)) {
          renewer = canidate;
          return renewer;
    LOG.warn("No TokenRenewer defined for token kind " + this.kind);
    return renewer;
And {{META-INF/services/org.apache.hadoop.security.token.TokenRenewer}} defines each implementation.
I didn't find any TokenRenewer implementation that handles kind {{kms-dt}}. I wanted to look
at {{HttpFSFileSystem}} for a reference since they reuse the same auth codes in hadoop-common,
but found out there's none.

(There's no TokenKind in HttpFS either, so may be that's not needed...) The main intention
of this jira is to allow KMS DTs to be able to get renewed.

> Implement TokenRenewer in KMS and HttpFS
> ----------------------------------------
>                 Key: HADOOP-13155
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13155
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Xiao Chen
>            Assignee: Xiao Chen
> DelegationToken is done in Yarn by {{DelegationTokenRenewer}}, where it calls {{Token#renew}}
and uses ServiceLoader to get the renewer class ([code|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java#L382])
> We seem to miss the token renewer class in KMS/HttpFSFileSystem, and hence Yarn defaults
to {{TrivialRenewer}} for DT of these kinds, resulting in the token not renewed.
> As a side note, {{HttpFSFileSystem}} does have a {{renewDelegationToken}} API, but I
don't see it invoked in hadoop code base. KMS does not have any renew hook.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message