hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuren Wu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-13173) SecurityUtil.buildtokenService broke token authentication in multi-home network env
Date Wed, 18 May 2016 14:13:12 GMT
Yuren Wu created HADOOP-13173:

             Summary: SecurityUtil.buildtokenService broke token authentication in multi-home
network env
                 Key: HADOOP-13173
                 URL: https://issues.apache.org/jira/browse/HADOOP-13173
             Project: Hadoop Common
          Issue Type: Bug
          Components: security
    Affects Versions: 2.6.0
         Environment: multi-homed network cluster on SLES 11

            Reporter: Yuren Wu

Hadoop cluster: multihomed network with KMS and Kerberos. 
Sympton: mapreduce mapper will fail with kerberos error. (stack trace later)

Let's assume hadoop cluster has two networks, PRIVATE_NET and PUBLIC_NET. 
When mapreduce job submitted to the cluster from an outside ndoe (node only has visibility
to PUBLIC_NET, it will acquire KMS tokens based on PUBLIC_NET ip address. Such token in Credential's
token list using IP will not be correct resolved in the cluster. Therefore, we are using hadoop.security.token.service.use_ip=false
in the client side core-site.xml file.  
However, once job is accepted by yarn, the mapper container still throws kerberos error. A
detailed trace shows that when DFSClient is calling decryptEncryptedDataencryptionKey function,
it will reach to org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection
method. This method will call SecurityUtil.buildtokenService to obtain the service name and
use it to lookup the token in Credentials.getToken(service) call. This call returned the service
in the IP:PORT format. Therefore, no token can be retrieved from credential's token hashmap.

After read SecurityUtil class, I think this class does not load configuration from configuration
file at all. It seems to me that this class create a default configuration object and the
useIpForTokenService variable is always set false. 

I do not have enough experience to provide a fix. In my mind, somewhere we need to provide
the correct value of useIpForTokenService based on job setup.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message