hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rushabh S Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14445) Delegation tokens are not shared between KMS instances
Date Tue, 03 Apr 2018 15:25:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16424166#comment-16424166
] 

Rushabh S Shah commented on HADOOP-14445:
-----------------------------------------

Thanks [~xiaochen] for the latest patch.
It looks good.
bq.  If KMS server is old, you'd get an old token. 
Thanks for catching that. I totally missed that.

There is one test failure in the latest run. 
{noformat}
org.apache.hadoop.conf.TestCommonConfigurationFields.testCompareXmlAgainstConfigurationClass

Failing for the past 1 build (Since Failed#14425 )
Took 0.2 sec.
Error Message
core-default.xml has 2 properties missing in  class org.apache.hadoop.fs.CommonConfigurationKeys
 class org.apache.hadoop.fs.CommonConfigurationKeysPublic  class org.apache.hadoop.fs.local.LocalConfigKeys
 class org.apache.hadoop.fs.ftp.FtpConfigKeys  class org.apache.hadoop.ha.SshFenceByTcpPort
 class org.apache.hadoop.security.LdapGroupsMapping  class org.apache.hadoop.ha.ZKFailoverController
 class org.apache.hadoop.security.ssl.SSLFactory  class org.apache.hadoop.security.CompositeGroupsMapping
 class org.apache.hadoop.io.erasurecode.CodecUtil  class org.apache.hadoop.security.RuleBasedLdapGroupsMapping
Entries:   hadoop.security.key.default.bitlength  hadoop.security.key.default.cipher expected:<0>
but was:<2>
{noformat}
I can't think of a way that your latest patch can introduce this failure.
The hadoop-common build is fairly stable compared to hadoop-hdfs. Can you please double check
whether your patch introduced this failure.
If not, can you please find out which jira is responsible ?

Also there are couple of checkstyle warnings in TestKMS.java regarding unused import.


If the test failure is not related, then you can make the checkstyle changes while committing.
Also can you upload the new patch after committing and resolving the jira.
 I know some people had concerns that it is difficult to co-relate the commit with the last
patch if they are not the same.

+1 (non-binding) pending confirming test failure.
Thanks a lot for the good work here.

> Delegation tokens are not shared between KMS instances
> ------------------------------------------------------
>
>                 Key: HADOOP-14445
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14445
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: kms
>    Affects Versions: 2.8.0, 3.0.0-alpha1
>         Environment: CDH5.7.4, Kerberized, SSL, KMS-HA, at rest encryption
>            Reporter: Wei-Chiu Chuang
>            Assignee: Xiao Chen
>            Priority: Major
>         Attachments: HADOOP-14445-branch-2.8.002.patch, HADOOP-14445-branch-2.8.patch,
HADOOP-14445.002.patch, HADOOP-14445.003.patch, HADOOP-14445.004.patch, HADOOP-14445.05.patch,
HADOOP-14445.06.patch, HADOOP-14445.07.patch, HADOOP-14445.08.patch, HADOOP-14445.09.patch,
HADOOP-14445.10.patch, HADOOP-14445.11.patch
>
>
> As discovered in HADOOP-14441, KMS HA using LoadBalancingKMSClientProvider do not share
delegation tokens. (a client uses KMS address/port as the key for delegation token)
> {code:title=DelegationTokenAuthenticatedURL#openConnection}
> if (!creds.getAllTokens().isEmpty()) {
>         InetSocketAddress serviceAddr = new InetSocketAddress(url.getHost(),
>             url.getPort());
>         Text service = SecurityUtil.buildTokenService(serviceAddr);
>         dToken = creds.getToken(service);
> {code}
> But KMS doc states:
> {quote}
> Delegation Tokens
> Similar to HTTP authentication, KMS uses Hadoop Authentication for delegation tokens
too.
> Under HA, A KMS instance must verify the delegation token given by another KMS instance,
by checking the shared secret used to sign the delegation token. To do this, all KMS instances
must be able to retrieve the shared secret from ZooKeeper.
> {quote}
> We should either update the KMS documentation, or fix this code to share delegation tokens.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message