hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Azuryy Yu <azury...@gmail.com>
Subject Re: Container authenticate failed after YARN-961
Date Sat, 14 Dec 2013 01:38:16 GMT
Hi Vinod,
I enabled DEBUG log on both RM and NM.
RM Log:

2013-12-14 09:17:47,093 DEBUG org.apache.hadoop.security.SaslRpcClient:
Sending sasl message state: NEGOTIATE

2013-12-14 09:17:47,208 DEBUG org.apache.hadoop.security.SaslRpcClient:
Received SASL message state: NEGOTIATE
auths {
  method: "TOKEN"
  mechanism: "DIGEST-MD5"
  protocol: ""
  serverId: "default"
  challenge:
"realm=\"default\",nonce=\"7+Tb5yJJQV0Mpd6r2OwezSWoeN07h7E0DelSSJJi\",qop=\"auth\",charset=utf-8,algorithm=md5-sess"
}

2013-12-14 09:17:47,216 DEBUG org.apache.hadoop.security.SaslRpcClient: Get
token info proto:interface
org.apache.hadoop.yarn.api.ContainerManagementProtocolPB info:null
2013-12-14 09:17:47,216 ERROR
org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
as:appattempt_1386983771970_0001_000001 (auth:SIMPLE)
cause:org.apache.hadoop.security.AccessControlException: Client cannot
authenticate via:[TOKEN]

NM Log:
2013-12-14 09:17:47,104 DEBUG
org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:hadoop
(auth:SIMPLE)
from:org.apache.hadoop.security.SaslRpcServer.create(SaslRpcServer.java:150)
2013-12-14 09:17:47,111 DEBUG org.apache.hadoop.security.SaslRpcServer:
Created SASL server with mechanism = DIGEST-MD5
2013-12-14 09:17:47,205 DEBUG org.apache.hadoop.ipc.Server: Sending sasl
message state: NEGOTIATE
auths {
  method: "TOKEN"
  mechanism: "DIGEST-MD5"
  protocol: ""
  serverId: "default"
  challenge:
"realm=\"default\",nonce=\"7+Tb5yJJQV0Mpd6r2OwezSWoeN07h7E0DelSSJJi\",qop=\"auth\",charset=utf-8,algorithm=md5-sess"
}

2013-12-14 09:17:47,208 DEBUG org.apache.hadoop.ipc.Server: IPC Server
Responder: responding to null from 10.2.8.148:34175 Call#-33 Retry#-1
2013-12-14 09:17:47,208 DEBUG org.apache.hadoop.ipc.Server: IPC Server
Responder: responding to null from 10.2.8.148:34175 Call#-33 Retry#-1 Wrote
166 bytes.
2013-12-14 09:17:47,223 DEBUG org.apache.hadoop.ipc.Server: Socket Reader
#1 for port 8021: disconnecting client 10.2.8.148:34175. Number of active
connections: 0

I am using HA based on trunk, there is in the log as:
13/12/14 09:17:42 DEBUG hdfs.BlockReaderLocal: dfs.domain.socket.path =
/home/hadoop/dn_socket
13/12/14 09:17:42 DEBUG hdfs.HAUtil: No HA service delegation token found
for logical URI hdfs://test-cluster

Please take a look, Thanks very much.



On Sat, Dec 14, 2013 at 8:56 AM, Azuryy Yu <azuryyyu@gmail.com> wrote:

> Thanks Vinod,
>
> I am not chang MR AppMaster. I'll enable debug logging then paste here.
>
>
> On Sat, Dec 14, 2013 at 2:36 AM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
>
>> Is it stock MR or your modified MR AppMaster?
>>
>> Can you enable debug logging on NodeManager as well as the MR
>> ApplicationMaster and trace what exactly is happening? We enabled TOKEN
>> based auth by default, so you should look if AM has the NMToken or not.
>> Also check the NM logs.
>>
>> Thanks,
>> +Vinod
>>
>> On Dec 13, 2013, at 3:31 AM, Azuryy Yu <azuryyyu@gmail.com> wrote:
>>
>> > Hi Dears,
>> >
>> > After YARN-961 committed to the trunk, we cannot sumbit MapReduce Job
>> now.
>> >
>> > I am using trunk in the test unsecure cluster, when submit Job, it
>> > complained:
>> > 386931155525_0001_000002. Got exception: java.io.IOException: Failed on
>> > local exception: java.io.IOException:
>> > org.apache.hadoop.security.AccessControlException: Client cannot
>> > authenticate via:[TOKEN]; Host Details : local host is: "
>> > 10.2.8.148/10.2.8.148"; destination host is: "test.152.com":8021;
>> > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
>> > at org.apache.hadoop.ipc.Client.call(Client.java:1410)
>> > at org.apache.hadoop.ipc.Client.call(Client.java:1359)
>> > at
>> >
>> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
>> > at com.sun.proxy.$Proxy23.startContainers(Unknown Source)
>> > at
>> >
>> org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.startContainers(ContainerManagementProtocolPBClientImpl.java:96)
>> > at
>> >
>> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:118)
>> > at
>> >
>> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:249)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> > at java.lang.Thread.run(Thread.java:744)
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message