hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bikas Saha <bi...@hortonworks.com>
Subject RE: blacklist additions/removals not propagated to RM
Date Mon, 25 Nov 2013 18:57:13 GMT
Please open a jira for the issue with the details you describe.


-----Original Message-----
From: Grandl Robert [mailto:rgrandl@yahoo.com]
Sent: Monday, November 25, 2013 10:38 AM
To: yarn-dev@hadoop.apache.org
Subject: Fw: blacklist additions/removals not propagated to RM

I am not sure I posted this question in the right thread.

Do you have some thoughts with the below mail ? Is there any configuration
parameter for RM to enable blacklisting ? Anyway, it seems my blacklist is
sent empty to RM, so I think is an AM problem though.

Please let me know about your thoughts,


On Saturday, November 23, 2013 2:31 PM, Grandl Robert <rgrandl@yahoo.com>


I am not 100% sure it's a bug in AM, but it seems so. I am blacklisting
some hosts, which are send through an AllocateRequest in makeRemoteRequest
method in RMContainerRequest.java.

However, the blacklist is empty at RM. I traced all over the places where
this is lost, and I found that in
ApplicationMasterProtocolPBClientImpl.java, in allocate(AllocateRequest
request) method, if I print the
request.getResourceBlacklistRequest().getBlacklistAdditiions() I see the
list, but after it does AllocateRequestProto requestProto =
(..).getProto(), and I print the
requestProto.getBlacklistRequest().getBlacklistAdditions() now is EMPTY.

So basically, AM send an empty black list to RM. I went even further down
and traced till YarnServiceProtos.java(in hadoop-yarn-api), but at this
point all the logs I have, are removed. it seems this code is regenerated
every time I am compiling yarn-api ....

Is there something I am missing guys, or is a bug ? Interesting is that
release containers list is propagated correctly but I think it comes on
other path. I am running hadoop-yarn 2.3.


NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

View raw message