hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhijie Shen <zs...@hortonworks.com>
Subject Re: Fw: blacklist additions/removals not propagated to RM
Date Mon, 25 Nov 2013 19:20:41 GMT
BTW, if anybody can help commit YARN-1301, which helps debugging the
blacklist issues. Thanks! - Zhijie


On Mon, Nov 25, 2013 at 11:08 AM, Zhijie Shen <zshen@hortonworks.com> wrote:

> Hi Grandl,
>
> Have you checked the following log message "Update the blacklist for ..."?
> What did it tell?
>
> BTW, YarnScheduler#allocate takes the blacklist additions/removes. You may
> also want to verify YarnScheduler#allocate gets the empty list or not.
>
> Thanks,
> Zhijie
>
>
> On Mon, Nov 25, 2013 at 10:38 AM, Grandl Robert <rgrandl@yahoo.com> wrote:
>
>> I am not sure I posted this question in the right thread.
>>
>>
>> Do you have some thoughts with the below mail ? Is there any
>> configuration parameter for RM to enable blacklisting ? Anyway, it seems my
>> blacklist is sent empty to RM, so I think is an AM problem though.
>>
>>
>> Please let me know about your thoughts,
>>
>> robert
>>
>>
>>
>>
>> On Saturday, November 23, 2013 2:31 PM, Grandl Robert <rgrandl@yahoo.com>
>> wrote:
>>
>> Hi,
>>
>> I am not 100% sure it's a bug in AM, but it seems so. I am blacklisting
>> some hosts, which are send through an AllocateRequest in makeRemoteRequest
>> method in RMContainerRequest.java.
>>
>> However, the blacklist is empty at RM. I traced all over the places where
>> this is lost, and I found that in
>> ApplicationMasterProtocolPBClientImpl.java, in allocate(AllocateRequest
>> request) method, if I print the
>> request.getResourceBlacklistRequest().getBlacklistAdditiions() I see the
>> list, but after it does AllocateRequestProto requestProto = (..).getProto(),
>> and I print the
>> requestProto.getBlacklistRequest().getBlacklistAdditions() now is EMPTY.
>>
>> So basically, AM send an empty black list to RM. I went even further down
>> and traced till YarnServiceProtos.java(in hadoop-yarn-api), but at this
>> point all the logs I have, are removed. it seems this code is regenerated
>> every time I am compiling yarn-api ....
>>
>> Is there something I am missing guys, or is a bug ? Interesting is that
>> release containers list is propagated correctly but I think it comes on
>> other path. I am running hadoop-yarn 2.3.
>>
>> Thanks,
>> robert
>>
>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>



-- 
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message