hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Devaraj K <deva...@apache.org>
Subject Re: Re:About ResourceRequest for a self-written AppMaster to ResourceManager
Date Sun, 20 Jul 2014 07:51:09 GMT
Yes, MRAppMaster also does the same way. It sends total resource requests
for map phase one time and will wait for RM to allocate containers for all
those requests in the further allocate responses. It also starts launching
containers immediately as on it gets from RM without waiting for all
resource requests to be allocated.

On Sun, Jul 20, 2014 at 8:15 AM, 欧阳晋(欧阳晋) <jin.oyj@alibaba-inc.com> wrote:

> Thank you very much Devaraj. It solves my question. And I wonder if this
> is the same thing the MapReduce ApplicationMaster do when it request
> resource to RM? Because MR is the App that can run map immediately after
> some containers for map are allocated, so for Map phase, MR AppMaster just
> send a total resource request for the first time and then just wait for
> allocation response?
> Thanks!
> Ouyang Jin
> ------------------------------------------------------------------
> 发件人:Devaraj K <devaraj@apache.org>
> 发送时间:2014年7月20日(星期日) 01:51
> 收件人:yarn-dev@hadoop.apache.org <yarn-dev@hadoop.apache.org>; 欧阳晋(风骖)
> jin.oyj@alibaba-inc.com>
> 主 题:Re: About ResourceRequest for a self-written AppMaster to
> ResourceManager
> Hi Jin,
>      You don't need to request containers for each allocate invocation.
> You can send containers request one time for 10 containers and then keep on
> checking the allocate response for containers allocation, RM will try to
> allocate and gives as part of allocate response whenever they become
> available.
> For ex: If we request for 10 containers, assume RM can allocate 2
> containers at that time then RM gives two containers and keeps the
> remaining 8 containers request information. In future RM continues to
> allocate remaining requests whenever resources available and gives to
> ApplicationMaster as part of allocate response.
> On Sat, Jul 19, 2014 at 10:01 PM, 欧阳晋(欧阳晋) <jin.oyj@alibaba-inc.com>
> wrote:
>> Hi all:I'm writing my own AppMaster to run in YARN. And I'm confused
>> about resource request between AppMaster and ResourceManager.My situation
>> is that I want total 10 containers, so in my AppMaster . after some
>> initiallization , my AppMaster send a 10 container resource request to
>> ResourceManager, call allocate interface, and wait for allocation in a
>> while loop. My app is such a type that it can run container immediately
>> after allocation happens . So after maybe 2 containers are allocated by
>> ResourceManager, I tell NodeManager to launch it . So my question is,
>> should I send a new ResourceReqeust to ResourceManager that I need 8
>> Container ?  It seems that after the first time I send a resource request
>> and waiting for allocation , I can get containers , but I don't know
>> whether that's suitable.Thanks!OuYang Jin
> --
> Thanks
> Devaraj K


Devaraj K

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message