hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gaurav <gau...@datatorrent.com>
Subject Re: Allocating Containers on a particular Node in Yarn
Date Thu, 14 Nov 2013 00:19:49 GMT
Hi Sandy,

If I set relaxlocality to true with host name, I don't get the container 
on specified host even though node has the resources.
I am using AMRMClient, only thing is I made following changes to get the 
containers on the specified node.
/
//Line: 361 //
//Original//
// addResourceRequest(req.getPriority(), node, req.getCapability(), req, 
true);//
//Modifiled//
// addResourceRequest(req.getPriority(), node, req.getCapability(), req, 
req.getRelaxLocality());//
//
//Line 374//
//Original//
// addResourceRequest(req.getPriority(), rack, req.getCapability(), req, 
true);//
//Modifiled//
// addResourceRequest(req.getPriority(), rack, req.getCapability(), req, 
req.getRelaxLocality());//
///


Thanks
-Gaurav

On 11/13/2013 4:02 PM, Sandy Ryza wrote:
> Yeah, specifying a host name with relaxLocality is meaningful.  Schedulers
> use delay scheduling (
> http://www.cs.berkeley.edu/~matei/talks/2010/eurosys_delaysched.pdf) to
> achieve locality when relaxLocality is on.  But it is turned off by
> default.  The individual scheduler docs have specifics on how to configure
> it.
>
> Guarav,
> Using ResourceRequests directly is not straightforward and error prone.  Is
> there a reason that AMRMClient is unsuitable for your needs?
>
> -Sandy
>
>
>
> On Wed, Nov 13, 2013 at 3:55 PM, Thomas Weise <thomas.weise@gmail.com>wrote:
>
>> Is it possible to specify a particular node and have RM fallback to an
>> different node only after making an attempt to allocate for the requested
>> node? In other words, is the combination of specific host name and
>> relaxLocality=TRUE meaningful at all?
>>
>> Thanks.
>>
>>
>> On Wed, Nov 13, 2013 at 3:23 PM, Alejandro Abdelnur <tucu@cloudera.com
>>> wrote:
>>> Gaurav,
>>>
>>> Setting relaxLocality to FALSE should do it.
>>>
>>> thanks.
>>>
>>>
>>> On Wed, Nov 13, 2013 at 2:58 PM, gaurav <gaurav@datatorrent.com> wrote:
>>>
>>>>   Hi,
>>>>   I am trying to allocate containers on a particular node in Yarn but
>> Yarn
>>>> is returning me containers on different node although the requested
>> node
>>>> has resources available.
>>>>
>>>> I checked into the allocate(AllocateRequest request) function of
>>>> ApplicationMasterService and my request is as follows
>>>>
>>>> *request: ask { priority { priority: 1 } resource_name: "h2"
>> capability {
>>>> memory: 1000 } num_containers: 2 } ask { priority { priority: 1 }
>>>> resource_name: "/default-rack" capability { memory: 1000 }
>>> num_containers:
>>>> 2 } ask { priority { priority: 1 } resource_name: "*" capability {
>>> memory:
>>>> 1000 } num_containers: 2 } response_id: 1 progress: 0.0*
>>>>
>>>> but the containers that I am getting back is as follows
>>>> [Container: [ContainerId: container_1384381084244_0001_01_000002,
>> NodeId:
>>>> h1:1234, NodeHttpAddress: h1:2, Resource: <memory:1024, vCores:1>,
>>>> Priority: 1, Token: Token { kind: ContainerToken, service: h1:1234 },
>> ],
>>>> Container: [ContainerId: container_1384381084244_0001_01_000003,
>> NodeId:
>>>> h1:1234, NodeHttpAddress: h1:2, Resource: <memory:1024, vCores:1>,
>>>> Priority: 1, Token: Token { kind: ContainerToken, service: h1:1234 },
>> ]]
>>>> I am attaching the test case that I have written along with the mail.
>> It
>>>> uses classes under org.apache.hadoop.yarn.server.resourcemanager
>> package.
>>>> Any pointers would be of great help
>>>>
>>>> Thanks
>>>> Gaurav
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>> Alejandro
>>>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message