hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wei Yan <ywsk...@gmail.com>
Subject Re: How to really add priority for job and queue?
Date Fri, 01 Aug 2014 00:38:07 GMT

YARN-2098, for supporting application priority in fair scheduler.

Wei

On Jul 31, 2014, at 5:33 PM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:

> Hi Cui,
> 
> Fair Scheduler queues can be given priorities via weights.  The Fair
> Scheduler documentation discusses how to configure these -
> http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/FairScheduler.html.
> Allowing applications to have priorities is planned, but I don't remember
> the JIRA off of the top of my head.
> 
> Tying queues to particular nodes is currently being discussed on YARN-796.
> There are differing opinions on whether we should include this feature.
> 
> -Sandy
> 
> 
> On Thu, Jul 31, 2014 at 5:26 PM, Cui Liqiang <cui.liqiang@gmail.com> wrote:
> 
>> Hi,
>> 
>> I am using Yarn and facing some custom requirements which seem not
>> supported yet:
>> Currently the getPriority() implementation in FSQueue and AppScheduler are
>> all hard coded to be 1. We’d like them to reflect the real priority, say,
>> from configuration file(or allocation file?).
>> We’d like to restrict container(or node?) assignments of some queues, e.g.
>> we want queueA only assigned to node with ip 10.10.10.10
>> 
>> From my understanding, if I want to implement these requirements, I have
>> to modify some codes in Yarn and probably do some extension as well. Thus I
>> may have troubles in resolving conflicts when upgrading Yarn version. So I
>> am wondering:
>> Is there any way to achieve those two requirements without change the code?
>> If not, does Yarn have any plans to implement these?
>> If not, does these requirements sound general enough so that I can
>> finished it by ourselves and try to merge it back to the trunk?
>> 
>> I am pretty new to Yarn, so If what I am asking sounds silly, please let
>> me know and give the direction.
>> 
>> 
>> Thanks.
>> 
>> --
>> Cui Liqiang
>> 
>> 


Mime
View raw message