hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cui Liqiang <cui.liqi...@gmail.com>
Subject Re: How to really add priority for job and queue?
Date Sat, 02 Aug 2014 00:28:56 GMT
Thanks guys, that’s really helpful.  

--  
Cui Liqiang


On Friday, August 1, 2014 at 8:38 AM, Wei Yan wrote:

>  
> YARN-2098, for supporting application priority in fair scheduler.
>  
> Wei
>  
> On Jul 31, 2014, at 5:33 PM, Sandy Ryza <sandy.ryza@cloudera.com (mailto:sandy.ryza@cloudera.com)>
wrote:
>  
> > Hi Cui,
> >  
> > Fair Scheduler queues can be given priorities via weights. The Fair
> > Scheduler documentation discusses how to configure these -
> > http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/FairScheduler.html.
> > Allowing applications to have priorities is planned, but I don't remember
> > the JIRA off of the top of my head.
> >  
> > Tying queues to particular nodes is currently being discussed on YARN-796.
> > There are differing opinions on whether we should include this feature.
> >  
> > -Sandy
> >  
> >  
> > On Thu, Jul 31, 2014 at 5:26 PM, Cui Liqiang <cui.liqiang@gmail.com (mailto:cui.liqiang@gmail.com)>
wrote:
> >  
> > > Hi,
> > >  
> > > I am using Yarn and facing some custom requirements which seem not
> > > supported yet:
> > > Currently the getPriority() implementation in FSQueue and AppScheduler are
> > > all hard coded to be 1. We’d like them to reflect the real priority, say,
> > > from configuration file(or allocation file?).
> > > We’d like to restrict container(or node?) assignments of some queues, e.g.
> > > we want queueA only assigned to node with ip 10.10.10.10
> > >  
> > > From my understanding, if I want to implement these requirements, I have
> > > to modify some codes in Yarn and probably do some extension as well. Thus I
> > > may have troubles in resolving conflicts when upgrading Yarn version. So I
> > > am wondering:
> > > Is there any way to achieve those two requirements without change the code?
> > > If not, does Yarn have any plans to implement these?
> > > If not, does these requirements sound general enough so that I can
> > > finished it by ourselves and try to merge it back to the trunk?
> > >  
> > > I am pretty new to Yarn, so If what I am asking sounds silly, please let
> > > me know and give the direction.
> > >  
> > >  
> > > Thanks.
> > >  
> > > --
> > > Cui Liqiang
> > >  
> >  
> >  
>  
>  
>  



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message