hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 牛兆捷 <nzjem...@gmail.com>
Subject Re: Yarn resource request
Date Thu, 21 Nov 2013 11:36:42 GMT
On receive node update event, scheduler will assign resources of this node
to containers, isn't it?

Why you say we can not know which one is satisfied?

I do not know what your code do? Testing resource request of response of
resource manager? How to run it?


2013/11/21 Steve Loughran <stevel@hortonworks.com>

> On 21 November 2013 09:04, 牛兆捷 <nzjemail@gmail.com> wrote:
>
> > Another question.
> >
> > For two request with the same priority, which one will be chosen?
> >
>
> No way to know. An equally interesting question is : how do you know which
> one has been satisfied, and which one is oustanding?
>
> I spent some time trying to do some of this for a service where I'd request
> some instances on specific nodes, some anywhere, and tried to see if I
> could determine which response satisified which request
>
>
> https://github.com/hortonworks/hoya/blob/master/src/site/markdown/rolehistory.md
>
> This all lives in a state library, which handles AMClientAsync events,
> generates lists of actions, but doesn't talk to the AM itself
>
>
> https://github.com/hortonworks/hoya/tree/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/appmaster/state
>
> This is matched by an in-VM mock YARN service
>
>
> https://github.com/hortonworks/hoya/tree/develop/hoya-core/src/test/groovy/org/apache/hadoop/hoya/yarn/model/mock
>
> I'd recommend this model-controller architecture purely for its testability
>
> Returning to request tracking I ended up doing was keeping a map of
> outstanding requests, and when they arrived I'd remove them from the list.
> When every outstanding request had been satisfied, I'd clear the list of
> any outstanding entries
>
>
> https://github.com/hortonworks/hoya/blob/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/appmaster/state/OutstandingRequestTracker.java
>
>
>
>
> >
> >
> > 2013/11/21 Bikas Saha <bikas@hortonworks.com>
> >
> > > That’s a known limitation of the current implementation of schedulers
> in
> > > YARN. The API allows different requests of different sizes within the
> > same
> > > priority but the implementations of the schedulers do not support this.
> > > For different sizes, one can either use different priorities or use the
> > > highest size amongst the sizes within a priority.
> > >
> > > Bikas
> > >
> > > -----Original Message-----
> > > From: 牛兆捷 [mailto:nzjemail@gmail.com]
> > > Sent: Wednesday, November 20, 2013 7:44 PM
> > > To: yarn-dev@hadoop.apache.org
> > > Subject: Yarn resource request
> > >
> > > Currently, resource requests for the same priority and locality are
> > > expected to all be the same size, how will Application Master do for
> > > different size? use different requests?
> > >
> > > --
> > > *Regards,*
> > > *Zhaojie*
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
> >
> >
> > --
> > *Regards,*
> > *Zhaojie*
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>



-- 
*Regards,*
*Zhaojie*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message