hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Tianyi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-4661) Per-queue preemption policy in FairScheduler
Date Sat, 30 Jan 2016 09:34:39 GMT
He Tianyi created YARN-4661:
-------------------------------

             Summary: Per-queue preemption policy in FairScheduler
                 Key: YARN-4661
                 URL: https://issues.apache.org/jira/browse/YARN-4661
             Project: Hadoop YARN
          Issue Type: Wish
          Components: fairscheduler
    Affects Versions: 2.6.0
            Reporter: He Tianyi
            Priority: Minor


When {{FairScheduler}} needs to preempt container, it tries to find a container by hierachically
sorting and selecting {{AppSchedulable}} with most 'over fairshare' (in {{FairSharePolicy}}),
and pick its latest launched container.

In some case, strategy above become non-optimal, one may want to kill latest container (not
{{AppSchedulable}}) launched in the queue for better trade-off between fairness and efficiency.
Since most app with over fairshare tend to be started longer ago than other apps, perhaps
even its latent launch container is running quite some time.

Maybe besides {{policy}}, we make it possible to also specify a {{preemptionPolicy}} only
for selecting container to preempt, without changing scheduling policy.

For example:
{quote}
<queue name="queueA">
  <preemptionPolicy>fifo</preemptionPolicy>
  <policy>fair</policy>
</queue>
{quote}

Any suggestions or comments?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message