hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yufei Gu <flyrain...@gmail.com>
Subject Re: Difference between Capacity scheduler and Fair Scheduler
Date Wed, 02 Jan 2019 06:54:21 GMT
Hi Ved,

Here is a blog for fair scheduler preemption,
https://blog.cloudera.com/blog/2018/06/yarn-fairscheduler-preemption-deep-dive/.
In terms of performance, I'm not aware of any document to compare both
schedulers. The easiest way to do it yourself is using SLS(
https://hadoop.apache.org/docs/r3.1.1/hadoop-sls/SchedulerLoadSimulator.html).
SLS focuses on the resource manager rather than anything else, like Node
managers, which is a good thing since FS and CS share the same NM
implantation. I would be surprise their performances drastically differ
from each other for the most common use cases, but their performances would
be very different in some cases.

Best,

Yufei

`This is not a contribution`


On Mon, Dec 31, 2018 at 3:36 AM Ved Gaurav <ved.gaurav@gslab.com> wrote:

> I have read some blogs about difference between them but I am not able to
> understand the difference. How both works , preemption in capacity vs
> preemption in fair scheduler , How both performs in terms of only one queue
> and hierarchical queues ,what are the practical scenarios where we can use
> them etc ?
>
> Any documentation if available will be nice.
>
> Regards,
> Ved Prakash
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message