hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Priyank rastogi <priyank.rast...@huawei.com>
Subject capacity-scheduler behavior
Date Tue, 01 Oct 2013 04:24:40 GMT

We are running a few tests with Hadoop-2.1.0/Capacity-Scheduler/preemption-enabled. The test
scenario and expected behavior is as below.

 +-- DEPT_X  / 50%
 +-- DEPT_Y  / 50%
     +-- TEAM_Y1  / 50%
     +-- TEAM_Y2  / 50%

<<Test Scenario>>
(1) Submit JOB_X to DEPT_X (it fully utilizes the cluster)
(2) Submit JOB_Y to TEAM_Y1 (same as JOB_X)

<<Expected behavior>>
(1) JOB_X tasks are expected to fully utilize the cluster,
(2) Once JOB_Y is submitted, JOB_Y tasks are expected to utilize the capacity allocated to
DEPT_Y (i.e. TEAM_Y1 queue expected running at 200% capacity)
    -> i.e. any unused capacity in DEPT_Y should be allocated to task submitted to its
sub-queues first (TEAM_Y1 or TEAM_Y2), and only thereafter to job submitted to DEPT_X queue.

The initial results are not as per this expectation, so first we want to confirm if the behavior
we expect is correct.

Priyank Rastogi

This e-mail and its attachments contain confidential information from HUAWEI, which is intended
only for the person or entity whose address is listed above. Any use of the information contained
herein in any way (including, but not limited to, total or partial disclosure, reproduction,
or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive
this e-mail in error, please notify the sender by phone or email immediately and delete it!

View raw message