tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "CruiseCheung (Jira)" <j...@apache.org>
Subject [jira] [Created] (TEZ-4260) DagAwareYarnTaskScheduler may assign unfittable container to task
Date Tue, 22 Dec 2020 07:51:00 GMT
CruiseCheung created TEZ-4260:
---------------------------------

             Summary: DagAwareYarnTaskScheduler may assign unfittable container to task
                 Key: TEZ-4260
                 URL: https://issues.apache.org/jira/browse/TEZ-4260
             Project: Apache Tez
          Issue Type: Bug
    Affects Versions: 0.9.2
            Reporter: CruiseCheung


When tasks in  two vertice require differrent memory size containers and   tez.am.container.reuse.enabled
is true, DagAwareYarnTaskSchedule may assign unfittable containers to tasks, cause out of
memory error. 

Drill down it, we found [https://github.com/apache/tez/blob/master/tez-dag/src/main/java/org/apache/tez/dag/app/rm/DagAwareYarnTaskScheduler.java#L1110] should
call hc.canFit() method
{code:java}
if (hc.canFit(request.getCapability()) && csig == null ||
        signatureMatcher.isSuperSet(csig, request.getContainerSignature())) 
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message