tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Created] (TEZ-3444) Handling of fetch-failures should consider time spent producing output
Date Thu, 22 Sep 2016 20:35:20 GMT
Jason Lowe created TEZ-3444:
-------------------------------

             Summary: Handling of fetch-failures should consider time spent producing output
                 Key: TEZ-3444
                 URL: https://issues.apache.org/jira/browse/TEZ-3444
             Project: Apache Tez
          Issue Type: Improvement
            Reporter: Jason Lowe


When handling fetch failures and deciding whether the upstream task should be re-run, we should
consider the duration of the upstream task that generated the data trying to be fetched. 
If the upstream task ran for a long time then we may want to retry a bit harder before deciding
to re-run.  If the upstream task executed in a few seconds then we should probably re-run
the upstream task more aggressively since that may be cheaper than multiple retries that timeout.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message