spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhenyu Hu <cszheny...@gmail.com>
Subject ExecutorMoniitor timeout
Date Tue, 10 Aug 2021 06:41:05 GMT
In private class Tracker of
org.apache.spark.scheduler.dynalloc.ExecutorMonitor, the method `
updateTimeout
<https://github.com/apache/spark/blob/e1a5d9411733437b5a18045bbd18b48f7aa40f46/core/src/main/scala/org/apache/spark/scheduler/dynalloc/ExecutorMonitor.scala#L567>`
will take the min of `_cacheTimeout` and `_shuffleTimeout`.
I think is more reasonable to take the max of `_cacheTimeout` and
`_shuffleTimeout`, since we need to wait both of cache and shuffle to be
timeout if they all be set.
Any one known about that?

Mime
View raw message