spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhenyu Hu <>
Subject Dynamic Allocation& ExecutorMonitor Shuffle Timeout & CacheTimeout
Date Wed, 11 Aug 2021 16:35:03 GMT
In private class Tracker of
org.apache.spark.scheduler.dynalloc.ExecutorMonitor, the method `
will take the min of `_cacheTimeout` and `_shuffleTimeout`.
I think is more reasonable to take the max of `_cacheTimeout` and
`_shuffleTimeout`, since we need to wait both of cache and shuffle to be
timeout if they all be set.
Any one known about that?

View raw message