spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhenyu Hu <cszheny...@gmail.com>
Subject Who can reply me about SPARK-36584
Date Thu, 02 Sep 2021 02:27:02 GMT
Hi All
        I recently raised an issue to the community about Spark's dynamic
scaling. When there is a broadcast and dynamic scaling is enabled at the
same time, the driver will send a BlockUpdate message when the broadcast is
completed. ExecutorMonitor#OnBlockUpdate will receive this event and add
the driver as an executor to its data structure. I think this is a bug
because the driver should not be treated as an executor. Can anyone answer
me and confirm this question?

The JIRA: https://issues.apache.org/jira/browse/SPARK-36584

Mime
View raw message