spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michal Sankot <>
Subject Strange WholeStageCodegen UI values
Date Thu, 09 Jul 2020 16:54:10 GMT
I'm checking execution of SQL queries in Spark UI, trying to find a 
bottleneck and values that are displayed in WholeStageCodegen blocks are 

In attached example whole query took 6.6 minutes and upper left 
WholeStageCodegen block says that median value is 7.8 minutes and 
maximum 7.27h :O

What does it mean? Do those number have any real meaning? Is there a way 
to find out how long individual blocks really took?

Spark 2.4.4 on AWS EMR

View raw message