spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michal Sankot <michal.san...@spreaker.com.INVALID>
Subject Strange WholeStageCodegen UI values
Date Thu, 09 Jul 2020 16:54:10 GMT
Hi,
I'm checking execution of SQL queries in Spark UI, trying to find a 
bottleneck and values that are displayed in WholeStageCodegen blocks are 
confusing.

In attached example whole query took 6.6 minutes and upper left 
WholeStageCodegen block says that median value is 7.8 minutes and 
maximum 7.27h :O

What does it mean? Do those number have any real meaning? Is there a way 
to find out how long individual blocks really took?

Thanks,
Michal
Spark 2.4.4 on AWS EMR

Mime
View raw message