spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jia, Ke A" <ke.a....@intel.com>
Subject Adaptive Query Execution performance results in 3TB TPC-DS
Date Wed, 12 Feb 2020 07:43:23 GMT
Hi all,
We have completed the Spark 3.0 Adaptive Query Execution(AQE) performance tests in 3TB TPC-DS
on 5 node Cascade Lake cluster. 2 queries bring about more than 1.5x performance and 37 queries
bring more than 1.1x performance with AQE.  There is no query has significant performance
degradations. The detail performance results and key configurations are shown in here<https://docs.google.com/spreadsheets/d/1uija2AFblciMcYzU4jnPiy6I8mU8-M0-HwSNNns5aLU/edit?usp=sharing>.
Based on the performance result, we recommend users to turn on AQE in spark 3.0. If encounter
any bug or improvement when enable AQE, please help to file related JIRAs. Thanks.

Regards,
Jia Ke


Mime
View raw message