Hello list, 

I'm having a little performance issue, with different Spark versions.

I've a spark embedded application written in scala, Initially I've use Spark 2.0.2, and works fine, with good speed response, but when I updated to 2.3.2 , with no any code changes It becomes slower.

Mainly what the application do is to gather information from a source, apply transformations with filters and performs aggregation over it. It's source is mainly parquet and no write is done just a serialization from the result.

Maybe I'm using  deprecated api functions or the order of the operations are not generating a good plan...

Can someone give me some idea of any change on the versions that could generate this behavior?


Ing. Ivaldi Andres