spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bartosz Konieczny <>
Subject Why Spark generates Java code and not Scala?
Date Sat, 09 Nov 2019 17:46:56 GMT
Hi there,

Few days ago I got an intriguing but hard to answer question:
"Why Spark generates Java code and not Scala code?"

Since I'm not sure about the exact answer, I'd like to ask you to confirm
or not my thinking. I was looking for the reasons in the JIRA and the
research paper "Spark SQL: Relational Data Processing in Spark" ( but
found nothing explaining why Java over Scala. The single task I found was
about why Scala and not Java but concerning data types ( That's why I'm writing

My guesses about choosing Java code are:
- Java runtime compiler libs are more mature and prod-ready than the
Scala's - or at least, they were at the implementation time
- Scala compiler tends to be slower than the Java's
- Scala compiler seems to be more complex, so debugging & maintaining it
would be harder
- it was easier to represent a pure Java OO design than mixed FP/OO in Scala

Thank you for your help.

Bartosz Konieczny
data engineer

View raw message