spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <sro...@gmail.com>
Subject Interesting implications of supporting Scala 2.13
Date Fri, 10 May 2019 18:41:38 GMT
While that's not happening soon (2.13 isn't out), note that some of
the changes to collections will be fairly breaking changes.

https://issues.apache.org/jira/browse/SPARK-25075
https://docs.scala-lang.org/overviews/core/collections-migration-213.html

Some of this may impact a public API, so may need to start proactively
fixing stuff for 2.13 before 3.0 comes out where possible.

Here's an example: Traversable goes away. We have a method
SparkConf.setAll(Traversable). We can't support 2.13 while that still
exists. Of course, we can decide to deprecate it with replacement (use
Iterable) and remove it in the version that supports 2.13. But that
would mean a little breaking change, and we either have to accept that
for a future 3.x release, or it waits until 4.x.

I wanted to put that on the radar now to gather opinions about whether
this will probably be acceptable, or whether we really need to get
methods like that changed before 3.0.

Also: there's plenty of straightforward but medium-sized changes we
can make now in anticipation of 2.13 support, like, make the type of
Seq we use everywhere explicit (will be good for a like 1000 file
change I'm sure). Or see if we can swap out Traversable everywhere.
Remove MutableList, etc.

I was going to start fiddling with that unless it just sounds too disruptive.

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Mime
View raw message