spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koert Kuipers <ko...@tresata.com>
Subject Re: Straw poll: dropping support for things like Scala 2.10
Date Thu, 02 Mar 2017 16:50:52 GMT
given the issues with scala 2.10 and java 8 i am in favor of dropping scala
2.10 in next release

On Sat, Feb 25, 2017 at 2:10 PM, Sean Owen <sowen@cloudera.com> wrote:

> I want to bring up the issue of Scala 2.10 support again, to see how
> people feel about it. Key opinions from the previous responses, I think:
>
> Cody: only drop 2.10 support when 2.12 support is added
> Koert: we need all dependencies to support 2.12; Scala updates are pretty
> transparent to IT/ops
> Ofir: make sure to deprecate 2.10 in Spark 2.1
> Reynold: let’s maybe remove support for Scala 2.10 and Java 7 in Spark 2.2
> Matei: let’s not remove things unless they’re burdensome for the project;
> some people are still on old environments that their IT can’t easily update
>
> Scala 2.10 support was deprecated in 2.1, and we did remove Java 7 support
> for 2.2. https://issues.apache.org/jira/browse/SPARK-14220 tracks the
> work to support 2.12, and there is progress, especially in dependencies
> supporting 2.12.
>
> It looks like 2.12 support may even entail a breaking change as documented
> in https://issues.apache.org/jira/browse/SPARK-14643 and will mean
> dropping Kafka 0.8, for example. In any event it’s going to take some
> surgery and a few hacks to make one code base work across 2.11 and 2.12. I
> don’t see this happening for Spark 2.2.0 because there are just a few weeks
> left.
>
> Supporting three versions at once is probably infeasible, so dropping 2.10
> should precede 2.12 support. Right now, I would like to make progress
> towards changes that 2.12 will require but that 2.11/2.10 can support. For
> example, we have to update scalatest, breeze, chill, etc. and can do that
> before 2.12 is enabled. However I’m finding making those changes tricky or
> maybe impossible in one case while 2.10 is still supported.
>
> For 2.2.0, I’m wondering if it makes sense to go ahead and drop 2.10
> support, and even get in additional prep work for 2.12, into the 2.2.0
> release. The move to support 2.12 in 2.3.0 would then be a smaller change.
> It isn’t strictly necessary. We could delay all of that until after 2.2.0
> and get it all done between 2.2.0 and 2.3.0. But I wonder if 2.10 is legacy
> enough at this stage to drop for Spark 2.2.0?
>
> I don’t feel strongly about it but there are some reasonable arguments for
> dropping it:
>
> - 2.10 doesn’t technically support Java 8, though we do have it working
> still even after requiring Java 8
> - Safe to say virtually all common _2.10 libraries has a _2.11 counterpart
> at this point?
> - 2.10.x was “EOL” in September 2015 with the final 2.10.6 release
> - For a vendor viewpoint: CDH only supports Scala 2.11 with Spark 2.x
>
> Before I open a JIRA, just soliciting opinions.
>
>
> On Tue, Oct 25, 2016 at 4:36 PM Sean Owen <sowen@cloudera.com> wrote:
>
>> I'd like to gauge where people stand on the issue of dropping support for
>> a few things that were considered for 2.0.
>>
>> First: Scala 2.10. We've seen a number of build breakages this week
>> because the PR builder only tests 2.11. No big deal at this stage, but, it
>> did cause me to wonder whether it's time to plan to drop 2.10 support,
>> especially with 2.12 coming soon.
>>
>> Next, Java 7. It's reasonably old and out of public updates at this
>> stage. It's not that painful to keep supporting, to be honest. It would
>> simplify some bits of code, some scripts, some testing.
>>
>> Hadoop versions: I think the the general argument is that most anyone
>> would be using, at the least, 2.6, and it would simplify some code that has
>> to reflect to use not-even-that-new APIs. It would remove some moderate
>> complexity in the build.
>>
>>
>> "When" is a tricky question. Although it's a little aggressive for minor
>> releases, I think these will all happen before 3.x regardless. 2.1.0 is not
>> out of the question, though coming soon. What about ... 2.2.0?
>>
>>
>> Although I tend to favor dropping support, I'm mostly asking for current
>> opinions.
>>
>

Mime
View raw message