spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Tank <amittankopensou...@gmail.com>
Subject Re: Straw poll: dropping support for things like Scala 2.10
Date Thu, 27 Oct 2016 15:49:53 GMT
+1 for Matei's point.

On Thursday, October 27, 2016, Matei Zaharia <matei.zaharia@gmail.com>
wrote:

> Just to comment on this, I'm generally against removing these types of
> things unless they create a substantial burden on project contributors. It
> doesn't sound like Python 2.6 and Java 7 do that yet -- Scala 2.10 might,
> but then of course we need to wait for 2.12 to be out and stable.
>
> In general, this type of stuff only hurts users, and doesn't have a huge
> impact on Spark contributors' productivity (sure, it's a bit unpleasant,
> but that's life). If we break compatibility this way too quickly, we
> fragment the user community, and then either people have a crappy
> experience with Spark because their corporate IT doesn't yet have an
> environment that can run the latest version, or worse, they create more
> maintenance burden for us because they ask for more patches to be
> backported to old Spark versions (1.6.x, 2.0.x, etc). Python in particular
> is pretty fundamental to many Linux distros.
>
> In the future, rather than just looking at when some software came out, it
> may be good to have some criteria for when to drop support for something.
> For example, if there are really nice libraries in Python 2.7 or Java 8
> that we're missing out on, that may be a good reason. The maintenance
> burden for multiple Scala versions is definitely painful but I also think
> we should always support the latest two Scala releases.
>
> Matei
>
> On Oct 27, 2016, at 12:15 PM, Reynold Xin <rxin@databricks.com
> <javascript:_e(%7B%7D,'cvml','rxin@databricks.com');>> wrote:
>
> I created a JIRA ticket to track this: https://issues.apache.
> org/jira/browse/SPARK-18138
>
>
>
> On Thu, Oct 27, 2016 at 10:19 AM, Steve Loughran <stevel@hortonworks.com
> <javascript:_e(%7B%7D,'cvml','stevel@hortonworks.com');>> wrote:
>
>>
>> On 27 Oct 2016, at 10:03, Sean Owen <sowen@cloudera.com
>> <javascript:_e(%7B%7D,'cvml','sowen@cloudera.com');>> wrote:
>>
>> Seems OK by me.
>> How about Hadoop < 2.6, Python 2.6? Those seem more removeable. I'd like
>> to add that to a list of things that will begin to be unsupported 6 months
>> from now.
>>
>>
>> If you go to java 8 only, then hadoop 2.6+ is mandatory.
>>
>>
>> On Wed, Oct 26, 2016 at 8:49 PM Koert Kuipers <koert@tresata.com
>> <javascript:_e(%7B%7D,'cvml','koert@tresata.com');>> wrote:
>>
>>> that sounds good to me
>>>
>>> On Wed, Oct 26, 2016 at 2:26 PM, Reynold Xin <rxin@databricks.com
>>> <javascript:_e(%7B%7D,'cvml','rxin@databricks.com');>> wrote:
>>>
>>>> We can do the following concrete proposal:
>>>>
>>>> 1. Plan to remove support for Java 7 / Scala 2.10 in Spark 2.2.0
>>>> (Mar/Apr 2017).
>>>>
>>>> 2. In Spark 2.1.0 release, aggressively and explicitly announce the
>>>> deprecation of Java 7 / Scala 2.10 support.
>>>>
>>>> (a) It should appear in release notes, documentations that mention how
>>>> to build Spark
>>>>
>>>> (b) and a warning should be shown every time SparkContext is started
>>>> using Scala 2.10 or Java 7.
>>>>
>>>>
>>
>
>

Mime
View raw message