spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Davies Liu <dav...@databricks.com>
Subject Re: Straw poll: dropping support for things like Scala 2.10
Date Thu, 27 Oct 2016 16:58:27 GMT
+1 for Matei's point.

On Thu, Oct 27, 2016 at 8:36 AM, Matei Zaharia <matei.zaharia@gmail.com> wrote:
> Just to comment on this, I'm generally against removing these types of
> things unless they create a substantial burden on project contributors. It
> doesn't sound like Python 2.6 and Java 7 do that yet -- Scala 2.10 might,
> but then of course we need to wait for 2.12 to be out and stable.
>
> In general, this type of stuff only hurts users, and doesn't have a huge
> impact on Spark contributors' productivity (sure, it's a bit unpleasant, but
> that's life). If we break compatibility this way too quickly, we fragment
> the user community, and then either people have a crappy experience with
> Spark because their corporate IT doesn't yet have an environment that can
> run the latest version, or worse, they create more maintenance burden for us
> because they ask for more patches to be backported to old Spark versions
> (1.6.x, 2.0.x, etc). Python in particular is pretty fundamental to many
> Linux distros.
>
> In the future, rather than just looking at when some software came out, it
> may be good to have some criteria for when to drop support for something.
> For example, if there are really nice libraries in Python 2.7 or Java 8 that
> we're missing out on, that may be a good reason. The maintenance burden for
> multiple Scala versions is definitely painful but I also think we should
> always support the latest two Scala releases.
>
> Matei
>
> On Oct 27, 2016, at 12:15 PM, Reynold Xin <rxin@databricks.com> wrote:
>
> I created a JIRA ticket to track this:
> https://issues.apache.org/jira/browse/SPARK-18138
>
>
>
> On Thu, Oct 27, 2016 at 10:19 AM, Steve Loughran <stevel@hortonworks.com>
> wrote:
>>
>>
>> On 27 Oct 2016, at 10:03, Sean Owen <sowen@cloudera.com> wrote:
>>
>> Seems OK by me.
>> How about Hadoop < 2.6, Python 2.6? Those seem more removeable. I'd like
>> to add that to a list of things that will begin to be unsupported 6 months
>> from now.
>>
>>
>> If you go to java 8 only, then hadoop 2.6+ is mandatory.
>>
>>
>> On Wed, Oct 26, 2016 at 8:49 PM Koert Kuipers <koert@tresata.com> wrote:
>>>
>>> that sounds good to me
>>>
>>> On Wed, Oct 26, 2016 at 2:26 PM, Reynold Xin <rxin@databricks.com> wrote:
>>>>
>>>> We can do the following concrete proposal:
>>>>
>>>> 1. Plan to remove support for Java 7 / Scala 2.10 in Spark 2.2.0
>>>> (Mar/Apr 2017).
>>>>
>>>> 2. In Spark 2.1.0 release, aggressively and explicitly announce the
>>>> deprecation of Java 7 / Scala 2.10 support.
>>>>
>>>> (a) It should appear in release notes, documentations that mention how
>>>> to build Spark
>>>>
>>>> (b) and a warning should be shown every time SparkContext is started
>>>> using Scala 2.10 or Java 7.
>>>>
>>
>
>

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Mime
View raw message