spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shane Knapp <skn...@berkeley.edu>
Subject Re: [DISCUSS] Deprecate Python < 3.6 in Spark 3.0
Date Fri, 08 Nov 2019 22:26:36 GMT
so...  i've updated the python testing framework to run on python 3.6, and
drop support for testing against python 2.7 (and pypy 2.5.1).

https://github.com/apache/spark/pull/26330

anyways:  this is ready to merge, but i'm more than happy to hold off for
the time being if we're still a little shy about dropping 2.7 so quickly.

thoughts?  opinions?

On Thu, Oct 31, 2019 at 1:00 PM Shane Knapp <sknapp@berkeley.edu> wrote:

> i'm currently testing PyPy3.6 v7.2.0 w/this pull request:
> https://github.com/apache/spark/pull/26330
>
> On Wed, Oct 30, 2019 at 2:31 PM Maciej Szymkiewicz <mszymkiewicz@gmail.com>
> wrote:
>
>> Could we upgrade to PyPy3.6 v7.2.0?
>> On 10/30/19 9:45 PM, Shane Knapp wrote:
>>
>> one quick thing:  we currently test against python2.7, 3.6 *and*
>> pypy2.5.1 (python2.7).
>>
>> what are our plans for pypy?
>>
>>
>> On Wed, Oct 30, 2019 at 12:26 PM Dongjoon Hyun <dongjoon.hyun@gmail.com>
>> wrote:
>>
>>> Thank you all. I made a PR for that.
>>>
>>> https://github.com/apache/spark/pull/26326
>>>
>>> On Tue, Oct 29, 2019 at 5:45 AM Takeshi Yamamuro <linguin.m.s@gmail.com>
>>> wrote:
>>>
>>>> +1, too.
>>>>
>>>> On Tue, Oct 29, 2019 at 4:16 PM Holden Karau <holden@pigscanfly.ca>
>>>> wrote:
>>>>
>>>>> +1 to deprecating but not yet removing support for 3.6
>>>>>
>>>>> On Tue, Oct 29, 2019 at 3:47 AM Shane Knapp <sknapp@berkeley.edu>
>>>>> wrote:
>>>>>
>>>>>> +1 to testing the absolute minimum number of python variants as
>>>>>> possible.  ;)
>>>>>>
>>>>>> On Mon, Oct 28, 2019 at 7:46 PM Hyukjin Kwon <gurwls223@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +1 from me as well.
>>>>>>>
>>>>>>> 2019년 10월 29일 (화) 오전 5:34, Xiangrui Meng <meng@databricks.com>님이
작성:
>>>>>>>
>>>>>>>> +1. And we should start testing 3.7 and maybe 3.8 in Jenkins.
>>>>>>>>
>>>>>>>> On Thu, Oct 24, 2019 at 9:34 AM Dongjoon Hyun <
>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Thank you for starting the thread.
>>>>>>>>>
>>>>>>>>> In addition to that, we currently are testing Python
3.6 only in
>>>>>>>>> Apache Spark Jenkins environment.
>>>>>>>>>
>>>>>>>>> Given that Python 3.8 is already out and Apache Spark
3.0.0 RC1
>>>>>>>>> will start next January
>>>>>>>>> (https://spark.apache.org/versioning-policy.html), I'm
+1 for the
>>>>>>>>> deprecation (Python < 3.6) at Apache Spark 3.0.0.
>>>>>>>>>
>>>>>>>>> It's just a deprecation to prepare the next-step development
cycle.
>>>>>>>>> Bests,
>>>>>>>>> Dongjoon.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Oct 24, 2019 at 1:10 AM Maciej Szymkiewicz <
>>>>>>>>> mszymkiewicz@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi everyone,
>>>>>>>>>>
>>>>>>>>>> While deprecation of Python 2 in 3.0.0 has been announced
>>>>>>>>>> <https://spark.apache.org/news/plan-for-dropping-python-2-support.html>,
>>>>>>>>>> there is no clear statement about specific continuing
support of different
>>>>>>>>>> Python 3 version.
>>>>>>>>>>
>>>>>>>>>> Specifically:
>>>>>>>>>>
>>>>>>>>>>    - Python 3.4 has been retired this year.
>>>>>>>>>>    - Python 3.5 is already in the "security fixes
only" mode and
>>>>>>>>>>    should be retired in the middle of 2020.
>>>>>>>>>>
>>>>>>>>>> Continued support of these two blocks adoption of
many new Python
>>>>>>>>>> features (PEP 468)  and it is hard to justify beyond
2020.
>>>>>>>>>>
>>>>>>>>>> Should these two be deprecated in 3.0.0 as well?
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Best regards,
>>>>>>>>>> Maciej
>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Shane Knapp
>>>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>>>>> https://rise.cs.berkeley.edu
>>>>>>
>>>>> --
>>>>> Twitter: https://twitter.com/holdenkarau
>>>>> Books (Learning Spark, High Performance Spark, etc.):
>>>>> https://amzn.to/2MaRAG9  <https://amzn.to/2MaRAG9>
>>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>>>
>>>>
>>>>
>>>> --
>>>> ---
>>>> Takeshi Yamamuro
>>>>
>>>
>>
>> --
>> Shane Knapp
>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>> https://rise.cs.berkeley.edu
>>
>> --
>> Best regards,
>> Maciej
>>
>>
>
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>


-- 
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

Mime
View raw message