spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyukjin Kwon <gurwls...@gmail.com>
Subject Re: [DISCUSS] Migrate development scripts under dev/ from Python2 to Python 3
Date Thu, 15 Aug 2019 09:38:22 GMT
I mean python 2 _will be_ deprecated in Spark 3.

On Thu, 15 Aug 2019, 18:37 Hyukjin Kwon, <gurwls223@gmail.com> wrote:

> Yeah, we will probably drop Python 2 entirely after 3.0.0. Python 2 is
> already deprecated.
>
> On Thu, 15 Aug 2019, 18:25 Driesprong, Fokko, <fokko@driesprong.frl>
> wrote:
>
>> Sorry for the late reply, was a bit busy lately, but I still would like
>> to share my thoughts on this.
>>
>> For Apache Airflow we're dropping support for Python 2 in the next major
>> release. We're now supporting Python 3.5+. Mostly because:
>>
>>    - Easier to maintain and test, and less if/else constructions for the
>>    different Python versions. Also, not having to test against Python 2.x
>>    reduces the build matrix.
>>    - Python 3 has support for typing. From Python 3.5 you can include
>>    provisional type hints. An excellent presentation by Guido himself:
>>    https://www.youtube.com/watch?v=2wDvzy6Hgxg. From Python 3.5 it is
>>    still provisional, but it is a really good idea. From Airflow we've noticed
>>    that using mypy is catching bugs early:
>>       - This will put less stress on the (boring part of the) reviewing
>>       process since a lot of this stuff is checked automatically.
>>       - For new developers, it is easier to read the code because of the
>>       annotations.
>>       - Can be used as an input for generated documentation (or check if
>>       it still in sync with the docstrings)
>>       - Easier to extend the code since you know what kind of types you
>>       can expect, and your IDE will also pick up the hinting.
>>    - Python 2.x will be EOL end this year
>>
>> I have a strong preference to migrate everything to Python 3.
>>
>> Cheers, Fokko
>>
>>
>> Op wo 7 aug. 2019 om 12:14 schreef Weichen Xu <weichen.xu@databricks.com
>> >:
>>
>>> All right we could support both Python 2 and Python 3 for spark 3.0.
>>>
>>> On Wed, Aug 7, 2019 at 6:10 PM Hyukjin Kwon <gurwls223@gmail.com> wrote:
>>>
>>>> We didn't drop Python 2 yet although it's deprecated. So I think It
>>>> should support both Python 2 and Python 3 at the current status.
>>>>
>>>> 2019년 8월 7일 (수) 오후 6:54, Weichen Xu <weichen.xu@databricks.com>님이
작성:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I would like to discuss the compatibility for dev scripts. Because we
>>>>> already decided to deprecate python2 in spark 3.0, for development scripts
>>>>> under dev/ , we have two choice:
>>>>> 1) Migration from Python 2 to Python 3
>>>>> 2) Support both Python 2 and Python 3
>>>>>
>>>>> I tend to option (2) which is more friendly to maintenance.
>>>>>
>>>>> Regards,
>>>>> Weichen
>>>>>
>>>>

Mime
View raw message