spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shane knapp <skn...@berkeley.edu>
Subject Re: Upgrading minimal PyArrow version to 0.12.x [SPARK-27276]
Date Mon, 01 Apr 2019 22:09:34 GMT
well now!  color me completely surprised...  i decided to whip up a fresh
python3.6.8 conda environment this morning to "see if things just worked".

well, apparently they do!  :)

regardless, this is pretty awesome news as i will be able to easily update
the 'py3k' python3.4 environment to a fresh, less bloated, but still
package-complete python3.6.8 environment (including pyarrow 0.12.0, pandas
0.24.2, scipy 1.2.1).

i tested this pretty extensively today on both the ubuntu and centos
workers, and i think i'm ready to pull the trigger for a build-system-wide
upgrade...   however, i'll be out wednesday through friday this week and
don't want to make a massive change before disappearing for a few days.

so:  how does early next week sound for the python upgrade?  :)

shane

On Mon, Apr 1, 2019 at 8:58 AM shane knapp <sknapp@berkeley.edu> wrote:

> i'd much prefer that we minimize the number of python versions that we
> test against...  would 2.7 and 3.6 be sufficient?
>
> On Fri, Mar 29, 2019 at 10:23 PM Felix Cheung <felixcheung_m@hotmail.com>
> wrote:
>
>> I don’t take it as Sept 2019 is end of life for python 3.5 tho. It’s just
>> saying the next release.
>>
>> In any case I think in the next release it will be great to get more
>> Python 3.x release test coverage.
>>
>>
>>
>> ------------------------------
>> *From:* shane knapp <sknapp@berkeley.edu>
>> *Sent:* Friday, March 29, 2019 4:46 PM
>> *To:* Bryan Cutler
>> *Cc:* Felix Cheung; Hyukjin Kwon; dev
>> *Subject:* Re: Upgrading minimal PyArrow version to 0.12.x [SPARK-27276]
>>
>> i'm not opposed to 3.6 at all.
>>
>> On Fri, Mar 29, 2019 at 4:16 PM Bryan Cutler <cutlerb@gmail.com> wrote:
>>
>>> PyArrow dropping Python 3.4 was mainly due to support going away at
>>> Conda-Forge and other dependencies also dropping it.  I think we better
>>> upgrade Jenkins Python while we are at it.  Are you all against jumping to
>>> Python 3.6 so we are not in the same boat in September?
>>>
>>> On Thu, Mar 28, 2019 at 7:58 PM Felix Cheung <felixcheung_m@hotmail.com>
>>> wrote:
>>>
>>>> 3.4 is end of life but 3.5 is not. From your link
>>>>
>>>> we expect to release Python 3.5.8 around September 2019.
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> *From:* shane knapp <sknapp@berkeley.edu>
>>>> *Sent:* Thursday, March 28, 2019 7:54 PM
>>>> *To:* Hyukjin Kwon
>>>> *Cc:* Bryan Cutler; dev; Felix Cheung
>>>> *Subject:* Re: Upgrading minimal PyArrow version to 0.12.x
>>>> [SPARK-27276]
>>>>
>>>> looks like the same for 3.5...
>>>> https://www.python.org/dev/peps/pep-0478/
>>>>
>>>> let's pick a python version and start testing.
>>>>
>>>> On Thu, Mar 28, 2019 at 7:52 PM shane knapp <sknapp@berkeley.edu>
>>>> wrote:
>>>>
>>>>>
>>>>>> If there was, it looks inevitable to upgrade Jenkins\s Python from
>>>>>> 3.4 to 3.5.
>>>>>>
>>>>>> this is inevitable.  3.4s final release was 10 days ago (
>>>>> https://www.python.org/dev/peps/pep-0429/) so we're basically EOL.
>>>>>
>>>>
>>>>
>>>> --
>>>> Shane Knapp
>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>>> https://rise.cs.berkeley.edu
>>>>
>>>
>>
>> --
>> Shane Knapp
>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>> https://rise.cs.berkeley.edu
>>
>
>
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>


-- 
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

Mime
View raw message