spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Davies Liu <dav...@databricks.com>
Subject Re: extended jenkins downtime, thursday april 9th 7am-noon PDT (moving to anaconda python & more)
Date Tue, 14 Apr 2015 18:41:15 GMT
Hey Shane,

Have you updated all the jenkins slaves?

There is a run with old configurations (no Python 3, with 130 minutes
timeout), see https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/666/consoleFull

Davies

On Thu, Apr 9, 2015 at 10:18 AM, shane knapp <sknapp@berkeley.edu> wrote:
> ok, we're looking good.  i'll keep an eye on this for the rest of the day,
> and if you happen to notice any infrastructure failures before i do (i
> updated a LOT), please let me know immediately!  :)
>
> On Thu, Apr 9, 2015 at 8:38 AM, shane knapp <sknapp@berkeley.edu> wrote:
>
>> things are looking pretty good and i expect to be done within an hour.
>>  i've got some test builds running right now, and will give the green light
>> when they successfully complete.
>>
>> On Thu, Apr 9, 2015 at 7:29 AM, shane knapp <sknapp@berkeley.edu> wrote:
>>
>>> and this is now happening.
>>>
>>> On Tue, Apr 7, 2015 at 4:38 PM, shane knapp <sknapp@berkeley.edu> wrote:
>>>
>>>> reminder!  this is happening thurday morning.
>>>>
>>>> On Fri, Apr 3, 2015 at 9:59 AM, shane knapp <sknapp@berkeley.edu> wrote:
>>>>
>>>>> welcome to python2.7+, java 8 and more!  :)
>>>>>
>>>>> i'll be doing a major upgrade to our build system next thursday
>>>>> morning.  here's a quick list of what's going on:
>>>>>
>>>>> * installation of anaconda python on all worker nodes
>>>>>
>>>>> * installation of pypy 2.5.1 (python 2.7) on all nodes
>>>>>
>>>>> * matching installation of python modules for the current system python
>>>>> (2.6), and anaconda python (2.6, 2.7 and 3.4)
>>>>>   - anaconda python 2.7 will be the default for all workers (this has
>>>>> stealthily been the case on amp-jenkins-worker-01 for the past two weeks,
>>>>> and i've noticed no test failures)
>>>>>   - you can now use anaconda environments to specify which version of
>>>>> python to use in your tests:  http://www.continuum.io/blog/conda
>>>>>
>>>>> * installation of new python 2.7 modules:  pymongo requests six pymongo
>>>>> requests six python-crontab
>>>>>
>>>>> * bare-bones mongodb installation on all workers
>>>>>
>>>>> * installation of java 1.6 and 1.8 internal to jenkins
>>>>>   - jobs will default to the system java, which is 1.7.0_75
>>>>>   - if you want to run your tests w/java 6 or 8, you can select the JDK
>>>>> version of your choice in the job configuration page (it'll be towards
the
>>>>> top)
>>>>>
>>>>> these changes have actually all been tested against a variety of builds
>>>>> (yay staging!) and while i'm certain that i have all of the kinks worked
>>>>> out, i'm going to schedule a longer downtime so that i have a chance
to
>>>>> identify and squash any problems that surface.
>>>>>
>>>>> thanks to josh rosen, k. shankari and davies liu for helping me test
>>>>> all of this and get it working.
>>>>>
>>>>> shane
>>>>>
>>>>
>>>>
>>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message