spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shane knapp <>
Subject extended jenkins downtime, thursday april 9th 7am-noon PDT (moving to anaconda python & more)
Date Fri, 03 Apr 2015 16:59:54 GMT
welcome to python2.7+, java 8 and more!  :)

i'll be doing a major upgrade to our build system next thursday morning.
 here's a quick list of what's going on:

* installation of anaconda python on all worker nodes

* installation of pypy 2.5.1 (python 2.7) on all nodes

* matching installation of python modules for the current system python
(2.6), and anaconda python (2.6, 2.7 and 3.4)
  - anaconda python 2.7 will be the default for all workers (this has
stealthily been the case on amp-jenkins-worker-01 for the past two weeks,
and i've noticed no test failures)
  - you can now use anaconda environments to specify which version of
python to use in your tests:

* installation of new python 2.7 modules:  pymongo requests six pymongo
requests six python-crontab

* bare-bones mongodb installation on all workers

* installation of java 1.6 and 1.8 internal to jenkins
  - jobs will default to the system java, which is 1.7.0_75
  - if you want to run your tests w/java 6 or 8, you can select the JDK
version of your choice in the job configuration page (it'll be towards the

these changes have actually all been tested against a variety of builds
(yay staging!) and while i'm certain that i have all of the kinks worked
out, i'm going to schedule a longer downtime so that i have a chance to
identify and squash any problems that surface.

thanks to josh rosen, k. shankari and davies liu for helping me test all of
this and get it working.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message