spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shane knapp <skn...@berkeley.edu>
Subject extended jenkins downtime, thursday april 9th 7am-noon PDT (moving to anaconda python & more)
Date Fri, 03 Apr 2015 16:59:54 GMT
welcome to python2.7+, java 8 and more!  :)

i'll be doing a major upgrade to our build system next thursday morning.
 here's a quick list of what's going on:

* installation of anaconda python on all worker nodes

* installation of pypy 2.5.1 (python 2.7) on all nodes

* matching installation of python modules for the current system python
(2.6), and anaconda python (2.6, 2.7 and 3.4)
  - anaconda python 2.7 will be the default for all workers (this has
stealthily been the case on amp-jenkins-worker-01 for the past two weeks,
and i've noticed no test failures)
  - you can now use anaconda environments to specify which version of
python to use in your tests:  http://www.continuum.io/blog/conda

* installation of new python 2.7 modules:  pymongo requests six pymongo
requests six python-crontab

* bare-bones mongodb installation on all workers

* installation of java 1.6 and 1.8 internal to jenkins
  - jobs will default to the system java, which is 1.7.0_75
  - if you want to run your tests w/java 6 or 8, you can select the JDK
version of your choice in the job configuration page (it'll be towards the
top)

these changes have actually all been tested against a variety of builds
(yay staging!) and while i'm certain that i have all of the kinks worked
out, i'm going to schedule a longer downtime so that i have a chance to
identify and squash any problems that surface.

thanks to josh rosen, k. shankari and davies liu for helping me test all of
this and get it working.

shane

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message