spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shane knapp <skn...@berkeley.edu>
Subject Re: JENKINS: downtime next week, wed and thurs mornings (9-23 and 9-24)
Date Thu, 24 Sep 2015 14:19:36 GMT
this is happening now.

On Tue, Sep 22, 2015 at 10:07 AM, shane knapp <sknapp@berkeley.edu> wrote:
> ok, here's the updated downtime schedule for this week:
>
> wednesday, sept 23rd:
>
> firewall maintenance cancelled, as jon took care of the update
> saturday morning while we were bringing jenkins back up after the colo
> fire
>
> thursday, sept 24th:
>
> jenkins maintenance is still scheduled, but abbreviated as some of the
> maintenance was performed saturday morning as well
> * new builds will stop being accepted ~630am PDT
>   - i'll kill any hangers-on at 730am, and after maintenance is done,
> i will retrigger any killed jobs
> * jenkins worker system package updates
>   - amp-jenkins-master was completed on saturday
>   - this will NOT include kernel updates as moving to
> 2.6.32-573.3.1.el6 bricked amp-jenkins-master
> * moving default system java for builds from jdk1.7.0_71 to jdk1.7.0_79
> * all systems get a reboot
> * expected downtime:  3.5 hours or so
>
> i'll post updates as i progress.
>
> also, i'll post a copy of our post-mortem once the dust settles.  it's
> been, shall we say, a pretty crazy few days.
>
> http://news.berkeley.edu/2015/09/19/campus-network-outage/
>
> :)
>
> On Mon, Sep 21, 2015 at 10:11 AM, shane knapp <sknapp@berkeley.edu> wrote:
>> quick update:  we actually did some of the maintenance on our systems
>> after the berkeley-wide outage caused by one of our (non-jenkins)
>> servers halting and catching fire.
>>
>> we'll still have some downtime early wednesday, but tomorrow's will be
>> cancelled.  i'll send out another update real soon now with what we'll
>> be covering on wednesday once we get our current situation more under
>> control.  :)
>>
>> On Wed, Sep 16, 2015 at 12:15 PM, shane knapp <sknapp@berkeley.edu> wrote:
>>>> 630am-10am thursday, 9-24-15:
>>>> * jenknins update to 1.629 (we're a few months behind in versions, and
>>>> some big bugs have been fixed)
>>>> * jenkins master and worker system package updates
>>>> * all systems get a reboot (lots of hanging java processes have been
>>>> building up over the months)
>>>> * builds will stop being accepted ~630am, and i'll kill any hangers-on
>>>> at 730am, and retrigger once we're done
>>>> * expected downtime:  3.5 hours or so
>>>> * i will also be testing out some of my shiny new ansible playbooks
>>>> for the system updates!
>>>>
>>> i forgot one thing:
>>>
>>> * moving default system java for builds from jdk1.7.0_71 to jdk1.7.0_79

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message