spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jungtaek Lim <kabh...@gmail.com>
Subject Re: Weird build failures in PR builder
Date Tue, 17 Sep 2019 04:13:16 GMT
Thanks for following up and providing the info! I'll also keep an eye after
clean-up to see whether it helps.

On Tue, Sep 17, 2019 at 1:08 PM Shane Knapp <sknapp@berkeley.edu> wrote:

> > Note that they're different workers (amp-jenkins-worker-0x) with same
> workspace /home/jenkins/workspace/SparkPullRequestBuilder@6.
> >
> the '@6' refers to the executor number that the build ran on (look at
> the jenkins ui and the numbers next to each slot for each worker).
>
> oddly, at the beginning of the build when we overload HOME to point to
> the .ivy dir, the executor doesn't seem to match up with what
> per-executor dir is being set to.  example:
>
>
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110688/consoleFull
> executor number is 8 (from
>
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110688/injectedEnvVars
> )
> HOME is set properly:  + HOME=/home/sparkivy/per-executor-caches/8
> but the dir the build runs from is indeed
> /home/jenkins/workspace/SparkPullRequestBuilder@6/
>
> > Any guess/suspect?
> >
> nope. no clue whatsoever.  ¯\_(ツ)_/¯
>
> what i CAN do first thing tomorrow is pause builds, wipe out the
> ivy/sbt caches and SparkPullRequestBuilder* dirs on all workers and
> see if that helps.
>
> shane
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>


-- 
Name : Jungtaek Lim
Blog : http://medium.com/@heartsavior
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Mime
View raw message