spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicholas Chammas <nicholas.cham...@gmail.com>
Subject Re: Unit test logs in Jenkins?
Date Thu, 02 Apr 2015 15:54:07 GMT
This is secondary to Marcelo’s question, but I wanted to comment on this:

Its main limitation is more cultural than technical: you need to get people
to care about intermittent test runs, otherwise you can end up with
failures that nobody keeps on top of

This is a problem that plagues Spark as well, but there *is* a technical
solution.

The solution is simple: *All* the builds that we care about run for *every*
proposed change. If *any* build fails, the change doesn’t make it into the
repository.

Spark already has a pull request builder that tests and reports back on
PRs. Committers don’t merge in PRs when this builder reports that it failed
some tests. That’s a good thing.

The problem is that there are several other builds that we run on a fixed
interval, independent of the pull request builder. These builds test
different configurations, dependency versions, and environments than what
the PR builder covers. If one of those builds fails, it fails on its own
little island, with no-one to hear it scream. The build failure is detached
from the PR that caused it to fail.

What should happen is that the whole matrix of stuff we care to test gets
run for every PR. No PR goes in if any build we care about fails for that
PR, and every build we care about runs for every commit of every PR.

Really, this is just an extension of the basic idea of the PR builder. It
doesn’t make much sense to test stuff *after* it has been committed and
potentially broken things. And it becomes exponentially more difficult to
find and fix a problem the longer it has been festering in the repo. It’s
best to keep such problems out in the first place.

With some more work on our CI infrastructure, I think this can be done.
Maybe even later this year.

Nick

On Thu, Apr 2, 2015 at 6:02 AM Steve Loughran stevel@hortonworks.com
<http://mailto:stevel@hortonworks.com> wrote:


> > On 2 Apr 2015, at 06:31, Patrick Wendell <pwendell@gmail.com> wrote:
> >
> > Hey Marcelo,
> >
> > Great question. Right now, some of the more active developers have an
> > account that allows them to log into this cluster to inspect logs (we
> > copy the logs from each run to a node on that cluster). The
> > infrastructure is maintained by the AMPLab.
> >
> > I will put you in touch the someone there who can get you an account.
> >
> > This is a short term solution. The longer term solution is to have
> > these scp'd regularly to an S3 bucket or somewhere people can get
> > access to them, but that's not ready yet.
> >
> > - Patrick
> >
> >>
>
>
> ASF Jenkins is always there to play with; committers/PMC members should
> just need to file a BUILD JIRA to get access.
>
> Its main limitation is more cultural than technical: you need to get
> people to care about intermittent test runs, otherwise you can end up with
> failures that nobody keeps on top of
> https://builds.apache.org/view/H-L/view/Hadoop/
>
> Someone really needs to own the "keep the builds working" problem -and
> have the ability to somehow kick others into fixing things. The latter is
> pretty hard cross-organisation
>
>
> >> That would be really helpful to debug build failures. The scalatest
> >> output isn't all that helpful.
> >>
>
> Potentially an issue with the test runner, rather than the tests
> themselves.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>  ​

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message