metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Allen <n...@nickallen.org>
Subject Re: [DISCUSS] Travis CI Build Time Limits
Date Thu, 23 May 2019 17:14:07 GMT
> (1) The build is really a tree of modules. With our recent de-coupling
   from Storm, this has become all the more exaggerated...

I agree that the approach I followed in my POC suffers from the risk of a
future refactor accidently causing some tests not to run.  But I just did
it this way because it was the simplest approach to see something work.  I
think we could easily find a way to mitigate or eliminate this risk in a
final solution.


> (2) I actually like the current limits because it has forced contributors and
reviewers to consider the overall build time as they design and add new
tests....

I agree with the goal.  I want the tests to run as quickly as possible too. But
spending time here worried about whether the builds will complete after the
cache is invalidated seems like a waste of everyone's time.  In addition,
I've seen times where my own feature branches CI builds fail because the
time limit is breached.  I think this tends to happen when Travis is under
greater load or when they have some resources down for maintenance or
outages (or so I assume).

Personally, I am in favor of anything that reduces the time for a developer
to get feedback on changes they've made.  I like the idea of
parallelizing/splitting the integration tests for this reason.









On Thu, May 23, 2019 at 11:33 AM Michael Miklavcic <
michael.miklavcic@gmail.com> wrote:

> I think this is a neat idea, however I have a couple concerns with this:
>
>    1. The build is really a tree of modules. With our recent de-coupling
>    from Storm, this has become all the more exaggerated. This is a good
> thing,
>    however the benefit of the parent pom approach we currently have is
> that it
>    does a pretty good job of guaranteeing all of the leaf modules get
> built as
>    well. Specifying the modules individually is going to turn this into a
> very
>    manual process any time a new module is added or refactored, and is more
>    likely to result in bugs. One other option to aid in solving that
> problem
>    is to use aggregator modules, and then have the full local build parent
> pom
>    use those aggregators instead of referring to the individual modules the
>    way it does now. You still get the top-down tree, but there's less of a
> gap
>    between Travis and our primary Maven structure.
>    2. I actually like the current limits because it has forced contributors
>    and reviewers to consider the overall build time as they design and add
> new
>    tests. Especially with the current set of upgrades and proposed
> integration
>    test and Docker changes, I think we should be extra vigilant on getting
>    those build/test times *down*, rather than enabling them to increase.
> Sure,
>    there are other options for solving that problem, but Travis is a simple
>    equalizer because it's impartial to whatever local hardware engineers
> are
>    using.
>
>
> On Wed, May 22, 2019 at 7:48 AM Nick Allen <nick@nickallen.org> wrote:
>
> > FYI - Here is a POC build of this concept running.  This only runs the
> > Parser and Enrichment integration tests, but as separate jobs in Travis.
> >
> > https://travis-ci.org/nickwallen/metron/builds/535791047
> >
> >
> >
> >
> > On Wed, May 22, 2019 at 9:20 AM Nick Allen <nick@nickallen.org> wrote:
> >
> > >
> > > > Justin Leet said
> > > <https://github.com/apache/metron/pull/1417#issuecomment-494464795>
> > [1]: Looking
> > > at our build times, I'm actually concerned that if we kill the caches
> our
> > > builds won't complete. The integration tests take >45 minutes and it's
> > very
> > > possible redownloading everything goes over our remaining time.
> > >
> > > To recap, our current Travis CI build is composed of multiple jobs.
> > Travis'
> > > time limit of 50 minutes
> > > <https://docs.travis-ci.com/user/customizing-the-build/#build-timeouts
> >
> > [2]
> > > is per job, rather than the total build time.  While our total build
> time
> > > is on the order of 2.5 hours, it is only our integration-test job which
> > is
> > > coming close to that 50 minute limit.
> > >
> > > We could try splitting our integration tests into multiple jobs.  Each
> of
> > > these would have the opportunity to run in parallel (given whatever
> > > resources Travis can allocate to us), but more importantly each one has
> > its
> > > own 50 minute limit. For example...
> > >
> > >    - Parser Integration Tests:
> > >       - `time mvn surefire:test@integration-tests -pl
> > >       "metron-platform/metron-parsing/metron-parsing-storm,
> > >       metron-platform/metron-parsing/metron-parsers,
> > >       metron-platform/metron-parsing/metron-parsers-common/"`
> > >    - Enrichment Integration Tests
> > >       - `time mvn surefire:test@integration-tests -pl
> > >
> >
> "metron-platform/metron-enrichment/metron-enrichment-common/,metron-platform/metron-enrichment/metron-enrichment-storm"`
> > >    - etc, etc
> > >
> > >
> > > We would just need to determine how to logically split the tests up.
> If
> > > this sounds reasonable, I've already got a start on a POC.
> > >
> > > ---
> > > [1] https://github.com/apache/metron/pull/1417#issuecomment-494464795
> > > [2]
> > https://docs.travis-ci.com/user/customizing-the-build/#build-timeouts
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message