metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Miklavcic <michael.miklav...@gmail.com>
Subject Re: [DISCUSS] Travis CI Build Time Limits
Date Thu, 23 May 2019 15:18:19 GMT
I think we're talking about killing the cache just 1 time for the purpose
of verifying that the https:// repo endpoints will work. It's possible I'm
misunderstanding how we've configured our caches, but it seems like we'd
just need to kick travis 1x after the first failed build that did all the
downloading, and then the cache will have been populated.

On Wed, May 22, 2019 at 7:21 AM Nick Allen <nick@nickallen.org> wrote:

> > Justin Leet said
> <https://github.com/apache/metron/pull/1417#issuecomment-494464795>
> [1]: Looking
> at our build times, I'm actually concerned that if we kill the caches our
> builds won't complete. The integration tests take >45 minutes and it's very
> possible redownloading everything goes over our remaining time.
>
> To recap, our current Travis CI build is composed of multiple jobs. Travis'
> time limit of 50 minutes
> <https://docs.travis-ci.com/user/customizing-the-build/#build-timeouts>
> [2]
> is per job, rather than the total build time.  While our total build time
> is on the order of 2.5 hours, it is only our integration-test job which is
> coming close to that 50 minute limit.
>
> We could try splitting our integration tests into multiple jobs.  Each of
> these would have the opportunity to run in parallel (given whatever
> resources Travis can allocate to us), but more importantly each one has its
> own 50 minute limit. For example...
>
>    - Parser Integration Tests:
>       - `time mvn surefire:test@integration-tests -pl
>       "metron-platform/metron-parsing/metron-parsing-storm,
>       metron-platform/metron-parsing/metron-parsers,
>       metron-platform/metron-parsing/metron-parsers-common/"`
>    - Enrichment Integration Tests
>       - `time mvn surefire:test@integration-tests -pl
>
> "metron-platform/metron-enrichment/metron-enrichment-common/,metron-platform/metron-enrichment/metron-enrichment-storm"`
>    - etc, etc
>
>
> We would just need to determine how to logically split the tests up.  If
> this sounds reasonable, I've already got a start on a POC.
>
> ---
> [1] https://github.com/apache/metron/pull/1417#issuecomment-494464795
> [2] https://docs.travis-ci.com/user/customizing-the-build/#build-timeouts
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message