metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Allen <n...@nickallen.org>
Subject Re: [DISCUSS] Travis CI Build Time Limits
Date Wed, 22 May 2019 13:47:24 GMT
FYI - Here is a POC build of this concept running.  This only runs the
Parser and Enrichment integration tests, but as separate jobs in Travis.

https://travis-ci.org/nickwallen/metron/builds/535791047




On Wed, May 22, 2019 at 9:20 AM Nick Allen <nick@nickallen.org> wrote:

>
> > Justin Leet said
> <https://github.com/apache/metron/pull/1417#issuecomment-494464795> [1]: Looking
> at our build times, I'm actually concerned that if we kill the caches our
> builds won't complete. The integration tests take >45 minutes and it's very
> possible redownloading everything goes over our remaining time.
>
> To recap, our current Travis CI build is composed of multiple jobs. Travis'
> time limit of 50 minutes
> <https://docs.travis-ci.com/user/customizing-the-build/#build-timeouts> [2]
> is per job, rather than the total build time.  While our total build time
> is on the order of 2.5 hours, it is only our integration-test job which is
> coming close to that 50 minute limit.
>
> We could try splitting our integration tests into multiple jobs.  Each of
> these would have the opportunity to run in parallel (given whatever
> resources Travis can allocate to us), but more importantly each one has its
> own 50 minute limit. For example...
>
>    - Parser Integration Tests:
>       - `time mvn surefire:test@integration-tests -pl
>       "metron-platform/metron-parsing/metron-parsing-storm,
>       metron-platform/metron-parsing/metron-parsers,
>       metron-platform/metron-parsing/metron-parsers-common/"`
>    - Enrichment Integration Tests
>       - `time mvn surefire:test@integration-tests -pl
>       "metron-platform/metron-enrichment/metron-enrichment-common/,metron-platform/metron-enrichment/metron-enrichment-storm"`
>    - etc, etc
>
>
> We would just need to determine how to logically split the tests up.  If
> this sounds reasonable, I've already got a start on a POC.
>
> ---
> [1] https://github.com/apache/metron/pull/1417#issuecomment-494464795
> [2] https://docs.travis-ci.com/user/customizing-the-build/#build-timeouts
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message