metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Allen <n...@nickallen.org>
Subject [DISCUSS] Travis CI Build Time Limits
Date Wed, 22 May 2019 13:20:58 GMT
> Justin Leet said
<https://github.com/apache/metron/pull/1417#issuecomment-494464795>
[1]: Looking
at our build times, I'm actually concerned that if we kill the caches our
builds won't complete. The integration tests take >45 minutes and it's very
possible redownloading everything goes over our remaining time.

To recap, our current Travis CI build is composed of multiple jobs. Travis'
time limit of 50 minutes
<https://docs.travis-ci.com/user/customizing-the-build/#build-timeouts> [2]
is per job, rather than the total build time.  While our total build time
is on the order of 2.5 hours, it is only our integration-test job which is
coming close to that 50 minute limit.

We could try splitting our integration tests into multiple jobs.  Each of
these would have the opportunity to run in parallel (given whatever
resources Travis can allocate to us), but more importantly each one has its
own 50 minute limit. For example...

   - Parser Integration Tests:
      - `time mvn surefire:test@integration-tests -pl
      "metron-platform/metron-parsing/metron-parsing-storm,
      metron-platform/metron-parsing/metron-parsers,
      metron-platform/metron-parsing/metron-parsers-common/"`
   - Enrichment Integration Tests
      - `time mvn surefire:test@integration-tests -pl
      "metron-platform/metron-enrichment/metron-enrichment-common/,metron-platform/metron-enrichment/metron-enrichment-storm"`
   - etc, etc


We would just need to determine how to logically split the tests up.  If
this sounds reasonable, I've already got a start on a POC.

---
[1] https://github.com/apache/metron/pull/1417#issuecomment-494464795
[2] https://docs.travis-ci.com/user/customizing-the-build/#build-timeouts

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message