metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Justin Leet <>
Subject [DISCUSS] Full-dev role in PR testign
Date Wed, 01 May 2019 14:59:09 GMT
Hi all,

I wanted to start a discussion on something near and dear to all of our
hearts: The role of full-dev in our testing cycle.

Right now, we require all PRs to have spun up the full-dev environment and
make sure that things flow through properly. In some cases, this is a
necessity, and in other cases, it's a fairly large burden on current and
potential contributors.

So what do we need to do to reduce our dependence on full dev and increase
our confidence in our CI process?

My initial thoughts on this encompass a few things
* Increasing our ability to easily write automated tests. In particular, I
think our integration testing is fairly hard and has some structural issues
(e.g. our tests spin up components per Test class, not for the overall test
suite being run, which also increases build times, etc.). How do we make
this easier and have less boilerplate?  I have one potential idea I'll
reply to this thread with.
* Unit tests in general. I'd argue that any area we thing need full-dev
spun up to be confident in needs more testing. Does anyone have any
opinions on which areas we have the least confidence in?  Which areas of
code are people afraid to touch?
* Our general procedures. Should we not be requiring full-dev on all PRs,
but maybe just asking for a justification why not (e.g. "I don't need
full-dev for this, because I added unit tests here and here and integration
tests for the these components?"). And then a reviewer can request a
full-dev spin up if needed?  Could we stage this rollout (e.g.
metron-platform modules require it, but others don't, etc.?) This'll add
more pressure onto the release phase, so maybe that involves fleshing out a
checklist of critical path items to check.
* Do we need to improve our docs? Publish Javadocs? After all, if docs are
better, hopefully we'd see less issues introduced and be more confident in
changes coming in.
* What environment do we even want testing to occur in? has seen a lot of work, and
getting it across the finish line may help make the dev environment less
onerous, even if still needed.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message