metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Casey Stella <ceste...@gmail.com>
Subject Re: [DISCUSS][PROPOSAL] Acceptance Tests
Date Fri, 03 Mar 2017 13:33:33 GMT
I also propose, once this is done, that we modify the developer bylaws and
the github PR script to ensure that PR authors:

   - Update the acceptance tests where appropriate
   - Run the tests as a smoketest



On Fri, Mar 3, 2017 at 8:21 AM, Casey Stella <cestella@gmail.com> wrote:

> Hi All,
>
> After doing METRON-744, where I had to walk through a manual test of every
> place that Stellar touched, it occurred to me that we should script this.
> It also occurred to me that some scripts that are run by the PR author to
> ensure no regressions and, eventually maybe, even run on an INFRA instance
> of Jenkins would give all of us some peace of mind.
>
> I am certain that this, along with a couple other manual tests from other
> PRs, could form the basis of a really great regression acceptance-test
> suite and I'd like to propose that we do that, as a community.
>
> What I'd like to see from such a suite has the following characteristics:
>
>    - Can be run on any Metron cluster, including but not limited to
>       - Vagrant
>       - AWS
>       - An existing deployment
>    - Can be *deployed* from ansible, but must be able to be deployed
>    manually
>       - With instructions in the readme
>    - Tests should be idempotent and independent
>       - Tear down what you set up
>
> I think between the Stellar REPL and the fundamental scriptability of the
> Hadoop services, we can accomplish these tests with a combination of shell
> scripts and python.
>
> I propose we break this into the following parts:
>
>    - Acceptance Testing Framework with a small smoketest
>    - Baseline Metron Test
>       - Send squid data through the squid topology
>       - Add an threat triage alert
>       - Ensure it gets through to the other side with alerts preserved
>    - + Enrichment
>       - Add an enrichment in the enrichment pipeline to the above
>    - + Profiler
>       - Add a profile with a tick of 1 minute to count per destination
>       address
>    - Base PCap test
>       - Something like the manual test for METRON-743 (
>       https://github.com/apache/incubator-metron/pull/467#issue-210285324
>       <https://github.com/apache/incubator-metron/pull/467#issue-210285324>
>       )
>
> Thoughts?
>
>
> Best,
>
> Casey
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message