metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Merriman <>
Subject [DISCUSS] Pcap panel architecture
Date Thu, 03 May 2018 18:35:53 GMT
We are planning on adding the pcap query feature to the Alerts UI.  Before
we start this work, I think it is important to get community buy in on the
architectural approach.  There are a couple different options.

One option is to leverage the existing metron-api module that exposes pcap
queries through a REST service.  The upsides are:

   - some work has already been done
   - it's part of our build so we know unit and integration tests pass

The downsides are:

   - It hasn't been used in a while and will need some end to end testing
   to make sure it still functions properly
   - It is synchronous and will block the UI, using up the limited number
   of concurrent connections available in a browser
   - It will require significant MPack work to properly set it up on install
   - It is a legacy module from OpenSOC and coding style is significantly

Another option would be moving to a micro-services architecture.  We have
experimented with a proof of concept and found it was too hard to add this
feature into our existing REST services because of all the dependencies
that must coexist in the same application.  The upsides are:

   - Would provide a platform for future Batch/MR/YARN type features
   - There would be fewer technical compromises since we are building it
   from the ground up

The downsides are:

   - Will require the most effort and will likely take a long time to plan
   and implement
   - Like the previous option, will require significant MPack work

A third option would be to add an endpoint to our existing REST service
that delegates to the script through the Java Process class.
The upsides to this approach are:

   - We know the script works and would require minimal
   - Minimal MPack work is required since our REST service is already

The downsides are:

   - Does not set us up to easily add other batch-oriented features in the
   - OS-level security becomes a concern since we are delegating to a
   script in a separate process

I feel like ultimately we want to transition to a micro-services
architecture because it will provide more flexibility and make it easier to
grow our set of features.  But in the meantime, wrapping the
script would allow us to add this feature with less work and fewer lines of
code.  If and when we decide to deploy a separate REST application for
batch features, the UI portion would require minimal changes.

What does everyone think?

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message