metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Allen <n...@nickallen.org>
Subject Re: [Discuss] Improve Alerting
Date Wed, 01 Feb 2017 19:33:33 GMT
*Problem*

Triage Calculated Values from the Profiler

The value being interrogated here, the number of inbound flows, is not a
static value contained within any single telemetry message.  This value is
calculated across multiple messages by the Profiler.  The current Threat
Triage process cannot be used to interrogate values calculated by the
Profiler.

*Proposed Solution*

What I am proposing here is that we treat the Profiler as a source of
telemetry.   The measurements captured by the Profiler would be enqueued
into a Kafka topic.  We would then treat those Profiler messages like any
other telemetry.  We would parse, enrich, triage, and index those messages.

While this isn't fully formed in my head, I am throwing this out there as I
think its rather interesting.  I think this would have the following
advantages.

1.  We would be able to resuse the same threat triage mechanism for values
calculated by the Profiler.

2.  We would be able to generate profiles from the profiled data - aka
meta-profiles anyone?  I am really curious if this helps the MAD use case
at all.

3.  We could also potentially have all the data produced by the Profiler
written to HBase using the same indexing mechanism as all the other
telemetry.

On Wed, Feb 1, 2017 at 2:11 PM, Nick Allen <nick@nickallen.org> wrote:

> I'd like to explore the functionality that we have in Metron using a
> motivating example.  I think this will help highlight some gaps where we
> can enhance Metron.
>
> The motivating example is that I would like to create an alert if the
> number of inbound flows to any host over a 15 minute interval is abnormal.
> I would like the alert to contain the specific information below to
> streamline the triage process.
>
> Rule: Abnormal number of inbound flows
> Bin: 15 mins
> Alert: The host 'powned.svr.bank.com' has '230' inbound flows, exceeding
> the threshold of '202'
>
>
> *What Works*
>
> In some ways, this example is similar to the "Outlier Detection" demo that
> I performed with the Profiler a few months back.   We have most of what we
> need to do this with a couple caveats.
>
> 1. An enrichment would be added to enrich the message with the correct
> internal hostname 'powned.svr.bank.com'.
>
> 2. With the Profiler, I can capture some idea of what "normal" is for the
> number of inbound flows across 15 minute intervals.
> 3. With Threat Triage, I can create rules that alert when a value exceeds
> what the Profiler defines as normal.
>
>
> *What's Missing*
>
> Its nice to know that we are almost all the way there with this example.
> Unfortunately, there are two gaps that fall out of this.
>
>  1. *Threat Triage Transparency*
>
> There is little transparency into the Threat Triage process itself.  When
> Threat Triage runs, all I get is a score.  I don't know how that score was
> arrived at, which rules were triggered, and the specific values that caused
> a rule to trigger.
>
> More specifically, there is no way to generate a message that looks like
> "The host 'powned.svr.bank.com' has '230' inbound flows, exceeding the
> threshold of '202'".
>
>
> 2. *Triage Calculated Values from the Profiler*
>
> Also, the value being interrogated here, the number of inbound flows, is
> not a static value contained within any single telemetry message.  This
> value is calculated across multiple messages by the Profiler.  The current
> Threat Triage process cannot be used to interrogate values calculated by
> the Profiler.
>
>
> To try and keep this email concise and digestible, I am going to send a
> follow-on discussing proposed solutions for each of these separately.
>
>
>
>
>
>


-- 
Nick Allen <nick@nickallen.org>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message