metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Allen <n...@nickallen.org>
Subject Re: [Discuss] Improve Alerting
Date Wed, 01 Feb 2017 19:35:06 GMT
Great.  I think we're thinking along the same lines.  I just sent a
follow-up of another proposal that takes this idea a little further.  What
if we treated the Profiler as another source of telemetry?

On Wed, Feb 1, 2017 at 2:23 PM, Casey Stella <cestella@gmail.com> wrote:

> Regarding point 2, could we enable the profiler to write data to kafka and
> the enrichment queue?
>
> I'm proposing the profiler do something like this:
>
>    - Count the number of inbound flows
>    - On the tick, send a message to the enrichment queue containing:
>       - the number of flows
>       - A source type of 'system_alert'
>       - is_alert set to true
>    - In enrichment, we enrich and triage system_alert source data in the
>    same way we do any other.
>
> This would not solve the transparency issue, but at least make it so we
> keep triage in one place in the architecture.  Also, enabling kafka writing
> would enable other types of use-cases, like situations where we find
> outliers *directly* in the profile and send the alerts directly to the
> indexing queue without triage.
>
> The only changes this proposal would require would be
>
>    1. a "write" section to a profile that takes a list of stellar
>    statements and gets run on the tick write
>    2. fixing the kafka writing stellar functions
>
> Casey
>
> On Wed, Feb 1, 2017 at 2:11 PM, Nick Allen <nick@nickallen.org> wrote:
>
> > I'd like to explore the functionality that we have in Metron using a
> > motivating example.  I think this will help highlight some gaps where we
> > can enhance Metron.
> >
> > The motivating example is that I would like to create an alert if the
> > number of inbound flows to any host over a 15 minute interval is
> abnormal.
> > I would like the alert to contain the specific information below to
> > streamline the triage process.
> >
> > Rule: Abnormal number of inbound flows
> > Bin: 15 mins
> > Alert: The host 'powned.svr.bank.com' has '230' inbound flows, exceeding
> > the threshold of '202'
> >
> >
> > *What Works*
> >
> > In some ways, this example is similar to the "Outlier Detection" demo
> that
> > I performed with the Profiler a few months back.   We have most of what
> we
> > need to do this with a couple caveats.
> >
> > 1. An enrichment would be added to enrich the message with the correct
> > internal hostname 'powned.svr.bank.com'.
> >
> > 2. With the Profiler, I can capture some idea of what "normal" is for the
> > number of inbound flows across 15 minute intervals.
> > 3. With Threat Triage, I can create rules that alert when a value exceeds
> > what the Profiler defines as normal.
> >
> >
> > *What's Missing*
> >
> > Its nice to know that we are almost all the way there with this example.
> > Unfortunately, there are two gaps that fall out of this.
> >
> >  1. *Threat Triage Transparency*
> >
> > There is little transparency into the Threat Triage process itself.  When
> > Threat Triage runs, all I get is a score.  I don't know how that score
> was
> > arrived at, which rules were triggered, and the specific values that
> caused
> > a rule to trigger.
> >
> > More specifically, there is no way to generate a message that looks like
> > "The host 'powned.svr.bank.com' has '230' inbound flows, exceeding the
> > threshold of '202'".
> >
> >
> > 2. *Triage Calculated Values from the Profiler*
> >
> > Also, the value being interrogated here, the number of inbound flows, is
> > not a static value contained within any single telemetry message.  This
> > value is calculated across multiple messages by the Profiler.  The
> current
> > Threat Triage process cannot be used to interrogate values calculated by
> > the Profiler.
> >
> >
> > To try and keep this email concise and digestible, I am going to send a
> > follow-on discussing proposed solutions for each of these separately.
> >
>



-- 
Nick Allen <nick@nickallen.org>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message