metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (METRON-141) The ability to do threat triage
Date Fri, 06 May 2016 23:00:15 GMT

    [ https://issues.apache.org/jira/browse/METRON-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15274884#comment-15274884
] 

ASF GitHub Bot commented on METRON-141:
---------------------------------------

GitHub user cestella opened a pull request:

    https://github.com/apache/incubator-metron/pull/108

    METRON-141: The ability to do threat triage

    We have the ability to mark messages as part of the enrichment topology as threat alerts,
but we have no ability to prioritize those alerts.
    
    We should allow for the prioritization of messages that have some threat intelligence
alert via a scoring mechanism. 
    
    The general idea is for each message from each sensor:
    1. Map threat conditions as defined by a lightweight query language to threat levels (a
number).
    2. More than one condition may match a given message, so allow for the choice of aggregation
functions.
    3. If the aggregated score is positive, then add a field (`threat.triage.level`) with
the level to the JSON message.
    
    The above configuration is done at the per-sensor level.
    
    As part of this PR, the following were added out of necessity:
    * A query language DSL which is used in defining the conditions which map to threat levels.
 Docs at [here](https://github.com/cestella/incubator-metron/tree/METRON-141/metron-platform/metron-common#query-language)
    * The addition of a `triageConfig` section to the sensor enrichment config.  Description
at [here](https://github.com/cestella/incubator-metron/tree/METRON-141/metron-platform/metron-common#the-threatintel-configuration)
    * Actual documentation of our enrichment configuration.  Documentation at [here](https://github.com/cestella/incubator-metron/tree/METRON-141/metron-platform/metron-common#enrichment-configuration)
    * A rewrite of the `zk_load_configs.sh` utility to support both pushing and pulling configs
(the threat triage rules are part of the configs and until we have a UI, we need to add them).
 Docs at [here](https://github.com/cestella/incubator-metron/tree/METRON-141/metron-platform/metron-common#management-utility)
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/cestella/incubator-metron METRON-141

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-metron/pull/108.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #108
    
----
commit cfdffd48d6a5ea7c228c4b66f23c16cc55691f79
Author: cstella <cestella@gmail.com>
Date:   2016-05-06T22:46:02Z

    METRON-141: The ability to do threat triage

----


> The ability to do threat triage
> -------------------------------
>
>                 Key: METRON-141
>                 URL: https://issues.apache.org/jira/browse/METRON-141
>             Project: Metron
>          Issue Type: New Feature
>            Reporter: Casey Stella
>            Assignee: Casey Stella
>
> We have the ability to mark messages as part of the enrichment topology as threat alerts,
but we have no ability to prioritize those alerts.
> We should allow for the prioritization of messages that have some threat intelligence
alert via a scoring mechanism.  The one implemented here allows the user to map conditions
expressed via a light-weight DSL to a score and allow a configurable aggregation strategy.
> The DSL to express conditions should allow for the following:
> * Referencing fields in the enriched JSON
> * Simple boolean operations: and, not, or
> * The ability to have parenthesis to make order of operations explicit
> * A fixed set of functions which take strings and return boolean (currently IN_SUBNET(ip,
cidr1, cidr2, ...), IS_EMPTY(str), STARTS_WITH(str, prefix), ENDS_WITH(str, suffix), REGEXP_MATCH(str,
pattern) )
> * A fixed set of string to string transformation functions:  TO_LOWER, TO_UPPER, TRIM
> For each message, if the rule as expressed by the DSL matches on the message, then we
are given a list of numbers to aggregate into a single score.  Aggregation functions supported
are as follows:
> * MAX
> * MEAN
> * POSITIVE_MEAN - the mean of the positive scores
> If an aggregated score that is positive is yielded, then a field 'threat.triage.level'
with the score is added to the indexed JSON.
> This configuration will be done on a per-sensor basis and added to the SensorEnrichmentConfig.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message