metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nickwallen <...@git.apache.org>
Subject [GitHub] incubator-metron pull request #438: METRON-686 Record Rule Set that Fired Du...
Date Fri, 03 Feb 2017 19:39:21 GMT
GitHub user nickwallen opened a pull request:

    https://github.com/apache/incubator-metron/pull/438

    METRON-686 Record Rule Set that Fired During Threat Triage

    The goal of threat triage is to prioritize the alerts that pose the greatest threat and
thus need urgent attention.  To perform threat triage, a set of rules are applied to each
message.  Each rule has a predicate to determine if the rule applies or not.  The threat score
from each applied rule is aggregated into a single threat triage score that can be used to
prioritize high risk threats.
    
    ### Changes
    * All of the threat triage rules that were applied to a message during threat triage are
captured and recorded.
    * A new 'reason' field was added to the threat triage rules to allow a rule author to
provide contextual information to assist in actioning a threat. 
    
    ### Implementation
    The `ThreatTriageProcessor` attempts to apply each `RiskLevelRule` to a message being
triaged.  If the rule's predicate returns true, the rule is applied to the message resulting
in a `RuleScore`.  The `RuleScore` represents the portion of the threat score contributed
by a single rule.  Each of the `RuleScore`s are aggregated into an overall `ThreatScore`,
which captures the outcome of the threat triage process.  This `ThreatScore` is then attached
to the message.
    
    ### Example
    
    Previously the result of threat triage was a single threat score represented as a double.
 
    ```
    "threat.triage.level": 10.0
    ```
    
    The result of threat triage now provides greater information about the triage process.
 The rule set...
    ```
        "triageConfig" : {
          "riskLevelRules" : [ {
            "name" : "Abnormal DNS Port",
            "rule" : "source.type == \"bro\" and protocol == \"dns\" and ip_dst_port != 53",
            "score" : 10.0,
            "reason" : "FORMAT(\"Abnormal DNS Port: expected: 53, found: %s:%d\", ip_dst_addr,
ip_dst_port)"
          } ],
          "aggregator" : "MAX",
          "aggregationConfig" : { }
        }
    ```
    
    produces a result like...
    ```
    "threat.triage.level":{
       "score":10.0,
       "rules":[
          { 
             "name":"Abnormal DNS Port",
             "comment":null
             "score":10.0,
             "reason":"Abnormal DNS Port: expected: 53, found: 224.0.0.251:5353",
          }
       ]
    }
    ```
    
    ### Testing
    Setup a Metron environment with the default Bro sensor running.  The following commands
can be executed in the Stellar REPL.
    
    ```
    # fetch the current config
    conf := CONFIG_GET("ENRICHMENT", "bro")
    
    # triage all bro telemetry
    is_alert := source.type == 'bro'
    conf := ENRICHMENT_STELLAR_TRANSFORM_ADD(conf, 'ENRICHMENT', SHELL_VARS2MAP('is_alert'))
    
    # create a rule that looks for DNS running on non-standard ports
    rule := source.type == "bro" and protocol == "dns" and ip_dst_port != 53
    triage := { "name":"Abnormal DNS Port", "rule": SHELL_GET_EXPRESSION('rule'), "reason":
SHELL_GET_EXPRESSION('reason'), "score":"10" }
    conf := THREAT_TRIAGE_ADD(conf, [triage])
    
    # save the configuration
    CONFIG_PUT("ENRICHMENT", conf, "bro")
    ```
    
    Monitor the messages being pushed to the 'indexing' topic in Kafka.  You should expect
to see a message where the 'threat.triage.level' is like the example shown above.
    
    Printing the threat triage rule in the REPL is going to look something like this.
    ```
    > THREAT_TRIAGE_PRINT(conf)
    ╔═══════════════════╤═════════╤══════════════════════════════════════════════════════════════════╤═══════╤═══════════════════════════════════════════════════════════════════════════════════╗
    ║ Name              │ Comment │ Triage Rule                                    
                 │ Score │ Reason                                                    
                       ║
    ╠═══════════════════╪═════════╪══════════════════════════════════════════════════════════════════╪═══════╪═══════════════════════════════════════════════════════════════════════════════════╣
    ║ Abnormal DNS Port │         │ source.type == "bro" and protocol == "dns" and ip_dst_port
!= 53 │ 10    │ FORMAT("Abnormal DNS Port: expected: 53, found: %s:%d", ip_dst_addr, ip_dst_port)
║
    ╚═══════════════════╧═════════╧══════════════════════════════════════════════════════════════════╧═══════╧═══════════════════════════════════════════════════════════════════════════════════╝
    
    ```

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/nickwallen/incubator-metron METRON-686

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-metron/pull/438.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #438
    
----
commit fe455187ff1f928edc43a6bfa509408695783a54
Author: Nick Allen <nick@nickallen.org>
Date:   2017-02-02T23:29:07Z

    METRON-686 Record Rule Set that Fired During Threat Triage

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message