metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dima Kovalyov <Dima.Koval...@sstech.us>
Subject Re: Discussion - Hostname Enrichment
Date Sat, 29 Oct 2016 19:19:58 GMT
Hello Tyler,

Can you send csv sample, extractor and enrichment/threatintel config and flatfile_loader command
that you using to load your enrichment/threatintel?

- Dima

On 10/29/2016 07:54 PM, Tyler Moore wrote:
Hey everyone,

I was having some trouble with creating a custom enrichment configuration for my bro sensor
and hopefully someone can clue me in on what i'm missing.

So basically I created a custom hostname enrichment config and an extractor config file that
I pushed to zookeeper that extract and data from a csv file i pushed into my enrichment table
in Hbase and maps this enrichment to the ip_src_addr and ip_dst_addr to see if they match.
If one of the fields matches the "ip" key from hbase the "host" value should be added via
the "hostname" enrichment mapping.

The problem is that the enrichment isn't being written. I attached screenshots of the bolts
and there stats after I pushed some data in via tcpreplay.
Also this is what my bro.json file looks like :

{
  "index" : "bro",
  "batchSize" : 5,
  "enrichment" : {
    "fieldMap" : {
      "geo" : [ "ip_dst_addr", "ip_src_addr" ],
      "host" : [ "ip_src_addr", "ip_dst_addr" ],
      "hbaseEnrichment" : [ "ip_src_addr", "ip_dst_addr" ]
    },
    "fieldToTypeMap" : {
      "ip_dst_addr" : [ "hostname" ],
      "ip_src_addr" : [ "hostname" ]
    },
    "config" : { }
  },
  "threatIntel" : {
    "fieldMap" : {
      "hbaseThreatIntel" : [ "ip_src_addr", "ip_dst_addr" ]
    },
    "fieldToTypeMap" : {
      "ip_src_addr" : [ "malicious_ip" ],
      "ip_dst_addr" : [ "malicious_ip" ]
    },
    "config" : { },
    "triageConfig" : {
      "riskLevelRules" : { },
      "aggregator" : "MAX",
      "aggregationConfig" : { }
    }
  },
  "configuration" : { }
}

The built in host enrichment works fine and is able to enrich via info in enrichment.host.known_hosts
but we will be adding a custom parser to this to stream host data.
If you have any ideas please let me know!

Regards,

Tyler Moore
Software Engineer
Flyball Labs


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message