metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tyler Moore <tmo...@goflyball.com>
Subject Re: Discussion - Hostname Enrichment
Date Sun, 30 Oct 2016 23:47:48 GMT
Dima,

Here are the CSV, extractor and enrichment configs that I uploaded via
flatfile_loader command.

Regards,

Tyler

Regards,

Tyler Moore
Software Engineer
Flyball Labs

On Sat, Oct 29, 2016 at 3:19 PM, Dima Kovalyov <Dima.Kovalyov@sstech.us>
wrote:

> Hello Tyler,
>
> Can you send csv sample, extractor and enrichment/threatintel config and
> flatfile_loader command that you using to load your enrichment/threatintel?
>
> - Dima
>
> On 10/29/2016 07:54 PM, Tyler Moore wrote:
> Hey everyone,
>
> I was having some trouble with creating a custom enrichment configuration
> for my bro sensor and hopefully someone can clue me in on what i'm missing.
>
> So basically I created a custom hostname enrichment config and an
> extractor config file that I pushed to zookeeper that extract and data from
> a csv file i pushed into my enrichment table in Hbase and maps this
> enrichment to the ip_src_addr and ip_dst_addr to see if they match. If one
> of the fields matches the "ip" key from hbase the "host" value should be
> added via the "hostname" enrichment mapping.
>
> The problem is that the enrichment isn't being written. I attached
> screenshots of the bolts and there stats after I pushed some data in via
> tcpreplay.
> Also this is what my bro.json file looks like :
>
> {
>   "index" : "bro",
>   "batchSize" : 5,
>   "enrichment" : {
>     "fieldMap" : {
>       "geo" : [ "ip_dst_addr", "ip_src_addr" ],
>       "host" : [ "ip_src_addr", "ip_dst_addr" ],
>       "hbaseEnrichment" : [ "ip_src_addr", "ip_dst_addr" ]
>     },
>     "fieldToTypeMap" : {
>       "ip_dst_addr" : [ "hostname" ],
>       "ip_src_addr" : [ "hostname" ]
>     },
>     "config" : { }
>   },
>   "threatIntel" : {
>     "fieldMap" : {
>       "hbaseThreatIntel" : [ "ip_src_addr", "ip_dst_addr" ]
>     },
>     "fieldToTypeMap" : {
>       "ip_src_addr" : [ "malicious_ip" ],
>       "ip_dst_addr" : [ "malicious_ip" ]
>     },
>     "config" : { },
>     "triageConfig" : {
>       "riskLevelRules" : { },
>       "aggregator" : "MAX",
>       "aggregationConfig" : { }
>     }
>   },
>   "configuration" : { }
> }
>
> The built in host enrichment works fine and is able to enrich via info in
> enrichment.host.known_hosts but we will be adding a custom parser to this
> to stream host data.
> If you have any ideas please let me know!
>
> Regards,
>
> Tyler Moore
> Software Engineer
> Flyball Labs
>
>

Mime
View raw message