metron-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Allen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (METRON-25) Create Bro Plugin to Send Logs Directly to Kafka
Date Thu, 04 Feb 2016 20:43:39 GMT

     [ https://issues.apache.org/jira/browse/METRON-25?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nick Allen updated METRON-25:
-----------------------------
    Description: 
Create a Bro plugin that will consume the logs produced by Bro and send them directly to a
Kafka topic.  The types of logs to send should be configurable, so that only a subset of them
are published to Kafka.  For example, I may only want DNS::LOG and HTTP::LOG sent to Kafka.
 This should not interfere with the existing file based logging which is useful for diagnostics
and troubleshooting.

The alternative solution to creating this Bro plugin is to use some means of tailing the log
files that are generated by Bro. Each stream in Bro is logged to a separate file, so you'd
have to tail each of these files independently. Tailing log files like this is problematic.

  was:Create a Bro plugin that will consume the logs produced by Bro and send them directly
to a Kafka topic.  The types of logs to send should be configurable, so that only a subset
of them are published to Kafka.  For example, I may only want DNS::LOG and HTTP::LOG sent
to Kafka.  This should not interfere with the existing file based logging which is useful
for diagnostics and troubleshooting.


> Create Bro Plugin to Send Logs Directly to Kafka
> ------------------------------------------------
>
>                 Key: METRON-25
>                 URL: https://issues.apache.org/jira/browse/METRON-25
>             Project: Metron
>          Issue Type: Improvement
>            Reporter: Nick Allen
>            Priority: Critical
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> Create a Bro plugin that will consume the logs produced by Bro and send them directly
to a Kafka topic.  The types of logs to send should be configurable, so that only a subset
of them are published to Kafka.  For example, I may only want DNS::LOG and HTTP::LOG sent
to Kafka.  This should not interfere with the existing file based logging which is useful
for diagnostics and troubleshooting.
> The alternative solution to creating this Bro plugin is to use some means of tailing
the log files that are generated by Bro. Each stream in Bro is logged to a separate file,
so you'd have to tail each of these files independently. Tailing log files like this is problematic.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message