metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zeolla@GMail.com" <zeo...@gmail.com>
Subject Re: Name conventions for parsers
Date Thu, 06 Oct 2016 12:32:27 GMT
If we don't do it by device I would be concerned that some more
appliance-based systems wouldn't allow the flexibility to split things up
to different destinations, nor would they allow external additions (NiFi,
etc.).  This where I am right now, where I can send from certain appliances
into my syslog infrastructure, then either force my syslog architecture to
selectively send onto Metron, or parse and then send into a generic JSON
parser (I will probably go the latter route).  In order to standardize and
simplify, I would suggest continuing down the device-based route.

Generally, I expect the community to grow and for parsers to just exist,
and some users to only do minor updates to them or throw together grok
parsers using GROK_PREDICT() where necessary.  In fact I would hope that is
the case, as it would indicate a broader user base.

Jon

On Thu, Oct 6, 2016 at 8:02 AM Simon Elliston Ball <
simon@simonellistonball.com> wrote:

> > On 6 Oct 2016, at 12:22, Yohann Lepage <yohann@lepage.info> wrote:
> >
> > 2016-10-06 12:21 GMT+02:00 Zeolla@GMail.com <zeolla@gmail.com>:
> >> I would think that instead we work to make each parser able to handle
> all
> >> the known outputs (and document explicitly what outputs per parser are
> >> supported) from a product and go back to vendor_product, with versions
> of
> >> the product supported/tested and version of the parser being stored in
> code
> >> and documentation only.
> > +1
> >
>
> +1 - this is similar to the evolving schema problem, and probably belongs
> in code.
>
> >> I'm currently working on mechanisms to get logs into Metron most
> >> efficiently because all of my syslog comes in one big pipe.
> > I have a similar use case. Most of the time, admins are ok to forward
> > logs from rsyslog/syslog-ng to the SIEM as they don't want to install
> > an agent  ( *.* @@siem.intra:514;).
> >
> > The result is that you receive a mix of log
> > (sudo/apache/mysql/audit/etc) from the same device and the SIEM have
> > to deals with it.
> >
> > So, it would be really useful that Metron could handle a syslog flow
> > and automatically apply the right parser for each log. In order to
> > help Metron, a config could be provide by the "Security Platform
> > Engineer" to preselect a list of parser per device (as you know what
> > type of logs a device  should send).  This feature exists in
> > commercial SIEM.
> >
>
> +1 for this too. One question though, do you think it’s viable to do this
> by device. I would expect multiple types of syslog coming from the same
> physical device, especially when dealing with things like server logs.
>
> This could be handled with minimal parse and routing in NiFi potentially,
> but that may make setup more complex than the sort of mapping you’re
> talking about here. Thoughts?
>
> Simon

-- 

Jon

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message