metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otto Fowler <ottobackwa...@gmail.com>
Subject Re: Develop some parsers
Date Fri, 07 Apr 2017 14:01:11 GMT
You’ll be able to use the archetype, even if you are doing configuration
only.
So - imagine this -

You want to do a grok ‘config’ only parser.

You :
* create a new parser extension with the archetype
* remove all the main/ java code
* setup your configuration in main/config ( for enrichments, indexing, and
parser )
* you add your grok patterns
* you can write unit and integration tests against your grok parser rules
etc
* you package it all up
* you deploy as a regular extension

You *could* do just a configuration, but I think being able to write unit
tests etc is better ;)




On April 7, 2017 at 09:35:20, Ali Nazemian (alinazemian@gmail.com) wrote:

Having a separate maven archetype is really great.

I prefer to use the Java one because performance is really important for
us...

On Fri, Apr 7, 2017 at 11:13 PM, Otto Fowler <ottobackwards@gmail.com>
wrote:

> I also believe that grok parsers can be added through configuration only,
> without having to
> compile a parser.
>
> You can add a parser configuration targeting the basic grok parser and
just
> provide the grok
> parser rules.
>
>
> Just as a heads up, I’m currently working on the parsers to allow for
> writing and maintaining parsers
> outside the metron code tree, including providing a maven archetype. This
> will allow you to create parsers
> without having to maintain a fork etc.
>
> Keep an eye out for METRON-258 as a PR on the list.
>
>
>
> On April 7, 2017 at 08:54:35, Justin Leet (justinjleet@gmail.com) wrote:
>
> My understanding of Grok vs Java is to provide a tradeoff for ease of
> implementation vs performance (plus Java can also handle parsing that
would
> be too complicated for Grok.
>
> Grok is less performant and handles less complex parsing, but it's easy
to
> get things going and potentially maintained without writing and compiling
> Java.
>
> The Java implementation will be better for performance and can handle
more
> complicated parsing Grok can't.
>
> I believe the preference has generally been for Grok parsers if
> appropriate, otherwise Java parsers.
>
> Justin
>
> On Fri, Apr 7, 2017 at 8:09 AM, Ali Nazemian <alinazemian@gmail.com>
> wrote:
>
> > Hi Mark,
> >
> > Yeah, that would be great. Can you please specify which devices you
have
> > developed so far?
> >
> > Cheers,
> > Ali
> >
> > On Fri, Apr 7, 2017 at 4:10 PM, Mark De Rijk <me.derijk@gmail.com>
> wrote:
> >
> > > Dear all,
> > >
> > > I am a heavy arcsight user and I have written quite a few parsers
over
> > > time.
> > > I am new to contributing to open source projects however.
> > > @Ali, would you like to cooperate on development of some parsers?
> > >
> > > Kind Regards,
> > > Mark de Rijk
> > >
> > >
> > > > On 7 Apr 2017, at 04:30, Ali Nazemian <alinazemian@gmail.com>
wrote:
> > > >
> > > > Hi all,
> > > >
> > > > We are going to develop some parsers and have some contribution to
> the
> > > > community as a start point. I was wondering what the reason is
behind
> > > > choosing Grok statements for some of the implementations and Java
> regex
> > > for
> > > > other ones? Is there any policy for that? Probably it would be
better
> > to
> > > > have the Java regex implementation due to performance concerns.
> > However,
> > > I
> > > > am sure there is a reason that some of them have been implemented
> with
> > > > using Grok statements.
> > > >
> > > > Regards,
> > > > Ali
> > >
> >
> >
> >
> > --
> > A.Nazemian
> >
>



-- 
A.Nazemian

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message