metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ali Nazemian <alinazem...@gmail.com>
Subject Re: performance benchmarks on the asa parser
Date Fri, 09 Jun 2017 04:45:31 GMT
Hi Simon,

We have noticed those issues as well. Can you share the changes you have
made? so we can merge it with our version. We have implemented about 40-50
more ciscotags so far. It would be great if we can optimize it and
contribute back to the community. However, we may end up reimplement it
using via Java Parser.

Cheers,
Ali

On Fri, Jun 9, 2017 at 12:55 PM, Simon Elliston Ball <
simon@simonellistonball.com> wrote:

> I thought about compile of first use and cache as an approach, but decided
> it would reduce the predictability of latency for a message, which is
> important in the metron enrichment context. As you say, we could end up
> growing a large number of Groks, but if the load of compile is all pushed
> to the (hopefully very rare) topology restart event, it feels like the
> performance trade off there is a good one, though the memory usage tradeoff
> could start to bite if we’re getting into the hundreds I guess.
>
> Simon
>
>
> > On 9 Jun 2017, at 03:32, Kyle Richardson <kylerichardson2@gmail.com>
> wrote:
> >
> > I like the pre-compile idea. One concern is I see the number of grok
> objects growing over time. This parser does not account for nearly all of
> the possible ASA message types, currently only the most common ones. Is
> there a middle ground implementation where we can compile on first use of a
> grok and then hold in memory? Avoids the up front burden but should also
> boost performance.
> >
> > -Kyle
> >
> >> On Jun 8, 2017, at 8:56 PM, Simon Elliston Ball <
> simon@simonellistonball.com> wrote:
> >>
> >> The changes are pretty simple (pre-compile the grok, duh). Most other
> grok parser just use a single expression, which is already pre-compiled
> (/checks assumption in code) so really it’s just the ASA one because of
> it’s strange two stage grok.
> >>
> >> Shame, it would have been nice to find some more low hanging fruit.
> >>
> >> Simon
> >>
> >>> On 9 Jun 2017, at 01:52, Otto Fowler <ottobackwards@gmail.com> wrote:
> >>>
> >>> Are these changes that all grok parsers can benefit from?  Are your
> changes to the base classes that they use or asa only?
> >>>
> >>>
> >>>
> >>>> On June 8, 2017 at 20:49:49, Simon Elliston Ball (
> simon@simonellistonball.com <mailto:simon@simonellistonball.com>) wrote:
> >>>>
> >>>> I got mildly interested in parser performance as a result of some
> recent work on tuning, and did some very quick benchmarking with Predfix on
> the ASA parser (which I hadn’t really cared about enough due to relatively
> low volume previously).
> >>>>
> >>>> That said, it’s not exactly perf optimised. 3 runs of 1000 iterations
> on my laptop as a micro-benchmark in Predfix (I know, scientific, right),
> with some changes (basically pushing all the grok statements up to
> pre-compile in init (the parser currently uses one grok to do the syslog
> bit and figure out which grok it needs for the second half, so this makes
> for a large number of Grok objects upfront, which I think we can live with.
> >>>>
> >>>> Do you think we should do this benchmarking properly, and extend?
> Anyone have thoughts about how to build parser benchmarks in to our test
> suite properly?
> >>>>
> >>>> Also, since these are showing approx 20 times improvement on the P95
> interval, do we think it’s worth the memory (not measured, but 39 Grok
> objects hanging around? If so I’ll get it JIRAed up and push my new version.
> >>>>
> >>>> Run results:-
> >>>>
> >>>> Base line (current master as is)
> >>>> |= Benchmark ==============================
> ====================================================|
> >>>> | - | unit | sum | min | max | avg | stddev | conf95 | runs |
> >>>> |========================================= TimeMeter
> ==========================================|
> >>>> |. AsaBenchmark ..............................
> .................................................|
> >>>> | parserBenchmark | ms | 5597.98 | 04.90 | 159.02 | 05.60 | 04.89 |
> [05.01-06.20] | 1000.00 |
> >>>> | parserBenchmark | ms | 5503.91 | 04.82 | 149.60 | 05.50 | 04.59 |
> [05.00-05.90] | 1000.00 |
> >>>> | parserBenchmark | ms | 5620.90 | 04.80 | 152.83 | 05.62 | 04.71 |
> [04.98-06.73] | 1000.00 |
> >>>> |===========================================================
> ===================================|
> >>>>
> >>>> Syslog element of Grok pulled out and pre-compiled
> >>>>
> >>>> |= Benchmark ==============================
> ====================================================|
> >>>> | - | unit | sum | min | max | avg | stddev | conf95 | runs |
> >>>> |========================================= TimeMeter
> ==========================================|
> >>>> |. AsaBenchmark ..............................
> .................................................|
> >>>> | parserBenchmark | ms | 4299.91 | 03.29 | 120.06 | 04.30 | 03.89 |
> [03.36-07.10] | 1000.00 |
> >>>> | parserBenchmark | ms | 4206.98 | 03.31 | 129.41 | 04.21 | 04.07 |
> [03.46-05.44] | 1000.00 |
> >>>> | parserBenchmark | ms | 3843.05 | 03.28 | 119.39 | 03.84 | 03.79 |
> [03.33-04.55] | 1000.00 |
> >>>> |===========================================================
> ===================================|
> >>>>
> >>>> With all precompiled in a hash map (more memory use, but not by a lot)
> >>>>
> >>>> |= Benchmark ==============================
> ===================================================|
> >>>> | - | unit | sum | min | max | avg | stddev | conf95 | runs |
> >>>> |========================================= TimeMeter
> =========================================|
> >>>> |. AsaBenchmark ..............................
> ................................................|
> >>>> | parserBenchmark | ms | 514.68 | 00.22 | 112.35 | 00.51 | 03.55 |
> [00.24-00.79] | 1000.00 |
> >>>> | parserBenchmark | ms | 472.42 | 00.22 | 105.19 | 00.47 | 03.32 |
> [00.23-00.70] | 1000.00 |
> >>>> | parserBenchmark | ms | 484.40 | 00.21 | 103.71 | 00.48 | 03.27 |
> [00.24-00.76] | 1000.00 |
> >>>> |===========================================================
> ===================================|
> >>>>
> >>>> Simon
> >>
>
>


-- 
A.Nazemian

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message