metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ali Nazemian <alinazem...@gmail.com>
Subject Re: performance benchmarks on the asa parser
Date Fri, 09 Jun 2017 10:30:39 GMT
Simon,

I have read all emails and now I understand what you are saying. However, I
couldn't understand the effect of predictability of latency on enrichments.

On Fri, Jun 9, 2017 at 2:45 PM, Ali Nazemian <alinazemian@gmail.com> wrote:

> Hi Simon,
>
> We have noticed those issues as well. Can you share the changes you have
> made? so we can merge it with our version. We have implemented about 40-50
> more ciscotags so far. It would be great if we can optimize it and
> contribute back to the community. However, we may end up reimplement it
> using via Java Parser.
>
> Cheers,
> Ali
>
> On Fri, Jun 9, 2017 at 12:55 PM, Simon Elliston Ball <
> simon@simonellistonball.com> wrote:
>
>> I thought about compile of first use and cache as an approach, but
>> decided it would reduce the predictability of latency for a message, which
>> is important in the metron enrichment context. As you say, we could end up
>> growing a large number of Groks, but if the load of compile is all pushed
>> to the (hopefully very rare) topology restart event, it feels like the
>> performance trade off there is a good one, though the memory usage tradeoff
>> could start to bite if we’re getting into the hundreds I guess.
>>
>> Simon
>>
>>
>> > On 9 Jun 2017, at 03:32, Kyle Richardson <kylerichardson2@gmail.com>
>> wrote:
>> >
>> > I like the pre-compile idea. One concern is I see the number of grok
>> objects growing over time. This parser does not account for nearly all of
>> the possible ASA message types, currently only the most common ones. Is
>> there a middle ground implementation where we can compile on first use of a
>> grok and then hold in memory? Avoids the up front burden but should also
>> boost performance.
>> >
>> > -Kyle
>> >
>> >> On Jun 8, 2017, at 8:56 PM, Simon Elliston Ball <
>> simon@simonellistonball.com> wrote:
>> >>
>> >> The changes are pretty simple (pre-compile the grok, duh). Most other
>> grok parser just use a single expression, which is already pre-compiled
>> (/checks assumption in code) so really it’s just the ASA one because of
>> it’s strange two stage grok.
>> >>
>> >> Shame, it would have been nice to find some more low hanging fruit.
>> >>
>> >> Simon
>> >>
>> >>> On 9 Jun 2017, at 01:52, Otto Fowler <ottobackwards@gmail.com>
wrote:
>> >>>
>> >>> Are these changes that all grok parsers can benefit from?  Are your
>> changes to the base classes that they use or asa only?
>> >>>
>> >>>
>> >>>
>> >>>> On June 8, 2017 at 20:49:49, Simon Elliston Ball (
>> simon@simonellistonball.com <mailto:simon@simonellistonball.com>) wrote:
>> >>>>
>> >>>> I got mildly interested in parser performance as a result of some
>> recent work on tuning, and did some very quick benchmarking with Predfix on
>> the ASA parser (which I hadn’t really cared about enough due to relatively
>> low volume previously).
>> >>>>
>> >>>> That said, it’s not exactly perf optimised. 3 runs of 1000
>> iterations on my laptop as a micro-benchmark in Predfix (I know,
>> scientific, right), with some changes (basically pushing all the grok
>> statements up to pre-compile in init (the parser currently uses one grok to
>> do the syslog bit and figure out which grok it needs for the second half,
>> so this makes for a large number of Grok objects upfront, which I think we
>> can live with.
>> >>>>
>> >>>> Do you think we should do this benchmarking properly, and extend?
>> Anyone have thoughts about how to build parser benchmarks in to our test
>> suite properly?
>> >>>>
>> >>>> Also, since these are showing approx 20 times improvement on the
P95
>> interval, do we think it’s worth the memory (not measured, but 39 Grok
>> objects hanging around? If so I’ll get it JIRAed up and push my new version.
>> >>>>
>> >>>> Run results:-
>> >>>>
>> >>>> Base line (current master as is)
>> >>>> |= Benchmark ==============================
>> ====================================================|
>> >>>> | - | unit | sum | min | max | avg | stddev | conf95 | runs |
>> >>>> |========================================= TimeMeter
>> ==========================================|
>> >>>> |. AsaBenchmark ..............................
>> .................................................|
>> >>>> | parserBenchmark | ms | 5597.98 | 04.90 | 159.02 | 05.60 | 04.89
|
>> [05.01-06.20] | 1000.00 |
>> >>>> | parserBenchmark | ms | 5503.91 | 04.82 | 149.60 | 05.50 | 04.59
|
>> [05.00-05.90] | 1000.00 |
>> >>>> | parserBenchmark | ms | 5620.90 | 04.80 | 152.83 | 05.62 | 04.71
|
>> [04.98-06.73] | 1000.00 |
>> >>>> |===========================================================
>> ===================================|
>> >>>>
>> >>>> Syslog element of Grok pulled out and pre-compiled
>> >>>>
>> >>>> |= Benchmark ==============================
>> ====================================================|
>> >>>> | - | unit | sum | min | max | avg | stddev | conf95 | runs |
>> >>>> |========================================= TimeMeter
>> ==========================================|
>> >>>> |. AsaBenchmark ..............................
>> .................................................|
>> >>>> | parserBenchmark | ms | 4299.91 | 03.29 | 120.06 | 04.30 | 03.89
|
>> [03.36-07.10] | 1000.00 |
>> >>>> | parserBenchmark | ms | 4206.98 | 03.31 | 129.41 | 04.21 | 04.07
|
>> [03.46-05.44] | 1000.00 |
>> >>>> | parserBenchmark | ms | 3843.05 | 03.28 | 119.39 | 03.84 | 03.79
|
>> [03.33-04.55] | 1000.00 |
>> >>>> |===========================================================
>> ===================================|
>> >>>>
>> >>>> With all precompiled in a hash map (more memory use, but not by
a
>> lot)
>> >>>>
>> >>>> |= Benchmark ==============================
>> ===================================================|
>> >>>> | - | unit | sum | min | max | avg | stddev | conf95 | runs |
>> >>>> |========================================= TimeMeter
>> =========================================|
>> >>>> |. AsaBenchmark ..............................
>> ................................................|
>> >>>> | parserBenchmark | ms | 514.68 | 00.22 | 112.35 | 00.51 | 03.55
|
>> [00.24-00.79] | 1000.00 |
>> >>>> | parserBenchmark | ms | 472.42 | 00.22 | 105.19 | 00.47 | 03.32
|
>> [00.23-00.70] | 1000.00 |
>> >>>> | parserBenchmark | ms | 484.40 | 00.21 | 103.71 | 00.48 | 03.27
|
>> [00.24-00.76] | 1000.00 |
>> >>>> |===========================================================
>> ===================================|
>> >>>>
>> >>>> Simon
>> >>
>>
>>
>
>
> --
> A.Nazemian
>



-- 
A.Nazemian

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message