metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Elliston Ball <si...@simonellistonball.com>
Subject performance benchmarks on the asa parser
Date Fri, 09 Jun 2017 00:49:36 GMT
I got mildly interested in parser performance as a result of some recent work on tuning, and
did some very quick benchmarking with Predfix on the ASA parser (which I hadn’t really cared
about enough due to relatively low volume previously).

That said, it’s not exactly perf optimised. 3 runs of 1000 iterations on my laptop as a
micro-benchmark in Predfix (I know, scientific, right), with some changes (basically pushing
all the grok statements up to pre-compile in init (the parser currently uses one grok to do
the syslog bit and figure out which grok it needs for the second half, so this makes for a
large number of Grok objects upfront, which I think we can live with. 

Do you think we should do this benchmarking properly, and extend? Anyone have thoughts about
how to build parser benchmarks in to our test suite properly?

Also, since these are showing approx 20 times improvement on the P95 interval, do we think
it’s worth the memory (not measured, but 39 Grok objects hanging around? If so I’ll get
it JIRAed up and push my new version.

Run results:-

Base line (current master as is)
|= Benchmark ==================================================================================|
| -               | unit | sum     | min   | max    | avg   | stddev | conf95        | runs
   |
|========================================= TimeMeter ==========================================|
|. AsaBenchmark ...............................................................................|
| parserBenchmark | ms   | 5597.98 | 04.90 | 159.02 | 05.60 | 04.89  | [05.01-06.20] | 1000.00
|
| parserBenchmark | ms   | 5503.91 | 04.82 | 149.60 | 05.50 | 04.59  | [05.00-05.90] | 1000.00
|
| parserBenchmark | ms   | 5620.90 | 04.80 | 152.83 | 05.62 | 04.71  | [04.98-06.73] | 1000.00
|
|==============================================================================================|

Syslog element of Grok pulled out and pre-compiled

|= Benchmark ==================================================================================|
| -               | unit | sum     | min   | max    | avg   | stddev | conf95        | runs
   |
|========================================= TimeMeter ==========================================|
|. AsaBenchmark ...............................................................................|
| parserBenchmark | ms   | 4299.91 | 03.29 | 120.06 | 04.30 | 03.89  | [03.36-07.10] | 1000.00
|
| parserBenchmark | ms   | 4206.98 | 03.31 | 129.41 | 04.21 | 04.07  | [03.46-05.44] | 1000.00
|
| parserBenchmark | ms   | 3843.05 | 03.28 | 119.39 | 03.84 | 03.79  | [03.33-04.55] | 1000.00
|
|==============================================================================================|

With all precompiled in a hash map (more memory use, but not by a lot)

|= Benchmark =================================================================================|
| -               | unit | sum    | min   | max    | avg   | stddev | conf95        | runs
   |
|========================================= TimeMeter =========================================|
|. AsaBenchmark ..............................................................................|
| parserBenchmark | ms   | 514.68 | 00.22 | 112.35 | 00.51 | 03.55  | [00.24-00.79] | 1000.00
|
| parserBenchmark | ms   | 472.42 | 00.22 | 105.19 | 00.47 | 03.32  | [00.23-00.70] | 1000.00
|
| parserBenchmark | ms   | 484.40 | 00.21 | 103.71 | 00.48 | 03.27  | [00.24-00.76] | 1000.00
|
|==============================================================================================|

Simon
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message