jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: svn commit: r1418666 - /jmeter/trunk/src/components/org/apache/jmeter/extractor/RegexExtractor.java
Date Mon, 10 Dec 2012 19:00:38 GMT
On 8 December 2012 22:10, Philippe Mouawad <philippe.mouawad@gmail.com> wrote:
> Hello sebb,
>
> I will put it in some utility class, which one would be the best in your
> opinion ?
>
> I didn't test yet with Java 6 but Java 6 class has a reset method although
> It has an text zone that contains matched test .
>
> But regarding the replacement you are talking about, it seems to me it's
> not that transparent as reading javadocs:
>
>    - http://docs.oracle.com/javase/6/docs/api/java/util/regex/Pattern.html
>    -
>    http://jakarta.apache.org/oro/api/org/apache/oro/text/regex/Perl5Matcher.html
>
> Java class does not seem to fully support PERL5 syntax no ? so it could
> break existing test plan.

AFAIK, ORO also does not support the full Perl5 syntax either.

So it would be a question of checking whether the ORO features [1] [2]
are supported by Java.

[1] http://jakarta.apache.org/oro/api/org/apache/oro/text/regex/package-summary.html
[2] http://www.savarese.org/oro/docs/OROMatcher/Syntax.html#Perl5Expressions

One obvious omission from Java 6 is embedded comments.
However, they aren't essential.

> So if we switch it seems to me we should have a configuration of regexp
> engine.

I suppose that might be necessary initially, but ideally we would drop
support for ORO (in a later release) if Java 6 is compatible.
If Java 6 is not fully compatible, then it depends on how likely the
features are to have been used, whether it's possible to detect that
it has been used (so we could potentially report a suitable error) and
whether there is a work-round

Java 6 has features not supported by ORO, and at least some of these
have been used by end-users.

>
> Anyway, I made a performance comparison on a test plan calling:
>
>    - 2 samples that generate pages of 280 and 200 Ko, gzipped respectively
>    to 1714 and 1475 bytes
>    - 200 threads
>    - 3 s pause between each sample
>    - 2 assertions
>    - 3 regexp
>
> Results are interesting:
>
>    - Memory and GC behaviours are greatly improved:
>       - much less CPU consumed by GC activity 3 to 15% with 2.8, 2 to 8%
>       with 2.9
>       - A Full GC drops memory to 100 Mb while with 2.8, it drops only to
>       200 Mb

Sorry, I don't follow - what is being compared with what?

> Plan is maybe a bit agressive regarding page sizes. 200 Kb.
>
> Maybe we should create some kind of Base Test Plan.
>
> I think there is another  potential improvement we could make which would
> be to cache in SampleResult#getResponseDataAsString the result of
> conversion.
>
> As if we have 3 regexp String will be decoded 3 times (CPU) and filled 3
> times (memory).

Not sure it uses extra memory - surely the decoded string will be
dropped (and made eligible for GC) once used?

>
> What do you think about it ?

OK, but remember that it will affect the total data stored.

>
> Regards
>
> Philippe
>
> On Sat, Dec 8, 2012 at 4:40 PM, sebb <sebbaz@gmail.com> wrote:
>
>> On 8 December 2012 13:03,  <pmouawad@apache.org> wrote:
>> > Author: pmouawad
>> > Date: Sat Dec  8 13:03:30 2012
>> > New Revision: 1418666
>> >
>> > URL: http://svn.apache.org/viewvc?rev=1418666&view=rev
>> > Log:
>> > Bug 54268 - Improve memory usage
>> >
>> > Bugzilla Id: 54268
>> >
>> > Modified:
>> >
>> jmeter/trunk/src/components/org/apache/jmeter/extractor/RegexExtractor.java
>> >
>> > Modified:
>> jmeter/trunk/src/components/org/apache/jmeter/extractor/RegexExtractor.java
>> > URL:
>> http://svn.apache.org/viewvc/jmeter/trunk/src/components/org/apache/jmeter/extractor/RegexExtractor.java?rev=1418666&r1=1418665&r2=1418666&view=diff
>> >
>> ==============================================================================
>> > ---
>> jmeter/trunk/src/components/org/apache/jmeter/extractor/RegexExtractor.java
>> (original)
>> > +++
>> jmeter/trunk/src/components/org/apache/jmeter/extractor/RegexExtractor.java
>> Sat Dec  8 13:03:30 2012
>> > @@ -110,11 +110,11 @@ public class RegexExtractor extends Abst
>> >          if (defaultValue.length() > 0){// Only replace default if it is
>> provided
>> >              vars.put(refName, defaultValue);
>> >          }
>> > -
>> > -
>> > +        Perl5Matcher matcher = JMeterUtils.getMatcher();
>> >          String regex = getRegex();
>> > +        Pattern pattern =
>> JMeterUtils.getPatternCache().getPattern(regex,
>> Perl5Compiler.READ_ONLY_MASK);
>> >          try {
>> > -            List<MatchResult> matches = processMatches(regex,
>> previousResult, matchNumber, vars);
>> > +            List<MatchResult> matches = processMatches(pattern, regex,
>> previousResult, matchNumber, vars);
>> >              int prevCount = 0;
>> >              String prevString = vars.get(refName + REF_MATCH_NR);
>> >              if (prevString != null) {
>> > @@ -162,6 +162,22 @@ public class RegexExtractor extends Abst
>> >              }
>> >          } catch (MalformedCachePatternException e) {
>> >              log.warn("Error in pattern: " + regex);
>> > +        } finally {
>> > +            clearMatcherMemory(matcher, pattern);
>> > +        }
>> > +    }
>> > +
>> > +    /**
>> > +     * Hack to make matcher clean the two internal buffers it keeps in
>> memory which size is equivalent to
>> > +     * the unzipped page size
>> > +     * @param matcher {@link Perl5Matcher}
>> > +     * @param pattern Pattern
>> > +     */
>> > +    private final void clearMatcherMemory(Perl5Matcher matcher, Pattern
>> pattern) {
>> > +        try {
>> > +            matcher.matches("", pattern); // $NON-NLS-1$
>> > +        } catch (Exception e) {
>> > +            // NOOP
>> >          }
>>
>> Perhaps needs to be a utility method?
>> May be useful from other classes.
>>
>> [However, if we end up dropping ORO it will become irrelevant.]
>>
>> Does the Java regex processor behave in the same way?
>> Or is it better at tidying up?
>>
>> >      }
>> >
>> > @@ -180,13 +196,12 @@ public class RegexExtractor extends Abst
>> >         return inputString;
>> >      }
>> >
>> > -    private List<MatchResult> processMatches(String regex, SampleResult
>> result, int matchNumber, JMeterVariables vars) {
>> > +    private List<MatchResult> processMatches(Pattern pattern, String
>> regex, SampleResult result, int matchNumber, JMeterVariables vars) {
>> >          if (log.isDebugEnabled()) {
>> >              log.debug("Regex = " + regex);
>> >          }
>> >
>> >          Perl5Matcher matcher = JMeterUtils.getMatcher();
>> > -        Pattern pattern =
>> JMeterUtils.getPatternCache().getPattern(regex,
>> Perl5Compiler.READ_ONLY_MASK);
>> >          List<MatchResult> matches = new ArrayList<MatchResult>();
>> >          int found = 0;
>> >
>> >
>> >
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.

Mime
View raw message