jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: Build failed in Jenkins: JMeter Ubuntu #929
Date Mon, 04 Nov 2019 22:12:59 GMT
On Mon, 4 Nov 2019 at 21:15, Vladimir Sitnikov
<sitnikov.vladimir@gmail.com> wrote:
>
> PS. I guess the problem with the test is
> that org.apache.jmeter.protocol.http.control.CacheManager#extractExpiresDateFromCacheControl
> uses System.currentTimeMillis() for max-age.
>
> >So is there a way to show the line number in the test log?
>
> This looks like a Gradle bug.
> By default it truncates the stacktrace to the first frame with "test
> class", and by default it does show the line.
>
> TestCacheManagerUrlConnection is different: it does not contain test
> methods,
> and the class is not present in the stacktrace. The test methods are
> inherited, and it seems to confuse output formatter.

So is there a way to fix this so the line number *is* shown?

> >- the test itself uses the same text for various different errors
>
> In practice, "test failures should look like a good bug report".
> The mere use of "unique messages" does not make the messages good or
> actionable or understandable.

Yes, but that is a separate issue entirely.

My point was that the message could have come from two different
places, making it impossible to debug without further info.

> For instance, the test in question is using the following:
>
> assertNull(getThreadCacheEntry(LOCAL_HOST), "Should not find entry");
> (1) assertFalse(this.cacheManager.inCache(url), "Should not find valid entry");
>
> ...
>
> cacheResult(sampleResultOK);
>
> ...
>
> assertNotNull(getThreadCacheEntry(LOCAL_HOST), "Should find entry");
> (2) assertTrue(this.cacheManager.inCache(url), "Should find valid entry");
>
> ...
>
> assertNotNull(getThreadCacheEntry(LOCAL_HOST), "Should find entry");
> (3) assertFalse(this.cacheManager.inCache(url), "Should not find valid entry");
>
>
> I would suggest:
> (1) "start of the test => url should not be in cacheManager"
> (2) "just cached the url => it should be in cacheManager"
> (3) "entry expires at " + ...  + ", current date is " + ...  + " => url
> should not be in cacheManager"

That would work of course as the messages are now unique.

> Messages for getThreadCacheEntry asserts could be adjusted as well.
> Note: I did not try to make the messages unique. I just put relevant
> explanations that clarify **why** the entry should be present/missing.

Yes, but that is missing my point.

If the same message can appear in two places in the same test method,
then it is impossible to determine what has failed without some other
context.
Unique messages are one way to achieve that if the test harness fails
to provide that information, as in this case.
Sometimes there may be other clues (e.g. intervening messages), but
that should not be necessary with a well-behaved test harness.

In this case the error message is not even correct:

org.apache.jmeter.protocol.http.control.TestCacheManagerUrlConnection
> testPrivateCache() FAILED
    org.opentest4j.AssertionFailedError: Should not find valid entry
==> expected: <false> but was: <true>

The method testPrivateCache is not actually in the class
TestCacheManagerUrlConnection.
As it happens there seems to be only one parent class that contains
the method, but it would be quite possible for the method to appear in
multiple ancestors.

As it stands, the test log output is rather unhelpful in this case.
Such an error is likely to occur again elsewhere, and it would be
useful not to have to waste time digging around in other files in the
workspace.

So I ask again: is there a way to ensure that the stack trace is shown?

> Vladimir

Mime
View raw message