jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: Design of Timeout test element / sampler interrupter
Date Tue, 01 Sep 2015 16:40:05 GMT
On 1 September 2015 at 17:23, Milamber <milamber@apache.org> wrote:
> Hello,
>
> I make some tests.

Thanks!

> * On basic scenario, with 2 http requests on the same url, Callable option
> seems most quick compare to Runnable (~70% more quick)

I'm surprised it makes that much difference.

What were you measuring?
How many threads?

> * Basic tests with the same url + Assertion Duration : the behavior is ok
>
> * Basic tests with the same url + a Constant Timer set to 2 sec (and
> Interrupt Timer set to 2 sec), the results seems not ok: the url is fully
> retrieve with a response time > 5 seconds (the request isn't interrupt, the
> response tab show the HTML code of the page). If I disable the Constant
> Timer, the request is interrupt in 2 sec. (the response tab show a (normal)
> exception causing by the interruption)
> Probably an issue...

No, this is expected because the InterruptTimer is called before the
delay is performed.

I have not yet implemented the use of sampleStarted/sampleStopped
which will allow the test element to be called after any other timers.

At present the interrupt test element has no way to allow for other
timer delays because it does not know what they will be.
These occur after it is called (and before the sample).
That is why sampleStarted/sampleStopped are needed.
Or some other change is needed to JMeterThread that will allow the
interrupt timer to schedule the interrupt at the correct time.

> * The fields Task type and Call delay isn't localized and the values aren't
> reset with your add a second Interrupt timer (I suppose you knows that,
> current branch is a PoC)

Yes, they are for testing purposes only and aren't needed in the final release.

> Milamber
>
> [the URL for test (deflate is disable in HTTP Header):
> https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Application_Platform/6.2/html-single/Administration_and_Configuration_Guide/index.html
> ]
>
>
>
>
> On 30/08/2015 17:37, sebb wrote:
>>
>> OK, the SampleTimeout branch now has a version that use Runnable by
>> default, but you can use Callable (type==CALL) with an optional delay
>> to simulate an interrupt that takes a long time.
>>
>> It works fine for me in simple tests.
>>
>> If you can break it, so much the better.
>>
>>
>> On 30 August 2015 at 16:36, sebb <sebbaz@gmail.com> wrote:
>>>
>>> On 30 August 2015 at 11:54, sebb <sebbaz@gmail.com> wrote:
>>>>
>>>> On 30 August 2015 at 07:12, Philippe Mouawad
>>>> <philippe.mouawad@gmail.com> wrote:
>>>>>
>>>>> On Sunday, August 30, 2015, sebb <sebbaz@gmail.com> wrote:
>>>>>
>>>>>> I've had a look at the classes that implement SampleListener, and
>>>>>> apart from ResultAction and TransactionSampler, only the Listeners
use
>>>>>> it. Since usage of these should be minimised in a production test,
>>>>>> it's likely that there won't be as many implementations as I had
>>>>>> feard.
>>>>>
>>>>> Implementation of SampleListener ?
>>>>> Usage of which should be minimised ?  SampleListener ?
>>>>
>>>> Usage of additional  Listeners should be minimised in a production test.
>>>>
>>>>>> Also if the implementation is empty, the overhead will be quite small.
>>>>>>
>>>>>> [There is a work-round if it does prove expensive: the SampleListener
>>>>>> interface could be split into two parent interfaces.]
>>>>>>
>>>>>> So assuming that JMeterThread implements sampleStarted/sampleStopped,
>>>>>> the Timeout element can use the Start to set up the timer and the
Stop
>>>>>> to cancel it. This will reduce the number outstanding as much as
>>>>>> possible.
>>>>>>
>>>>>> The timeouts have to be implemented using separate threads for two
>>>>>> reasons:
>>>>>> - it's obviously not possible to interrupt a sampler from the same
>>>>>> thread as the sampler
>>>>>> - depending on the sampler, and its state, the interrupt may take
a
>>>>>> while to complete, so each interrupt must be done in its own thread
>>>>>
>>>>> Are you sure, calling interrupt is usually just about setting a flag
>>>>> no?
>>>>
>>>> Ultimately yes, I guess a flag will be set.
>>>> However I'm not sure that it is always instantaneous, as there may be
>>>> locks to aquire.
>>>>
>>>>> Having 1 thread for each interruption, could lead to hundred of
>>>>> threads running for high throughput threads (500 res/s for example),
it
>>>>> won't scale.
>>>>
>>>> That assumes that all the samples in all the threads have timeouts
>>>> enabled.
>>>
>>> It also assumes that the threads are running for the life of the
>>> timeout, which I've just realised is not the case.
>>>>>
>>>>> Why can't we have 1 Thread (TimeoutChecker) called every N milliseconds
>>>>> that checks all registered JMeterThreads to check and call interrupt
if
>>>>> necessary ?
>>>>>
>>>> That's quite a bit of work to code, but if there is already such a
>>>> queing mechanism it would be worth trying.
>>>
>>> Actually the ScheduledExecutorService does just that.
>>> It uses one thread to handle the timeouts in sequence, and then starts
>>> a new Runnable when the timeout expires.
>>>
>>> Now that I have created the service as a singleton, there will only be
>>> one extra thread (the executor) most of the time.
>>> Since the timer tasks are cancelled if the sample completes in time,
>>> there will only ever be at most one extra thread for each overdue
>>> sample.
>>>
>>> I think it is a reasonable assumption to assume that the number of
>>> such samples will generally be small.
>>> At the very most it could only be one per thread.
>>> And remember that the new thread is only started when the timeout
>>> expires.
>>> It is extremely unlikely that they will all expire at once, and anyway
>>> (according to you) they don't take long to run so they won't build up.
>>>
>>> I've just noticed that there is a version of the Executor Service that
>>> uses Callable rather than Runnable (not sure why I missed it before).
>>> I asume this means the tasks are run in the same thread (I'll check
>>> this).
>>> If the interrupts really do happen quickly, this might be a better
>>> choice.
>>> Even if not, then it may not matter so long as any delays don't
>>> continue to build up.
>>>
>>>>>> It should be possible to use a single shared instance of the
>>>>>> ScheduledExecutorService; that could be lazily created using IODH.
[I
>>>>>> can try that with the current implementation]
>>>>
>>>> I've already tried it.
>>>>
>>>>> See my note above
>>>>>>
>>>>>> As to whether the Timeout class should be a Timer or some other type
>>>>>> of test element - that does not matter so long as it can be applied
to
>>>>>> the samplers individually or when in scope.
>>>>>>
>>>>>> I chose Timer because it was already called in the right place, but
I
>>>>>> assume JMeterThread can call any Test class provided that it
>>>>>> implemented the SampleInterface.
>>>>>>
>>>>>> It must be one of the existing Test Element classes that are handled
>>>>>> by the Menu system otherwise it will need special handling.
>>>>>>
>>>>>> The scope requirement rules out Config elements and Logic Controllers.
>>>>>
>>>>> It does not seem like a pre-processor to me, nor a post-procesor, nor
a
>>>>>>
>>>>>> Listener
>>>>>>
>>>>>> So AFAICT the only remaining options are the Timer and Assertions.
>>>>>>
>>>>>> I think both are justifiable.
>>>>>
>>>>> Why isn't it part of Sampler abstract class and as such a field in
>>>>> Sampler?
>>>>
>>>> How does the user indicate that a Timeout should be applied to a
>>>> particular Sampler?
>>>> It's a lot of work to add Timeout fields to every GUI.
>>>> Whereas being able to add a child test element to each applicable
>>>> sampler is already supported.
>>>> Further, such a test element can be applied to multiple samplers in
>>>> scope.
>>>> Much easier to enable and disable a single element that having to
>>>> update each Sampler.
>>>>
>>>> That's why it needs to be a separate test element.
>>>>
>>>>>
>>>>> For me none of Timer nor Assertions are conceptually valid.
>>>>> The behaviour is not a pause(so not Timer for me), it's not an
>>>>> Assertion
>>>>> neither as for me an Assertion only checks something.
>>>>> Although not fully satisfying it look to me more of a PreProcessor as
>>>>> it
>>>>> sets a timeout on the Sampler , it can also be considered as a Post
>>>>> Processor.
>>>>
>>>> OK, I could live with it being a Pre-Procesor.
>>>> That has the correct scoping rules.
>>>>
>>>>>
>>>>>> The name of the class can of course be changed from InterruptTimer
- I
>>>>>> think that is probably not the best choice. Maybe something like
>>>>>> SamplerTimeout?
>>>>>>
>>>>> Yes or SamplerTimeouter or SamplerInterrupter
>>>>>
>>>> Or even SampleTimeout - that's what it does, it applies a timeout to a
>>>> sample.
>>>>
>>>>> --
>>>>> Cordialement.
>>>>> Philippe Mouawad.
>
>

Mime
View raw message