jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milamber <milam...@apache.org>
Subject Re: Design of Timeout test element / sampler interrupter
Date Tue, 01 Sep 2015 16:23:58 GMT
Hello,

I make some tests.

* On basic scenario, with 2 http requests on the same url, Callable 
option seems most quick compare to Runnable (~70% more quick)

* Basic tests with the same url + Assertion Duration : the behavior is ok

* Basic tests with the same url + a Constant Timer set to 2 sec (and 
Interrupt Timer set to 2 sec), the results seems not ok: the url is 
fully retrieve with a response time > 5 seconds (the request isn't 
interrupt, the response tab show the HTML code of the page). If I 
disable the Constant Timer, the request is interrupt in 2 sec. (the 
response tab show a (normal) exception causing by the interruption)
Probably an issue...

* The fields Task type and Call delay isn't localized and the values 
aren't reset with your add a second Interrupt timer (I suppose you knows 
that, current branch is a PoC)

Milamber

[the URL for test (deflate is disable in HTTP Header):
https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Application_Platform/6.2/html-single/Administration_and_Configuration_Guide/index.html

]



On 30/08/2015 17:37, sebb wrote:
> OK, the SampleTimeout branch now has a version that use Runnable by
> default, but you can use Callable (type==CALL) with an optional delay
> to simulate an interrupt that takes a long time.
>
> It works fine for me in simple tests.
>
> If you can break it, so much the better.
>
>
> On 30 August 2015 at 16:36, sebb <sebbaz@gmail.com> wrote:
>> On 30 August 2015 at 11:54, sebb <sebbaz@gmail.com> wrote:
>>> On 30 August 2015 at 07:12, Philippe Mouawad <philippe.mouawad@gmail.com>
wrote:
>>>> On Sunday, August 30, 2015, sebb <sebbaz@gmail.com> wrote:
>>>>
>>>>> I've had a look at the classes that implement SampleListener, and
>>>>> apart from ResultAction and TransactionSampler, only the Listeners use
>>>>> it. Since usage of these should be minimised in a production test,
>>>>> it's likely that there won't be as many implementations as I had
>>>>> feard.
>>>> Implementation of SampleListener ?
>>>> Usage of which should be minimised ?  SampleListener ?
>>> Usage of additional  Listeners should be minimised in a production test.
>>>
>>>>> Also if the implementation is empty, the overhead will be quite small.
>>>>>
>>>>> [There is a work-round if it does prove expensive: the SampleListener
>>>>> interface could be split into two parent interfaces.]
>>>>>
>>>>> So assuming that JMeterThread implements sampleStarted/sampleStopped,
>>>>> the Timeout element can use the Start to set up the timer and the Stop
>>>>> to cancel it. This will reduce the number outstanding as much as
>>>>> possible.
>>>>>
>>>>> The timeouts have to be implemented using separate threads for two reasons:
>>>>> - it's obviously not possible to interrupt a sampler from the same
>>>>> thread as the sampler
>>>>> - depending on the sampler, and its state, the interrupt may take a
>>>>> while to complete, so each interrupt must be done in its own thread
>>>> Are you sure, calling interrupt is usually just about setting a flag no?
>>> Ultimately yes, I guess a flag will be set.
>>> However I'm not sure that it is always instantaneous, as there may be
>>> locks to aquire.
>>>
>>>> Having 1 thread for each interruption, could lead to hundred of
>>>> threads running for high throughput threads (500 res/s for example), it
>>>> won't scale.
>>> That assumes that all the samples in all the threads have timeouts enabled.
>> It also assumes that the threads are running for the life of the
>> timeout, which I've just realised is not the case.
>>>> Why can't we have 1 Thread (TimeoutChecker) called every N milliseconds
>>>> that checks all registered JMeterThreads to check and call interrupt if
>>>> necessary ?
>>>>
>>> That's quite a bit of work to code, but if there is already such a
>>> queing mechanism it would be worth trying.
>> Actually the ScheduledExecutorService does just that.
>> It uses one thread to handle the timeouts in sequence, and then starts
>> a new Runnable when the timeout expires.
>>
>> Now that I have created the service as a singleton, there will only be
>> one extra thread (the executor) most of the time.
>> Since the timer tasks are cancelled if the sample completes in time,
>> there will only ever be at most one extra thread for each overdue
>> sample.
>>
>> I think it is a reasonable assumption to assume that the number of
>> such samples will generally be small.
>> At the very most it could only be one per thread.
>> And remember that the new thread is only started when the timeout expires.
>> It is extremely unlikely that they will all expire at once, and anyway
>> (according to you) they don't take long to run so they won't build up.
>>
>> I've just noticed that there is a version of the Executor Service that
>> uses Callable rather than Runnable (not sure why I missed it before).
>> I asume this means the tasks are run in the same thread (I'll check this).
>> If the interrupts really do happen quickly, this might be a better choice.
>> Even if not, then it may not matter so long as any delays don't
>> continue to build up.
>>
>>>>> It should be possible to use a single shared instance of the
>>>>> ScheduledExecutorService; that could be lazily created using IODH. [I
>>>>> can try that with the current implementation]
>>> I've already tried it.
>>>
>>>> See my note above
>>>>> As to whether the Timeout class should be a Timer or some other type
>>>>> of test element - that does not matter so long as it can be applied to
>>>>> the samplers individually or when in scope.
>>>>>
>>>>> I chose Timer because it was already called in the right place, but I
>>>>> assume JMeterThread can call any Test class provided that it
>>>>> implemented the SampleInterface.
>>>>>
>>>>> It must be one of the existing Test Element classes that are handled
>>>>> by the Menu system otherwise it will need special handling.
>>>>>
>>>>> The scope requirement rules out Config elements and Logic Controllers.
>>>> It does not seem like a pre-processor to me, nor a post-procesor, nor a
>>>>> Listener
>>>>>
>>>>> So AFAICT the only remaining options are the Timer and Assertions.
>>>>>
>>>>> I think both are justifiable.
>>>> Why isn't it part of Sampler abstract class and as such a field in Sampler?
>>> How does the user indicate that a Timeout should be applied to a
>>> particular Sampler?
>>> It's a lot of work to add Timeout fields to every GUI.
>>> Whereas being able to add a child test element to each applicable
>>> sampler is already supported.
>>> Further, such a test element can be applied to multiple samplers in scope.
>>> Much easier to enable and disable a single element that having to
>>> update each Sampler.
>>>
>>> That's why it needs to be a separate test element.
>>>
>>>>
>>>> For me none of Timer nor Assertions are conceptually valid.
>>>> The behaviour is not a pause(so not Timer for me), it's not an Assertion
>>>> neither as for me an Assertion only checks something.
>>>> Although not fully satisfying it look to me more of a PreProcessor as it
>>>> sets a timeout on the Sampler , it can also be considered as a Post
>>>> Processor.
>>> OK, I could live with it being a Pre-Procesor.
>>> That has the correct scoping rules.
>>>
>>>>
>>>>> The name of the class can of course be changed from InterruptTimer -
I
>>>>> think that is probably not the best choice. Maybe something like
>>>>> SamplerTimeout?
>>>>>
>>>> Yes or SamplerTimeouter or SamplerInterrupter
>>>>
>>> Or even SampleTimeout - that's what it does, it applies a timeout to a sample.
>>>
>>>> --
>>>> Cordialement.
>>>> Philippe Mouawad.


Mime
View raw message