jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milamber <milam...@apache.org>
Subject Re: Design of Timeout test element / sampler interrupter
Date Tue, 01 Sep 2015 22:48:59 GMT

>> * On basic scenario, with 2 http requests on the same url, Callable option
>> seems most quick compare to Runnable (~70% more quick)
> I'm surprised it makes that much difference.
>
> What were you measuring?

The URL below (in redhat documentation website)

> How many threads?

One VU with 10 loops

The latest version of the branch seems have some differences:


2015/09/01 23:41:14 WARN  - jmeter.timers.InterruptTimer: Run Done 
interrupting HTTPSamplerProxy @469583928 'HTTP Request 1'  took 
3.15911E-4 secs
2015/09/01 23:41:16 WARN  - jmeter.timers.InterruptTimer: Call Done 
interrupting HTTPSamplerProxy @508309015 'HTTP Request 2'  took 
7.38335E-4 secs
2015/09/01 23:41:18 WARN  - jmeter.timers.InterruptTimer: Run Done 
interrupting HTTPSamplerProxy @469583928 'HTTP Request 1'  took 
7.26091E-4 secs
2015/09/01 23:41:20 WARN  - jmeter.timers.InterruptTimer: Call Done 
interrupting HTTPSamplerProxy @508309015 'HTTP Request 2'  took 
7.4593E-4 secs
2015/09/01 23:41:22 WARN  - jmeter.timers.InterruptTimer: Run Done 
interrupting HTTPSamplerProxy @469583928 'HTTP Request 1'  took 
0.00181855 secs
2015/09/01 23:41:24 WARN  - jmeter.timers.InterruptTimer: Call Done 
interrupting HTTPSamplerProxy @508309015 'HTTP Request 2'  took 
5.33232E-4 secs
2015/09/01 23:41:26 WARN  - jmeter.timers.InterruptTimer: Run Done 
interrupting HTTPSamplerProxy @469583928 'HTTP Request 1'  took 
4.05846E-4 secs
2015/09/01 23:41:28 WARN  - jmeter.timers.InterruptTimer: Call Done 
interrupting HTTPSamplerProxy @508309015 'HTTP Request 2'  took 
5.50154E-4 secs
2015/09/01 23:41:30 WARN  - jmeter.timers.InterruptTimer: Run Done 
interrupting HTTPSamplerProxy @469583928 'HTTP Request 1'  took 
6.93174E-4 secs
2015/09/01 23:41:32 WARN  - jmeter.timers.InterruptTimer: Call Done 
interrupting HTTPSamplerProxy @508309015 'HTTP Request 2'  took 
6.11455E-4 secs
2015/09/01 23:41:34 WARN  - jmeter.timers.InterruptTimer: Run Done 
interrupting HTTPSamplerProxy @469583928 'HTTP Request 1'  took 
6.1518E-4 secs
2015/09/01 23:41:36 WARN  - jmeter.timers.InterruptTimer: Call Done 
interrupting HTTPSamplerProxy @508309015 'HTTP Request 2'  took 
6.24097E-4 secs
2015/09/01 23:41:38 WARN  - jmeter.timers.InterruptTimer: Run Done 
interrupting HTTPSamplerProxy @469583928 'HTTP Request 1'  took 
7.53767E-4 secs
2015/09/01 23:41:40 WARN  - jmeter.timers.InterruptTimer: Call Done 
interrupting HTTPSamplerProxy @508309015 'HTTP Request 2'  took 
6.84471E-4 secs
2015/09/01 23:41:42 WARN  - jmeter.timers.InterruptTimer: Run Done 
interrupting HTTPSamplerProxy @469583928 'HTTP Request 1'  took 
7.34266E-4 secs
2015/09/01 23:41:44 WARN  - jmeter.timers.InterruptTimer: Call Done 
interrupting HTTPSamplerProxy @508309015 'HTTP Request 2'  took 
5.88123E-4 secs
2015/09/01 23:41:46 WARN  - jmeter.timers.InterruptTimer: Run Done 
interrupting HTTPSamplerProxy @469583928 'HTTP Request 1'  took 
3.34427E-4 secs
2015/09/01 23:41:48 WARN  - jmeter.timers.InterruptTimer: Call Done 
interrupting HTTPSamplerProxy @508309015 'HTTP Request 2'  took 
6.3848E-4 secs
2015/09/01 23:41:50 WARN  - jmeter.timers.InterruptTimer: Run Done 
interrupting HTTPSamplerProxy @469583928 'HTTP Request 1'  took 
7.77866E-4 secs
2015/09/01 23:41:52 WARN  - jmeter.timers.InterruptTimer: Call Done 
interrupting HTTPSamplerProxy @508309015 'HTTP Request 2'  took 
7.13642E-4 secs

The record times is very close (in nano seconds), perhaps, not important 
(significant).


>
>> * Basic tests with the same url + Assertion Duration : the behavior is ok
>>
>> * Basic tests with the same url + a Constant Timer set to 2 sec (and
>> Interrupt Timer set to 2 sec), the results seems not ok: the url is fully
>> retrieve with a response time > 5 seconds (the request isn't interrupt, the
>> response tab show the HTML code of the page). If I disable the Constant
>> Timer, the request is interrupt in 2 sec. (the response tab show a (normal)
>> exception causing by the interruption)
>> Probably an issue...
> No, this is expected because the InterruptTimer is called before the
> delay is performed.
>
> I have not yet implemented the use of sampleStarted/sampleStopped
> which will allow the test element to be called after any other timers.
>
> At present the interrupt test element has no way to allow for other
> timer delays because it does not know what they will be.
> These occur after it is called (and before the sample).
> That is why sampleStarted/sampleStopped are needed.
> Or some other change is needed to JMeterThread that will allow the
> interrupt timer to schedule the interrupt at the correct time.
>
>> * The fields Task type and Call delay isn't localized and the values aren't
>> reset with your add a second Interrupt timer (I suppose you knows that,
>> current branch is a PoC)
> Yes, they are for testing purposes only and aren't needed in the final release.
>
>> Milamber
>>
>> [the URL for test (deflate is disable in HTTP Header):
>> https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Application_Platform/6.2/html-single/Administration_and_Configuration_Guide/index.html
>> ]
>>
>>
>>
>>
>> On 30/08/2015 17:37, sebb wrote:
>>> OK, the SampleTimeout branch now has a version that use Runnable by
>>> default, but you can use Callable (type==CALL) with an optional delay
>>> to simulate an interrupt that takes a long time.
>>>
>>> It works fine for me in simple tests.
>>>
>>> If you can break it, so much the better.
>>>
>>>
>>> On 30 August 2015 at 16:36, sebb <sebbaz@gmail.com> wrote:
>>>> On 30 August 2015 at 11:54, sebb <sebbaz@gmail.com> wrote:
>>>>> On 30 August 2015 at 07:12, Philippe Mouawad
>>>>> <philippe.mouawad@gmail.com> wrote:
>>>>>> On Sunday, August 30, 2015, sebb <sebbaz@gmail.com> wrote:
>>>>>>
>>>>>>> I've had a look at the classes that implement SampleListener,
and
>>>>>>> apart from ResultAction and TransactionSampler, only the Listeners
use
>>>>>>> it. Since usage of these should be minimised in a production
test,
>>>>>>> it's likely that there won't be as many implementations as I
had
>>>>>>> feard.
>>>>>> Implementation of SampleListener ?
>>>>>> Usage of which should be minimised ?  SampleListener ?
>>>>> Usage of additional  Listeners should be minimised in a production test.
>>>>>
>>>>>>> Also if the implementation is empty, the overhead will be quite
small.
>>>>>>>
>>>>>>> [There is a work-round if it does prove expensive: the SampleListener
>>>>>>> interface could be split into two parent interfaces.]
>>>>>>>
>>>>>>> So assuming that JMeterThread implements sampleStarted/sampleStopped,
>>>>>>> the Timeout element can use the Start to set up the timer and
the Stop
>>>>>>> to cancel it. This will reduce the number outstanding as much
as
>>>>>>> possible.
>>>>>>>
>>>>>>> The timeouts have to be implemented using separate threads for
two
>>>>>>> reasons:
>>>>>>> - it's obviously not possible to interrupt a sampler from the
same
>>>>>>> thread as the sampler
>>>>>>> - depending on the sampler, and its state, the interrupt may
take a
>>>>>>> while to complete, so each interrupt must be done in its own
thread
>>>>>> Are you sure, calling interrupt is usually just about setting a flag
>>>>>> no?
>>>>> Ultimately yes, I guess a flag will be set.
>>>>> However I'm not sure that it is always instantaneous, as there may be
>>>>> locks to aquire.
>>>>>
>>>>>> Having 1 thread for each interruption, could lead to hundred of
>>>>>> threads running for high throughput threads (500 res/s for example),
it
>>>>>> won't scale.
>>>>> That assumes that all the samples in all the threads have timeouts
>>>>> enabled.
>>>> It also assumes that the threads are running for the life of the
>>>> timeout, which I've just realised is not the case.
>>>>>> Why can't we have 1 Thread (TimeoutChecker) called every N milliseconds
>>>>>> that checks all registered JMeterThreads to check and call interrupt
if
>>>>>> necessary ?
>>>>>>
>>>>> That's quite a bit of work to code, but if there is already such a
>>>>> queing mechanism it would be worth trying.
>>>> Actually the ScheduledExecutorService does just that.
>>>> It uses one thread to handle the timeouts in sequence, and then starts
>>>> a new Runnable when the timeout expires.
>>>>
>>>> Now that I have created the service as a singleton, there will only be
>>>> one extra thread (the executor) most of the time.
>>>> Since the timer tasks are cancelled if the sample completes in time,
>>>> there will only ever be at most one extra thread for each overdue
>>>> sample.
>>>>
>>>> I think it is a reasonable assumption to assume that the number of
>>>> such samples will generally be small.
>>>> At the very most it could only be one per thread.
>>>> And remember that the new thread is only started when the timeout
>>>> expires.
>>>> It is extremely unlikely that they will all expire at once, and anyway
>>>> (according to you) they don't take long to run so they won't build up.
>>>>
>>>> I've just noticed that there is a version of the Executor Service that
>>>> uses Callable rather than Runnable (not sure why I missed it before).
>>>> I asume this means the tasks are run in the same thread (I'll check
>>>> this).
>>>> If the interrupts really do happen quickly, this might be a better
>>>> choice.
>>>> Even if not, then it may not matter so long as any delays don't
>>>> continue to build up.
>>>>
>>>>>>> It should be possible to use a single shared instance of the
>>>>>>> ScheduledExecutorService; that could be lazily created using
IODH. [I
>>>>>>> can try that with the current implementation]
>>>>> I've already tried it.
>>>>>
>>>>>> See my note above
>>>>>>> As to whether the Timeout class should be a Timer or some other
type
>>>>>>> of test element - that does not matter so long as it can be applied
to
>>>>>>> the samplers individually or when in scope.
>>>>>>>
>>>>>>> I chose Timer because it was already called in the right place,
but I
>>>>>>> assume JMeterThread can call any Test class provided that it
>>>>>>> implemented the SampleInterface.
>>>>>>>
>>>>>>> It must be one of the existing Test Element classes that are
handled
>>>>>>> by the Menu system otherwise it will need special handling.
>>>>>>>
>>>>>>> The scope requirement rules out Config elements and Logic Controllers.
>>>>>> It does not seem like a pre-processor to me, nor a post-procesor,
nor a
>>>>>>> Listener
>>>>>>>
>>>>>>> So AFAICT the only remaining options are the Timer and Assertions.
>>>>>>>
>>>>>>> I think both are justifiable.
>>>>>> Why isn't it part of Sampler abstract class and as such a field in
>>>>>> Sampler?
>>>>> How does the user indicate that a Timeout should be applied to a
>>>>> particular Sampler?
>>>>> It's a lot of work to add Timeout fields to every GUI.
>>>>> Whereas being able to add a child test element to each applicable
>>>>> sampler is already supported.
>>>>> Further, such a test element can be applied to multiple samplers in
>>>>> scope.
>>>>> Much easier to enable and disable a single element that having to
>>>>> update each Sampler.
>>>>>
>>>>> That's why it needs to be a separate test element.
>>>>>
>>>>>> For me none of Timer nor Assertions are conceptually valid.
>>>>>> The behaviour is not a pause(so not Timer for me), it's not an
>>>>>> Assertion
>>>>>> neither as for me an Assertion only checks something.
>>>>>> Although not fully satisfying it look to me more of a PreProcessor
as
>>>>>> it
>>>>>> sets a timeout on the Sampler , it can also be considered as a Post
>>>>>> Processor.
>>>>> OK, I could live with it being a Pre-Procesor.
>>>>> That has the correct scoping rules.
>>>>>
>>>>>>> The name of the class can of course be changed from InterruptTimer
- I
>>>>>>> think that is probably not the best choice. Maybe something like
>>>>>>> SamplerTimeout?
>>>>>>>
>>>>>> Yes or SamplerTimeouter or SamplerInterrupter
>>>>>>
>>>>> Or even SampleTimeout - that's what it does, it applies a timeout to
a
>>>>> sample.
>>>>>
>>>>>> --
>>>>>> Cordialement.
>>>>>> Philippe Mouawad.
>>


Mime
View raw message