jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milamber <milam...@apache.org>
Subject Re: Design of Timeout test element / sampler interrupter
Date Tue, 01 Sep 2015 22:37:47 GMT

I've tested with the updated branch, and with the new version of 
JMeterThread, the behaviors of InterruptTimer & ConstantTimer have 
better predictable (in my mind). All http requests stop by the Interrupt 
Timer after 2sec.



On 01/09/2015 18:45, sebb wrote:
> OK, the branch now has a version of JMeterThread that supports
> sampleStarted/Stopped.
>
> Disable it by setting the property
>
> temp.sampleStarted=true
>
>
> If enabled, it allows interrupts to work in conjunction with timers.
>
>
> On 1 September 2015 at 17:40, sebb <sebbaz@gmail.com> wrote:
>> On 1 September 2015 at 17:23, Milamber <milamber@apache.org> wrote:
>>> Hello,
>>>
>>> I make some tests.
>> Thanks!
>>
>>> * On basic scenario, with 2 http requests on the same url, Callable option
>>> seems most quick compare to Runnable (~70% more quick)
>> I'm surprised it makes that much difference.
>>
>> What were you measuring?
>> How many threads?
>>
>>> * Basic tests with the same url + Assertion Duration : the behavior is ok
>>>
>>> * Basic tests with the same url + a Constant Timer set to 2 sec (and
>>> Interrupt Timer set to 2 sec), the results seems not ok: the url is fully
>>> retrieve with a response time > 5 seconds (the request isn't interrupt, the
>>> response tab show the HTML code of the page). If I disable the Constant
>>> Timer, the request is interrupt in 2 sec. (the response tab show a (normal)
>>> exception causing by the interruption)
>>> Probably an issue...
>> No, this is expected because the InterruptTimer is called before the
>> delay is performed.
>>
>> I have not yet implemented the use of sampleStarted/sampleStopped
>> which will allow the test element to be called after any other timers.
>>
>> At present the interrupt test element has no way to allow for other
>> timer delays because it does not know what they will be.
>> These occur after it is called (and before the sample).
>> That is why sampleStarted/sampleStopped are needed.
>> Or some other change is needed to JMeterThread that will allow the
>> interrupt timer to schedule the interrupt at the correct time.
>>
>>> * The fields Task type and Call delay isn't localized and the values aren't
>>> reset with your add a second Interrupt timer (I suppose you knows that,
>>> current branch is a PoC)
>> Yes, they are for testing purposes only and aren't needed in the final release.
>>
>>> Milamber
>>>
>>> [the URL for test (deflate is disable in HTTP Header):
>>> https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Application_Platform/6.2/html-single/Administration_and_Configuration_Guide/index.html
>>> ]
>>>
>>>
>>>
>>>
>>> On 30/08/2015 17:37, sebb wrote:
>>>> OK, the SampleTimeout branch now has a version that use Runnable by
>>>> default, but you can use Callable (type==CALL) with an optional delay
>>>> to simulate an interrupt that takes a long time.
>>>>
>>>> It works fine for me in simple tests.
>>>>
>>>> If you can break it, so much the better.
>>>>
>>>>
>>>> On 30 August 2015 at 16:36, sebb <sebbaz@gmail.com> wrote:
>>>>> On 30 August 2015 at 11:54, sebb <sebbaz@gmail.com> wrote:
>>>>>> On 30 August 2015 at 07:12, Philippe Mouawad
>>>>>> <philippe.mouawad@gmail.com> wrote:
>>>>>>> On Sunday, August 30, 2015, sebb <sebbaz@gmail.com> wrote:
>>>>>>>
>>>>>>>> I've had a look at the classes that implement SampleListener,
and
>>>>>>>> apart from ResultAction and TransactionSampler, only the
Listeners use
>>>>>>>> it. Since usage of these should be minimised in a production
test,
>>>>>>>> it's likely that there won't be as many implementations as
I had
>>>>>>>> feard.
>>>>>>> Implementation of SampleListener ?
>>>>>>> Usage of which should be minimised ?  SampleListener ?
>>>>>> Usage of additional  Listeners should be minimised in a production
test.
>>>>>>
>>>>>>>> Also if the implementation is empty, the overhead will be
quite small.
>>>>>>>>
>>>>>>>> [There is a work-round if it does prove expensive: the SampleListener
>>>>>>>> interface could be split into two parent interfaces.]
>>>>>>>>
>>>>>>>> So assuming that JMeterThread implements sampleStarted/sampleStopped,
>>>>>>>> the Timeout element can use the Start to set up the timer
and the Stop
>>>>>>>> to cancel it. This will reduce the number outstanding as
much as
>>>>>>>> possible.
>>>>>>>>
>>>>>>>> The timeouts have to be implemented using separate threads
for two
>>>>>>>> reasons:
>>>>>>>> - it's obviously not possible to interrupt a sampler from
the same
>>>>>>>> thread as the sampler
>>>>>>>> - depending on the sampler, and its state, the interrupt
may take a
>>>>>>>> while to complete, so each interrupt must be done in its
own thread
>>>>>>> Are you sure, calling interrupt is usually just about setting
a flag
>>>>>>> no?
>>>>>> Ultimately yes, I guess a flag will be set.
>>>>>> However I'm not sure that it is always instantaneous, as there may
be
>>>>>> locks to aquire.
>>>>>>
>>>>>>> Having 1 thread for each interruption, could lead to hundred
of
>>>>>>> threads running for high throughput threads (500 res/s for example),
it
>>>>>>> won't scale.
>>>>>> That assumes that all the samples in all the threads have timeouts
>>>>>> enabled.
>>>>> It also assumes that the threads are running for the life of the
>>>>> timeout, which I've just realised is not the case.
>>>>>>> Why can't we have 1 Thread (TimeoutChecker) called every N milliseconds
>>>>>>> that checks all registered JMeterThreads to check and call interrupt
if
>>>>>>> necessary ?
>>>>>>>
>>>>>> That's quite a bit of work to code, but if there is already such
a
>>>>>> queing mechanism it would be worth trying.
>>>>> Actually the ScheduledExecutorService does just that.
>>>>> It uses one thread to handle the timeouts in sequence, and then starts
>>>>> a new Runnable when the timeout expires.
>>>>>
>>>>> Now that I have created the service as a singleton, there will only be
>>>>> one extra thread (the executor) most of the time.
>>>>> Since the timer tasks are cancelled if the sample completes in time,
>>>>> there will only ever be at most one extra thread for each overdue
>>>>> sample.
>>>>>
>>>>> I think it is a reasonable assumption to assume that the number of
>>>>> such samples will generally be small.
>>>>> At the very most it could only be one per thread.
>>>>> And remember that the new thread is only started when the timeout
>>>>> expires.
>>>>> It is extremely unlikely that they will all expire at once, and anyway
>>>>> (according to you) they don't take long to run so they won't build up.
>>>>>
>>>>> I've just noticed that there is a version of the Executor Service that
>>>>> uses Callable rather than Runnable (not sure why I missed it before).
>>>>> I asume this means the tasks are run in the same thread (I'll check
>>>>> this).
>>>>> If the interrupts really do happen quickly, this might be a better
>>>>> choice.
>>>>> Even if not, then it may not matter so long as any delays don't
>>>>> continue to build up.
>>>>>
>>>>>>>> It should be possible to use a single shared instance of
the
>>>>>>>> ScheduledExecutorService; that could be lazily created using
IODH. [I
>>>>>>>> can try that with the current implementation]
>>>>>> I've already tried it.
>>>>>>
>>>>>>> See my note above
>>>>>>>> As to whether the Timeout class should be a Timer or some
other type
>>>>>>>> of test element - that does not matter so long as it can
be applied to
>>>>>>>> the samplers individually or when in scope.
>>>>>>>>
>>>>>>>> I chose Timer because it was already called in the right
place, but I
>>>>>>>> assume JMeterThread can call any Test class provided that
it
>>>>>>>> implemented the SampleInterface.
>>>>>>>>
>>>>>>>> It must be one of the existing Test Element classes that
are handled
>>>>>>>> by the Menu system otherwise it will need special handling.
>>>>>>>>
>>>>>>>> The scope requirement rules out Config elements and Logic
Controllers.
>>>>>>> It does not seem like a pre-processor to me, nor a post-procesor,
nor a
>>>>>>>> Listener
>>>>>>>>
>>>>>>>> So AFAICT the only remaining options are the Timer and Assertions.
>>>>>>>>
>>>>>>>> I think both are justifiable.
>>>>>>> Why isn't it part of Sampler abstract class and as such a field
in
>>>>>>> Sampler?
>>>>>> How does the user indicate that a Timeout should be applied to a
>>>>>> particular Sampler?
>>>>>> It's a lot of work to add Timeout fields to every GUI.
>>>>>> Whereas being able to add a child test element to each applicable
>>>>>> sampler is already supported.
>>>>>> Further, such a test element can be applied to multiple samplers
in
>>>>>> scope.
>>>>>> Much easier to enable and disable a single element that having to
>>>>>> update each Sampler.
>>>>>>
>>>>>> That's why it needs to be a separate test element.
>>>>>>
>>>>>>> For me none of Timer nor Assertions are conceptually valid.
>>>>>>> The behaviour is not a pause(so not Timer for me), it's not an
>>>>>>> Assertion
>>>>>>> neither as for me an Assertion only checks something.
>>>>>>> Although not fully satisfying it look to me more of a PreProcessor
as
>>>>>>> it
>>>>>>> sets a timeout on the Sampler , it can also be considered as
a Post
>>>>>>> Processor.
>>>>>> OK, I could live with it being a Pre-Procesor.
>>>>>> That has the correct scoping rules.
>>>>>>
>>>>>>>> The name of the class can of course be changed from InterruptTimer
- I
>>>>>>>> think that is probably not the best choice. Maybe something
like
>>>>>>>> SamplerTimeout?
>>>>>>>>
>>>>>>> Yes or SamplerTimeouter or SamplerInterrupter
>>>>>>>
>>>>>> Or even SampleTimeout - that's what it does, it applies a timeout
to a
>>>>>> sample.
>>>>>>
>>>>>>> --
>>>>>>> Cordialement.
>>>>>>> Philippe Mouawad.
>>>


Mime
View raw message