jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Sitnikov <sitnikov.vladi...@gmail.com>
Subject On-demand thread creation
Date Fri, 24 Apr 2015 07:18:55 GMT
Hi,


I want JMeter to create threads only when it is required for the next iteration.

Currently JMeter has only two options:

O1) pre-create all threads at startup — not an option really due to
OOM and due to the fact it cannot provision new threads on failures.
O2) create threads with given delays (ramp up with fixed delays). Does
not work for me (see below). Ramp up may lag behind the need of
threads (for instance, if threads fail faster than created in
ramp-up). Ramp up may create excessive threads if threads existing
operate fast enough to drive the load.

I want something like the following:

S1) A certain timer/test plan element is marked as «start of test». It
must have a timer that shapes the load.
S2) Thread group creates threads only in case there is not enough
threads waiting _at_ «start of test» element. In other words, TG would
stop creating new threads if certain number of threads become waiting
on the shaping timer. «The number of prepared» threads should probably
be configurable.
S3) «Ramp up» configuration,  would become irrelevant.
S4) «Number of threads» configuration would become «maximum number of
active threads» and «target number of prepared threads at start of
test element» settings. For instance, TG would automatically spawn
threads if they die.
S5) The proposed solution works for both looping and non-looping scenarios

I understand, it might be too intimate cooperation of TG and test plan
elements, however it is the simplest approach I see to explain what I
want.

Do you think this can be integrated into JMeter?

I think it is good to decouple thread group and shaping timer,
otherwise the number of combinations would explode.

I cannot use existing solutions due to:

P1) JMeter OOB does not have «on demand thread creation»
P2) UltimateThreadGroup (from jmeter plugins) cannot be used to shape
the load since the number of required threads is not known in advance.
UTG cannot create realistic workloads (poisson workloads)
P3) Current timers (both in-core and out-of core) cannot prevent
creating of new threads

The closest workaround I see is calling "stop thread" from within a timer.
For instance, a timer might stop thread instead of issuing a delay in
case the delay would be "too big".
The idea is "if thread should be delayed much, then TG would anyway
create a new thread soon, so we kill current one and hope a new one
would arrive later".
It does not look nice since TG would have to spawn threads.

Background.


My test plan has «shaping timer» in the beginning to make sure the
overall load is kept constant.
So, if too many threads are created, they just wait.


For instance, if I want to test 60 orders/hour and and it takes ~30
seconds to create an order, I specify: 120 threads, ramp up=15seconds
and enable «delayed thread creation».
This saves a bit of memory since JMeter does not create all the threads upfront.

However, particular test executions might stall for certain reasons,
so I have to be conservative on the «maximum» number of threads.

I do not want to test «each order at 0, 60, 120, 180 sec» pattern, so
my shaping timer issues fuzzy delays (~poisson process in fact).
For instance, a «60 orders/hour» pattern could be «0, 15, 150, 180».

So far, so good.

Unfortunately, ramp up approach fails for durability testing and high
throughput tests.

In durability testing, the number of threads is not known in advance:
threads might fail, so I have to setup something like 100’000 threads,
ramp up 7 days, then at the end of the test this fails with OOM.

In high-throughput tests (>1/sec), «per thread ramp up delay» have to
be small, otherwise I would lack enough threads to drive the desired
load.
On the other hand, too little ramp up value could result in too many
waiting threads, thus memory issues.

-- 
Regards,
Vladimir Sitnikov

Mime
View raw message