jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: Bug 56119 - dealing with idle connection timeout and dropped connection
Date Wed, 26 Feb 2014 01:59:27 GMT
On 23 February 2014 04:38, sebb <sebbaz@gmail.com> wrote:
> On 19 February 2014 21:51, sebb <sebbaz@gmail.com> wrote:
>> On 19 February 2014 21:03, Philippe Mouawad <philippe.mouawad@gmail.com> wrote:
>>> On Wed, Feb 19, 2014 at 9:58 PM, sebb <sebbaz@gmail.com> wrote:
>>>
>>>> On 19 February 2014 18:49, sebb <sebbaz@gmail.com> wrote:
>>>> > It looks as though the problem reported in Bug 56119 was due to the
>>>> > server dropping connections that have been idle too long.
>>>> >
>>>> > There may also be servers that only allow a connection to be reused
a
>>>> > certain number of times (this does not seem to have been the case
>>>> > here).
>>>> >
>>>> > This email is to discuss what JMeter could perhaps do to make it
>>>> > easier to test such servers.
>>>> >
>>>> > The two existing work-rounds are:
>>>> > - disable Keep-Alive
>>>> > - enable staleCheck
>>>> >
>>>> > Neither is ideal; the first is awkward to use, and staleCheck can
>>>> > generate unnecessary additional traffic (which is why it was disabled
>>>> > in 2.11).
>>>> >
>>>> > I can think of two possible approaches:
>>>> >
>>>> > 1) proactively shut connections. This would be easy for servers that
>>>> > limit reuse.
>>>> > Just count reuses and turn off keep-alive when a specified limit is
>>>> reached.
>>>> > Not so easy for idle timeouts; one cannot retroactively disable
>>>> keep-alive.
>>>> >
>>>> > 2) Deal with the disconnects when they occur.
>>>> > The code needs to distinguish which errors are retriable, and may need
>>>> > to distinguish at what point the failure occurs. For example, even a
>>>> > POST ought to be retriable if JMeter is unable to send any data on the
>>>> > connection.
>>>> >
>>>> > Also need to consider how one might report retries.
>>>> > I think the tester needs to be able to find out if additional requests
>>>> > have been made by JMeter.
>>>>
>>>> Further testing against the ASF servers shows that HC 4.2.x does
>>>> handle idle timeouts without needing to use the staleCheck option.
>>>> This relies on the server sending a header of the form:
>>>>
>>>> Keep-Alive: timeout=5, max=100
>>>>
>>>> In this case, the connection is automatically recreated if necessary
>>>> when the next sampler runs.
>>>> If the server fails to send the header, then the connection may be
>>>> dropped unexpectedly (which is what was happening with Bug 56119).
>>>> So another approach might be to allow an optional keep-alive timeout
>>>> in case the server does not provide one.
>>>>
>>>> Or we could take the view that there is nothing to fix in JMeter.
>>>> The Keep-Alive header is there for a reason, it tells the client when
>>>> it is safe to reuse the connectiion.
>>>> If the server fails to send it, then it is broken, and so the failed
>>>> samples are to be expected.
>>>>
>>>
>>> I think we need to make something at least for servers like Amazon S3 which
>>> close connections after number of uses.
>>> Did you check to see if this kind of server send a keep alive header ?
>>
>> I just tested again with jmeter.a.o.
>> It returns headers of the form:
>>
>> Keep-Alive: timeout=5, max=100
>> Connection: Keep-Alive
>> ...
>> Keep-Alive: timeout=5, max=99
>> Connection: Keep-Alive
>> ...
>> etc
>> ...
>> Keep-Alive: timeout=5, max=1
>> Connection: Keep-Alive
>> ...
>> Connection: close
>>
>> So the HC connection manager does not need to keep track of the
>> remaining re-use count; the server disconnects at the end of the last
>> request.
>> Nice and simple.
>>
>> I assume S3 does the same as jmeter.a.o if it is well-behaved.
>
> I asked on the HC dev list, and found out that the Keep-Alive header
> is optional.
> So servers that don't send it are not misconfigured - though of course
> it helps the client if they send the header.
>
> Also, the HC default retry processing does handle the disconnect case,
> i.e. where the server disconnects whilst starting to send the next
> request.
> Setting retry count to 1 allows the request to be retried.
>
> JMeter previously used to enable retries, but we found this had
> adverse effects, as it could generate extra requests.
> It turns out that the default retry handler always retries failed
> idempotent requests (e.g. GET), even if they were successfully sent.
> This is perhaps what caused the additional traffic.
>
> So I think we need an amended version of the retry handler which only
> retries failed sends, if the retry count is 0.
> We can use the HC version for retry counts > 0.

Unfortunately this does not work, as the disconnect is only detected
once the request has been sent.

However, it is possible to specify an idle timeout to deal with a
missing Keep-Alive header.
I've tried that and it works, but we need some decisions on how to implement it.
I'll send a separate e-mail for that.

It may also be possible to code a conditional stale connection check
which is not applied every single time.
e.g. it could be done after N requests or when the connection has been
idle for T seconds.
I'll look into this a bit more and report back in another mail.

>>> Anyway on my side I think what has been changed in 2.11 should not be
>>> reverted, because for servers correctly configured you don't get these
>>> errors, I made 3 campaigns on different servers  with 2.11 and never got
>>> this kind of issues.
>>
>> Agreed, no need to revert.
>>
>>> But maybe we should document it better somewhere.
>>
>> Yes, the error and likely cause should be documented.
>> Probably easiest to start as a Wiki page.
>>

Mime
View raw message