activemq-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiram Chirino (JIRA)" <>
Subject [jira] [Commented] (APLO-241) Apollo becomes unresponsive under stress
Date Thu, 16 Aug 2012 03:41:37 GMT


Hiram Chirino commented on APLO-241:

Apollo was not optimally handling buffers in the case were clients were sending small messages
between small pauses.  Each message in apollo would hold on to about a 64k chunk of memory.
 In the case when there is no pause, multiple messages would share that 64k chunk of memory
so it was a non-issue. 

The responsiveness issue was due to the broker hitting an out of memory condition.

This should now be fixed in the following build:

I've also update the stomp-benchmark to more gracefully establish large numbers of connections
against a broker.  I recommend you pull the new stomp-benchmark source and do an 'sbt update'.
> Apollo becomes unresponsive under stress
> ----------------------------------------
>                 Key: APLO-241
>                 URL:
>             Project: ActiveMQ Apollo
>          Issue Type: Bug
>         Environment: apollo-99-trunk-20120813.171747-82
>            Reporter: Lionel Cons
>         Attachments: APLO-241.stack, APLO-241.xml
> When trying to reproduce APLO-238, I found another problem :-(
> I ran stomp-benchmark with the attached scenario to simulate one topic consumer with
many producers. As expected, stomp-benchmark reported many errors like:
> Connection timed out
> Connection reset by peer
> However, according to netstat, more than 10k connections have been established. stomp-benchmark
eventually stopped, with some results:
> c_c1 samples: [ 1450,140,261,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
> p_p1 samples: [ 75052,447310,430373,431496,406670,436637,455825,436305,451396,449173,408920,491221,527663,556973,605508,580931,616605,576667,589194,606230,567179,426264,256924,152997,82039,42061,23405,11416,5965,3874,2660,1834,1740,1298,1026,660,742,639,533,496,400,175,124,79,124,124,123,124,123,87,0,0,0,0,0,0,0,0,0,0
> e_p1 samples: [ 1832,76,0,0,0,3,664,1721,1,1,0,3,19,659,1704,0,5,8,10,670,910,785,4,10,8,26,654,1138,555,12,7,12,24,656,1674,13,11,8,17,20,655,1670,11,17,14,14,29,651,1664,15,16,16,16,669,901,772,19,16,15,25
> p_p2 samples: [ 68831,398643,391879,389710,365791,393488,406712,382222,398429,407385,370296,439194,465552,496801,507263,412671,328124,216992,123357,77492,43132,19195,10099,6064,4780,2969,1255,766,886,879,849,677,995,932,860,486,551,552,551,415,247,149,110,117,330,330,330,294,419,444,394,441,196,220,158,217,221,148,111,97
> e_p2 samples: [ 1704,82,0,0,0,1,767,1613,1,2,0,8,24,767,1601,0,1,1,12,786,782,814,7,3,8,28,770,984,604,7,10,8,24,773,1566,14,11,16,8,27,775,1557,17,8,15,10,32,775,1547,15,10,18,15,786,767,789,19,11,25,35
> However, after the end of the test, Apollo does not respond anymore. Its REST API cannot
be contacted (read timeout) and it cannot be stopped via the service script, only kill -9
works. Strangely, it's only using 100% of CPU (on multi-core) and 35% of memory.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message