qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Conway" <acon...@redhat.com>
Subject Re: Review Request: WIP: Improve multi-thread performance by reducing the duration the Queue message lock is held.
Date Thu, 08 Mar 2012 14:52:43 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4232/#review5723
-----------------------------------------------------------


Looks good overall, some questions & comments in line.


/trunk/qpid/cpp/src/qpid/broker/Queue.h
<https://reviews.apache.org/r/4232/#comment12458>

    Need to make it clear what state each lock covers. My preference would be to actually
make 3 structs, each with a lock and the state that lock covers. At least we should indicate
by comments.



/trunk/qpid/cpp/src/qpid/broker/Queue.h
<https://reviews.apache.org/r/4232/#comment12457>

    I prefer the ScopedLock& convention over the LH convention but I guess that's cosmetic.



/trunk/qpid/cpp/src/qpid/broker/Queue.cpp
<https://reviews.apache.org/r/4232/#comment12464>

    why did this change from consumerLock?



/trunk/qpid/cpp/src/qpid/broker/Queue.cpp
<https://reviews.apache.org/r/4232/#comment12465>

    Does the snapshot have to be atomic with setting deleted = true?


- Alan


On 2012-03-07 19:23:36, Kenneth Giusti wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4232/
> -----------------------------------------------------------
> 
> (Updated 2012-03-07 19:23:36)
> 
> 
> Review request for qpid, Andrew Stitcher, Alan Conway, Gordon Sim, and Ted Ross.
> 
> 
> Summary
> -------
> 
> In order to improve our parallelism across threads, this patch reduces the scope over
which the Queue's message lock is held.
> 
> I've still got more testing to do, but the simple test cases below show pretty good performance
improvements on my multi-core box.  I'd like to get some early feedback, as there are a lot
of little changes to the queue locking that will need vetting.
> 
> As far as performance is concerned - I've run the following tests on an 8 core Xeon X5550
2.67Ghz box.  I used qpid-send and qpid-receive to generate traffic.  All traffic generated
in parallel.
> All numbers are in messages/second - sorted by best overall throughput.
> 
> Test 1: Funnel Test (shared queue, 2 producers, 1 consumer)
> 
> test setup and execute info:
> qpidd --auth no -p 8888 --no-data-dir --worker-threads=3
> qpid-config -b $SRC_HOST add queue srcQ1 --max-queue-size=1200000000 --max-queue-count=4000000
--flow-stop-size=0 --flow-stop-count=0
> qpid-receive -b $SRC_HOST -a srcQ1 -f -m 4000000 --capacity 2000 --ack-frequency 1000
--print-content no --report-total &
> qpid-send -b $SRC_HOST -a srcQ1 -m 2000000 --content-size 300 --capacity 2000 --report-total
--sequence no --timestamp no &
> qpid-send -b $SRC_HOST -a srcQ1 -m 2000000 --content-size 300 --capacity 2000 --report-total
--sequence no --timestamp no &
> 
> trunk:                        patched:
> tp(m/s)                       tp(m/s)
> 
> S1    :: S2    :: R1          S1    :: S2    :: R1
> 73144 :: 72524 :: 112914      91120 :: 83278 :: 133730
> 70299 :: 68878 :: 110905      91079 :: 87418 :: 133649
> 70002 :: 69882 :: 110241      83957 :: 83600 :: 131617
> 70523 :: 68681 :: 108372      83911 :: 82845 :: 128513
> 68135 :: 68022 :: 107949      85999 :: 81863 :: 125575
> 
> 
> Test 2: Quad Client Shared Test (1 queue, 2 senders x 2 Receivers)
> 
> test setup and execute info:
> qpidd --auth no -p 8888 --no-data-dir --worker-threads=4
> qpid-config -b $SRC_HOST add queue srcQ1 --max-queue-size=1200000000 --max-queue-count=4000000
--flow-stop-size=0 --flow-stop-count=0
> qpid-receive -b $SRC_HOST -a srcQ1 -f -m 2000000 --capacity 2000 --ack-frequency 1000
--print-content no --report-total &
> qpid-receive -b $SRC_HOST -a srcQ1 -f -m 2000000 --capacity 2000 --ack-frequency 1000
--print-content no --report-total &
> qpid-send -b $SRC_HOST -a srcQ1 -m 2000000 --content-size 300 --capacity 2000 --report-total
--sequence no --timestamp no &
> qpid-send -b $SRC_HOST -a srcQ1 -m 2000000 --content-size 300 --capacity 2000 --report-total
--sequence no --timestamp no &
> 
> trunk:                               patched:
> tp(m/s)                              tp(m/s)
> 
> S1    :: S2    :: R1    :: R2        S1    :: S2    :: R1    :: R2
> 52826 :: 52364 :: 52386 :: 52275     64826 :: 64780 :: 64811 :: 64638
> 51457 :: 51269 :: 51399 :: 51213     64630 :: 64157 :: 64562 :: 64157
> 50557 :: 50432 :: 50468 :: 50366     64023 :: 63832 :: 64092 :: 63748
> 
> **Note: pre-patch, qpidd ran steadily at about 270% cpu during this test.  With the patch,
qpidd's cpu utilization was steady at 300%
> 
> 
> Test 3: Federation Funnel (two federated brokers, 2 producers upstream, 1 consumer downstream)
> 
> test setup and execute info:
> qpidd --auth no -p 8888 --no-data-dir --worker-threads=3 -d
> qpidd --auth no -p 9999 --no-data-dir --worker-threads=2 -d
> qpid-config -b $SRC_HOST add queue srcQ1 --max-queue-size=600000000  --max-queue-count=2000000
--flow-stop-size=0 --flow-stop-count=0
> qpid-config -b $SRC_HOST add queue srcQ2 --max-queue-size=600000000  --max-queue-count=2000000
--flow-stop-size=0 --flow-stop-count=0
> qpid-config -b $DST_HOST add queue dstQ1 --max-queue-size=1200000000 --max-queue-count=4000000
--flow-stop-size=0 --flow-stop-count=0
> qpid-config -b $DST_HOST add exchange fanout dstX1
> qpid-config -b $DST_HOST bind dstX1 dstQ1
> qpid-route --ack=1000 queue add $DST_HOST $SRC_HOST dstX1 srcQ1
> qpid-route --ack=1000 queue add $DST_HOST $SRC_HOST dstX1 srcQ2
> qpid-receive -b $DST_HOST -a dstQ1 -f -m 4000000 --capacity 2000 --ack-frequency 1000
--print-content no --report-total &
> qpid-send -b $SRC_HOST -a srcQ1 -m 2000000 --content-size 300 --capacity 2000 --report-total
--sequence no --timestamp no &
> qpid-send -b $SRC_HOST -a srcQ2 -m 2000000 --content-size 300 --capacity 2000 --report-total
--sequence no --timestamp no &
> 
> 
> trunk:                        patched:
> tp(m/s)                       tp(m/s)
> 
> S1    :: S2    :: R1          S1    :: S2    :: R1
> 86150 :: 84146 :: 87665       90877 :: 88534 :: 108418
> 89188 :: 88319 :: 82469       87014 :: 86753 :: 107386
> 88221 :: 85298 :: 82455       89790 :: 88573 :: 104119
> 
> Still TBD:
>   - fix message group errors
>   - verify/fix cluster
> 
> 
> This addresses bug qpid-3890.
>     https://issues.apache.org/jira/browse/qpid-3890
> 
> 
> Diffs
> -----
> 
>   /trunk/qpid/cpp/src/qpid/broker/Queue.h 1297185 
>   /trunk/qpid/cpp/src/qpid/broker/Queue.cpp 1297185 
>   /trunk/qpid/cpp/src/qpid/broker/QueueListeners.h 1297185 
>   /trunk/qpid/cpp/src/qpid/broker/QueueListeners.cpp 1297185 
> 
> Diff: https://reviews.apache.org/r/4232/diff
> 
> 
> Testing
> -------
> 
> unit test (non-cluster)
> performance tests as described above.
> 
> 
> Thanks,
> 
> Kenneth
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message