qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Flavio Baronti (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (QPID-5334) Low throughput with thousands of queues
Date Tue, 12 Nov 2013 16:46:20 GMT

     [ https://issues.apache.org/jira/browse/QPID-5334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Flavio Baronti updated QPID-5334:
---------------------------------

    Attachment: JMSReceiver.java
                JMSGenerator.java

Two test clients in Java, one to generate data, one to receive.
Playing with the parameters (number of topics, number of sessions/connections) shows that
while the generator can maintain its output throughput, the receiver cannot sustain a high
throughput with many topics and few sessions.

> Low throughput with thousands of queues
> ---------------------------------------
>
>                 Key: QPID-5334
>                 URL: https://issues.apache.org/jira/browse/QPID-5334
>             Project: Qpid
>          Issue Type: Improvement
>          Components: C++ Broker, Java Client
>    Affects Versions: 0.24
>         Environment: Broker: Linux 2.6.32 64bit
> Client: Windows 7, Java 1.7.0_21 64bit
>            Reporter: Flavio Baronti
>              Labels: performance
>         Attachments: JMSGenerator.java, JMSReceiver.java
>
>
> I have an use case where I need to create hundreds of thousands of queues,
> each one subscribed to a different topic (therefore I have as many topics
> as queues).
> I set up a test with a single producer generating data on a randomly
> chosen topic, and a receiver retrieving data from the queues (and throwing
> it away).
> I'm using the JMS api, and doing the obvious thing makes the throughput
> drop dramatically from 10k msg/sec with a single topic/queue (around the
> top my network adapter can sustain) to 20 msg/sec with 100k topics/queues.
> I found out that I can recover performance by using more JMS sessions and
> connections - e.g. create 4 connections with 100 sessions each, and
> randomly distributing the receiving queues on them.
> This however is less than ideal, since with the JMS client a thread is
> created for each session, I can manage the workload with 1 thread only when it's over
a single queue.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Mime
View raw message