qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rafael Schloming <...@alum.mit.edu>
Subject Re: dispatch router handles 100,000 addresses
Date Fri, 02 May 2014 12:04:25 GMT
I think a lot of the extra overhead is coming from some overly generous
default allocation sizes that ended up getting used in a lot of places.
I've adjusted these down. If you can retry your testing with trunk I'm
hoping you should see some improvement.

--Rafael


On Fri, May 2, 2014 at 6:29 AM, Rafael Schloming <rhs@alum.mit.edu> wrote:

> On Thu, May 1, 2014 at 8:14 PM, Michael Goulish <mgoulish@redhat.com>wrote:
>
>>
>>
>>
>>
>> ----- Original Message -----
>> > On 05/01/2014 08:55 PM, Rafael Schloming wrote:
>> > > On Thu, May 1, 2014 at 3:42 PM, Michael Goulish <mgoulish@redhat.com>
>> > > wrote:
>> > >> I tried firing up my messenger-based receivers, each subscribing to
>> 100
>> > >> addresses, then 200, 300, 400, 500.  The results are consistent
>> across
>> > >> that range, and show that each extra address costs 115 KB.  ( Looking
>> > >> only at resident-set size. )
>> > >>
>> > >> So when I tried to do a total of 1,000,000 addrs on one box, I did
>> > >> indeed overwhelm my memory.  That would come to 115 GB, which
>> > >> would have been more than double my physical mem.
>> > >>
>> > >> Please note I did not actually send any messages.  A router was
>> running
>> > >> for these receivers to attach to, but no senders were running.
>> > >>
>> > >> Does 115 KB per subscribed addr seem fairly reasonable?
>> > >>
>> > >
>> > > No, that seems quite excessive. Can you trace where the memory is
>> actually
>> > > coming from?
>> >
>> > Just for comparison, a qpid::messaging process with 1000 subscriptions
>> > over AMQP 1.0 uses 48MB on my laptop. A similar process using AMQP 0-10
>> > uses 8MB.
>> >
>> >
>>
>> Yikes.  A 40 KB per subscription added cost.
>> That's because 1.0 is ten times as good as 0.10   .
>>
>> Just off the top of your head -- is there anything inherent
>> in 1.0 that would make you expect that kind of difference?
>>
>
> Definitely not, the overhead should be comparable.
>
> --Rafael
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message