qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Goulish <mgoul...@redhat.com>
Subject Re: dispatch router handles 100,000 addresses
Date Thu, 01 May 2014 13:36:44 GMT




----- Original Message -----
> On 05/01/2014 02:09 AM, Michael Goulish wrote:
> >
> > I just had a successful test in which a single dispatch router
> > handled 100,000 unique addresses.
> >
> > The router was running on its own box.  On a separate box, I
> > started 100 receiving clients ( proton messenger based ) each
> > receiving  1000 unique addresses.  A single sending client
> > ( qpid-messaging based ) sent the 100,000 messages to the
> > router -- 1 to each address.
> >
> > All the receivers got their expected 1000 messages each,
> > and retired happy.
> >
> > I tried to scale up to 1,000,000 addresses.  The router seemed
> > OK with that, but my client-box melted down to slag so now I'm
> > a little shy to scale up further...
> 
> Was this with 100 clients each receiving on 10,000 unique addresses? Or
> 1000 clients each receiving on 1000 unique addresses?
> 

  1000 * 1000

> On the client-box, was it a cpu issue? a memory issue? something else?
> 


Unfortunately, I don't know.  It became so unresponsive that I couldn't
get a prompt, couldn't run top.  It felt like swapping.

I suspect that it was the messenger 1-link-per-address thing.

Since I reported earlier that 1 messenger-based sender grew to 
3.4 GB after sending to 30,000 unique addrs, it seems reasonable
that 1000 messenger-based receivers, attempting to receive from a total 
of 1,000,000 addrs, would have attempted to grow to a total of more
than 100 GB.  Which would account very nicely for the behavior I saw.
( The box had 45 GB mem. )

This kind of extreme scale-up with addresses on a single box is not 
what messenger was intended for -- I am retooling to use qpid-messaging 
based clients on both sides of the router.



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Mime
View raw message