qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Goulish <mgoul...@redhat.com>
Subject Re: dispatch router handles 100,000 addresses
Date Thu, 01 May 2014 14:23:50 GMT




----- Original Message -----
> On Thu, May 1, 2014 at 9:36 AM, Michael Goulish <mgoulish@redhat.com> wrote:
> 
> >
> >
> >
> >
> > ----- Original Message -----
> > > On 05/01/2014 02:09 AM, Michael Goulish wrote:
> > > >
> > > > I just had a successful test in which a single dispatch router
> > > > handled 100,000 unique addresses.
> > > >
> > > > The router was running on its own box.  On a separate box, I
> > > > started 100 receiving clients ( proton messenger based ) each
> > > > receiving  1000 unique addresses.  A single sending client
> > > > ( qpid-messaging based ) sent the 100,000 messages to the
> > > > router -- 1 to each address.
> > > >
> > > > All the receivers got their expected 1000 messages each,
> > > > and retired happy.
> > > >
> > > > I tried to scale up to 1,000,000 addresses.  The router seemed
> > > > OK with that, but my client-box melted down to slag so now I'm
> > > > a little shy to scale up further...
> > >
> > > Was this with 100 clients each receiving on 10,000 unique addresses? Or
> > > 1000 clients each receiving on 1000 unique addresses?
> > >
> >
> >   1000 * 1000
> >
> > > On the client-box, was it a cpu issue? a memory issue? something else?
> > >
> >
> >
> > Unfortunately, I don't know.  It became so unresponsive that I couldn't
> > get a prompt, couldn't run top.  It felt like swapping.
> >
> > I suspect that it was the messenger 1-link-per-address thing.
> >
> > Since I reported earlier that 1 messenger-based sender grew to
> > 3.4 GB after sending to 30,000 unique addrs, it seems reasonable
> > that 1000 messenger-based receivers, attempting to receive from a total
> > of 1,000,000 addrs, would have attempted to grow to a total of more
> > than 100 GB.  Which would account very nicely for the behavior I saw.
> > ( The box had 45 GB mem. )
> >
> > This kind of extreme scale-up with addresses on a single box is not
> > what messenger was intended for -- I am retooling to use qpid-messaging
> > based clients on both sides of the router.
> >
> 
> I wouldn't say messenger isn't intended for this sort of scale up. The
> fundamental issue here if you need to scale is multiplexing a single link
> with messages to different addresses. This is something you can trivially
> instruct messenger to do with the appropriate route instruction. This
> should let you scale up indefinitely on the send side at least.

 Ah, good.   Care to give a free example?

> 
> The receive side is a bit different, and using qpid-messaging will not
> necessarily help you scale up. Fundamentally in order to receive messages
> from N different addresses you need to create N subscriptions. That's going
> to be just as expensive regardless of which API you use to do it. The one
> option you might have with qpid-messaging is that you can close your
> receivers explicitly, whereas messenger doesn't have a way to cancel
> subscriptions. This isn't really a matter of intent though, it's just a
> missing feature.

Good point.  1 million subscriptions, no matter what I use.

Dang.   I wish there were a way for a receiver to multiplex many 
addresses through a single link, and a single subscription -- and 
if I want to know what addr a given msg came from, I could look it
up in the message header or something.

> 
> I don't know if cancelling subscriptions will help you much or not, but I'm
> guessing it would be of similar difficulty (and at least more generally
> useful) to supply a patch for canceling subscriptions rather than retooling
> your receivers.
> 

No, that wouldn't help me right now -- although it does sound like a very good 
feature.  I need all the zillion subscriptions set up before I ever start sending.





---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Mime
View raw message