qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rafael Schloming <...@alum.mit.edu>
Subject Re: dispatch router handles 100,000 addresses
Date Thu, 01 May 2014 15:40:54 GMT
On Thu, May 1, 2014 at 10:23 AM, Michael Goulish <mgoulish@redhat.com>wrote:

>  Ah, good.   Care to give a free example?
>

The big hammer would be:

  messenger.route("*", "amqp://hostname")

This will funnel every single address you use into a single link to the
anonymous terminus on the given hostname. You can do finer grained stuff
too, e.g.:

  messenger.route("foo/*", "amqp://hostname1/foo")
  messenger.route("bar/*", "amqp://hostname2/bar")

The key thing to understand is that route doesn't at all impact the address
carried on the message, simply what underlying connection and link are
established and used to deliver the message, and because the rule on the
right doesn't include a substitution variable for the part of the address
that matches the wildcard, you will get the entire class of addresses
matching the pattern on the left funneled down whatever you specify on the
right.

Note that this does assume that whatever you are sending the message to on
the right is capable of understanding the address specified on the message
and routing based on this address.


>
> >
> > The receive side is a bit different, and using qpid-messaging will not
> > necessarily help you scale up. Fundamentally in order to receive messages
> > from N different addresses you need to create N subscriptions. That's
> going
> > to be just as expensive regardless of which API you use to do it. The one
> > option you might have with qpid-messaging is that you can close your
> > receivers explicitly, whereas messenger doesn't have a way to cancel
> > subscriptions. This isn't really a matter of intent though, it's just a
> > missing feature.
>
> Good point.  1 million subscriptions, no matter what I use.
>
> Dang.   I wish there were a way for a receiver to multiplex many
> addresses through a single link, and a single subscription -- and
> if I want to know what addr a given msg came from, I could look it
> up in the message header or something.
>

The address is carried in the message header as you suggest, and in
principle you could create a subscription to the anonymous terminus with a
filter expressing interest in a broad category of messages. I believe the
dispatch router would need to support this for it to work though. There is
some work going on in the AMQP TC to specify some standard capabilities
around the anonymous terminus to make this sort of thing interoperable.


>
> >
> > I don't know if cancelling subscriptions will help you much or not, but
> I'm
> > guessing it would be of similar difficulty (and at least more generally
> > useful) to supply a patch for canceling subscriptions rather than
> retooling
> > your receivers.
> >
>
> No, that wouldn't help me right now -- although it does sound like a very
> good
> feature.  I need all the zillion subscriptions set up before I ever start
> sending.
>

Yeah, I realized that after I posted. If you don't have all zillion
subscriptions up simultaneously, you aren't really testing scale. ;-)

--Rafael

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message