qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lichtner <licht...@bway.net>
Subject Re: clustering
Date Fri, 22 Dec 2006 18:57:18 GMT

On Fri, 22 Dec 2006, Tim Fox wrote:

> > But I am proposing that all queues be available on all nodes. So all the
> > nodes do need to see all the messages.
> That's only true if you want to maintain exact copies of the queues on
> each node. If you want to replicate for high availability that might be
> useful.

But up to a certain number of nodes the number of clients increases.

As I said, you can put 25,000 1400-byte messages through a queue in this
manner. The limiting factor on the number of clients is the latency,
because since each node can only process, say, C clients, in order to add
clients you have to add more nodes. But I think that with 3-4 nodes you
are already going to have an award-winning jms server.

If you want to scale beyond then you can use the multiple-ring protocol.

> However, if you're intention is to be able to scale your system by
> adding nodes to the cluster then maintaining exact replicas is not going
> to help, since as we've both tacitly agreed each node would receive the
> traffic from all other nodes, so you make no gains in adding extra nodes
> to the cluster. (I.e. clustering is pointless)

As explained above, by adding nodes you get to service a greater number of
clients. I hope it's clear. But the price is that you increase the latency
of all messages, a little bit. You just have to figure out how much is too

But the implementation is so much easier than anything else that it should
be good enough for a first implementation.

> This boils down to the fact that the work in adding/removing an entry
> from a strict FIFO queue is not inherently parallelizable (see Amdahl's
> law).

But the cost of adding to the queue is a marginal cost in the complete use
case of adding to a queue. A machine has to do a context switch, read the
incoming command from the client, and then add the message to the queue.

> So how do you solve this problem? One way (and this is how several of
> the popular messaging systems do it) do it is to relax the strict
> FIFOness of the queue.

You don't need to do that.

> For many messaging applications/flavours, e.g. JMS, strict FIFO is not
> necessary, it is only necessary to maintain order of messages produced
> from a session. Therefore you can implement the "logical" clustered
> queue, as a set of quasi-independent partial queues - one on each node
> of the cluster. When you are attached at a particular node and sending a
> message to the clustered queue, the particular partial queue it ends up
> in is determined by a routing policy - typically you would always favour
> the local queue to avoid extra network traffic. Consumers can be
> attached at any node and consume from the partial queue. Then you
> implement a "message redistribution" algorithm on top of this which
> pulls messages from one partial quuee to another to avoid starvation of
> consumers on a particular partial queue, or to spread load due to
> consumers faster or slower on different nodes.
> Theoretically this has linear scalability.

Is it easy to implement correctly?

View raw message