qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Conway <acon...@redhat.com>
Subject Re: Sender flow control via resource-limit-exceeded
Date Fri, 18 Dec 2009 13:24:38 GMT
On 12/17/2009 08:13 AM, Rafael Schloming wrote:
> Andrew Stitcher wrote:
>> On Wed, 2009-12-16 at 09:56 -0500, Alan Conway wrote:
>>> I'm looking at some code that does sender flow control as follows:
>>> - set a queue size limit
>>> - sender catches 'resource-limit-exceeded' exception, sleeps for a
>>> bit, creates a new session & re-tries
>>
>> The essential problem with this approach is that the queue size is not
>> directly caused by any single producer. So it's entirely possible in
>> general for a producer that is producing at a low rate to be entirely
>> killed by another producer sending messages into the same queue.
>>
>> The only way that this overall scheme can work fairly is if there is a
>> 1-1 producer-queue mapping. If that is the case then it should work.
>
> Why wouldn't we use flow control as specified by the protocol? It
> doesn't help you figure out *who* to flow control, but once you've made
> that decision, it provides a much better way to actually flow control an
> individual producer.
>

Agreed, I'm talking about what a user can do with the code as it stands today. 
We definitely need to implement some producer flow control.

I'm still not sure how this would work. As you point out its tricky to give any 
exact guarantees since you have no idea which queues a client may publish to (at 
least in 0.10) You can bound some notion of "total messages in the system", but 
you need to figure out
a) what should that bound be? It probably varies with number of queues, 
producers/consumers and queue limits but I don't know what the relationship 
would be.
b) how to divide up the credit so fast producers with slow consumers can't hog 
the show.

I don't think it's possible with the protocol flow control to enforce a depth 
limit on individual queues which is unfortunate. So retry after 
resource-limit-exceeded is still probably necessary for a robust producer.

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org


Mime
View raw message