qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Conway <acon...@redhat.com>
Subject Re: Proposed QIP: Support for Message Grouping in the broker.
Date Tue, 28 Jun 2011 10:56:59 GMT
On 06/28/2011 06:39 AM, Gordon Sim wrote:
> On 06/28/2011 09:57 AM, Marnie McCormack wrote:
>> In JMS, the sequence id would be used to maintain order, but the expectation
>> is that the group are processed as a whole on receipt of the final message
>> in the group rather than simply shared out between consumers and processed
>> in order.
>
> I think that is a third variation (further restriction really) on the use case.
> I.e. where message group is used to piece together a large unit from discrete
> messages.
>
> So I think the requirements are:
>
> (1) message group must always be processed in order
>
> (2) all messages in a group are processed by the same consumer
>
> (3) the consumer will only complete processing having received the 'final'
> message in the group
>

What happens if the consumer dies or cancel's mid-group in this case? Do we
- replay the group from the beginning to another consumer?
- drop the rest of the group?
- continue sending to a new consumer from where the first consumer left off?
   - In which case do we provide a flag to let the new consumer know it's 
starting mid-group?

> Where (3) implies (2) and (2) implies (1).
>
> The third requirement essentially rules out an implementation of the first
> requirement that refuses to deliver the next message in the group until the
> previous message is accepted.
>
> The second requirement is important where there is a desire to contain all the
> necessary state for processing a group of messages within a single consumer. I
> agree that is important and should be supported. However I think it is not
> always required and it should be possible to drop that if not needed as that
> could lead to more adaptive load-balancing while meeting the first requirement.
>
> So I think the configuration involved is:
>
> (a) identification of the header that signifies a group within a given queue,
> which implicitly enables requirement (1)
>
> (b) an indication that 'stickiness' of group to consumer is required (beyond
> simply meeting the first requirement)
>
> I think we can avoid the need for any explicit configuration of requirement (3).
> Refusing to deliver the next message in the group until the previous message is
> accepted is in my view unnecessary to meet (1) and could be emulated more or
> less by controlling prefetch.
>

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org


Mime
View raw message