qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Crist" <colincr...@hermesjms.com>
Subject RE: Message history and replay
Date Thu, 18 Jan 2007 16:30:11 GMT

Hi Marnie,

There are certainly opertational issues to consider when deploying such
behaviour and its unlikely that it would be enabled out of the box :)

I think I'd go for a header property to mark the message as "recoverable" or
"replayable" so that I could run both types of behaviour concurrently and
even maybe keep the recoverable messages in a separate message store so I
can better monitor its size. There would be no need to try and cache any of
these messages anywhere, recovering clients would be happy to take the disk
read hit. Having two stores would complicate recovery as you'd have to
ensure that all messages in transit between the stores were flushed when a
client requests recovery so an exchange implementation would need support in
the core broker to do this - even if all it did was call something to say a
message has reached zero references and should it be deleted.

I've used this kind of recovery before and it worked very well but you need
the transport to support it natively and very few do. It would be real Qpid
differentiator and I'd be keen to bash out a prototype when I've got some
time next month.

Anyway, thanks for your brain time, talk l8r.

Colin.

> -----Original Message-----
> From: Marnie McCormack [mailto:marnie.mccormack@googlemail.com] 
> Sent: 18 January 2007 16:02
> To: qpid-dev@incubator.apache.org
> Subject: Re: Message history and replay
> 
> Hi Colin,
> 
> I guess my immediate reading of this is that it could be 
> mighty dangerous in the wrong hands i.e. a very large message 
> store might result if applied too liberally.
> 
> When we talked about archiving messages previously we had 
> imagined that timeToLive could come in to play here, though I 
> might well be abusing it's intended purpose.
> 
> I guess we might also need to be careful around anything 
> which might slow up the 'normal' startup model i.e. when we 
> need to point a new broker instance at an existing BDB store 
> and get the messages in queues for recovery.
> 
> I'll try to give it some more thought :-)
> 
> Bfn,
> Marnie
> 
> On 1/18/07, Colin Crist <colincrist@hermesjms.com> wrote:
> >
> >
> > Hi,
> >
> > A quick question...
> >
> > A common recovery scenario is requesting replay on a queue from a 
> > given, known message, either by its ID or by something in 
> the header. 
> > Its saves all that mucking about with XA and makes recovery 
> the normal 
> > way of system startup rather then some exceptional case.
> >
> > This means when a message has reached all its recipients it should 
> > never be marked for removal from the durable message store.
> >
> > As a user, I would also like to control when messages get 
> removed as 
> > part of my end of day or even weekly processs, probably based on 
> > header property selectors.
> >
> > I don't believe this is supported in AMQP in the wire protocol (nor 
> > perhaps should it be) - is it maybe something that could be 
> done via 
> > some standardised command messages to a "replay" exchange?
> >
> > Any thoughts on implementation difficulty?
> >
> > Regards,
> >
> > Colin.
> >
> >
> >
> >
> >
> 



Mime
View raw message