qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arnaud Simon <asi...@redhat.com>
Subject Re: DTX for BDB Desgin Question.
Date Mon, 21 May 2007 18:37:22 GMT

Are you suggesting that we use a single BDB tx for several concurrent
prepare/commit operations? In such a case we would have to change the
code as we are allocating one BDB tx object per Xid. Note that we do not
do that in the case of the JDBC store. I am a little bit concern about
the sync operation gradually slowing down though. If you start to sync 2
messages then 4 may have been queued then when syncing the 4 queued then
6 may be queued and so on. 
I have nothing against any optimization and I would be please to include
your trick (if load tests show that this is significantly faster). 

BTW, I have cleaned the BDB code and added LGPL license text in all



On Mon, 2007-05-21 at 17:37 +0100, Rupert Smith wrote:
> Unless the BDB Transaction.commitSync() method is already capable of doing
> this? It doesn't seem impossible, but I'm assuming it doesn't or else the
> batch synch queueing scheme wouldn't have been used in the old BDB store.
> Rupert
> On 21/05/07, Rupert Smith <rupertlssmith@googlemail.com> wrote:
> >
> > Hi Arnaud,
> >
> > The old BDBMessageStore had a feature whereby commits in different threads
> > could batch their transactions into the same disk synch. Implemented by the
> > Commit and CommitThread inner class. The idea is that there is one hand-off
> > thread that is doing the writes to disk, and committing threads hand over
> > their writes to it, but block until it completes them. The hand-off thread
> > is in one of two states: busy writing to the disk, in which case requests
> > queue up, or taking the built up requests to write them to disk. I call this
> > a batch synched queue. It can boost performance considerably by ammortizing
> > the cost of disk seeks accross many transactions, assuming there is enough
> > concurrency accross the commits to take advantage of.
> >
> > Just a suggestion, but you could copy (and perhaps neaten up) the old code
> > for this? If not, its something I may do, depending on priorities. I've also
> > got a bit of code somewhere that wraps this 'batch synched queue' pattern up
> > and makes it look like any other Queue implementation (puts block till
> > taken, or even beyond being taken, till explicitly released). I have a
> > feeling it needs a little bit of work, but can add it to a
> > org.apache.qpid.utils when done.
> >
> > Rupert
> >
> >

View raw message