qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carl Trieloff <cctriel...@redhat.com>
Subject Re: DTX for BDB Desgin Question.
Date Tue, 22 May 2007 13:59:57 GMT

we can use dev@etp.108.redhat.com  It is also a public list.
Carl.

Rupert Smith wrote:
> Apologies, that was my fault. Is there a list for etp that we could use?
>
> I'll just email the answer direct to Arnaud, anyone else shout if you 
> want a
> CC.
>
> Rupert
>
> On 21/05/07, Robert Godfrey <rob.j.godfrey@gmail.com> wrote:
>>
>> all - we should keep discussions of the BDB store off this list I
>> think, it is not part of the Apache project.
>>
>> -- Rob
>>
>> On 21/05/07, Arnaud Simon <asimon@redhat.com> wrote:
>> > Rupert,
>> >
>> > Are you suggesting that we use a single BDB tx for several concurrent
>> > prepare/commit operations? In such a case we would have to change the
>> > code as we are allocating one BDB tx object per Xid. Note that we 
>> do not
>> > do that in the case of the JDBC store. I am a little bit concern about
>> > the sync operation gradually slowing down though. If you start to 
>> sync 2
>> > messages then 4 may have been queued then when syncing the 4 queued 
>> then
>> > 6 may be queued and so on.
>> > I have nothing against any optimization and I would be please to 
>> include
>> > your trick (if load tests show that this is significantly faster).
>> >
>> > BTW, I have cleaned the BDB code and added LGPL license text in all
>> > classes.
>> >
>> > Cheers
>> >
>> > Arnaud
>> >
>> > On Mon, 2007-05-21 at 17:37 +0100, Rupert Smith wrote:
>> > > Unless the BDB Transaction.commitSync() method is already capable of
>> doing
>> > > this? It doesn't seem impossible, but I'm assuming it doesn't or 
>> else
>> the
>> > > batch synch queueing scheme wouldn't have been used in the old BDB
>> store.
>> > >
>> > > Rupert
>> > >
>> > > On 21/05/07, Rupert Smith <rupertlssmith@googlemail.com> wrote:
>> > > >
>> > > > Hi Arnaud,
>> > > >
>> > > > The old BDBMessageStore had a feature whereby commits in different
>> threads
>> > > > could batch their transactions into the same disk synch. 
>> Implemented
>> by the
>> > > > Commit and CommitThread inner class. The idea is that there is one
>> hand-off
>> > > > thread that is doing the writes to disk, and committing threads 
>> hand
>> over
>> > > > their writes to it, but block until it completes them. The 
>> hand-off
>> thread
>> > > > is in one of two states: busy writing to the disk, in which case
>> requests
>> > > > queue up, or taking the built up requests to write them to disk. I
>> call this
>> > > > a batch synched queue. It can boost performance considerably by
>> ammortizing
>> > > > the cost of disk seeks accross many transactions, assuming 
>> there is
>> enough
>> > > > concurrency accross the commits to take advantage of.
>> > > >
>> > > > Just a suggestion, but you could copy (and perhaps neaten up) the
>> old code
>> > > > for this? If not, its something I may do, depending on priorities.
>> I've also
>> > > > got a bit of code somewhere that wraps this 'batch synched queue'
>> pattern up
>> > > > and makes it look like any other Queue implementation (puts block
>> till
>> > > > taken, or even beyond being taken, till explicitly released). I 
>> have
>> a
>> > > > feeling it needs a little bit of work, but can add it to a
>> > > > org.apache.qpid.utils when done.
>> > > >
>> > > > Rupert
>> > > >
>> > > >
>> >
>> >
>>
>


Mime
View raw message