directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Selcuk AYA <>
Subject Re: Txn Work Update(2)
Date Thu, 15 Dec 2011 16:04:06 GMT
On Thu, Dec 15, 2011 at 2:27 PM, Emmanuel Lecharny <> wrote:
> On 12/15/11 12:25 PM, Selcuk AYA wrote:
>> Hi all,
>> update on my status:
>> * There is going to be a big change in my life soon and I will do a
>> big move. So I will be MIA for some time( around 2 months) and after
>> that I will have less time to contribute but I will try to keep being
>> involved in development and discussions.
> Ok. I wish this is a move for a better life :)
>> * I did a recent check in. There are some tests that fail. I will list
>> them below. The basic reason that those tests are failing is that they
>> do queries and updates of which  some are transactional, some are not
>> and this leads to wrong results.
> I spent some time yesterday on the failing tests. What I need here is a clue
> about how to make some operation transactionals. If you can update us, that
> would be great.

please take a look at ldap protocol handlers.
>> And the basic reason they do queries
>> and updates in a non transactional way is that I did not put
>> transaction begin/commif/abort( ie, txn demarcation) at CoreSession.
>> This is the layer we start the execution of  the basic ldap operations
>> but I thought there might be clients of CoreSession which want to
>> group operations together and execute them as a single txn(If you
>> think this is not necessary, please let me know).
> I don't think it's anymore necessary. We have refactored all the operations
> last month so that we never have to re-enter the chain twice when processing
> an operation.

I was thinking of clients of CoreSession within ApacheDs. For example,
revert of changelog or changepassword. There might be other cases.  I
am not talking about the interceptor chain here. If these kind of
clients do not need to group ldap operations together as a
transaction, then move txn begin/commit/abort to CoreSession, and for
search close the transaction when entry filtering cursor is closed.

Note that, there is also the case that all over the server, cursors
are not closed or not closed in a finally clause. I think these should
be fixed.

>>  Also, there is at
>> least search, for which the cursor is kept open above CoreSession, So
>> for search, the caller has to handle txn commit, abort.Also there
>> might be some cases where operations do not use session at all.
> Well, we do some lookups directly on the nexus, as we are inside the chain
> already, and I guess we are using the ongping transaction here. If it's not
> the case, then we should probably have a way to make those operations part
> of the ongoing txn (don't know how difficult it is)

Operations within interceptor chain is part of the same transaction,
But we have some cases which do not use sessions and which directly
interact with nexus outside interceptor chain. For example some
interceptor initializations.
>> As a result, I decided to handle each  caller of CoreSession
>> operations separately and this increased the number changes that had
>> to be done. I handled ldap protocol handlers and change password.
>> Right now StockServerSuite is passing except three tests.
>> StockCoreSuite and MigratedStockCoreSuite is failing. The reason is
>> they use LDAPCoreConnection and this is not transactional yet.
> I have some failures in core-integ. Do you have them too ?

StockCoreSuite and MigratedStockCoreSuite  => Yes . These are using
LdapCoreConnection. If you move txn begin/end to CoreSession you wont
need to handle LDAPCoreConnection separately.

>> For anybody who wants to get involved, here is the list of todo:
>> * Figure out why SaslBind and DelegatedAuthIT in stockserver suite is
>> failing. We get a KRBException here and I do not know this module well
>> enough.
> Will check.
>> *Handle LdapCoreConnection. This is used in applications which embed
>> apacheds i guess?
> Yes.
>> The basic difficulty is that users of this class get
>> a cursor and the transaction should be comitted or aborted only when
>> this cursor closes . Moreover, when a cursor is open, user of
>> ldapcoreconnection can call other ldap operations and it might make
>> sense to group these in a transaction. If this is true, then it should
>> be the responsibility of the callers of LdapCoreSession to start and
>> end txns. If we are not going to let the user do this, we need to
>> store a pointer to txn context in the cursor and end the txn when
>> cursor is closed taking into account that the txn might have to be
>> upgraded to a readwrite txn from a read only txn if the user does
>> changess while holding the cursor open.
> I don't think we should expose the txns. It should remain an internal
> mechanism. Also I don't mind if a user has an issue because he is doing some
> changes in the server while processing a search : it's his problem, not our.
> It is very clear that LDAP is not transactionnal, and the suer should be
> aware of this fact. (that just means a user can't expect that a long search
> can return entries which have been modified or deleted since the search has
> started. But the user should not get an exception just because an entry does
> not exists anymore when the cursor is moving forward).
> At least, this is my undertsanding. Alex, do you agree on that ?
>> * There are some tests which call session operations directly, these
>> should be wrapped in txn begin/end.
> ok.

See above comment about moving txn beging/commit/end to CoreSession.

>> *I put some tests to AVL and jdbm partitions to test txns without the
>> effect of interceptors and logical caches. These are multithreaded and
>> are good to test the robustness of the data change and query layer and
>> txn subsystem. Add more tests like this.
> Ok.
>> * Handle logical caches. As we talked handle these with read write
>> locks taking cleanup in  case of an aborted txn into account.
>> Please let me know if anyone wants to attack any of these. I might
>> provide more input if needed.
> I will spend some time by the end of this year on those things. We need to
> get a branch that builds and passes tests with flying colors, even if it's
> not perfect.
> Also I see that there are TODO into the DefaultOperationExecutionManager
> class :
>    public void delete( Partition partition, DeleteOperationContext
> deleteContext ) throws LdapException
>    {
>        Dn dn = deleteContext.getDn();
>        partition.delete( deleteContext );
> What should we do ?
I mentioned previously that I was trying to remove
add/delete/modify/moveDn interfaces from partitions. This reason I
couldnt do that was SchemaPartition was triggning some updates with
these mehtods. Since you mentioned those are moved, these calls can be
removed and partition interface can be changed.

> --
> Regards,
> Cordialement,
> Emmanuel Lécharny

View raw message