quetz-mod_python-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Gallacher <jg.li...@sympatico.ca>
Subject Re: FileSession.py - improved? version
Date Mon, 11 Apr 2005 13:02:28 GMT
Hi Nicolas,
Nicolas Lehuen wrote:
> OK, I've checked your code in, but I have a remark : the number of
> global locks is limited in  mod_python. That's why you have something
> like "mod_python: Creating 32 session mutexes based on 0 max processes
> and 250 max threads." in the error.log file.

I've been wondering about that. Thanks for confirming. I'm very aware 
that the locking scheme for my FileSession is less than ideal.

> By having a look at _global_lock and _global_unlock in
> _apachemodule.c, you will see that any request for a lock will be
> brought down to one of the N-1 (here 32 - 1 = 31) pre-allocated global
> locks (the -1 stands for the lock 0 which is special, see the code).
> Hence, with more than 31 concurrent requests, you WILL have
> collisions. What I fear is that those collisions are deadlock-prone...

To say nothing of lock starvation, where some application programing 
error causes a request to not return and thus the lock for that session 
is never released. I wonder if this would explain the problems I've seen 
with session locks turned. The file locks created should always be 
released since the file manipulations are surrounded by a try - finally 
clause, whereas the session lock release is depending on the method 
registered by the req.register_cleanup in BaseSession.lock(). Is 
anything registered with register_clean() absolutely  guaranteed to run, 
no matter what? I'll need to read the code.

> In that case, I wonder if it is worthy to have two different locking
> mechanisms (at the session level and a the file level) since it will
> make collisions even more probable. You solve this by disabling
> session locking by default, but session locking can truly be useful
> (for example, when you have a site which loads a frameset in which
> each frame requires a session).

I realize session locking is useful, but I turned it off by default 
since it seemed to be causing too many problems. This was intended as a 
temporary measure until I could figure out what was going on. I can 
imagine someone grabbing FileSession.py from svn, using it as a drop-in 
replacement for DbmSession on a production machine, and killing apache 
as a result.

> I think we should find a way to have reentrant session locking, so
> that if a process/thread already holds a session lock, a new call to
> BaseSession.lock() does not block the process/thread. This way, we
> could simply enable session locking by default and re-use the same
> locks for file-level locking. This would enable to have file-level
> locking even if session-level locking is disabled, while keeping the
> number of locks (and lock collisions) low.

This would be ideal.

> Or, we could try to re-evaluate the need for this finite global lock
> pool. I guess Grisha implemented this for the sake of performance
> (apparently global mutexes can be expensive in some implementations of
> the APR). Why not create a distinct lock for each lock name (and thus
> exceed the 32 limit) ?

Well, it's always going to be finite limit, and some idiot like me will 
always find a way to exceed that limit. :)


> On Apr 11, 2005 6:41 AM, Jim Gallacher <jim@jgassociates.ca> wrote:
>>Hello All,
>>I've attached an alternative version of FileSession.py. A couple of
>>changes from the original version:
>>1. Uses PythonOption FileSessionDir to determine directory where session
>>files will be save. If this option does not exists, it defaults to
>>2. Does file locking independent of the session locking. File locking
>>always occurs, while session locking is still optional. I'm finding that
>>session locking still causes some problems, so I've changed the default
>>for session locking to 0.
>>3. Fixed a bug in do_cleanup which was calling time(), not time.time()
>>and added import time statement.
>>4. Added do_cleanup() optimizations. Two parameters now control the
>>behaviour of do_cleanup(). The session file mtime can be used to
>>determine if the file is a deletion candidate. If it is it can either be
>>deleted immediately or unpickled and the true session expiration
>>verified. The speedup can be significant for more than 10,000 sessions.

View raw message