quetz-mod_python-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Lehuen <nicolas.leh...@gmail.com>
Subject Re: [mod_python] Sessions performance and some numbers
Date Sat, 09 Apr 2005 09:00:19 GMT
On Apr 9, 2005 3:00 AM, Jim Gallacher <jg.lists@sympatico.ca> wrote:
> Gregory (Grisha) Trubetskoy wrote:
> >
> > [also moved this to python-dev]
> >
> >
> > On Fri, 8 Apr 2005, Nicolas Lehuen wrote:
> >
> >> Well, I had a go last night, but my brain was a little fuzzy too, so I
> >> don't mind a double-check. I didn't use the global lock n° 0, which
> >> would serialize all access to all sessions, but a lock named after the
> >> session, just like it was done in BaseSession.lock() and
> >> BaseSession.unlock().
> >>
> >> BTW, I'm not sure I understand the current locking system in
> >> BaseSession : it looks to me as if the lock on the session was held
> >> for the duration of the whole request (the lock being released during
> >> the request cleanup). Grisha, is this true ? If this is so, then there
> >> is no need to lock anything in FileSession, as the BaseSession already
> >> ensure a proper level of locking.
> >
> >
> > Yes, there is a lock for the duration of the session. This is to make
> > sure that no two requests for the same session are processed at the same
> > time (this is a little documented artifact of sessions in other
> > environments, e.g. jsp).
> >
> > The lock 0 is there to serialize access to the DBM. So we have "session
> > locks" and "dbm store lock" which serve different purposes.
> >
> > Since there is a one-to-one correspondence between sessions and files in
> > a FileSession, you're right the lock 0 use is probably not appropriate
> > (although disk I/O is ultimately serialized by the OS).
> >
> > However, keep in mind that BaseSession (session locks) is optional (it's
> > an __init__ argument), so when session locking is off, you still (and
> > especially more so!) need a store lock.
> >
> > So I guess the bottom line is that FileSession locking is going to be a
> > little different :-)
> >
> > Grisha
> Sorry for the long post here - just kind of thinking out loud.
> I've been playing with the code Nicolas committed and found the file
> locking is not working quite right. I couldn't figure out what was going
> wrong until I re-read Grisha's comments.
> There is a deadlock when accessing an existing session with session
> locking on. A DOS results for that session, and in the worst case a
> complete DOS for any connections. The following bash script demonstrates
> the effect:
> #!/bin/sh
> ab -n 1 http://localhost/session_test.py
> ab -n 1 -C pysid=723b98c0abf885a97b8bdc8d806b4bd8 \
>     http://localhost/session_test.py
> where pysid is a valid session id.
> The first call to the url will succeed, while the second one will fail
> with ab timing out. Here is the FileSession program flow as I see it:
> sess = FileSession.FileSession.__init__(req,lock=1)
>    - calls BaseSession.__init__(req,lock=1)
> BaseSession.__init__()
>    - gets the existing session id from the request cookie
>    - acquires a lock for this session
>    - registers the unlock_session_cleanup
>    - calls BaseSession.load()
> BaseSession.load()
>    - calls self.do_load() which is overriden in FileSession
> FileSession.do_load()
>    - attempts to acquire a lock on the session, but is blocked since
> session was previously locked in BaseSession.__init__()
>    - client times out
>    - unlock_session_cleanup is never run so lock is never released
>    - this apache thread or child is deadlocked
>    - access to this session is blocked forever
> Call the url enough times for an exisiting session, and apache will
> reach MaxClients and refuse additional connections. DOS. oops.
> I assume that overriding BaseSession.load() is the best way to change
> the locking behaviour and fix the problem. I haven't worked out the
> implications of that yet - I guess I need a little more head scratching
> time.
> On another note, I also wonder if this type of locking problem might
> also be a factor in http://issues.apache.org/jira/browse/MODPYTHON-31
> Regards,
> Jim

Given the fact that getting the session object holds a lock for the
whole duration of the request (from the session instantiation to the
request cleanup), the locking code I've wrote in FileSession.do_load
is redundant. What I don't understand, though, is why I didn't see any
deadlock when I tested the code, whereas you see one. I assumed that
the APR locking facility was reentrant, but maybe it's not. Or maybe
it's reentrant on Win32 with threads, and not in an over environment.

Anyway... let's get rid of this redundant locking code in FileSession
and the problem will be solved. The only problem, as Grisha pointed
out, is that locking is an optional feature, you can disable it by
passing lock=0 in the session constructor call. If locking is
disabled, there is a potential for trouble in FileSession. For now,
I'll just say that if you want to disable locking, then you know what
you are doing...

As for MODPYTHON-31, as Graham noted in the comments, it looks like a
problem in the reporter's configuration file. I've closed the bug


View raw message