quetz-mod_python-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Lehuen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MODPYTHON-2) multiple/redundant interpreter creation
Date Tue, 14 Dec 2004 21:59:00 GMT
     [ http://nagoya.apache.org/jira/browse/MODPYTHON-2?page=comments#action_56673 ]
     
Nicolas Lehuen commented on MODPYTHON-2:
----------------------------------------

The patch for mod_python.c solves a problem with mod_python creating multiple interpreters
with the same name, but there is stil a problem in the mod_python.apache module, since it
reloads the same publisher many times.

Indeed, having a look at apache.py, I didn't see any locking done to prevent the same module
from being reloaded multiple times concurrently. What we need is to put a lock in the import_module
method, so that two threads cannot check the freshness of the publisher simultaneously. This
can be done in a rather brutal way by calling imp.acquire_lock() at the beginning of import_module,
put a try/finally block around the code of the function and call imp.release_lock() in the
finally block : 

def import_module(module_name, autoreload=1, log=0, path=None):
    """
    Get the module to handle the request. If
    autoreload is on, then the module will be reloaded
    if it has changed since the last import.
    """

    imp.acquire_lock()
    try:
	# previous code of apache.import_module
    finally:
        imp.release_lock()

I tested this, and it works as expected. BTW, maybe it wouldn't work without the patch to
mod_python.c, since I could suddenly see two interpreters with their own copy of the mod_python.apache
module. But the two fixes are required for it to work properly.

Why did I wrote "in a rather brutal way" above ? Because this locking scheme is indeed brutal.
It is a global lock, so each and every thread who want to get its handler must acquire it
for a brief amount of time. The problem is that if a handler module is reloaded, it locks
each and every other thread during its reloading, even if they should not be handled by the
same module. A smarter way to lock would be to use a two-levels locking scheme, as I described
in my caching recipe in the Python Cookbook. I can implement it there if it's ok for everybody.

I suggest we solve this promptly with a big lock and try to have a finer locking scheme later
on.

> multiple/redundant interpreter creation
> ---------------------------------------
>
>          Key: MODPYTHON-2
>          URL: http://nagoya.apache.org/jira/browse/MODPYTHON-2
>      Project: mod_python
>         Type: Bug
>     Versions: 3.1.3
>  Environment: mod_python 3.1.3 + a threaded MPM (observed on Win32 and Mac OS X)
>     Reporter: Nicolas Lehuen
>     Assignee: Nicolas Lehuen
>      Fix For: 3.1.3
>  Attachments: mod_python.c.patch
>
> A small bug in mod_python.c allows the creation of many Python interpreters, where there
should be only one. As a result, modules can be loaded multiple times (once per interpreter)
and some higher level bugs can occur (beginning with higher memory usage). Graham Dumpleton
found the bug in mod_python.c, and I completed the fix with a patch to apache.py (in the import_module)
function.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://nagoya.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
   http://www.atlassian.com/software/jira


Mime
View raw message