quetz-mod_python-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Deron Meranda (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MODPYTHON-217) Python 2.3 and simplified GIL state API still causes problems.
Date Wed, 29 Aug 2007 20:40:30 GMT

    [ https://issues.apache.org/jira/browse/MODPYTHON-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12523668

Deron Meranda commented on MODPYTHON-217:

Another real-world case where this comes up is with the Xapian
search engine (http://xapian.org/).  It also is SWIG generated.
See their bug #185,

> Python 2.3 and simplified GIL state API still causes problems.
> --------------------------------------------------------------
>                 Key: MODPYTHON-217
>                 URL: https://issues.apache.org/jira/browse/MODPYTHON-217
>             Project: mod_python
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 3.3.x, 3.2.10
>            Reporter: Graham Dumpleton
> There are still problems in mod_python when third party extension modules are used which
make use of the Python simplified GIL state API. Specifically, if C code makes calls to PyGILState_Ensure()
when it already holds the Python GIL and has an active thread state, but the active thread
was one which was originally created outside of Python, such as by Apache with mod_python,
then the call to PyGILState_Ensure() results in a dead lock. This is occurring because the
PyGILState_Ensure() function finds it does not know anything about the current thread ID and
therefore assumes it needs to create a new thread state for it and make that the active thread
state. In doing that though it tries to acquire the GIL when the thread already holds it,
resulting in the deadlock.
> At this stage it is believed this only occurs with Python 2.3 and shouldn't be a problem
with later versions of Python. This is the case as later versions of Python were modified
so that PyThreadState_New() will register the thread with the underpinnings of the PyGILState_???()
mechanism and because it already knows about it, it will not attempt to create a new thread
state object and so the deadlock will not occur.
> When C code is hand crafted it is unlikely that anyone would call PyGILState_Ensure()
when they already know they hold the GIL, ie., when calling out of Python code, but SWIG when
used with the -threads option does exactly this. For example:
> SWIGINTERN PyObject *_wrap_ap_get_server_version(PyObject *SWIGUNUSEDPARM(self), PyObject
*args) {  PyObject *resultobj = 0;
>   char *result = 0 ;
>   if (!PyArg_ParseTuple(args,(char *)":ap_get_server_version")) SWIG_fail;
>   {
>     result = (char *)ap_get_server_version();
>   }
>   resultobj = SWIG_FromCharPtr((const char *)result);
>   return resultobj;
> fail:
>   return NULL;
> }
> where SWIG_PYTHON_THREAD_BEGIN_BLOCK eventually expands to a call to PyGILState_Ensure().
> The only solution to this problem would be for mod_python to behave differently when
it is acquiring a thread state against the main Python interpreter, ie. 'main_interpreter'.
Specifically, rather than use PyThreadState_New() etc, it should call PyGILState_Ensure()
instead. When needing to release the main Python interpreter it should call PyGILState_Release().
It should continue to work as before for all other interpreters and anyone with GILState code
would need to ensure they are using 'main_interpreter'.
> Unfortunately at the moment making this change is not completely straight forward as
when an interpreter is being released it isn't known that it is the main Python interpreter.
Thus some knowledge that it is the main interpreter would need to be carried into the call.
The only other option is that the list of all active interpreters is traversed and the last
one in the list is assumed to be the main interpreter. Then if the interpreter being released
matches that then act differently.
> Note that at present the acquire/release methods are exported as optional functions so
that other Apache modules use them. It is unlikely that anyone is making use of them, so changing
the in/out parameters of the functions would possibly be acceptable.
> Also note that maintaining support for threaded and non threaded Python and now this
GIL state variation is going to make the code even more messy. As such it might be worthwhile
considering dropping support for non threaded Python. These days Python defaults to being
threaded anyway, and highly unlikely that someone would want to use non thread Python with
mod_python anyway. When ever any development of mod_python is done, no testing is ever done
with a non threaded Python anyway and so it is possibly questionable if that support still
> FWIW, this issue came up as a result of queries expressed in:
>   http://www.modpython.org/pipermail/mod_python/2007-April/023445.html
> The GIL state issue has attempted to be addressed before in MODPYTHON-77 but this particular
issue not captured in that.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message