quetz-mod_python-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Graham Dumpleton <grah...@dscpl.com.au>
Subject Re: Need confirmation of memory leak using Apache 2.2.2.
Date Mon, 31 Jul 2006 09:52:42 GMT
Okay, found the source of the memory leak. The problem goes right back
to 3.1.4 which also has the problem when tested.

The problem code is in python_handler() in 'src/mod_python.c'.  
Specifically
the code does:

     if (!hle) {
         /* create a handler list object from dynamically registered  
handlers */
         request_obj->hlo = (hlistobject *)MpHList_FromHLEntry 
(dynhle, 1);
     }
     else {
         /* create a handler list object */
         request_obj->hlo = (hlistobject *)MpHList_FromHLEntry(hle, 1);

         /* add dynamically registered handlers, if any */
         if (dynhle) {
             MpHList_Append(request_obj->hlo, dynhle);
         }
     }

Problem is that request_obj->hlo can already be set by a prior phase's
handler and by simply assigning to request_obj->hlo you get a memory
leak as it is a Python object and it isn't being decref'd.

Thus, before this 'if' statement, it would appear that the following  
should
be inserted.

     if (request_obj->hlo)
         Py_DECREF(request_obj->hlo);

or:

     Py_XDECREF(request_obj->hlo);

Still need to do some more checking, but adding that seems to get rid of
the problem.

Now what do we do about 3.2.10? Given that this thing leaks really badly
when triggered shows that no one must be using multiple handler phases
at the same time, so may be safe to still release 3.2.10 and we fix  
it in next
backport release and 3.3.

Comments.

Graham

On 31/07/2006, at 7:24 PM, Graham Dumpleton wrote:

> The good news is that this isn't a problem introduced with  
> mod_python 3.2.10
> so doesn't necessarily have to stop that being released.
>
> The bad news though is that is is also in mod_python 3.2.8, so it  
> has been
> there for some time.
>
> The question now is whether anyone else sees it on other platforms  
> or whether
> it is something specific to my machine.
>
> Graham
>
> On 31/07/2006, at 4:53 PM, Graham Dumpleton wrote:
>
>> I get it on Apache 2.0.59 as well. :-(
>>
>> I will thus be interested to see what others get, as appears to be  
>> an existing
>> mod_python issue.
>>
>> BTW, this is with worker MPM.
>>
>> Graham
>>
>> Graham Dumpleton wrote ..
>>> I am using Apache 2.2.2 and when using mod_python in a certain  
>>> way, I am
>>> seeing
>>> significant ongoing increases in memory use by Apache child  
>>> processes.
>>> Initially I thought I had stuffed up recent changes in mod_python  
>>> 3.3 out
>>> of
>>> subversion trunk that I had been making, but I went back to  
>>> mod_python
>>> 3.2.10
>>> and am seeing the same problem.
>>>
>>> Can some one please run the following test and use "top" or some  
>>> other
>>> means of
>>> monitoring memory use to see if you can duplicate the problem.  
>>> The test
>>> is really
>>> quite simple actually.
>>>
>>> # .htaccess
>>>
>>> PythonFixupHandler handlers
>>> AddHandler mod_python .py
>>> PythonHandler handlers
>>>
>>> # handlers.py
>>>
>>> from mod_python import apache
>>>
>>> def handler(req):
>>>   req.content_type = 'text/plain'
>>>   req.write('handler')
>>>   return apache.OK
>>>
>>> def fixuphandler(req):
>>>   return apache.OK
>>>
>>> The command I have been using to test is:
>>>
>>>   ab -c 1 -n 5000 http://localhost:8082/~grahamd/MODPYTHON-155/ 
>>> hello.py
>>>
>>> What is strange is that if I have only the fixup handler or  
>>> response handler
>>> invoked and not both, then memory leak isn't present. It is only  
>>> when both
>>> are
>>> being invoked that it leaks memory.
>>>
>>> This occurs for me on Mac OS X 10.4, Python 2.3.5 and Apache  
>>> 2.2.2, with
>>> either mod_python 3.2.10 or mod_python 3.3. I haven't yet tested  
>>> with older
>>> Apache 2.0.58 as yet as don't have it on my box at present, but  
>>> will download
>>> it and see if that makes a difference.
>>>
>>> I guess what I am trying to working out is if this is an issue  
>>> with Apache
>>> 2.2.2
>>> or whether it is mod_python and problem has been there for a  
>>> while. Thus
>>> if
>>> people with both Apache 2.0.X and Apache 2.2.X could test it it  
>>> would be
>>> great.
>>>
>>> Any help appreciated.
>>>
>>> Thanks.
>>>
>>> Graham


Mime
View raw message