quetz-mod_python-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dharana <dhar...@dharana.net>
Subject Re: [mod_python] Help with memory problems
Date Tue, 12 Jul 2005 23:54:45 GMT
Uh oh, no it doesn't goes away. Check this (custom handler):

------ controller.py ---------------
import os
from time import time
from mod_python import apache
from mod_python import Session

def handler(req):
     start_time = time()
     req.sess = Session.FileSession(req)
     req.content_type = 'text/html'
     end_time = time()
     totaltime = (end_time - start_time)*1000
     req.write('%.0f ms' % ((totaltime)))
     return apache.OK
------------------------------------


This takes 75ms on a 1Ghz with 1Gb RAM server. After some page refreshes the 
memory load is over 70Mb. Is this right? (I'm asking for both the load time and 
memory usage)


dharana wrote:
> Please, ignore my previous email. The massive leak was being caused by 
> FileSession. I was using an earlier snapshot and after upgrading to:
> 
> http://svn.apache.org/snapshots/mod_python/mod_python_20050712224304.tar.gz
> 
> Everything went back normal. After checking everything works go upgrade 
> production. Load problem solved. I feel much better now, python handles 
> the heavy complexity I imposed to him with no problems.
> 
> 
> How much is left for 3.2? I can't use an stable mp release right now 
> because I need several things included in 3.2 (segfaults for psp, 
> filesession, etc) and using svn HEAD in a prod server is scary.
> 
> dharana wrote:
> 
>> look at this screenshot:
>>
>> http://dharana.net/memory_concerns.png
>>
>> It shows two screenshots of the same page of my webapp. The lower one 
>> was shoot just after an apachectl start. The second one was shot after 
>> 6 consecutive refreshes where memory usage increased every time by a 
>> lot of Mbs.
>>
>> I use a lot of custom modules, from db calls to xslt transformations 
>> (with libxml). But in the page shown in the screenshot it does little. 
>> The strange thing is that after a few more page refreshes the memory 
>> load goes back to 23Mb and then increases again.
>>
>> I use some light module-level caching (I cache the 3 queries executed 
>> in the lower page). But I don't think this is causing the heavy memory 
>> usage because after it goes back to 23Mb it does query again (ie: the 
>> caching mechanism still has it's cache).
>>
>> I use apache prefork and Maxrequestperchild 0. After every page 
>> refresh the unr obj var shown on the page doesn't changes. It's either 
>> 63 or 0.
>>
>> mem, resident and stack come from this script:
>>
>> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286222
>>
>> unr obj is just the output of gc.collect() after everything else is 
>> done (the debug print is printed just before return apache.OK.
>>
>> What I am asking is for some help or pointers in how to track down who 
>> is causing this or where to look for reducing the memory usage.
>>
>>
>> Thanks,
>>
> 

-- 
dharana


Mime
View raw message