quetz-mod_python-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Gardner <jgard...@smoothcorp.com>
Subject Re: [mod_python] Using shared memory to do global persistence
Date Fri, 27 Jun 2003 13:46:00 GMT
On Thursday 26 June 2003 16:58, Jack Diederich wrote:
> I think your use of dicts is dead on, something like:
> def do_calc():
>   cache = mod_python.shared
>   if ('foo' not in cache):
>     cache['foo'] = 'bar'
>   return cache['foo']
>
> As I mentioned during the long thread on what frameworks
> should do, we could add two dicts to mod_python,
> mod_python.page_cache # caching dict created per-request
> mod_python.perm_cache # cache valid accross all processes for ever
>

Caching pages in memory is not such a good idea. Depending on the operating 
system, you can cache pages in physical memory just by storing their contents 
to the disk. The OS is left to figure out what is the most effective use of 
resources. The algorithms that decide which files should be cached are going 
to be much smarter than any algorithm you and I could come up with. This is 
especially true because those algorithms are implemented at the kernel level, 
where some information about the operating system can be gathered and 
accessed far easier than out in user-space.

The permanent cache, however, is a good idea. But rather than hardcoding it 
into mod_python, I think we should leave it open as an option that is easily 
implemented. Certain database modules can cache the results of common queries 
that have small result sets in the same way that the OS caches files. You can 
put memoized functions into the cache as well, to save memory all across the 
board while tremendously boosting performance.

One more tremendous use would be shared modules, in the same way that mod_perl 
handles it. At the time of start up, mod_perl loads all the modules you 
specify in startup.pl, and these modules are shared among all the child 
processes. This saves a ton of memory, and increases performance in things 
like spawning new children.

-- 
Jonathan M. Gardner
Smooth Corporation - Perl Programmer
jgardner@smoothcorp.com - (425) 460-4780
Live Free, Use Linux!

Mime
View raw message