quetz-mod_python-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jack Diederich" <jack_dieder...@email.com>
Subject Re: Initializing the interpreter in python_init() instead of python_handler()
Date Mon, 16 Sep 2002 20:52:27 GMT
> On Mon, 16 Sep 2002, Jack Diederich wrote:
> > I just threw some of the python_handler() code into python_init() and it works!
> >
> > This means that it is possible to do all the imports etc only once,
> > instead of once per child.
[Grisha's Reply]
> The main the problem with this is that python_init(), which executes as a
> post_config hook, runs as root. Technically speaking this is MPM-specific,
> but at least looking at prefork, the sequence of events is:
> 	post_config
> 	...
> 	fork
> 	setuid
> 	run_child_init
> So child_init is the earliest non-root execution.
> Yes, you can import things once, but do you want to run a risk of some
> Python script reeking havoc on your system as root?

I'm willing to take that risk because I wrote the code :)
But more seriously, It is worth it for me because of the speedup.  I let mod_python processes
handle a finite number of hits before they are terminated and respawned, and with the initialization
being done in child_init() this means the new fork needs to import and parse ten thousand
plus lines of python code and run the main setup in each module to boot.  So when a new child
is forked, the response time for the process exceeds my personal goal of < 1/2 a second
because 1/2 second is added just via the importing.  The server will be getting potentially
hundreds of thousands of hits a day, so every little bit helps. (btw, that is half a second
extra on a dual AthlonMP 2000+, that is alot of extra cycles!)

That is my personal problem, here is a more general one.
OSSPMM http://www.ossp.org/pkg/lib/mm/ is a cross platform shared memory library.
I would like to write a python wrapper for it, but in order to be cross platform the author
had to add the limitation that the shared memory segment be created _before_ process forking.
 This would mean in python_init() instead of child_init().

Most people don't need this feature to run a wiki, but there are some that might need it for
other reasons.

I would suggest adding a ./configure option --enable-importprefork
or similar that would just include the code in python_init() to do imports, etc.
Then add config handlers to the apache module that specifies imports at python_init() time
instead of child init time to remove any confusion.  configure could even spit out 'THIS OPTION
IS EVIL, PROVE YOU NEED IT TO THREE OTHER PEOPLE' when the option is specified as a reminder.

Admins who want the PythonImportPrefork 'modulefoo, modulebar' to be legal would have to specifically
compile mod_python with the --enable-importprefork option.  Everyone else would get the default
and more secure method.

This of course requires that someone reshuffle the code in child_init so it can also be called
from python_init.  And test it, and document it, and stuff.

I won't be that guy in the near future, but I'd like to know if other people on python-dev
even like the idea.



Sign-up for your own FREE Personalized E-mail at Mail.com

View raw message