quetz-mod_python-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jack Diederich" <jack_dieder...@email.com>
Subject Re: Prefork import solutions/problems
Date Sun, 22 Sep 2002 19:26:14 GMT
Okay, I should have run my question through google first...
http://www.geocrawler.com/archives/3/192/1999/6/0/2385317/

you can never garuntee that python_init() will be called only once.
All memory allocated from the original pool is freed before the next call to python_init()
just having the single python_imports table seems to avoid this for some reason, but it isn't
garunteed (I don't want to know what happens on USR1).

we need a cleanup func that just resets the python_imports state between calls to python_init()

/* This func can go anywhere above directive_PythonImports */
void cleanup_python_imports(void *data)
{
  python_imports = NULL;
}
/* This line needs to go in directive_PythonImports in the if block where we call ap_make_table
*/
ap_register_cleanup(cmd->pool, NULL, cleanup_python_preforkimports, ap_null_cleanup);


The end result is that we remake the table if python_init() is called more than once.  This
is OK because the configuration is also reread.  We don't have to worry about the global 'interpreters'
variable because it is allocated by Python outside of apache's memory management, but we might
have a small memory leak if we get USR1'd over and over again because it is never Py_DECREF'd
in python_finalize.  On the other hand, Py_Finalize() is probably smart enough to take of
that for us.

-jack
----- Original Message -----
From: "Jack Diederich" <jack_diederich@email.com>
Date: Sun, 22 Sep 2002 13:52:40 -0500


> Okay, I've figured out the symptoms, but not the problem.
> 
> If a PreforkImport directive is specified, python_init() ends up being called _TWICE_
> The same code without a PreforkImport directive in httpd.conf will only call python_init()
_ONCE_
> The two globals we allocated in the first pass have been freed by the second pass and
we segfault when we try to read them.
> 
> Something is telling apache to try again, I don't know what.  We do more ap_* alloc'ing
on the server pool if a PreforkImport is specified, mainly through string manipulations in
get_interpreter_name() as well as one call to python_create_dir_config().
> 
> Bad work arounds for this are to NULL our two globals after we read them, they will be
realloced on the secdond pass and do everything again.  But on the second pass apache seems
happy.
> We could also use malloc() instead of the ap_* pool access routines.
> Both of these are hackish, but work.
> 
> Anyone know what we do that makes apache do python_init() twice?
> 
> additional info, in the case of the twice-called python_init(), the second time is with
a PID that will become the real process group leader.  The process from the first call is
discarded somewhere along the line.  
> 
> -jack
> 
> ----- Original Message -----
> From: "Jack Diederich" <jack_diederich@email.com>
> Date: Sat, 21 Sep 2002 16:34:25 -0500
> 
> > Below is a patch I cooked up in a few hours, it cores 99% of the time.
> > the global static python_imports gets corrupted.  These are created via ap_make_table
with the server mem pool.  They shouldn't get freed until server exit time .. but they do.
 Specifically the memory is freed sometime after the parent inits the tables and the children
try to read them.  Creating these tables with malloc() makes this work 100% of the time, but
then that [small] amount of memory only gets cleaned up at shutdown.  For some reason apache
cleans up the memory when we make two tables, but not for one (the existing python_imports).
> 
> -- 
> __________________________________________________________
> Sign-up for your own FREE Personalized E-mail at Mail.com
> http://www.mail.com/?sr=signup
> 
> 

-- 
__________________________________________________________
Sign-up for your own FREE Personalized E-mail at Mail.com
http://www.mail.com/?sr=signup


Mime
View raw message