quetz-mod_python-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Gallacher <...@jgassociates.ca>
Subject Re: Apache log messages
Date Sun, 10 Dec 2006 17:49:40 GMT
Eric Brunson wrote:
> Platform: FreeBSD 6.1, Apache 2.0.55 (mpm_prefork), Python 2.4.3,
> mod_python 3.3 (20061130232253 snapshot).
> 
> Well, I see the problem is that httpd is getting started up close enough
> to the same time every boot cycle that the worker processes are getting
> the same semaphore file names.  For some reason FreeBSD doesn't clean
> out /tmp on boot by default.  I can't seem to find any cleanup mechanism
> for the files on shutdown, but the 80 or so mutex files seem to
> correspond to server crashes 

Yep. As a rule that is were the mutexes get consumed. I'm sure I don't 
need to tell you the ultimate solution is to figure out why apache is 
crashing and make it stop. :)

My notes on this issue follow. They apply to Linux and may not be of any 
use on FreeBSD.
-------------------------------------------------------------------------

Under some conditions Apache may crap out and the semaphores it reserved 
for the mod_python mutexes may not have been released. This may happen 
if it does not cleanly shutdown due to a hung mod_python instance.

Each time apache is restarted it will grab additional semaphores until 
the server is starved for this resource.

Check the number of semaphores available:

# sysctl kernel.sem
kernel.sem = 250        32000   32      128


Get information about the semaphores
# ipcs -s

------ Semaphore Arrays --------
key        semid      owner      perms      nsems
0x0052e2c1 0          postgres  600        17
0x0052e2c2 32769      postgres  600        17
0x0052e2c3 65538      postgres  600        17
0x0052e2c4 98307      postgres  600        17
0x0052e2c5 131076     postgres  600        17
0x0052e2c6 163845     postgres  600        17
0x0052e2c7 196614     postgres  600        17
0x00000000 3145735    www-data  600        1
0x00000000 3178504    www-data  600        1
0x00000000 3211273    www-data  600        1
0x00000000 3244042    www-data  600        1
0x00000000 3276811    www-data  600        1
0x00000000 3309580    www-data  600        1
0x00000000 3342349    www-data  600        1
0x00000000 3375118    www-data  600        1
0x00000000 3407887    www-data  600        1
0x00000000 3440656    www-data  600        1


The solution is to stop apache and clean out the dead semaphores, 
substituting the the correct apache user name for apache-user. Make sure 
apache is stopped before running this - we only want to remove the dead 
semaphores. On debian apache is run as the "www-data" user.

~localhost# ipcs -s | awk '/www-data/ { system("ipcrm sem " $2) }'

Another one liner using perl to do the cleanup:

~localhost# ipcs -s | perl -ane '/^0x00000000/ && `ipcrm -s $F[1]`'

Jim

Mime
View raw message