www-apache-bugdb mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lenny Foner <fo...@media.mit.edu>
Subject general/4809: It would be nice if Apache could write compressed logfiles.
Date Tue, 03 Aug 1999 18:02:55 GMT

>Number:         4809
>Category:       general
>Synopsis:       It would be nice if Apache could write compressed logfiles.
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    apache
>State:          open
>Class:          change-request
>Submitter-Id:   apache
>Arrival-Date:   Tue Aug  3 11:10:01 PDT 1999
>Originator:     foner@media.mit.edu
>Release:        1.3
Currently HPUX 10.20/gcc, but this is an
architecture-independent feature request.
(I currently administrate an Apache server
that serves about 1/3 of the Media Lab.)
Apache currently writes uncompressed logs everywhere, which take up
lots of room and are much slower to serve over NFS than compressed logs
would be.  We routinely compress our logs when they are rotated, and
searching them with gzip can be 10x faster because gunzip on a modern
CPU can be far faster than NFS on most LANs---the fact that so much less
data need cross the network more than makes up for the overhead of doing
the decompression.  Further, the compression saves a tremendous amount
of space---most logfiles are highly compressible.  Keeping them compressed
while written means that sites would not have to have enough logfile space
for the log before it gets rotated and -then- compressed, for example.
Look at any Apache's logfiles.
Build the gzip library into Apache; it's already available in autoconf'ed
form and so forth and is used in many GNU projects.  (You should of course
check, but I believe it's covered by the LGPL, which should mean you can
use it without fear of infection even though Apache itself doesn't appear
to be GPL'ed.)

Given that, it should be a relatively simple matter to stream logging output
into the compressor subroutines, and then write -those- to disk.  You'd have
to provide configuration options that tell whether to do this on a per-logfile
(probably) basis, and probably also the compression level desired.  Webmasters
would also need to understand that the buffering imposed by the compression
library means that the latency between a request and seeing it in the compressed
output would be significantly larger than with uncompressed logs, of course.

I've used the gzip libraries in projects of mine and could conceivably offer
some help, but I have -not- looked at the innards of the Apache logging code,
so that help might be somewhat limited.  It's an easy library to use, however.
[In order for any reply to be added to the PR database, you need]
[to include <apbugs@Apache.Org> in the Cc line and make sure the]
[subject line starts with the report component and number, with ]
[or without any 'Re:' prefixes (such as "general/1098:" or      ]
["Re: general/1098:").  If the subject doesn't match this       ]
[pattern, your message will be misfiled and ignored.  The       ]
["apbugs" address is not added to the Cc line of messages from  ]
[the database automatically because of the potential for mail   ]
[loops.  If you do not include this Cc, your reply may be ig-   ]
[nored unless you are responding to an explicit request from a  ]
[developer.  Reply only with text; DO NOT SEND ATTACHMENTS!     ]

View raw message