httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eike Frost ...@kefro.st>
Subject Re: [users@httpd] Reliable combined log sorting
Date Sun, 09 May 2010 15:36:25 GMT
Hi Jason,

On Sun, May 9, 2010 at 5:41 AM, Jason Nunnelley <jason@jasonn.com> wrote:

> In our configuration of Apache we push all logs to a single file via
> syslog-ng. I need to create some basic log analysis (like AWstats style) for
> a single domain out of several domains that post to that file. Any
> suggestions on a simple solution?
>

What LogFormat are you using, precisely ? And what, precisely, is the
problem you are having (you mention sorting in the subject, but nothing
related to it in the body).

If all you require is the ability to filter for a single vhost, have a look
at http://httpd.apache.org/docs/2.2/mod/mod_log_config.html, make sure your
LogFormat-string contains "%v" somewhere, and filter by the respective field
in the syslog-ng file -- which can trivially be done with awk or similar. If
your line looks something like this :

yourvhost.example.com 66.249.xx.xx - - [09/May/2010:15:25:56 +0000] "GET
/xxx HTTP/1.1" 200 8284 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +
http://www.google.com/bot.html)" 261 8619

due to a LogFormat of  "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\"
\"%{User-Agent}i\" %I %O" combined_2 (or something similar), then

awk '$1 ~ /yourvhost\.example\.com/ { print }' < logfilename

will filter out just those lines where the first field ($1) matches your
VirtualHost ServerName of yourvhost.example.com -- which you are then free
to pipe to whatever log analysis method you want. If you want this to also
filter out the syslog-related timestamps, awk can do that as well;
http://www.gnu.org/manual/gawk/gawk.html should provide sufficient detail on
how manipulate the print-statement.

If your problem is that the log entries are not in strictly chronological
order (which most decent log analysis software can deal with as long as the
delta is not too large), the problem becomes different beast altogether --
one which you'd likely want to employ a more sophisticated perl or python
script since you require more state -- code for which I don't have handy at
the moment ;-)

If this does not answer your question, please provide some more example log
and code.

--Eike

Mime
View raw message