logging-log4j-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bruno Melloni" <Bruno.Mell...@wnco.com>
Subject RE: New Appender: PatternFileAppender
Date Wed, 03 Sep 2008 19:12:04 GMT
Seems that several of us have been looking at similar things.  

I looked at the MultiFile appender but it wasn't quite flexible enough,
so I created another appender similar to this
(DynamicTimestampedFileAppender)  and was debating whether to submit it.
It is complete but has only been in use for under a week.  Please let me
know if I should submit it, or perhaps the three appenders ought to be
compared and eventually create a hybrid appender that replaces them all.

Key features:

- Designed as a full lifecycle file appender.  
- Instead of sequence numbers uses timestamps of file creation (much
easier to find entries).  Rollovers are really nothing more than closing
a file and creating a new one.
- Zips and deletes old files.  Zips and deletes happen at night and
server start (minimizes load when server is running).
- Multiple logging file sets are dynamically generated.
- Has built-in variables that can be used in defining the file names.
- Additional variables can be defined by parsing the message through
regular expressions.
- Has a maximum file size, but it can be overridden with a minimum
amount of time between rollovers (i.e.: 1 hour).
- Extra rollovers occur at midnight and at a specified time (i.e.:
3:00am), allowing for clear separation of logs by 'operating window'.

log4j.properties instructions and sample setup data:

#   basePath            = The base folder for the server log
(./log/server)
#   parseExpression     = A RegularExpression to match.  Each group will
become a variable {%i},
#                         with i=1,2,... in the order shown in the
regex, with its value parsed
#                         from the message.  Currently
Station:ClientPC:TransactionID.  Only
#                         the station is being used, the other variables
in the expression 
#                         help prevent mismatches.
#   filespecFormat      = Format to use for the path and file name.  
#           {%T}        = Timestamp is required somewhere IN the file
name.
#           {%d}        = Current Day.  Optional.  May be in path, file
name, or nowhere.
#           {%s}        = Server Name.  Optional.  Where this appender
is running.
#                         May be in path, file name, or nowhere.
#           {%i}        = Where i is a variable number from the
parseExpression.  Optional.
#                         May be in path, file name, or nowhere.
#   defaultFilespec     = Format to use for the path and file name when
parsing fails to find a match.
#                         For example, when there is no Station.
#                         It MUST include {%T} IN the file name.  It may
not have any {%i} values.
#                         {%d} and {%s} are allowed as in
filespecFormat.
#   maxDesiredSize      = Maximum size desired for individual log files.
I.e.: 1GB, 2MB or 4KB.
#                         It may be ignored based on minElapsedMinutes.
#   minElapsedMinutes   = A minimum time that must pass before rolling,
even if the maxDesiredSize has
#                         been reached.  This will prevent excessive
rolling when volume/hr is high.
#                         To ignore it, set it to a low value (i.e.: 1).
The default limits rolls to
#                         at most once an hour, or less, depending on
the maxDesiredSize.
#   forceRollTime       = A roll is forced on all open files at the time
specified.
#   zipAfterDays        = Zip all files that are older than this number
of days.
#   disposeAfterDays    = Delete all files that are older than this
number of days.
#
# RELEVANT FEATURES:
#   - There will be a roll of all open files at 00:00:00 every day, and
possibly one when a file
#     is being opened for the first time in the day if it had not
previously rolled.
#   - zipping and deleting of old files happens during the daily roll.
#   - Logs are created as .txt files.  After zipping they are converted
to .zip.
# LIMITATIONS (intentional):
#   Currently using lazy algorithms that have higher performance but may
not trigger a roll until
#   later than normally expected:  
#     - Elapsed time is being measured since start or last roll.
#     - Only open files are rolled.
log4j.appender.LOGFILE=com...logging.DynamicTimestampedFileAppender
log4j.appender.LOGFILE.layout=com...logging.PCIPatternLayout
log4j.appender.LOGFILE.layout.ConversionPattern=%d{yyyy-MM-dd
HH:mm:ss,SSS} [%t] [%c] %-5p - %m%n
log4j.appender.LOGFILE.basePath=${serverLog}
log4j.appender.LOGFILE.parseExpression=^(\\w{3}):([\\w\\-]{1,}):([\\d_]{
1,})
log4j.appender.LOGFILE.filespecFormat={%d}/server_{%1}_{%T}.{%s}
log4j.appender.LOGFILE.defaultFilespec={%d}/default.{%T}.{%s}
log4j.appender.LOGFILE.maxDesiredSize=2MB
log4j.appender.LOGFILE.minElapsedMinutes=60
log4j.appender.LOGFILE.forceRollTime=03:00
log4j.appender.LOGFILE.zipAfterDays=1
log4j.appender.LOGFILE.disposeAfterDays=6

Note: My appender does not limit the number of concurrent open files.
May I ask what is the limit in Unix/Linux if anyone knows?

bruno

-----Original Message-----
From: David Britton [mailto:dpb@hp.com] 
Sent: Tuesday, September 02, 2008 4:33 PM
To: Log4J Users List
Subject: Re: New Appender: PatternFileAppender

On Tue, Sep 02, 2008 at 09:04:30PM +0000, Curt Arnold wrote:
> 
> 
> Could you compare it with the MultiFileAppender that has been 
> languishing in the sandbox for a while?  Search the archives for 
> MultiFileAppender.  Source code at 
> http://svn.apache.org/repos/asf/logging/sandbox/log4j/multifile .

Hi Curt --

Glad I asked!  It looks to be nearly identical in base functionality,
but of course has a much more developed interface, and has many more
options to it.

It looks like it has two main "policies", a Pattern policy and a Size
policy.

The "Size" policy looks like it provides duplicate functionality that
the RollingFileAppender provides.  It increments an index appended to
the file after they get to a certain size.  I may be over simplifying
here -- but I think that RollingFileAppender provides this and more.

The "Pattern" policy is what my appender was supposed to solve, and I
think is what is "missing" from log4j right now.  Just from inspection,
it appears as though it would be slightly more complex than my
PatternFileAppender, but has more configuration options, and of course
is 1.2 compatible. :)


Is there a reason this has not made it into the official distribution?

--
David Britton <dpb@hp.com>

---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org


Mime
View raw message