logging-log4j-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Muller <...@online.no>
Subject RE: Dual logging problem - outofmemoryerror... heap space
Date Wed, 08 Oct 2008 21:21:13 GMT
... and here is the implementation as as well ;)

--

Thomas 


| -----Original Message-----
| From: Thomas Muller [mailto:ttm@online.no] 
| Sent: 08 October 2008 19:07
| To: 'Log4J Users List'
| Cc: 'Log4J Developers List'
| Subject: RE: Dual logging problem - outofmemoryerror... heap space
| 
| Hi,
| 
| I think the functionality you're after is interesting and 
| useful in a Log4j toolbox - hence I added it to ours.
| 
| My solution (IMHO) offers a much more generic solution to the 
| problem. Instead of "spying" on the Logger, I've implemented 
| an Appender which again implements the AppenderAttachable 
| interface, i.e. it stands between a Logger (the source) and 
| the "real" appender(s) (the sink). The ErrorAppender 
| implements (I believe) exactly the functionality you're after 
| - it uses a rotating buffer and is triggered on a 
| configurable log event priority (level).
| 
| This solution is non-intrusive on existing code - you just 
| log as usual, no extra/artificial calls to an "ErrorLogger" instance.
| 
| You can compare to the AsyncAppender which is also included 
| with Log4j - this also buffers and distributes log-events to 
| underlying appenders, but the functionality you get here is 
| that the thread actually sending the log statements to the 
| appenders (sinks) is not the same as the one logging, i.e. 
| you speed up logging at the expense of higher memory usage. 
| 
| You could combine AsyncAppender and ErrorAppender for this 
| purpose, just use an AsyncAppender as your main-appender and 
| ErrorAppender as one of the underlying appenders (using 
| appender-ref in the log4j.xml file). If you use the 
| ErrorAppender directly on a Logger you take a hit everytime a 
| log-statement has pri > trigger-pri: the buffer will be 
| drained to all the underlying appenders you might have added 
| (which again logs to files, syslog, a database, by 
| pigeon-post, whatever) - even worse - it's synchronized based 
| on the buffer itself, so all other Loggers using the same 
| appender will block on calling the doAppend method until the 
| draining/distribution procedure is done. You are warned.
| 
| ErrorAppender might not be the best name - TriggerAppender is 
| maybe better?
| WARNING: Only basic-tested.
| 
| FYI: I've cc'ed the log4j-dev list in case they want this 
| Appender in an official release. Use and abuse.
| 
| NOTE: All the annotations can safely be removed if you don't 
| have the JSR 305 release in your classpath.
| 
| 
| --
| 
| Thomas
| 
|  
| 
|  
| 
| | -----Original Message-----
| | From: Johan Sandgren [mailto:jsa@svep.se]
| | Sent: 08 October 2008 10:20
| | To: log4j-user@logging.apache.org
| | Subject: Dual logging problem - outofmemoryerror... heap space
| | 
| |  
| | 
| | Hi,
| | 
| |  
| | 
| | Need advice from professionals here. I'm stuck! L
| | 
| |  
| | 
| | I'm using log4j, and my logs are rotating quite fast, so if 
| an error 
| | occurs, for example in the middle of the night, it will not 
| be present 
| | in the log in the morning when I arrive to check for errors.
| | 
| |  
| | 
| | So I decided to extract the errors live, when they occur, and make 
| | them go to another file.
| | 
| | But since errors just on their own does not say much of where the 
| | program were and what was the variables right then, I'd 
| also like to 
| | attach some history before the error in the error.log-file.
| | 
| |  
| | 
| | So I wrote a class to support this feature. It's a spy really. It's 
| | attaching to an existing logger, and adding another 
| appender, which I 
| | use to create history and outputting to another 
| error.log-file when I 
| | need to.
| | 
| |  
| | 
| |  
| | 
| | A little more details on the spyclass are:
| | 
| |  
| | 
| | I'm calling my usual logger X, and I've created another 
| loggerclass Y, 
| | which attaches another appender on my X.
| | 
| |  
| | 
| | For each logger call with X, I also call my Y which is getting the 
| | string too, because it's snooping on X, via this appender I 
| added. Y 
| | is also keeping a circularbuffer of history of all logging it gets.
| | 
| | When Y is called with error, fatal or warn, it will do a 
| dump of the 
| | history and lastly, the erranous line, and all this to 
| another file, 
| | error.log.
| | 
| |  
| | 
| | That way I will get my usual log in blabla.log, and for each warn, 
| | error /fatal, I get a few lines of history + the 
| error/warn/fatal-line 
| | that is to be noticed.
| | 
| |  
| | 
| |  
| | 
| | My problem:
| | 
| | BUT, I get out of heapmemory when using it after a while.
| | 
| | I cannot find the error causing it. Mayber I'm misusing the 
| appender 
| | or log4j?
| | 
| |  
| | 
| | I've attached the historymaking class "ErrorLogger.java", and an 
| | example (test.java) of how the calls are made.
| | 
| |  
| | 
| |  
| | 
| |  
| | 
| | Exception in thread "Thread-5" java.lang.OutOfMemoryError: 
| | Java heap space
| | 
| |         at
| | java.lang.AbstractStringBuilder.expandCapacity(AbstractStringB
| | uilder.java:99)
| | 
| |         at
| | 
| java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:393)
| | 
| |         at java.lang.StringBuffer.append(StringBuffer.java:225)
| | 
| |         at java.io.StringWriter.write(StringWriter.java:79)
| | 
| |         at
| | org.apache.log4j.helpers.QuietWriter.write(QuietWriter.java:47)
| | 
| |         at
| | org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:301)
| | 
| |         at
| | org.apache.log4j.WriterAppender.append(WriterAppender.java:159)
| | 
| |         at
| | 
| org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:230)
| | 
| |         at
| | org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAp
| | penders(AppenderAttachableImpl.java:65)
| | 
| |         at 
| org.apache.log4j.Category.callAppenders(Category.java:203)
| | 
| |         at org.apache.log4j.Category.forcedLog(Category.java:388)
| | 
| |         at org.apache.log4j.Category.debug(Category.java:257)
| | 
| |  
| | 
| |         at
| | com.sttcare.alarmSwitch.JmsTester.activeMqIsUp(JmsTester.java:
| | 88) calls connection.stop(); So it really has nothing with log4j, I 
| | guess the heap is just ended right then.
| | 
| |  
| | 
| |         at
| | com.sttcare.alarmSwitch.systemWatch.ProgramChecker.activeMQWor
| | ks(ProgramChecker.java:143)
| | 
| |         at
| | com.sttcare.alarmSwitch.systemWatch.ProgramChecker.run(Program
| | Checker.java:119)
| | 
| |         at java.lang.Thread.run(Thread.java:595)
| | 
| |  
| | 
| |  
| | 
| | Seems to me something is exandCapacity too much.
| | 
| |  
| | 
| | /Johan
| | 
| |  
| | 
| | _______________
| | 
| | Johan Sandgren    
| | 
| | Svep Design Center AB                                
| | 
| | Phone +46 46 192 722
| | 
| | Mobile +46 70 173 4152
| | 
| | Box 1233, 221 05 Lund, Sweden                                  
| | 
| | E-mail   jsa@svep.se
| | 
| | Website www.svep.se <http://www.svep.se/>
| | 
| |  
| | 
| | 
| 


Mime
View raw message