logging-log4j-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ramakrishna menon <rmenon...@gmail.com>
Subject Re: Creating a log4j appender that helps in "batching up" messages before logging into the file
Date Fri, 26 Feb 2010 14:32:23 GMT
Thank you, Mike
Yes - the OS is probably buffering but as I mentioned I did see
difference in my quick tests (just pure timing based.) I will check
out the setBuffered option you mention in the next email and would
also perhaps write my own appender and see whether it performs better
in an actual environment. I did check out the source code before
posting to this thread - just wanted to see if others have encountered
this issue and their solutions.

Best Regards
Menon
-----------------------------------------------------------
R. M. Menon - A Rafi fan(www.mohdrafi.com)
Author, Expert Oracle JDBC Programming,
http://www.amazon.com/exec/obidos/tg/detail/-/159059407X/
-----------------------------------------------------------



On Fri, Feb 26, 2010 at 1:09 AM, Michael Erskine <msemtd@googlemail.com> wrote:
> On 26 February 2010 01:13, ramakrishna menon <rmenon.us@gmail.com> wrote:
>> We use log4j extensively in our application and need to reduce the
>> overhead of log4j writes into file. We can trade off memory at this
>> stage for cpu cycles. One way I was thinking of was that we log into
>> memory for a given "batch" of messages and then we flush this memory
>> structure when we hit a configured batch size where we write all the
>> messages in one shot into the file.
>
> Surely your operating system should be buffering filesystem writes for you!
>
> Seriously though, you wouldn't be trading off memory for CPU cycles:
> you'd be deferring the writes until a more convenient time and most
> probably using additional CPU cycles as a result!
>
> If you have performance issues in your application then I'd suggest
> you address them directly. If logging is slowing your application then
> switch it off. If you require an audit trail, however, then basic
> Log4j is probably not the right way to go: at the very least, write a
> custom appender that does _exactly_ what you need. AppenderSkeleton is
> a good starting point: study the code of the other appenders to see
> how they achieve what they need.
>
> I have written quite a few custom appenders that operate under heavy
> load in heavily multithreaded systems and they all follow the same
> pattern: quickly append a lightweight representation of the
> LoggingEvent* to a java.util.Deque<> and perform writes (to file,
> JDBC, socket, etc.) in a daemon thread.
>
> [*] not the LoggingEvent itself - it has excess baggage!
>
> I recommend the book Java Performance Tuning by Jack Shirazi published
> by O'Reilly. Not just for Java performance geeks but for ALL Java
> developers! And ALWAYS read the code of any libraries you're thinking
> of using!
>
> Regards,
> Michael Erskine
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org


Mime
View raw message