logging-log4j-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Remko Popma <remko.po...@gmail.com>
Subject Re: log4j2 + FastFileAppender + Tomcat logrotate problem
Date Sun, 01 Sep 2013 06:32:57 GMT
Kamil,

Did you have a chance to look at the comments on
https://issues.apache.org/jira/browse/LOG4J2-354 ?
Essentially I would recommend you use the log4j built-in mechanism for
rolling over log files instead of an external mechanism. Would that work
for you?

-Remko


On Fri, Aug 16, 2013 at 11:17 AM, Remko Popma <remko.popma@gmail.com> wrote:

> I raised https://issues.apache.org/jira/browse/LOG4J2-354 to track this
> issue.
> I'll add questions in the comments of that JIRA ticket.
>
> Remko
>
>
> On Fri, Aug 16, 2013 at 7:17 AM, Kamil Mroczek <kamil@thinknear.com>wrote:
>
>> Hello,
>>
>> We decided to try out log4j2-beta8 to see if we could improve our
>> logging performance.  We were testing with the disruptor 3.1.1.
>> library to make all our appenders async.
>>
>>
>> -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector
>>
>> We are running tomcat 7 with Java 1.6.  We use the SLF4J library as well.
>>
>> The appender that we were using in this case was the Fast File
>> Appender with a definition like:
>>
>> <FastFile name="RequestLog" fileName="requests.log"
>> immediateFlush="false" append="true">
>>   <PatternLayout>
>>     <pattern>%m%n</pattern>
>>   </PatternLayout>
>> </FastFile>
>>
>> And logger was..
>>
>> <logger name="a.namespace" level="info" additivity="false">
>>   <appender-ref ref="RequestLog"/>
>> </logger>
>>
>> So the system was designed to allow log4j to do the logging and then
>> have logrotate rotate the log files from the host to an external
>> destination.
>>
>> We rotate the logs every 5 minutes with these params (with LZO
>> compression).
>>
>> compress
>> compresscmd /bin/lzop
>> compressoptions "-U"
>> compressext .lzo
>>
>> What we were seeing was that after a log rotation happened the new
>> file would start with a massive chunk of binary data at the start.
>> Many times on the order of 100-200MB.  This would turn the logs from
>> being on the order of 50-100MB to 200-350MB.
>>
>> My guess was that it had something to do with the byte buffer flushing
>> mid-rotate since these chunks always come at the start of the file.
>> But I also saw LOG4J2-295 (Fast(Rolling)FileAppender now correctly
>> handles messages exceeding the buffer size. ) which was fixed in beta8
>> which my discredit that idea.
>>
>> We were able to fix the issue by using the regular FileAppender like this:
>>
>> <File name="RequestLog" fileName="requests.log" immediateFlush="true"
>> append="true" bufferedIO="false">
>>       <PatternLayout>
>>         <pattern>%m%n</pattern>
>>       </PatternLayout>
>>   </File>
>>
>> I can't remember for certain, but I am pretty sure that even if we had
>> bufferedIO="true" on the FileAppender everything worked okay as well.
>>
>> We could reproduce it pretty consistently.  I wanted to post to the
>> group to see if anyone has seen anything like this before.  Any ideas
>> on what the issue could be?
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>> For additional commands, e-mail: log4j-user-help@logging.apache.org
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message