logging-log4j-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sebastien Tardif" <SebTar...@ncf.ca>
Subject RE: Re: RE: Re: performance problem in multithread environment
Date Fri, 17 Dec 2010 13:20:10 GMT
The only improvement that can be added beside what everybody mentioned is to
improve the locking mechanism in Category to use instead a ReadWriteLock
<http://download.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks
/ReadWriteLock.html> . However, it’s hard to justify the change. It should
be a bottleneck only in an application logging like 50+ log statements per
second, which is likely only happening when debugging, which by definition
do not care much about performance.

 

In your real application, you likely do one of the following:

-          Your application do almost nothing else than logging

-          Your application do not use async appender and/or buffering

>From the information you provided it seems you execute on a single threaded
machine, something like 100 threads, since we see in your screen shot
“http8080-Processor99”. It’s not the optimum configuration to get the
best throughput. To give you an example, if you use asychnonous socket IO,
and you estimate that 10% of the time your application is waiting for IO
(disk, socket), so mainly CPU bound, then you just need 2 threads, then
obviously you will have lot less synchronization contention.

 

 

 

From: 刘东 [mailto:betterjonah@hotmail.com] 
Sent: Friday, December 17, 2010 1:40 AM
To: Log4J Users List
Subject: Re: Re: RE: Re: performance problem in multithread environment

 

Hi,I use jprofiler to monitor appliaction,the result shows that,at the same
time only on thread is runing,the others are blocked,why? 

pls see attachment,thanks.

 

 

 

2010-12-17 

  _____  

刘东 

  _____  

发件人: 刘东 

发送时间: 2010-12-17  10:51:48 

收件人: Log4J Users List 

抄送: 

主题: Re: RE: Re: performance problem in multithread environment 

Hi,thank you very much.

I test asyncappender again,the result is as follows:

thread number |time cost |asyn BufferSize

1                   |28282     |1024

2                   |56359     |1024

4                   |107188   |1024

16                  |421547   |1024

my log4j.xml is:

<?xml version="1.0" encoding="UTF-8" ?>

<!DOCTYPE log4j:configuration PUBLIC "-//log4j/log4j Configuration//EN"
"log4j.dtd">

<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">

<appender name="file-log" class="org.apache.log4j.RollingFileAppender">

<param name="bufferSize" value="262144" />

<param name="bufferedIO" value="true" />

<!-- <param name="fileAppend" value="true" /> -->

<param name="File" value="log4jtest.log" />

<layout class="org.apache.log4j.PatternLayout">

    <param name="ConversionPattern" value="%-d{yyyy-MM-dd HH:mm:ss} [%-5p]
[%t] [%c.%M:%L] - %m%n" />

</layout>

</appender>

<appender name="async" class="org.apache.log4j.AsyncAppender">

<param name="BufferSize" value="1024" />

<appender-ref ref="file-log" />

</appender>

<!-- for all other loggers log only debug and above log messages -->

<root>

<priority value="DEBUG" />

<!-- <appender-ref ref="STDOUT" /> -->

<!-- <appender-ref ref="file-log" /> -->

<appender-ref ref="async" />

</root>

</log4j:configuration>

I turned asyn BufferSize to 10240,but the test result is no changing.

why?

2010-12-17 

刘东 

发件人: Sebastien Tardif 

发送时间: 2010-12-17  02:32:38 

收件人: 'Log4J Users List' 

抄送: 

主题: RE: Re: performance problem in multithread environment 

 

File IO is slow, you need to have a asyncappender between your application

and file i/o.

-----Original Message-----

From: 刘东 [mailto:betterjonah@hotmail.com] 

Sent: Thursday, December 16, 2010 10:02 AM

To: Log4J Users List

Subject: Re: Re: performance problem in multithread environment

tanks for you reply.

I write test program to show my  appliaction case.

package com.iss.cnooc.test.ebank;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

public class LoggerTest

{

    Logger logger = LoggerFactory.getLogger(LoggerTest.class);

    

    public static void main(String[] args)

    {

        int i = 0;

        

        while (i < 2)

        {

            LoggerThread t = new LoggerThread();

            t.start();

            i++;

        }

        

    }

}

package com.iss.cnooc.test.ebank;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

public class LoggerThread extends Thread

{

    Logger logger = LoggerFactory.getLogger(LoggerThread.class);

    

    @Override

    public void run()

    {

        int i = 0;

        long start = System.currentTimeMillis();

        while (i < 1000000)

        {

            logger.debug("i = " + i);            

            i++;

        }

        

        long ecl = System.currentTimeMillis() - start;

        

        System.out.println(this.getName() + "-" + this.getId() + " spend "

                + ecl);

    }

}

The test reslut is

thread number      |     cost

1                        |     46859 

2                        |     96593 

4                        |     196921 

16                      |      717703 

I use fileappender.

why?

2010-12-16 

刘东 

发件人: Curt Arnold 

发送时间: 2010-12-16  14:15:03 

收件人: Log4J Users List 

抄送: 

主题: Re: performance problem in multithread environment 

On Dec 13, 2010, at 10:54 AM, Jacob Kjome wrote:

> You will get better performance by not logging at all, no question.

However, there are tuning possibilities.  I see you mention "logger.info()".

In production, I generally only have "warn()" and above for the vast

majority of loggers.  In fact, I configure the root logger up with the

"WARN" level and selectively set other loggers to something less than WARN

(if need be).

> 

> The other thing you might look at is whether you are concatenating strings

in your logging statements.  For instance the following will incur an

unnecessary cost in concatenating strings even when the "DEBUG" level is not

enabled....

> 

> logger.debug("product: " + someProduct + ", price: " + somePrice);

> 

> A more efficient way to define this in your code is....

> 

> if (logger.isDebugEnabled()) {

>     logger.debug("product: " + someProduct + ", price: " + somePrice);

> }

> 

alternatively use LogMF or LogSF (in the extras companion or in the SVN

HEAD)

LogMF.debug(logger, "product: {0}, price: {1}", price,somePrice);

will have performance generally comparable to using logger.isDebugEnabled

when logging is disabled since any conversion and concatenation is deferred

until after the logging level is checked.

---------------------------------------------------------------------

To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org

For additional commands, e-mail: log4j-user-help@logging.apache.org

---------------------------------------------------------------------

To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org

For additional commands, e-mail: log4j-user-help@logging.apache.org


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message