logging-log4j-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Remko Popma <remko.po...@gmail.com>
Subject Re: why is AsyncLoggerContextSelector faster?
Date Mon, 21 Jul 2014 15:12:29 GMT
Mariano, to clarify, I would recommend that you keep your custom context
selector simple and only use it for the log separation that you mentioned.
I would not recommend that you also somehow try to merge the functionality
of AsyncLoggerContextSelector in your context selector. From a performance
perspective that should not be necessary. If you use <AsyncRoot> and
<AsyncLogger> in your configuration then I doubt that logging will be a
performance bottleneck.


On Mon, Jul 21, 2014 at 11:47 PM, Remko Popma <remko.popma@gmail.com> wrote:

> Matt, not sure what you mean. Mariano needs to develop his own
> ContextSelector to achieve log separation.
> So he cannot use the AsyncLoggerContextSelector.
>
> Mariano, I would not be overly concerned with the performance results
> mentioned on the async logger manual page. Bear in mind that in that
> performance test, _all_ that the program does is logging. In the real world
> (your application) that will obviously not be true: your application will
> do many other things.
>
> Also, mixed async loggers performed much, much better than AsyncAppenders.
>
> So, unless you have actual measurement numbers for _your_ application, not
> the theoretical numbers from the async logger manual page, I would keep it
> simple at first, just get your custom selector to work well, and then start
> measuring performance.
>
> If you are getting strange results, like AsyncAppenders performing better
> than AsyncLoggers, then let us know and we'll try to help figure out what
> the problem is.
>
>
>
> On Mon, Jul 21, 2014 at 11:38 PM, Matt Sicker <boards@gmail.com> wrote:
>
>> Well, the way to go that's similar to the async version is to use the
>> BasicContextSelector which contains a singleton LoggerContext. Otherwise,
>> you'll have to keep your own registry that can be looked up dynamically
>> such as through reflection of the caller stack and other fun design
>> problems. :)
>>
>>
>> On 21 July 2014 09:35, Mariano Gonzalez <mariano.l.gonzalez@gmail.com>
>> wrote:
>>
>> > Hello Remko,
>> >
>> > I'm still a couple of days away from starting my own performance
>> testing.
>> > I'm taking about the difference in the async loggers manual page, more
>> > specifically, the charts that compare sync loggers, to mixed async
>> loggers
>> > against purely async loggers. Since I need to build my own selector, I'm
>> > trying to be clear on how this works internally in order to implement
>> the
>> > less latency possible selector and try to minimize the performance
>> testing
>> > effort.
>> >
>> > Thanks for the clarifications!
>> >
>> >
>> >
>> > On Mon, Jul 21, 2014 at 11:22 AM, Remko Popma <remko.popma@gmail.com>
>> > wrote:
>> >
>> > > Hi,
>> > > No that is incorrect.
>> > > If you do not specify AsyncLoggerContextSelector but instead configure
>> > with
>> > > <AsyncRoot> and <AsyncLogger> loggers, you _do_ need the disruptor
>> jar on
>> > > the classpath and this does _not_ use AsyncAppender. AsyncAppender is
>> > > completely separate from Async Loggers. Async Loggers (mixed or all
>> > async)
>> > > use the disruptor and both need the disruptor jar.
>> > >
>> > > You keep mentioning a performance difference. I was assuming you were
>> > > talking about the performance test results mentioned on the Async
>> Logger
>> > > manual page, but perhaps I was wrong? Are you experiencing a
>> performance
>> > > difference between the two flavors of Async Loggers in your
>> application?
>> > >
>> > >
>> > >
>> > >
>> > > On Mon, Jul 21, 2014 at 11:10 PM, Mariano Gonzalez <
>> > > mariano.l.gonzalez@gmail.com> wrote:
>> > >
>> > > > Hello Remko,
>> > > >
>> > > > I think I found the difference. AsyncLoggerContextSelector always
>> > returns
>> > > > the same instance of AsyncLoggerContext, which in turns always
>> returns
>> > > > instances of AsyncLogger, which uses disruptor to handle
>> concurrency.
>> > > >
>> > > > However, with any other selector, a standard Logger instance is
>> > returned
>> > > > and parallelism is achieved through an AsyncAppender. AsyncAppender
>> > use a
>> > > > standard blocking queue instead of using disruptor which explains
>> the
>> > > > performance difference (there's also the fact that
>> > > > AsyncLoggerContextSelector always returns the same context instance
>> and
>> > > > does not spend cycles in the lookup, but I think that is not a
>> > > significant
>> > > > cost once everything was warmed out).
>> > > >
>> > > > http://logging.apache.org/log4j/2.x/manual/async.html says that
>> when
>> > > using
>> > > > mixed type loggers disruptor is needed on the classpath. That seems
>> to
>> > be
>> > > > an error. For what I see disruptor is only used when setting all
>> > loggers
>> > > > asynchronous.
>> > > >
>> > > > Does this make sense? Anyway around this? Do you have a disruptor
>> > > appender
>> > > > somewhere?
>> > > >
>> > > > Thanks!
>> > > >
>> > > >
>> > > > On Sat, Jul 19, 2014 at 11:55 PM, Remko Popma <
>> remko.popma@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > To be honest, I haven't investigated in detail the reason for
the
>> > > > > difference in throughput in the performance test.
>> > > > >
>> > > > > Are you measuring the performance of your application container,
>> and
>> > > can
>> > > > > you see an improvement when using Async Loggers?
>> > > > > Do you see a large difference in performance _in your application_
>> > > > between
>> > > > > making all loggers Asynchronous and using mixed synchronous and
>> > > > > Asynchronous loggers?
>> > > > >
>> > > > >
>> > > > > On Sun, Jul 20, 2014 at 7:45 AM, Mariano Gonzalez <
>> > > > > mariano.l.gonzalez@gmail.com> wrote:
>> > > > >
>> > > > > > Hello Remko,
>> > > > > >
>> > > > > > Thanks for the insight. I guess my case falls into the wrong
>> end of
>> > > the
>> > > > > > pareto law. My project is a low latency application container,
>> so I
>> > > > need
>> > > > > to
>> > > > > > have:
>> > > > > >
>> > > > > >
>> > > > > >    - low latency
>> > > > > >    - log separation (I actually had to implement my own
context
>> > > > selector
>> > > > > >    because my logic is more complicated than the standard
>> > > > > >    ClassLoaderContextSelector case)
>> > > > > >    - I want async loggers by default, but deployed apps
need to
>> be
>> > > able
>> > > > > to
>> > > > > >    specify sync loggers
>> > > > > >
>> > > > > >
>> > > > > > Right now I'm kinda meeting those requirements using config
>> file,
>> > > > > AsyncRoot
>> > > > > > and my custom selector, but it'd be really great to achieve
a
>> > > > performance
>> > > > > > level like the one that AsyncContextSelector promises.
>> > > > > >
>> > > > > > Is there a way of doing that? For what I see on the code,
the
>> > > > > > AsyncLoggerContextSelector's secret sauce is just to always
>> return
>> > an
>> > > > > > AsyncLogger on the newInstance() method. Why is that so
much
>> faster
>> > > > than
>> > > > > > what ClassLoaderLoggerContextSelector does?
>> > > > > >
>> > > > > > Thanks!
>> > > > > >
>> > > > > >
>> > > > > > On Fri, Jul 18, 2014 at 1:20 PM, Remko Popma <
>> > remko.popma@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > The Async Loggers created with the context selector
have a
>> > slightly
>> > > > > > > different mechanism. One of the differences is that
LogEvent
>> > > objects
>> > > > > are
>> > > > > > > re-used.
>> > > > > > >
>> > > > > > > However, unless your application is in the low-latency
space,
>> I
>> > > would
>> > > > > not
>> > > > > > > worry too much about the performance difference. Both
flavors
>> of
>> > > > Async
>> > > > > > > Loggers are much faster than the alternative (Async
>> Appenders).
>> > > > > > >
>> > > > > > > Your point is valid though. I've been thinking about
an
>> > alternative
>> > > > way
>> > > > > > to
>> > > > > > > configure Async Loggers than via system properties.
The work
>> in
>> > > > > progress
>> > > > > > is
>> > > > > > > tracked in Jira ticket LOG4J2-321
>> > > > > > > <https://issues.apache.org/jira/browse/LOG4J2-321>.
This is
>> > still
>> > > in
>> > > > > the
>> > > > > > > concept phase though. Meanwhile your best option is
probably
>> to
>> > use
>> > > > > > > ClassLoaderContextSelector and configure with <AsyncRoot>
and
>> > > > > > > <AsyncLogger>.
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > On Fri, Jul 18, 2014 at 10:57 PM, Mariano Gonzalez
<
>> > > > > > > mariano.l.gonzalez@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Hello,
>> > > > > > > >
>> > > > > > > > According to the performance charts in the documentation,
>> > log4j2
>> > > > has
>> > > > > a
>> > > > > > > > significantly higher throughput when using
>> > > > AsyncLoggerContextSelector
>> > > > > > > than
>> > > > > > > > when using all async loggers with any different
selector.
>> > > > > > > >
>> > > > > > > > Why is that? Is it just because the same context
is always
>> > reused
>> > > > and
>> > > > > > > > there's no lookup like in the ClassLoaderContextSelector
>> case?
>> > > > > > > >
>> > > > > > > > If I need functionality similar to
>> ClassLoaderContextSelector,
>> > is
>> > > > > there
>> > > > > > > any
>> > > > > > > > way to get a throughput similar to
>> AsyncLoggerContextSelector?
>> > > > > > > >
>> > > > > > > > Thanks!
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>>
>>
>> --
>> Matt Sicker <boards@gmail.com>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message