logging-log4j-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ralph Goers <ralph.go...@dslextreme.com>
Subject Re: MessageLayout
Date Tue, 04 Apr 2017 05:11:06 GMT
No. JMS does not guarantee delivery in the way your would want. Sure, if the JMS service says
it accepted the event then it is guaranteed, but if it doesn’t what are you going to do
with the event? It is stuck in the client. You would have to invent some way to queue the
events and then retry them later.

The Flume Appender writes the event into a channel in the client and then gives control back
to you. From that point on delivery is guaranteed downstream. It uses a separate thread to
pull from the channel and write to a downstream flume agent. That agent can use any of the
sinks that Flume provides to write the data to Kafka, Hadoop, ElasticSearch, etc. or you can
write a custom Sink to do whatever you want. I used Flume at my previous employer and am using
it currently and I can tell you that it is extremely reliable and the throughput is pretty
good.

Yes, logging asynchronously is unsafe because all in-flight events will be lost. That is fine
for normal debug logs but is unacceptable for auditing. Again, because Flume writes to the
file channel that won’t happen unless you disk is full.

If you use JMS then you have to handle the case where event delivery fails for whatever reason,
or you have to require that your JMS service is 100% reliable.

Ralph

> On Apr 3, 2017, at 5:45 PM, Gary Gregory <garydgregory@gmail.com> wrote:
> 
> Wait a sec. A JMS provider can guarantee message delivery. How can events be lost once
they are in the provider? Are you saying that using an async logger is 'unsafe' because the
events in the ring buffer go away if the JVM goes down?
> 
> Ideally I want to publish and forget, with the publish part asynchronous, call the API,
and it returns right away.
> 
> Is there no way to do that with our JMS Appender?
> 
> Gary
> 
> On Mon, Apr 3, 2017 at 5:38 PM, Ralph Goers <ralph.goers@dslextreme.com <mailto:ralph.goers@dslextreme.com>>
wrote:
> What’s the point though when all you have to do is specify the pattern layout with
“%m%ex{none}”?
> 
> How can you do auditing with the async logger? You may lose events with that.
> 
> Your use case is exactly why I wrote the FlumeAppender. It is very much like JMS but
a whole lot faster and guarantees events aren’t lost.
> 
> Ralph
> 
>> On Apr 3, 2017, at 5:03 PM, Gary Gregory <garydgregory@gmail.com <mailto:garydgregory@gmail.com>>
wrote:
>> 
>> Right. I want to post to JMS message objects I've serialized to JSON. All I care
about is the message.
>> 
>> I log these events to a specially named logger with a specially named marker. These
are the only events that should be published to JMS (I use a filter and the one marker). I
am in charge of the message format and it is consumed on the other JMS side by a specialized
agent which I also control.
>> 
>> IOW, I am using the Log4j infrastructure as the simplest way to send custom messages
over to JMS. These are not traditional logging events, rather a kind of auditing system. I
leverage the Log4j async logger as well. No need to deal with the JMS API.
>> 
>> Gary
>> 
>> On Mon, Apr 3, 2017 at 4:07 PM, Remko Popma <remko.popma@gmail.com <mailto:remko.popma@gmail.com>>
wrote:
>> So, exceptions are swallowed and no newlines are rendered? Interesting. 
>> What's the use case?
>> 
>> Sent from my iPhone
>> 
>> On Apr 4, 2017, at 7:30, Gary Gregory <garydgregory@gmail.com <mailto:garydgregory@gmail.com>>
wrote:
>> 
>>> Hi All,
>>> 
>>> I am considering a new layout called "MessageLayout" which would be synonymous
with:<PatternLayout pattern="%m" />. Thoughts? 
>>> 
>>> Gary
>>> 
>>> -- 
>>> E-Mail: garydgregory@gmail.com <mailto:garydgregory@gmail.com> | ggregory@apache.org
 <mailto:ggregory@apache.org>
>>> Java Persistence with Hibernate, Second Edition <https://www.amazon.com/gp/product/1617290459/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1617290459&linkCode=as2&tag=garygregory-20&linkId=cadb800f39946ec62ea2b1af9fe6a2b8> 
<http://ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=am2&o=1&a=1617290459>
>>> JUnit in Action, Second Edition <https://www.amazon.com/gp/product/1935182021/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1935182021&linkCode=as2&tag=garygregory-20&linkId=31ecd1f6b6d1eaf8886ac902a24de418%22> 
<http://ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=am2&o=1&a=1935182021>
>>> Spring Batch in Action <https://www.amazon.com/gp/product/1935182951/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1935182951&linkCode=%7B%7BlinkCode%7D%7D&tag=garygregory-20&linkId=%7B%7Blink_id%7D%7D%22%3ESpring+Batch+in+Action>
 <http://ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=am2&o=1&a=1935182951>
>>> Blog: http://garygregory.wordpress.com <http://garygregory.wordpress.com/>

>>> Home: http://garygregory.com/ <http://garygregory.com/>
>>> Tweet! http://twitter.com/GaryGregory <http://twitter.com/GaryGregory>
>> 
>> 
>> -- 
>> E-Mail: garydgregory@gmail.com <mailto:garydgregory@gmail.com> | ggregory@apache.org
 <mailto:ggregory@apache.org>
>> Java Persistence with Hibernate, Second Edition <https://www.amazon.com/gp/product/1617290459/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1617290459&linkCode=as2&tag=garygregory-20&linkId=cadb800f39946ec62ea2b1af9fe6a2b8> 
<http://ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=am2&o=1&a=1617290459>
>> JUnit in Action, Second Edition <https://www.amazon.com/gp/product/1935182021/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1935182021&linkCode=as2&tag=garygregory-20&linkId=31ecd1f6b6d1eaf8886ac902a24de418%22> 
<http://ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=am2&o=1&a=1935182021>
>> Spring Batch in Action <https://www.amazon.com/gp/product/1935182951/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1935182951&linkCode=%7B%7BlinkCode%7D%7D&tag=garygregory-20&linkId=%7B%7Blink_id%7D%7D%22%3ESpring+Batch+in+Action>
 <http://ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=am2&o=1&a=1935182951>
>> Blog: http://garygregory.wordpress.com <http://garygregory.wordpress.com/>

>> Home: http://garygregory.com/ <http://garygregory.com/>
>> Tweet! http://twitter.com/GaryGregory <http://twitter.com/GaryGregory>
> 
> 
> 
> -- 
> E-Mail: garydgregory@gmail.com <mailto:garydgregory@gmail.com> | ggregory@apache.org
 <mailto:ggregory@apache.org>
> Java Persistence with Hibernate, Second Edition <https://www.amazon.com/gp/product/1617290459/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1617290459&linkCode=as2&tag=garygregory-20&linkId=cadb800f39946ec62ea2b1af9fe6a2b8> 
<http://ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=am2&o=1&a=1617290459>
> JUnit in Action, Second Edition <https://www.amazon.com/gp/product/1935182021/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1935182021&linkCode=as2&tag=garygregory-20&linkId=31ecd1f6b6d1eaf8886ac902a24de418%22> 
<http://ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=am2&o=1&a=1935182021>
> Spring Batch in Action <https://www.amazon.com/gp/product/1935182951/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1935182951&linkCode=%7B%7BlinkCode%7D%7D&tag=garygregory-20&linkId=%7B%7Blink_id%7D%7D%22%3ESpring+Batch+in+Action>
 <http://ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=am2&o=1&a=1935182951>
> Blog: http://garygregory.wordpress.com <http://garygregory.wordpress.com/> 
> Home: http://garygregory.com/ <http://garygregory.com/>
> Tweet! http://twitter.com/GaryGregory <http://twitter.com/GaryGregory>

Mime
View raw message