logging-log4j-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikael Ståldal <mikael.stal...@magine.com>
Subject Re: Layouts requiring headers vs. message based appenders
Date Mon, 09 Nov 2015 17:09:38 GMT
For JSON, a single message (an object) is also a valid complete unit, and I
guess that is also true for XML.

BTW, I believe that JsonLayout is broken. When you use <JsonLayout complete=
"true"/>, you get output like this:

[
{
  "timeMillis" : 1447088521228,
  "thread" : "main",
  "level" : "INFO",
  "loggerName" : "com.twitter.finagle.http.HttpMuxer$",
  "message" : "HttpMuxer[/admin/metrics.json] =
com.twitter.finagle.stats.MetricsExporter(<function1>)",
  "endOfBatch" : false,
  "loggerFqcn" : "org.apache.logging.log4j.jul.ApiLogger"
}
{
  "timeMillis" : 1447088521349,
  "thread" : "main",
  "level" : "INFO",
  "loggerName" : "com.magine.contenturl.ContentUrlServer$",
  "message" : "/admin => com.twitter.server.handler.SummaryHandler",
  "endOfBatch" : false,
  "loggerFqcn" : "org.apache.logging.log4j.jul.ApiLogger"
}
{
  "timeMillis" : 1447088521350,
  "thread" : "main",
  "level" : "INFO",
  "loggerName" : "com.magine.contenturl.ContentUrlServer$",
  "message" : "/admin/server_info => com.twitter.finagle.Filter$$anon$2",
  "endOfBatch" : false,
  "loggerFqcn" : "org.apache.logging.log4j.jul.ApiLogger"
}

]

which is not valid JSON. There needs to be a comma (,) between the items in
the array (but not after the last one).


On Mon, Nov 9, 2015 at 5:53 PM, Mikael Ståldal <mikael.staldal@magine.com>
wrote:

> So how should we handle this? Make KafkaAppender support SerializedLayout
> as a special case?
>
> On Mon, Nov 9, 2015 at 5:35 PM, Ralph Goers <ralph.goers@dslextreme.com>
> wrote:
>
>> I don’t think the header and footer make any sense with some appenders.
>> Even with XML, the JMSAppender or KafkaAppender would want to send each
>> message as a complete unit, so I would expect that every message would
>> contain the complete XML message.
>>
>> Ralph
>>
>> On Nov 9, 2015, at 8:47 AM, Mikael Ståldal <mikael.staldal@magine.com>
>> wrote:
>>
>> It currently does not work with KafkaAppender since it does not use
>> header / footer. We could change that, but is that the proper way to do it?
>> That would mean that for JsonLayout each message would look like "[{...}]",
>> I think it make more sense with just "{...}".
>>
>> BTW, does it work for JmsAppender? From what I can see in the code,
>> JmsAppende does not support header / footer either. Should it? It works
>> with SerializedLayout as a special case, but I don't think it will work
>> with an arbitrary Layout which uses header/footer. Have we tested using
>> JmsAppender with another layout than SerializedLayout?
>>
>> On Mon, Nov 9, 2015 at 4:20 PM, Remko Popma <remko.popma@gmail.com>
>> wrote:
>>
>>> I think the layout header/footer idea was originally designed with
>>> HTML/XML format log files in mind. For a File appender (where the "unit
>>> of work" is a file) that means write the layout's header at startup and
>>> write the layout's footer at rollover or shutdown.
>>>
>>> Still, the concept is general enough to work with all kinds of
>>> appender/layout combinations. It's just that different appenders have
>>> different "units of work".
>>>
>>> If the unit of work for a certain appender is a message, it seems
>>> reasonable to prefix each message with the header and postfix each message
>>> with the layout's footer.
>>>
>>> Why does this not work with KafkaAppender? (Sorry, I haven't checked the
>>> source...)
>>>
>>> If there really is a mismatch here, we can make use of the fact that we
>>> know how things work under the hood (for the built-in appenders and
>>> layouts): each appender can check if it knows the layout and whether it
>>> should ask the layout for its header/footer  or whether it should ignore
>>> the layout header/footer and write some byte sequence specific to the
>>> appender instead.
>>>
>>> For an unknown layout, the appender has to make a reasonable choice,
>>> which we need to document on the site).
>>>
>>> Does this answer your question?
>>>
>>> Remko
>>>
>>> On 2015/11/09, at 18:18, Mikael Ståldal <mikael.staldal@magine.com>
>>> wrote:
>>>
>>> This JIRA issue: https://issues.apache.org/jira/browse/LOG4J2-1195
>>>
>>> points out that we have kind of a conceptual problem. Some layouts, such
>>> as SerializedLayout, require a header. This does not work with message
>>> based appenders with layout support like KafkaAppender (and I guess
>>> JmsAppender and JeroMqAppender have the same issue).
>>>
>>> Should those appenders write layout header in every message, or should
>>> we just document that certain combinations of layouts and appenders are not
>>> supported?
>>>
>>> --
>>> [image: MagineTV]
>>>
>>> *Mikael Ståldal*
>>> Senior software developer
>>>
>>> *Magine TV*
>>> mikael.staldal@magine.com
>>> Grev Turegatan 3  | 114 46 Stockholm, Sweden  |   www.magine.com
>>>
>>> Privileged and/or Confidential Information may be contained in this
>>> message. If you are not the addressee indicated in this message
>>> (or responsible for delivery of the message to such a person), you may
>>> not copy or deliver this message to anyone. In such case,
>>> you should destroy this message and kindly notify the sender by reply
>>> email.
>>>
>>>
>>
>>
>> --
>> [image: MagineTV]
>>
>> *Mikael Ståldal*
>> Senior software developer
>>
>> *Magine TV*
>> mikael.staldal@magine.com
>> Grev Turegatan 3  | 114 46 Stockholm, Sweden  |   www.magine.com
>>
>> Privileged and/or Confidential Information may be contained in this
>> message. If you are not the addressee indicated in this message
>> (or responsible for delivery of the message to such a person), you may
>> not copy or deliver this message to anyone. In such case,
>> you should destroy this message and kindly notify the sender by reply
>> email.
>>
>>
>>
>
>
> --
> [image: MagineTV]
>
> *Mikael Ståldal*
> Senior software developer
>
> *Magine TV*
> mikael.staldal@magine.com
> Grev Turegatan 3  | 114 46 Stockholm, Sweden  |   www.magine.com
>
> Privileged and/or Confidential Information may be contained in this
> message. If you are not the addressee indicated in this message
> (or responsible for delivery of the message to such a person), you may not
> copy or deliver this message to anyone. In such case,
> you should destroy this message and kindly notify the sender by reply
> email.
>



-- 
[image: MagineTV]

*Mikael Ståldal*
Senior software developer

*Magine TV*
mikael.staldal@magine.com
Grev Turegatan 3  | 114 46 Stockholm, Sweden  |   www.magine.com

Privileged and/or Confidential Information may be contained in this
message. If you are not the addressee indicated in this message
(or responsible for delivery of the message to such a person), you may not
copy or deliver this message to anyone. In such case,
you should destroy this message and kindly notify the sender by reply
email.

Mime
View raw message