logging-log4j-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ralph Goers <ralph.go...@dslextreme.com>
Subject Re: Custom JSONLayout
Date Fri, 14 Aug 2020 17:21:24 GMT
Is the Message passed to the LogEvent a MapMessage?

Ralph

> On Aug 14, 2020, at 9:40 AM, Naz S <sn1.nazerke@gmail.com> wrote:
> 
> Volkan,
> 
> Wondering if the following case is supported by the logstash layout:
> 
> Let's say logEvent.getMessage() contains the message like " key1=value1
> key2=value2 someString". Then I want to parse that message and put to json:
>      {
>         "timestamp" :    "x",
>          "key1" :     "value1",
>           "key2" :    "value2",
>           "message" :   "someString",
>            ....
>      }
> 
> On Fri, Aug 14, 2020 at 5:23 PM Volkan Yazıcı <volkan.yazici@gmail.com>
> wrote:
> 
>> JsonTemplateLayout definitely supports access to MapMessage. Just try
>> searching for "MapMessage" in the documentation:
>> 
>> https://github.com/apache/logging-log4j2/blob/master/src/site/asciidoc/manual/json-template-layout.adoc
>> 
>> On Fri, Aug 14, 2020 at 5:14 PM Ralph Goers <ralph.goers@dslextreme.com>
>> wrote:
>> 
>>> Volkan - Do you not support the MapMessage?  That is the standard way to
>>> handle this and is what Log4j Audit does. It uses an AuditMessage that
>>> extends MapMessage.  In fact, I should be able to construct a template
>> for
>>> AuditMessage to use with JsonTemplateLayout in Log4j-Audit. If I cannot
>>> then there is a problem.  I should point out that with auditing I might
>>> have a hundred or more potential keys depending on how many audit events
>> I
>>> have defined. I would not want to create a layout that explicitly
>> specifies
>>> them all. But for a MapMessage I don’t see why I should have to.
>>> 
>>> Ralph
>>> 
>>>> On Aug 14, 2020, at 7:07 AM, Volkan Yazıcı <volkan.yazici@gmail.com>
>>> wrote:
>>>> 
>>>> Hello Naz,
>>>> 
>>>> [Sorry for my late response.]
>>>> 
>>>> I am the author of LogstashLayout and also its successor
>>> JsonTemplateLayout
>>>> which will be merged into the release-2.x branch of Log4j (hopefully)
>> in
>>> a
>>>> week or two.
>>>> 
>>>> LogstashLayout **does** support blank value elimination, you just need
>> to
>>>> set emptyPropertyExclusionEnabled to true.
>>>> 
>>>> I have excluded this feature in JsonTemplateLayout for two main
>> reasons:
>>>> 
>>>>  1. It incurs an extra runtime cost.
>>>>  2. Most of the time you don't care. You persist logs in a storage
>>> system,
>>>>  e.g., ELK stack, and there null value elimination is provided out of
>>> the
>>>>  box by the storage engine in the most efficient way.
>>>> 
>>>> Let me know if I can help you further.
>>>> 
>>>> Kind regards.
>>>> 
>>>> 
>>>> On Fri, Aug 14, 2020 at 2:19 PM Naz S <sn1.nazerke@gmail.com> wrote:
>>>> 
>>>>> So far I have looked at the logstash layout  (
>>>>> https://github.com/vy/log4j2-logstash-layout). It seems it doesn't
>> work
>>>>> properly for my use case.
>>>>> 
>>>>> Basically  if I have key,value pairs I log them into the LogEvent
>>> message
>>>>> field, then I parse them while generating JSON objects. The key,value
>>> pairs
>>>>> are not known in advance; sometimes they are null; If the keys are
>>> null, I
>>>>> don't want to log them at all.  Are there any other ways to do this?
>>>>> 
>>>>> On Thu, Aug 13, 2020 at 3:38 PM Naz S <sn1.nazerke@gmail.com> wrote:
>>>>> 
>>>>>> Thank you for your swift response, really appreciate it.
>>>>>> 
>>>>>> Wondering when the  2.14.0 release will come out?
>>>>>> 
>>>>>> On Wed, Aug 12, 2020 at 4:59 PM Ralph Goers <
>>> ralph.goers@dslextreme.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> JsonTemplateLayout hasn’t been merged to the release-2.x branch
but
>> it
>>>>>>> will be so that it can be included in the 2.14.0 release.
>>>>>>> 
>>>>>>> Ralph
>>>>>>> 
>>>>>>>> On Aug 12, 2020, at 7:47 AM, Carter Kozak <ckozak@ckozak.net>
>> wrote:
>>>>>>>> 
>>>>>>>> Hi Naz,
>>>>>>>> 
>>>>>>>> Volkan has contributed a fantastic json template layout that
I
>> expect
>>>>>>> will do what you want, however it has not been released yet.
>>>>>>>> https://github.com/apache/logging-log4j2/pull/335
>>>>>>>> 
>>>>>>>> This discussion from the mailing list provides several options
>> which
>>>>>>> may be relevant to your use case:
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> https://lists.apache.org/thread.html/r538bb480c6d92813224b6a207e34210cccebc01e8d1781943d0ce32d%40%3Clog4j-user.logging.apache.org%3E
>>>>>>>> 
>>>>>>>>> But my custom json layout doesn't write into a file.
Do I have to
>>>>>>> write a file appender for that?
>>>>>>>> 
>>>>>>>> Layout implementations can be configured with any appender,
the
>>>>>>> existing file based appenders
>>>>>>>> work with custom layouts. The layout is responsible for converting
>> a
>>>>>>> LogEvent into a format (e.g. bytes) that
>>>>>>>> can be written to an appender (file, socket, standard out,
etc)
>>>>> without
>>>>>>> being aware of the appenders
>>>>>>>> implementation details.
>>>>>>>> 
>>>>>>>> Best,
>>>>>>>> -ck
>>>>>>>> 
>>>>>>>> On Wed, Aug 12, 2020, at 09:56, Naz S wrote:
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> I'm trying to use JSON layout for a logging format. When
I use the
>>>>>>> default
>>>>>>>>> JSON Layout, I see that there are some fields which are
>> unnecessary
>>>>> to
>>>>>>> me
>>>>>>>>> (for example, loggerFqcn, endOfBatch, etc).
>>>>>>>>> 
>>>>>>>>> Then I decided to implement a custom JSON layout. But
I see that
>>>>>>>>> JacksonFactory is not visible outside the package. Now
I get only
>>>>>>> necessary
>>>>>>>>> fields from LogEvent and generate JSON format using JSON
>> generator.
>>>>>>> Every
>>>>>>>>> configuration is done by code. But ultimately I want
to configure
>>>>>>> fields in
>>>>>>>>> log4j2.xml file. Any suggestions?
>>>>>>>>> 
>>>>>>>>> Also I would like to write logs into a log file. But
my custom
>> json
>>>>>>> layout
>>>>>>>>> doesn't write into a file. Do I have to write a file
appender for
>>>>> that?
>>>>>>>>> 
>>>>>>>>> Many thanks,
>>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>>>>>>> For additional commands, e-mail: log4j-user-help@logging.apache.org
>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>>> For additional commands, e-mail: log4j-user-help@logging.apache.org
>>> 
>>> 
>> 



---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org


Mime
View raw message