logging-log4j-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ralph Goers <ralph.go...@dslextreme.com>
Subject Re: Custom JSONLayout
Date Fri, 14 Aug 2020 22:58:30 GMT
I would suggest you consider using MapMessage. The usage you are describing is why it exists.

Ralph

> On Aug 14, 2020, at 12:48 PM, Naz S <sn1.nazerke@gmail.com> wrote:
> 
> No, it's not.
> 
> We just log it as a string.
> 
> On Fri, Aug 14, 2020 at 7:22 PM Ralph Goers <ralph.goers@dslextreme.com>
> wrote:
> 
>> Is the Message passed to the LogEvent a MapMessage?
>> 
>> Ralph
>> 
>>> On Aug 14, 2020, at 9:40 AM, Naz S <sn1.nazerke@gmail.com> wrote:
>>> 
>>> Volkan,
>>> 
>>> Wondering if the following case is supported by the logstash layout:
>>> 
>>> Let's say logEvent.getMessage() contains the message like " key1=value1
>>> key2=value2 someString". Then I want to parse that message and put to
>> json:
>>>     {
>>>        "timestamp" :    "x",
>>>         "key1" :     "value1",
>>>          "key2" :    "value2",
>>>          "message" :   "someString",
>>>           ....
>>>     }
>>> 
>>> On Fri, Aug 14, 2020 at 5:23 PM Volkan Yazıcı <volkan.yazici@gmail.com>
>>> wrote:
>>> 
>>>> JsonTemplateLayout definitely supports access to MapMessage. Just try
>>>> searching for "MapMessage" in the documentation:
>>>> 
>>>> 
>> https://github.com/apache/logging-log4j2/blob/master/src/site/asciidoc/manual/json-template-layout.adoc
>>>> 
>>>> On Fri, Aug 14, 2020 at 5:14 PM Ralph Goers <ralph.goers@dslextreme.com
>>> 
>>>> wrote:
>>>> 
>>>>> Volkan - Do you not support the MapMessage?  That is the standard way
>> to
>>>>> handle this and is what Log4j Audit does. It uses an AuditMessage that
>>>>> extends MapMessage.  In fact, I should be able to construct a template
>>>> for
>>>>> AuditMessage to use with JsonTemplateLayout in Log4j-Audit. If I cannot
>>>>> then there is a problem.  I should point out that with auditing I might
>>>>> have a hundred or more potential keys depending on how many audit
>> events
>>>> I
>>>>> have defined. I would not want to create a layout that explicitly
>>>> specifies
>>>>> them all. But for a MapMessage I don’t see why I should have to.
>>>>> 
>>>>> Ralph
>>>>> 
>>>>>> On Aug 14, 2020, at 7:07 AM, Volkan Yazıcı <volkan.yazici@gmail.com>
>>>>> wrote:
>>>>>> 
>>>>>> Hello Naz,
>>>>>> 
>>>>>> [Sorry for my late response.]
>>>>>> 
>>>>>> I am the author of LogstashLayout and also its successor
>>>>> JsonTemplateLayout
>>>>>> which will be merged into the release-2.x branch of Log4j (hopefully)
>>>> in
>>>>> a
>>>>>> week or two.
>>>>>> 
>>>>>> LogstashLayout **does** support blank value elimination, you just
need
>>>> to
>>>>>> set emptyPropertyExclusionEnabled to true.
>>>>>> 
>>>>>> I have excluded this feature in JsonTemplateLayout for two main
>>>> reasons:
>>>>>> 
>>>>>> 1. It incurs an extra runtime cost.
>>>>>> 2. Most of the time you don't care. You persist logs in a storage
>>>>> system,
>>>>>> e.g., ELK stack, and there null value elimination is provided out
of
>>>>> the
>>>>>> box by the storage engine in the most efficient way.
>>>>>> 
>>>>>> Let me know if I can help you further.
>>>>>> 
>>>>>> Kind regards.
>>>>>> 
>>>>>> 
>>>>>> On Fri, Aug 14, 2020 at 2:19 PM Naz S <sn1.nazerke@gmail.com>
wrote:
>>>>>> 
>>>>>>> So far I have looked at the logstash layout  (
>>>>>>> https://github.com/vy/log4j2-logstash-layout). It seems it doesn't
>>>> work
>>>>>>> properly for my use case.
>>>>>>> 
>>>>>>> Basically  if I have key,value pairs I log them into the LogEvent
>>>>> message
>>>>>>> field, then I parse them while generating JSON objects. The key,value
>>>>> pairs
>>>>>>> are not known in advance; sometimes they are null; If the keys
are
>>>>> null, I
>>>>>>> don't want to log them at all.  Are there any other ways to do
this?
>>>>>>> 
>>>>>>> On Thu, Aug 13, 2020 at 3:38 PM Naz S <sn1.nazerke@gmail.com>
wrote:
>>>>>>> 
>>>>>>>> Thank you for your swift response, really appreciate it.
>>>>>>>> 
>>>>>>>> Wondering when the  2.14.0 release will come out?
>>>>>>>> 
>>>>>>>> On Wed, Aug 12, 2020 at 4:59 PM Ralph Goers <
>>>>> ralph.goers@dslextreme.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> JsonTemplateLayout hasn’t been merged to the release-2.x
branch but
>>>> it
>>>>>>>>> will be so that it can be included in the 2.14.0 release.
>>>>>>>>> 
>>>>>>>>> Ralph
>>>>>>>>> 
>>>>>>>>>> On Aug 12, 2020, at 7:47 AM, Carter Kozak <ckozak@ckozak.net>
>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hi Naz,
>>>>>>>>>> 
>>>>>>>>>> Volkan has contributed a fantastic json template
layout that I
>>>> expect
>>>>>>>>> will do what you want, however it has not been released
yet.
>>>>>>>>>> https://github.com/apache/logging-log4j2/pull/335
>>>>>>>>>> 
>>>>>>>>>> This discussion from the mailing list provides several
options
>>>> which
>>>>>>>>> may be relevant to your use case:
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>> 
>>>> 
>> https://lists.apache.org/thread.html/r538bb480c6d92813224b6a207e34210cccebc01e8d1781943d0ce32d%40%3Clog4j-user.logging.apache.org%3E
>>>>>>>>>> 
>>>>>>>>>>> But my custom json layout doesn't write into
a file. Do I have to
>>>>>>>>> write a file appender for that?
>>>>>>>>>> 
>>>>>>>>>> Layout implementations can be configured with any
appender, the
>>>>>>>>> existing file based appenders
>>>>>>>>>> work with custom layouts. The layout is responsible
for converting
>>>> a
>>>>>>>>> LogEvent into a format (e.g. bytes) that
>>>>>>>>>> can be written to an appender (file, socket, standard
out, etc)
>>>>>>> without
>>>>>>>>> being aware of the appenders
>>>>>>>>>> implementation details.
>>>>>>>>>> 
>>>>>>>>>> Best,
>>>>>>>>>> -ck
>>>>>>>>>> 
>>>>>>>>>> On Wed, Aug 12, 2020, at 09:56, Naz S wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>> 
>>>>>>>>>>> I'm trying to use JSON layout for a logging format.
When I use
>> the
>>>>>>>>> default
>>>>>>>>>>> JSON Layout, I see that there are some fields
which are
>>>> unnecessary
>>>>>>> to
>>>>>>>>> me
>>>>>>>>>>> (for example, loggerFqcn, endOfBatch, etc).
>>>>>>>>>>> 
>>>>>>>>>>> Then I decided to implement a custom JSON layout.
But I see that
>>>>>>>>>>> JacksonFactory is not visible outside the package.
Now I get only
>>>>>>>>> necessary
>>>>>>>>>>> fields from LogEvent and generate JSON format
using JSON
>>>> generator.
>>>>>>>>> Every
>>>>>>>>>>> configuration is done by code. But ultimately
I want to configure
>>>>>>>>> fields in
>>>>>>>>>>> log4j2.xml file. Any suggestions?
>>>>>>>>>>> 
>>>>>>>>>>> Also I would like to write logs into a log file.
But my custom
>>>> json
>>>>>>>>> layout
>>>>>>>>>>> doesn't write into a file. Do I have to write
a file appender for
>>>>>>> that?
>>>>>>>>>>> 
>>>>>>>>>>> Many thanks,
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>>>>>>>>> For additional commands, e-mail:
>> log4j-user-help@logging.apache.org
>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>>>>> For additional commands, e-mail: log4j-user-help@logging.apache.org
>>>>> 
>>>>> 
>>>> 
>> 
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>> For additional commands, e-mail: log4j-user-help@logging.apache.org
>> 
>> 



---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org


Mime
View raw message