james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From René Cordier (Jira) <server-...@james.apache.org>
Subject [jira] [Commented] (JAMES-3399) Document structured logging with FluentBit
Date Fri, 02 Oct 2020 08:09:00 GMT

    [ https://issues.apache.org/jira/browse/JAMES-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17206024#comment-17206024
] 

René Cordier commented on JAMES-3399:
-------------------------------------

[https://github.com/linagora/james-project/pull/3850] contributed this documentation

> Document structured logging with FluentBit
> ------------------------------------------
>
>                 Key: JAMES-3399
>                 URL: https://issues.apache.org/jira/browse/JAMES-3399
>             Project: James Server
>          Issue Type: Improvement
>          Components: Documentation, guice
>    Affects Versions: master
>            Reporter: Benoit Tellier
>            Priority: Major
>
> == Context
> As any application, James outputs logs.
> The team uses the SLF4J logging facade, along with the logback-classic log producer library.
> The team had been setting up contextual logging: the context of the execution is recorded
through the use of the 
> MDC (Mapped Diagnostic Context) on the behalf of the developper. Once the context set
up, all logging action performed in the scope will implicilty include the MDC context.
> Contextual logging is beneficial as it enable dispaying efficiently request parameters,
tracking users, request ids and so on. It greatly enhence the debugging experience.
> Historically, we relyed on a custom logback appender to output logs directly to ElasticSearch,
in a structured fashion, that could be directly accessed via Kibana.
> The historical stack had been proven brittle:
>  - James needs to be pushing logs, eventual consummiung some of its resources
>  - Being push based, upon bolks or temporary ElasticSearch failures, some logs are lost.
> This conversation was marked as resolved by chibenwa
>  - Logs are duplicated, both to the standard output and ElasticSearch
>  - Changes to the logging stack requires a James restart, that could cause inconsistency
and harm availability.
> This conversation was marked as resolved by chibenwa
> == Desirable propoerties of the logging stack
> We see the following properties as desirable:
>  - Having structured logging
>  - Being pull based
>  - Uncorrelating the logging stack from the James configuration
> == Definition of Done
> Document how to rely on FluentD to collect James logs. The James logging configuration
can be simplified to log all logs to the standard output.
> Document how to emmit logs in a JSON format, in order for fluentD and Kibana to allow
preserving the structured logging experience.
> == Consequences
> We expect our logging stack to be more resilient, while keeping structured logging.
> https://github.com/linagora/logback-elasticsearch-appender project can be deprecated
- if desired.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message