So how should we handle this? Make KafkaAppender support SerializedLayout as a special case?

On Mon, Nov 9, 2015 at 5:35 PM, Ralph Goers <ralph.goers@dslextreme.com> wrote:
I don’t think the header and footer make any sense with some appenders. Even with XML, the JMSAppender or KafkaAppender would want to send each message as a complete unit, so I would expect that every message would contain the complete XML message.

Ralph

On Nov 9, 2015, at 8:47 AM, Mikael Ståldal <mikael.staldal@magine.com> wrote:

It currently does not work with KafkaAppender since it does not use header / footer. We could change that, but is that the proper way to do it? That would mean that for JsonLayout each message would look like "[{...}]", I think it make more sense with just "{...}".

BTW, does it work for JmsAppender? From what I can see in the code, JmsAppende does not support header / footer either. Should it? It works with SerializedLayout as a special case, but I don't think it will work with an arbitrary Layout which uses header/footer. Have we tested using JmsAppender with another layout than SerializedLayout?

On Mon, Nov 9, 2015 at 4:20 PM, Remko Popma <remko.popma@gmail.com> wrote:
I think the layout header/footer idea was originally designed with HTML/XML format log files in mind. For a File appender (where the "unit of work" is a file) that means write the layout's header at startup and write the layout's footer at rollover or shutdown. 

Still, the concept is general enough to work with all kinds of appender/layout combinations. It's just that different appenders have different "units of work".

If the unit of work for a certain appender is a message, it seems reasonable to prefix each message with the header and postfix each message with the layout's footer. 

Why does this not work with KafkaAppender? (Sorry, I haven't checked the source...)

If there really is a mismatch here, we can make use of the fact that we know how things work under the hood (for the built-in appenders and layouts): each appender can check if it knows the layout and whether it should ask the layout for its header/footer  or whether it should ignore the layout header/footer and write some byte sequence specific to the appender instead. 

For an unknown layout, the appender has to make a reasonable choice, which we need to document on the site).

Does this answer your question?

Remko 

On 2015/11/09, at 18:18, Mikael Ståldal <mikael.staldal@magine.com> wrote:


points out that we have kind of a conceptual problem. Some layouts, such as SerializedLayout, require a header. This does not work with message based appenders with layout support like KafkaAppender (and I guess JmsAppender and JeroMqAppender have the same issue).

Should those appenders write layout header in every message, or should we just document that certain combinations of layouts and appenders are not supported?

--
MagineTV

Mikael Ståldal
Senior software developer

Magine TV
Grev Turegatan 3  | 114 46 Stockholm, Sweden  |   www.magine.com

Privileged and/or Confidential Information may be contained in this message. If you are not the addressee indicated in this message
(or responsible for delivery of the message to such a person), you may not copy or deliver this message to anyone. In such case, 
you should destroy this message and kindly notify the sender by reply email.   



--
MagineTV

Mikael Ståldal
Senior software developer

Magine TV
Grev Turegatan 3  | 114 46 Stockholm, Sweden  |   www.magine.com

Privileged and/or Confidential Information may be contained in this message. If you are not the addressee indicated in this message
(or responsible for delivery of the message to such a person), you may not copy or deliver this message to anyone. In such case, 
you should destroy this message and kindly notify the sender by reply email.   




--
MagineTV

Mikael Ståldal
Senior software developer

Magine TV
Grev Turegatan 3  | 114 46 Stockholm, Sweden  |   www.magine.com

Privileged and/or Confidential Information may be contained in this message. If you are not the addressee indicated in this message
(or responsible for delivery of the message to such a person), you may not copy or deliver this message to anyone. In such case, 
you should destroy this message and kindly notify the sender by reply email.