kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neha Narkhede <neha.narkh...@gmail.com>
Subject Re: KafkaLog4jAppender with AsyncProducer?
Date Mon, 19 Mar 2012 21:16:42 GMT
Edward,

The kafka log4j in trunk takes in either a zookeeper connection url or
a list of brokers. It just means that the log4j messages will be sent
in a synchronous manner with no buffering. This will mean lower
throughput. But if no broker is available, both sync and async
producers will not be able to send the data at all.

>From a throughput standpoint, it does seem reasonable to change
KafkaLog4jAppender to take in the producer.type config and be able to
pick either the sync or the async producers. Would you mind filing a
JIRA for this ?

Thanks,
Neha

On Mon, Mar 19, 2012 at 12:14 PM, Edward Buck <ebuck@choicestream.com> wrote:
> I see that the KafkaLog4jAppender uses a SyncProducer to send messages. I'm assuming
the means that if no broker is available,  the client program will throw an error (rather
then enqueue/cache the message locally until a broker is available). Would an AsyncProducer
be a good fit for use with a slightly modified KafkaLog4jAppender? If so, this would make
the producer code more resilient to network outages.
>
> Thanks,
> Ed
>
> ________________________________
> The information transmitted is intended only for the person or entity to which it is
addressed and may contain confidential, proprietary, and/or privileged material. Any review,
retransmission, dissemination or other use of, or taking of any action in reliance upon this
information by persons or entities other than the intended recipient is prohibited. If you
received this in error, please contact the sender and delete the material from all computers.

Mime
View raw message