kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manikumar Reddy <ku...@nmsworks.co.in>
Subject Re: Issue with log4j Kafka Appender.
Date Thu, 18 Jun 2015 11:14:41 GMT
You can enable producer  debug log and verify. In 0.8.2.0, you can set
 compressionType
, requiredNumAcks,  syncSend producer config properties to log4j.xml. Trunk
build can take additional retries property .


Manikumar

On Thu, Jun 18, 2015 at 1:14 AM, Madhavi Sreerangam <
madhavi.sreerangam@gmail.com> wrote:

> I have configured my log4j with Kafka Appender.(Kafka version 0.8.2.0)
> Following are entries from my log4j file
>
> log4j.appender.KAFKA=kafka.producer.KafkaLog4jAppender
> log4j.appender.KAFKA.BrokerList=iakafka301p.dev.ch3.s.com:9092,
> iakafka302p.dev.ch3.s.com:9092,iakafka303p.dev.ch3.s.com:9092
> log4j.appender.KAFKA.Topic=dev-1.0_audit
> log4j.appender.KAFKA.Serializer=kafka.test.AppenderStringSerializer
> log4j.appender.KAFKA.layout=org.apache.log4j.PatternLayout
> log4j.appender.KAFKA.layout.ConversionPattern=%m-%d
>
> Kafka is configured with 3 servers, 3 partitions and 3 replicas.
> I have created a test method to publish the messages to kafka topic as
> follows
>
>     private void testKAFKAlog(int noOfMessages){
>         for(int i=0; i < noOfMessages; i++){
>             KAFKA_LOG.info("Test Message: " + i);
>         }
>     }
> I could not see any messages published into the topic. Then I have modified
> the test method to introduce some wait between the requests as follows
>
>     private void testKAFKAlog(int noOfMessages){
>         for(int i=0; i < noOfMessages; i++){
>             try {
>                 Thread.sleep(10);
>             } catch (InterruptedException e) {
>                 e.printStackTrace();
>             }
>             KAFKA_LOG.info("Test Message: " + i);
>         }
>     }
>
> Then all the messages started publishing. I did this exercise couple of
> times with and without sleep between the requests. Messages got published
> only when there is sleep in between the requests.
> Does any one help me here, what is wrong with the configurations I am
> using. (I can't afford 10ms wait for each message, as my application logs
> few Million messages for each run).
> Is there any way that I can override the default ProducerConfig for log4j
> kafka appender.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message