kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lu Xuechao <lux...@gmail.com>
Subject Unable to send and consume compressed events.
Date Thu, 29 Aug 2013 08:40:07 GMT
Hi ,

I am trying to enable gzip compression for my events. But after I switched
compression.codec to "1" I found the produced events were even not be
persisted to disk log file. Of course, the consumer could not receive any
compressed events. I sent 10,000 or more events but the broker's log file
not changed. Seems no events were actually send to broker? Below is my
producer's code:

        Properties props = new Properties();
        props.put("serializer.class", "kafka.serializer.StringEncoder");
        props.put("metadata.broker.list", "");
        props.put("partitioner.class", "kafka.producer.DefaultPartitioner");
        props.put("queue.enqueue.timeout.ms", "-1");
        props.put("request.required.acks", "0");
        props.put("producer.type", "async");

        props.put("batch.num.messages", "100");

        props.put("compression.codec", "1");

        ProducerConfig config = new ProducerConfig(props);
        producer = new Producer<String, String>(config);

        KeyedMessage<String, String> data = new KeyedMessage<String,
String>("topic1", messageStr, msg);

If I comment out this line of code : props.put("compression.codec", "1");
then everything works fine. Did I miss something?


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message