kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maung Than <maung_t...@apple.com>
Subject Re: Question on output of kafka-producer-perf-test.sh
Date Wed, 28 May 2014 04:16:54 GMT
Any idea when and which release that could be included in?

We would like to have it sooner and can we do something about it?

Thanks,
Maung

On May 27, 2014, at 5:10 PM, Guozhang Wang <wangguoz@gmail.com> wrote:

> Maung,
> 
> Yes, the throughput computed here is based on pre-compression bytes. In the
> old producer we do not have a metric exposing the compression ratio, in the
> new producer available in current trunk we can easily add that metric. I
> have just created a ticket for this a moment ago.
> 
> https://issues.apache.org/jira/browse/KAFKA-1472
> 
> Guozhang
> 
> 
> 
> On Tue, May 27, 2014 at 2:41 PM, Maung Than <maung_than@apple.com> wrote:
> 
>> 
>> Hi All,
>> 
>> We are seeing total data sent below is the same with or without
>> compression.
>> Is it always raw data?
>> If so is there a way we can get the compressed volume?
>> 
>> start.time, end.time, compression, message.size, batch.size,
>> total.data.sent.in.MB, MB.sec, total.data.sent.in.nMsg, nMsg.sec
>> [2014-05-27 14:10:37,471] WARN Property reconnect.interval is not valid
>> (kafka.utils.VerifiableProperties)
>> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
>> SLF4J: Defaulting to no-operation (NOP) logger implementation
>> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
>> details.
>> [2014-05-27 14:10:37,547] WARN Property reconnect.interval is not valid
>> (kafka.utils.VerifiableProperties)
>> [2014-05-27 14:10:37,549] WARN Property reconnect.interval is not valid
>> (kafka.utils.VerifiableProperties)
>> [2014-05-27 14:10:37,552] WARN Property reconnect.interval is not valid
>> (kafka.utils.VerifiableProperties)
>> [2014-05-27 14:10:37,560] WARN Property reconnect.interval is not valid
>> (kafka.utils.VerifiableProperties)
>> 2014-05-27 14:10:37:347, 2014-05-27 14:11:15:546, 0, 420, 1000, 2002.72,
>> 52.4285, 5000000, 130893.4789
>> 
>> 
>> -bash-4.1$
>> /Users/worun/kafkabuild/kafka-0.8.1-src/bin/kafka-producer-perf-test.sh
>> --broker-list vp21q12ic-hpaj020921:9092 --messages 5000000 --topic imessage
>> --threads 5 --message-size 420 --batch-size 1000 --compression-codec 2
>> start.time, end.time, compression, message.size, batch.size,
>> total.data.sent.in.MB, MB.sec, total.data.sent.in.nMsg, nMsg.sec
>> [2014-05-27 13:33:31,616] WARN Property reconnect.interval is not valid
>> (kafka.utils.VerifiableProperties)
>> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
>> SLF4J: Defaulting to no-operation (NOP) logger implementation
>> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
>> details.
>> [2014-05-27 13:33:31,692] WARN Property reconnect.interval is not valid
>> (kafka.utils.VerifiableProperties)
>> [2014-05-27 13:33:31,694] WARN Property reconnect.interval is not valid
>> (kafka.utils.VerifiableProperties)
>> [2014-05-27 13:33:31,697] WARN Property reconnect.interval is not valid
>> (kafka.utils.VerifiableProperties)
>> [2014-05-27 13:33:31,705] WARN Property reconnect.interval is not valid
>> (kafka.utils.VerifiableProperties)
>> 2014-05-27 13:33:31:493, 2014-05-27 13:34:11:862, 2, 420, 1000, 2002.72,
>> 49.6102, 5000000, 123857.4153
>> 
>> Thanks,
>> Maung
>> 
> 
> 
> 
> -- 
> -- Guozhang


Mime
View raw message