kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: Question on output of kafka-producer-perf-test.sh
Date Wed, 28 May 2014 00:10:38 GMT
Maung,

Yes, the throughput computed here is based on pre-compression bytes. In the
old producer we do not have a metric exposing the compression ratio, in the
new producer available in current trunk we can easily add that metric. I
have just created a ticket for this a moment ago.

https://issues.apache.org/jira/browse/KAFKA-1472

Guozhang



On Tue, May 27, 2014 at 2:41 PM, Maung Than <maung_than@apple.com> wrote:

>
> Hi All,
>
> We are seeing total data sent below is the same with or without
> compression.
> Is it always raw data?
> If so is there a way we can get the compressed volume?
>
> start.time, end.time, compression, message.size, batch.size,
> total.data.sent.in.MB, MB.sec, total.data.sent.in.nMsg, nMsg.sec
> [2014-05-27 14:10:37,471] WARN Property reconnect.interval is not valid
> (kafka.utils.VerifiableProperties)
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> [2014-05-27 14:10:37,547] WARN Property reconnect.interval is not valid
> (kafka.utils.VerifiableProperties)
> [2014-05-27 14:10:37,549] WARN Property reconnect.interval is not valid
> (kafka.utils.VerifiableProperties)
> [2014-05-27 14:10:37,552] WARN Property reconnect.interval is not valid
> (kafka.utils.VerifiableProperties)
> [2014-05-27 14:10:37,560] WARN Property reconnect.interval is not valid
> (kafka.utils.VerifiableProperties)
> 2014-05-27 14:10:37:347, 2014-05-27 14:11:15:546, 0, 420, 1000, 2002.72,
> 52.4285, 5000000, 130893.4789
>
>
> -bash-4.1$
> /Users/worun/kafkabuild/kafka-0.8.1-src/bin/kafka-producer-perf-test.sh
> --broker-list vp21q12ic-hpaj020921:9092 --messages 5000000 --topic imessage
> --threads 5 --message-size 420 --batch-size 1000 --compression-codec 2
> start.time, end.time, compression, message.size, batch.size,
> total.data.sent.in.MB, MB.sec, total.data.sent.in.nMsg, nMsg.sec
> [2014-05-27 13:33:31,616] WARN Property reconnect.interval is not valid
> (kafka.utils.VerifiableProperties)
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> [2014-05-27 13:33:31,692] WARN Property reconnect.interval is not valid
> (kafka.utils.VerifiableProperties)
> [2014-05-27 13:33:31,694] WARN Property reconnect.interval is not valid
> (kafka.utils.VerifiableProperties)
> [2014-05-27 13:33:31,697] WARN Property reconnect.interval is not valid
> (kafka.utils.VerifiableProperties)
> [2014-05-27 13:33:31,705] WARN Property reconnect.interval is not valid
> (kafka.utils.VerifiableProperties)
> 2014-05-27 13:33:31:493, 2014-05-27 13:34:11:862, 2, 420, 1000, 2002.72,
> 49.6102, 5000000, 123857.4153
>
> Thanks,
> Maung
>



-- 
-- Guozhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message