kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chanchal Chatterji <chanchal.chatte...@infosys.com>
Subject RE: Kafka compression - results
Date Wed, 12 Sep 2018 09:33:10 GMT
It's not just lz4 , except in case if gzip everything else increases the directory size. 

-----Original Message-----
From: darekAsz <darekkafka@gmail.com> 
Sent: Wednesday, September 12, 2018 2:43 PM
To: users@kafka.apache.org
Subject: Kafka compression - results

Hi
I made some tests of compression in kafka. First I want to check speed of producer with compression.
There are my results:
with no compression: 112.58 MB/s
with gzip compression: 63.24 MB/s
with snappy compression: 132.43 MB/s
with lz4 compression: 136.66 MB/s

Then I want to check how looks size of data on broker. I sent 500 mln 64B records. There are
my results:
directory size before sending data: 4KB
directory size after sending data compressed with gzip: 1.5 GB directory size after sending
data compressed with snappy: 366 MB directory size after sending data compressed with lz4:
2.4 GB

And there is my question: why lz4 compression make data size bigger than data without compression?
I think that i made all tests correctly.

Best regards
Mime
View raw message