kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitry Minkovsky <dminkov...@gmail.com>
Subject Why is compression disabled by default?
Date Mon, 11 Dec 2017 00:44:45 GMT
This is hopefully my final question for a while.

I noticed that compression is disabled by default. Why is this? My best
guess is that compression doesn't work well for short messages
<https://www.elastic.co/blog/store-compression-in-lucene-and-elasticsearch>,
which was maybe identified as the majority use-case for Kafka. But,
producers batch records based on buffer/linger, and in my understanding the
whole record batch is compressed together.

So, what's the deal? Should I turn on compression in production? Does it
depends on my anticipated batch size? I am using Kafka Streams with a very
low linger, so most of my batches will likely be very small.

Thank you!

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message