kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Stein <joe.st...@stealth.ly>
Subject Re: Compression in Kafka: GZIP or Snappy
Date Fri, 16 May 2014 21:20:24 GMT
I created a ticket for the patch
https://issues.apache.org/jira/browse/KAFKA-1456

/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/


On Thu, May 15, 2014 at 1:59 PM, Steven Schlansker <
sschlansker@opentable.com> wrote:

>
> On May 7, 2014, at 7:16 AM, Maung Than <maung_than@apple.com> wrote:
>
> > Hi  All,
> >
> > I have read this posting from linkedIn Team member;
> http://geekmantra.wordpress.com/2013/03/28/compression-in-kafka-gzip-or-snappy/; Thanks.
> >
> > I have few questions and thoughts:
> >
> > 4) Has any one else done Snappy Vs. GZIP comparison and chosen one over
> the other. If so what are your findings and why if you could share as we
> are going through this expertise.
>
> I compared GZIP vs Snappy vs LZ4 for HTTP data a little while ago.
>
> We found that GZIP was so slow that we could not saturate our network
> connection with GZIP enabled - it just takes too much CPU!  We decided that
> GZIP might make sense over WAN links but there’s no way it makes sense over
> a LAN.
>
> Snappy was much faster.
>
> LZ4 was faster than GZIP and smaller than Snappy.  We switched to LZ4 and
> never looked back.  It would be cool if Kafka supported it :)
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message