kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vincent Dautremont <vincent.dautrem...@olamobile.com.INVALID>
Subject Re: is a topic compressed?
Date Thu, 21 Sep 2017 08:08:46 GMT
Hi,
Snappy keeps a lot of parts in plain text :
look that example where only "pedia" is encoded/tokenized in the sentence.
https://en.wikipedia.org/wiki/Snappy_(compression)

> Wikipedia is a free, web-based, collaborative, multilingual encyclopedia
> project.


your data is then probably compressed with snappy.

Another try would be to change compression to other values (or remove
compression) and compare the tcp dump with the one you already have.


Vincent.

On Wed, Sep 20, 2017 at 11:58 PM, Wesley Chow <wes@chartbeat.com> wrote:

> I have a producer configured to snappy compress data sent to a cluster. Is
> there some way to verify that the data indeed is being compressed? If I
> peek at the .log files on the broker, I can read some plain text amongst
> binary. Similarly, tcpdump shows plain text readable data. I do not know if
> this is evidence that compression is not working, but is there a better way
> to verify that compression is turned on?
>
> Wes
>

-- 
The information transmitted is intended only for the person or entity to 
which it is addressed and may contain confidential and/or privileged 
material. Any review, retransmission, dissemination or other use of, or 
taking of any action in reliance upon, this information by persons or 
entities other than the intended recipient is prohibited. If you received 
this in error, please contact the sender and delete the material from any 
computer.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message