spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject Re: Event Log Compression
Date Tue, 26 Jul 2016 14:08:16 GMT
Hi,

Guess it's spark.io.compression.codec with lz4 being default.
Supported are lzf or snappy.

Pozdrawiam,
Jacek Laskowski
----
https://medium.com/@jaceklaskowski/
Mastering Apache Spark http://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski


On Tue, Jul 26, 2016 at 3:33 PM, Bryan Jeffrey <bryan.jeffrey@gmail.com> wrote:
> All,
>
> I am running Spark 1.6.1. I enabled 'spark.eventLog.compress', and the data
> is now being compressed using lz4.  I would like to move that back to Snappy
> as I have some third party tools that require using Snappy.
>
> Is there a variable used to control Spark eventLog compression algorithm?
>
> Thank you,
>
> Bryan Jeffrey

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message