spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangchangchun (A)" <wangchangchu...@huawei.com>
Subject [ compress in-memory column storage used in sparksql cache table ]
Date Wed, 02 Sep 2015 04:23:41 GMT
Hi,  I have an idea, can someone give me some advice?



I want to compress data in in-memory column storage which is used by cache table in spark.
This will make cache table use less memory.



I will set an conf to this function, so if anyone want to use this function, he can set this
conf to true.



Compress algorithom I want to use Dictionary Encoding.



Do you think this method worth a try ?


Mime
View raw message