spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Goyal <nitin2go...@gmail.com>
Subject Re: [ compress in-memory column storage used in sparksql cache table ]
Date Wed, 02 Sep 2015 07:58:38 GMT
I think spark sql's in-memory columnar cache already does compression. Check
out classes in following path :-

https://github.com/apache/spark/tree/master/sql/core/src/main/scala/org/apache/spark/sql/columnar/compression

Although compression ratio is not as good as Parquet.

Thanks
-Nitin



--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/compress-in-memory-column-storage-used-in-sparksql-cache-table-tp13932p13937.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message