spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nirav Patel <>
Subject Compress individual RDD
Date Tue, 15 Mar 2016 11:03:36 GMT

I see that there's following spark config to compress an RDD.  My guess is
it will compress all RDDs of a given SparkContext, right?  If so, is there
a way to instruct spark context to only compress some rdd and leave others
uncompressed ?


spark.rdd.compress false Whether to compress serialized RDD partitions
(e.g. forStorageLevel.MEMORY_ONLY_SER). Can save substantial space at the
cost of some extra CPU time.


[image: What's New with Xactly] <>

<>  [image: LinkedIn] 
<>  [image: Twitter] 
<>  [image: Facebook] 
<>  [image: YouTube] 

View raw message