spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörn Franke <>
Subject Re: AVRO File size when caching in-memory
Date Mon, 14 Nov 2016 09:36:16 GMT
spark version? Are you using tungsten?

> On 14 Nov 2016, at 10:05, Prithish <> wrote:
> Can someone please explain why this happens?
> When I read a 600kb AVRO file and cache this in memory (using cacheTable), it shows up
as 11mb (storage tab in Spark UI). I have tried this with different file sizes, and the size
in-memory is always proportionate. I thought Spark compresses when using cacheTable. 

To unsubscribe e-mail:

View raw message