spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <>
Subject Re: How can a "deserialized Java object" be stored on disk?
Date Sun, 31 Aug 2014 08:29:37 GMT
Yes, there's no such thing as writing a deserialized form to disk.
However there are other persistence levels that store *serialized*
forms in memory. The meaning here is that the objects are not
serialized in memory in the JVM. Of course, they are serialized on

On Sun, Aug 31, 2014 at 5:02 AM, Tao Xiao <> wrote:
> Reading about RDD Persistency, I learned that the storage level
> "MEMORY_AND_DISK" means that " Store RDD as deserialized Java objects in the
> JVM. If the RDD does not fit in memory, store the partitions that don't fit
> on disk, and read them from there when they're needed. "
> But how can a "deserialized Java object" be stored on disk? As far as I
> know, a Java object should be stored as an array of bytes on disk, which
> means that Java object should be firtly converted into an array of bytes (a
> serialized object).
> Thanks .

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message