spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: How can a "deserialized Java object" be stored on disk?
Date Sun, 31 Aug 2014 08:29:37 GMT
Yes, there's no such thing as writing a deserialized form to disk.
However there are other persistence levels that store *serialized*
forms in memory. The meaning here is that the objects are not
serialized in memory in the JVM. Of course, they are serialized on
disk.

On Sun, Aug 31, 2014 at 5:02 AM, Tao Xiao <xiaotao.cs.nju@gmail.com> wrote:
> Reading about RDD Persistency, I learned that the storage level
> "MEMORY_AND_DISK" means that " Store RDD as deserialized Java objects in the
> JVM. If the RDD does not fit in memory, store the partitions that don't fit
> on disk, and read them from there when they're needed. "
>
> But how can a "deserialized Java object" be stored on disk? As far as I
> know, a Java object should be stored as an array of bytes on disk, which
> means that Java object should be firtly converted into an array of bytes (a
> serialized object).
>
> Thanks .

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message