spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Larry Liu <>
Subject Re: where storagelevel DISK_ONLY persists RDD to
Date Mon, 26 Jan 2015 07:08:15 GMT
Hi, Charles

Thanks for your reply.

Is it possible to persist RDD to HDFS? What is the default location to
persist RDD with storagelevel DISK_ONLY?

On Sun, Jan 25, 2015 at 6:26 AM, Charles Feduke <>

> I think you want to instead use `.saveAsSequenceFile` to save an RDD to
> someplace like HDFS or NFS it you are attempting to interoperate with
> another system, such as Hadoop. `.persist` is for keeping the contents of
> an RDD around so future uses of that particular RDD don't need to
> recalculate its composite parts.
> On Sun Jan 25 2015 at 3:36:31 AM Larry Liu <> wrote:
>> I would like to persist RDD TO HDFS or NFS mount. How to change the
>> location?

View raw message