spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <>
Subject Re: Using persistent hdfs on spark ec2 instanes
Date Thu, 23 Jan 2014 00:26:47 GMT
> 1. It seems by default spark ec2 uses ephemeral hdfs, how to switch this to
> persistent hdfs?
You can stop the ephemeral one using


and start the persistent one using


> 2. By default persistent hdfs server is not up, is this meant to be like
> this?

Yes - it starts only an ephemeral one:

"The spark-ec2 script already sets up a HDFS instance for you. It’s
installed in /root/ephemeral-hdfs"

View raw message