spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From TANG Gen <gen.t...@keyrus.com>
Subject Re: The question about mount ephemeral disk in slave-setup.sh
Date Fri, 03 Oct 2014 21:18:51 GMT
I have taken a look at the code of mesos spark-ec2 and documentation of AWS.
I think that maybe I found the answer. 

In fact, there are two types AMI in AWS EBS backed AMI and instance store
backed AMI. For EBS backed AMI, we can add instance store volume when we
create the images(The details can be founded in 
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/creating-an-ami-ebs.html
<http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/creating-an-ami-ebs.html> 
). And then by default when we launch an instance from this AMI, the default
instance store volume will be formatted(ext3) and mounted at
/media/ephemeral0... etc 

The images provided by mesos spark-ec2 is EBS backed AMI and it is already
added instance store volume(I guess). However, it is modified the file
etc/fstab to mount the ephemeral disks to /mnt...etc (But I don't know how
they modify dynamically the file etc/fstab)

At last, as described in slave-setup.sh, for r3*, ext4 has the best
performance. Hence, they reformat the ephemeral disk to ext4 and mount it to
/mnt...etc.

Hope this could help someone else. 



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/The-question-about-mount-ephemeral-disk-in-slave-setup-sh-tp15675p15704.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message