spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <matei.zaha...@gmail.com>
Subject Re: Suggestion for ec2 script
Date Fri, 24 Jan 2014 22:37:49 GMT
The issue is that “start” is designed to restart the cluster after pausing the instances
with “stop”, and because some instances’ DNS names changed, we need to modify the master
URL and the list of slaves in the various config files. If you make other changes in these
config files, they will indeed be lost. One solution might be to create your own setup script
to run on the instances after “start”.

Matei 

On Jan 24, 2014, at 2:19 PM, Mingyu Kim <mkim@palantir.com> wrote:

> Hi all,
> 
> I found it confusing that "./spark-ec2 start” actually reinstalls the cluster, which
ends up wiping out all the configurations. How about renaming “start” to “install”
and add a real light-weight “start” for frequently starting and stopping ec2 instances
for mostly cost reasons? The light-weight “start” can simply bring up the instances and
invoke “bin/start-all.sh”. Thoughts?
> 
> Mingyu


Mime
View raw message