spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shivaram Venkataraman <shivaram.venkatara...@gmail.com>
Subject Re: Deploying spark on EC2 failed
Date Mon, 07 Oct 2013 16:39:35 GMT
Ah - This was a recently introduced bug in deploy_templates.py. Should
be fixed when we merge https://github.com/mesos/spark-ec2/pull/24

Thanks
Shivaram

On Mon, Oct 7, 2013 at 7:40 AM, Hao REN <julien19890118@gmail.com> wrote:
> Hi,
>
> I am trying to deploy spark 0.8.0 on EC2 with the CDH4 prebuild patch.
>
> Here is the cmd:
>
> ./spark-ec2 -k sparkkp -i sparkkp.pem -s 3 -t m1.medium -v 0.8.0
> --ebs-vol-size=100 launch spark-test
>
> I just added -v 0.8.0. However, during the deployment, there is an error
> thrown as below :
>
> ~/spark-ec2
> Initializing spark-standalone
> Creating local config files...
>   File "./deploy_templates.py", line 29
>     if system_ram_mb > 100*1024;
>                                                   ^
> SyntaxError: invalid syntax
>
> I have also checked repo on github:
> https://github.com/mesos/spark-ec2/blob/master/deploy_templates.py
>
> I notice that there are somehow some modification done there. It's not the
> exacte one that the launching script uses.
>
> In the end, the cluster is launched, but without hdfs running on it.
>
> I've tried to start hdfs manually. And the it says: JAVA_HOME is not set.
>
> I thought there mb other config not done yet, so I terminated my deployment.
>
> I believe that the launching script does not use the latest deploying
> script.
>
> Could someone help me out ? or some fix here ?
>
> Thank you in advance.
>
> Hao.
> --
> REN Hao
>
> Data Engineer @ ClaraVista
>
> Paris, France
>
> Tel:  +33 06 14 54 57 24

Mime
View raw message