spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jian Feng <freedaf...@yahoo.com.INVALID>
Subject Re: how to run python app in yarn?
Date Wed, 14 Jan 2015 23:35:47 GMT
Thanks Josh and Marcelo! It now works!
BTW, just wondering, is there any perf difference between running spark in standalone mode
and under yarn? The only goal that I created this cluster is to run spark jobs. So I can set
up spark in standalone mode if it runs slow in yarn.
best.
      From: Josh Rosen <rosenville@gmail.com>
 To: Marcelo Vanzin <vanzin@cloudera.com> 
Cc: freedafeng <freedafeng@yahoo.com>; "user@spark.apache.org" <user@spark.apache.org>

 Sent: Wednesday, January 14, 2015 3:20 PM
 Subject: Re: how to run python app in yarn?
   
There's an open PR for supporting yarn-cluster mode in PySpark: https://github.com/apache/spark/pull/3976
(currently blocked on reviewer attention / time)


On Wed, Jan 14, 2015 at 3:16 PM, Marcelo Vanzin <vanzin@cloudera.com> wrote:

As the error message says...

On Wed, Jan 14, 2015 at 3:14 PM, freedafeng <freedafeng@yahoo.com> wrote:
> Error: Cluster deploy mode is currently not supported for python
> applications.

Use "yarn-client" instead of "yarn-cluster" for pyspark apps.


--
Marcelo

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org





  
Mime
View raw message