spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jian Feng <>
Subject Re: how to run python app in yarn?
Date Wed, 14 Jan 2015 23:35:47 GMT
Thanks Josh and Marcelo! It now works!
BTW, just wondering, is there any perf difference between running spark in standalone mode
and under yarn? The only goal that I created this cluster is to run spark jobs. So I can set
up spark in standalone mode if it runs slow in yarn.
      From: Josh Rosen <>
 To: Marcelo Vanzin <> 
Cc: freedafeng <>; "" <>

 Sent: Wednesday, January 14, 2015 3:20 PM
 Subject: Re: how to run python app in yarn?
There's an open PR for supporting yarn-cluster mode in PySpark:
(currently blocked on reviewer attention / time)

On Wed, Jan 14, 2015 at 3:16 PM, Marcelo Vanzin <> wrote:

As the error message says...

On Wed, Jan 14, 2015 at 3:14 PM, freedafeng <> wrote:
> Error: Cluster deploy mode is currently not supported for python
> applications.

Use "yarn-client" instead of "yarn-cluster" for pyspark apps.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message