spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <>
Subject Re: Is it possible to disable AM page proxy in Yarn client mode?
Date Mon, 03 Aug 2015 17:14:49 GMT
the reason that redirect is there is for security reasons; in a kerberos enabled cluster the
RM proxy does the authentication, then forwards the requests to the running application. There's
no obvious way to disable it in the spark application master, and I wouldn't recommend doing
this anyway, as it only gets you into a situation where your code works until you flip the
security bit on.

The Spark Web UI installs a new filter, (AmIpFilter), which 302's all HTTP requests coming
in from anywhere other than the host running the RM Proxy.

1. if you make requests from that host (curl, browser, whatever), then they go through without
the redirect.
2. If you don't have an RM proxy (why not?) then you can configure the spark AM to treat your
client IP address as the proxy -and again, no redirect.

YARN-2031 covers the ongoing work to have that proxy/IP filter handle REST API properly. Currently
it only handles GET operations & assumes a human visiting the application in a web browser

> On 3 Aug 2015, at 01:52, Rex Xiong <> wrote:
> In Yarn client mode, Spark driver URL will be redirected to Yarn web proxy server, but
I don't want to use this dynamic name, is it possible to still use <host>:<port>
as standalone mode?

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message