spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oliver Koeth (JIRA)" <>
Subject [jira] [Commented] (SPARK-20044) Support Spark UI behind front-end reverse proxy using a path prefix
Date Tue, 28 Mar 2017 09:09:41 GMT


Oliver Koeth commented on SPARK-20044:

The field "webuiaddress" in the <master>/json response does not reflect the proxy, but
returns the internal worker host/port. I left this unchanged form SPARK-15487.
The underlying field WorkerInfo.webUiAddress does not reflect proxy on purpose, because this
is what the master uses to build the proxy rounting
if (reverseProxy) {
   webUi.addProxyTargets(, worker.webUiAddress)

If the returned JSON should rather show the "external" proxy address, one could either fix
up the JSON (without affecting workerInfo) in MasterPage.renderJson or add a new property
"externalwebuiaddress" to WorkerInfo (normally equal to webuiaddress, except in case of reverse

> Support Spark UI behind front-end reverse proxy using a path prefix
> -------------------------------------------------------------------
>                 Key: SPARK-20044
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: Web UI
>    Affects Versions: 2.1.0
>            Reporter: Oliver Koeth
>            Priority: Minor
>              Labels: reverse-proxy, sso
> Purpose: allow to run the Spark web UI behind a reverse proxy with URLs prefixed by a
context root, like In particular, this allows to access multiple Spark
clusters through the same virtual host, only distinguishing them by context root, like,, and it allows to run the Spark UI in a common cookie domain (for
SSO) with other services.
> [SPARK-15487] introduced some support for front-end reverse proxies by allowing all Spark
UI requests to be routed through the master UI as a single endpoint and also added a spark.ui.reverseProxyUrl
setting to define a another proxy sitting in front of Spark. However, as noted in the comments
on [SPARK-15487], this mechanism does not currently work if the reverseProxyUrl includes a
context root like the examples above: Most links generated by the Spark UI result in full
path URLs (like /proxy/app-"id"/...) that do not account for a path prefix (context root)
and work only if the Spark UI "owns" the entire virtual host. In fact, the only place in the
UI where the reverseProxyUrl seems to be used is the back-link from the worker UI to the master
> The discussion on [SPARK-15487] proposes to open a new issue for the problem, but that
does not seem to have happened, so this issue aims to address the remaining shortcomings of
> The problem can be partially worked around by doing content rewrite in a front-end proxy
and prefixing src="/..." or href="/..." links with a context root. However, detecting and
patching URLs in HTML output is not a robust approach and breaks down for URLs included in
custom REST responses. E.g. the "allexecutors" REST call used from the Spark 2.1.0 application/executors
page returns links for log viewing that direct to the worker UI and do not work in this scenario.
> This issue proposes to honor spark.ui.reverseProxyUrl throughout Spark UI URL generation.
Experiments indicate that most of this can simply be achieved by using/prepending spark.ui.reverseProxyUrl
to the existing spark.ui.proxyBase system property. Beyond that, the places that require adaption
> - worker and application links in the master web UI
> - webui URLs returned by REST interfaces
> Note: It seems that returned redirect location headers do not need to be adapted, since
URL rewriting for these is commonly done in front-end proxies and has a well-defined interface

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message