spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From benbongalon <>
Subject Re: WebUI on yarn through ssh tunnel affected by AmIpfilter
Date Fri, 20 Mar 2015 23:25:56 GMT

I ran into a similar issue. What's happening is that when Spark is running
in YARN client mode, YARN automatically launches a  Web Application Proxy
to reduce hacking attempts. In doing so, it adds the AmIpFilter to the
proxy. You can see this is the example log snippet below:

15/03/20 21:33:14 INFO cluster.YarnClientSchedulerBackend: ApplicationMaster
registered as
15/03/20 21:33:14 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter.
org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS
15/03/20 21:33:14 INFO ui.JettyUtils: Adding filter:
15/03/20 21:33:15 INFO yarn.Client: Application report for
application_1426881405719_0009 (state: RUNNING)
15/03/20 21:33:15 INFO yarn.Client: 
         client token: N/A
         diagnostics: N/A
         ApplicationMaster host:
         ApplicationMaster RPC port: 0
         queue: default
         start time: 1426887190001
         final status: UNDEFINED
         *tracking URL:*

While I haven't found a way to disable it (the  Spark doc
<>   may help), you can view
the Web UI by forwarding the proxy's port (9046 by default). Then point your
browser the the tracking URL with the host IP replaced with localhost, eg:


Hope that helps.

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message