spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Haberman <>
Subject accessing spark ui over an ssh tunnel
Date Sat, 26 Oct 2013 02:40:15 GMT

The new Spark UI looks awesome. Unfortunately, it's hard for me to use
in a real browser (on my desktop, vs. lynx on the cluster), since our
(EMR) clusters are basically only accessible via SSH.

I can port forward the master:8080 web UI port locally, and it looks
great, but I can't follow any links through to the slaves. The links
technically use the public AWS host names, but we don't have those
ports opened in the security group.

Two things come to mind:

1) Maybe the entire UI could be rendered on the master? Either based on
the master's data or making actor/JSON/whatever calls to the slaves.

2) Somehow proxy links through the master to each slave. Like
master:8080/slave/<id>/<some-path> ends up proxied to the
slave:8081/<some-path>, and the master just sends the HTML as-is back
to the client.

I'm not very familiar with the web UI, so I'm not sure which would be
easier. I also don't mean to bikeshed on the really great UI.

But, at least with our current setup, this would demonstrably increase
the usability for me.

Is this something that you guys have considered? Do you have any
thoughts on it? WIP? Open to contributions?


- Stephen

View raw message