spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Williams <>
Subject Spree: a live-updating web UI for Spark
Date Mon, 27 Jul 2015 22:14:31 GMT
Probably relevant to people on this list: on Friday I released a clone of
the Spark web UI built using Meteor <> so that
everything updates in real-time, saving you from endlessly refreshing the
page while jobs are running :) It can also serve as the UI for running as
well as completed applications, so you don't have to mess with the separate
history-server process if you don't want to.

*This blog post*
*the github repo* <> have lots of
information on how to use it.

It has two sub-components, JsonRelay
<> and Slim
<>; the former sends SparkListenerEvent
out of Spark via a network socket, while the latter receives those events
and writes various stats about them to Mongo (like an external
that persists info to a database). You might find them to offer a better
way of storing information about running and completed Spark applications
than the event log files that Spark uses, and they can be used with or
without the real-time web UI.

Give them a try if they sound useful to you, and let me know if you have
questions or comments!


View raw message