spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Du Li <>
Subject spark eventLog and history server
Date Mon, 08 Jun 2015 23:57:20 GMT
Event log is enabled in my spark streaming app. My code runs in standalone mode and the spark
version is 1.3.1. I periodically stop and restart the streaming context by calling ssc.stop().
However, from the web UI, when clicking on a past job, it says the job is still in progress
and does not show the event log. The event log files have suffix .inprogress. Removing the
suffix does not solve the problem. Do I need to do anything here in order to view the event
logs of finished jobs? Or do I need to stop ssc differently?
In addition, the documentation seems to suggest history server is used for Mesos or YARN mode.
Does it mean I don't need to start history server if I only use spark in standalone mode?
View raw message