spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brett Meyer <Brett.Me...@crowdstrike.com>
Subject Re: View executor logs on YARN mode
Date Wed, 14 Jan 2015 15:50:39 GMT
You can view the logs for the particular containers on the YARN UI if you go
to the page for a specific node, and then from the Tools menu on the left,
select Local Logs.  There should be a userlogs directory which will contain
the specific application ids for each job that you run.  Inside the
directories for the applications, you can find the specific container logs.


From:  lonely Feb <lonely8658@gmail.com>
Date:  Wednesday, January 14, 2015 at 8:07 AM
To:  "user@spark.apache.org" <user@spark.apache.org>
Subject:  View executor logs on YARN mode

Hi all, i sadly found on YARN mode i cannot view executor logs on YARN web
UI nor on SPARK history web UI. On YARN web UI i can only view AppMaster
logs and on SPARK history web UI i can only view Application metric
information. If i want to see whether a executor is being full GC i can only
use "yarn logs" command. It's very unfriendly. BTW, "yarn logs" command
requires a option of containerID which i could not found on YARN web UI
either. I need to grep it in AppMaster log.

I just wonder how do you handle this situation?



Mime
View raw message