spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wenxing zheng <wenxing.zh...@gmail.com>
Subject How to export the Spark SQL jobs from the HiveThriftServer2
Date Wed, 06 Dec 2017 06:08:43 GMT
Dear all,

I have a HiveThriftServer2 serer running and most of our spark SQLs will go
there for calculation. From the Yarn GUI, I can see the application id and
the attempt ID of the thrift server. But with the REST api described on the
page (https://spark.apache.org/docs/latest/monitoring.html#rest-api), I
still can't get the jobs for a given application with the endpoint:
*/applications/[app-id]/jobs*

Can anyone kindly advice how to dump the spark SQL jobs for audit? Just
like the one for the MapReduce jobs (
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html
).

Thanks again,
Wenxing

Mime
View raw message