spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stavros Kontopoulos <stavros.kontopou...@lightbend.com>
Subject [DISCUSS][CORE] Exposing application status metrics via a source
Date Wed, 12 Sep 2018 20:34:50 GMT
Hi all,

I have a PR https://github.com/apache/spark/pull/22381 that exposes
application status
metrics (related jira: SPARK-25394).

So far metrics tooling needs to scrape the metrics rest api to get metrics
like job delay, stages failed, stages completed etc.
>From devops perspective it is good to standardize on a unified way of
gathering metrics.
The need came up on the K8s side where jmx prometheus exporter is commonly
used to scrape metrics for several components such as kafka, cassandra, but
the need is not limited there.

Check comment here
<https://github.com/apache/spark/pull/22381#issuecomment-420029771>:
"The rest api is great for UI and consolidated analytics, but monitoring
through it is not as straightforward as when the data emits directly from
the source like this. There is all kinds of nice context that we get when
the data from this spark node is collected directly from the node itself,
and not proxied through another collector / reporter. It is easier to build
a monitoring data model across the cluster when node, jmx, pod, resource
manifests, and spark data all align by virtue of coming from the same
collector. Building a similar view of the cluster just from the rest api,
as a comparison, is simply harder and quite challenging to do in general
purpose terms."

The PR is ok to be merged but the major concern here is the mirroring of
the metrics. I think that mirroring is ok since people may dont want to
check the ui and they just want to integrate with jmx only (my use case)
and gather metrics in grafana (common case out there).

Does any of the committers or the community have an opinion on this?
Is there an agreement about moving on with this? Note that the addition
does not change much and can always be refactored if we come up with a new
plan for the metrics story in the future.

Thanks,
Stavros

Mime
View raw message