flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chesnay Schepler (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4840) Measure latency/delay of record processing and expose it as a metric
Date Thu, 20 Oct 2016 06:45:59 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15591000#comment-15591000

Chesnay Schepler commented on FLINK-4840:

There are some things that are concerning: A histogram, by it's very nature, tends to be expensive.
Similarly, calling System.nanoTime() for each record (twice actually) probably wouldn't make
it through a review either.

What calculation is made when the MetricFetcher is doing a snapshot? (I assume average but
want to make sure)
Why is this calculation only made for the MetricFetcher but not reporters?

> Measure latency/delay of record processing and expose it as a metric
> --------------------------------------------------------------------
>                 Key: FLINK-4840
>                 URL: https://issues.apache.org/jira/browse/FLINK-4840
>             Project: Flink
>          Issue Type: Improvement
>          Components: Metrics
>            Reporter: zhuhaifeng
>            Assignee: zhuhaifeng
>            Priority: Minor
>             Fix For: 1.2.0
> We should expose the following Metrics on the OperatorIOMetricGroup:
>  1. latency(ms): record processing time as a histogram metric.
>  2. procTimeCostPerSecond(ms): record processing time accumulated per second, it measure
record processing percentage to check if it's a main time costing
>  3. delay(ms): the time between upstream emit record and downstream start record processing

This message was sent by Atlassian JIRA

View raw message