flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2944) Collect, expose and display operator-specific stats
Date Tue, 03 Nov 2015 00:03:27 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14986320#comment-14986320

Stephan Ewen commented on FLINK-2944:

Yes, this is very valuable.

We should pass reporter classes similar to the ones used by the readers/writers that count
bytes/records. We can initially display them in the web UI as part of the accumulators and
later create a dedicates section for core metrics.

> Collect, expose and display operator-specific stats
> ---------------------------------------------------
>                 Key: FLINK-2944
>                 URL: https://issues.apache.org/jira/browse/FLINK-2944
>             Project: Flink
>          Issue Type: New Feature
>          Components: Local Runtime, Webfrontend
>            Reporter: Fabian Hueske
>              Labels: requires-design-doc
> It would be nice to collect operator-specific stats such as:
> - HashJoin: bytes spilled build side, bytes spilled probe side, num keys (build / probe)
> - GroupBy: bytes spilled, num keys, max group size, avg. group size
> - Combiner: combine rate (in/out ratio)
> - Streaming Ops: avg. throughput
> - etc.
> and display these stats in the webUI. 

This message was sent by Atlassian JIRA

View raw message