flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6464) Metric name is not stable
Date Mon, 22 Jan 2018 14:08:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16334297#comment-16334297

ASF GitHub Bot commented on FLINK-6464:

GitHub user zentol opened a pull request:


    [FLINK-6464][streaming] Stabilize default window operator names

    ## What is the purpose of the change
    This PR modifies the generation of default operator names for window operators to be more
concise, and more importantly, stable across job submissions.
    Example: SocketTextWordCount
    TriggerWindow(TumblingProcessingTimeWindows(5000), ReducingStateDescriptor{serializer=org.apache.flink.api.java.typeutils.runtime.PojoSerializer@591ae253,
ProcessingTimeTrigger(), WindowedStream.reduce(WindowedStream.java:241))
    Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger, WindowedStream.reduce(WindowedStream.java:243))
    ## Brief change log
    * create shared static utility method to generate names
    * remove state-descriptor from operator name
    * replace `TriggerWindow` with `Window`
    * trigger/assigner are now included with their simple class name instead of fully qualified
class name + object reference
    ## Verifying this change
    This change is a trivial rework / code cleanup without any test coverage.
    ## Does this pull request potentially affect one of the following parts:
      - Dependencies (does it add or upgrade a dependency): (no)
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
      - The serializers: (no)
      - The runtime per-record code paths (performance sensitive): (no)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing,
Yarn/Mesos, ZooKeeper: (no)
      - The S3 file system connector: (no)
    ## Documentation
      - Does this pull request introduce a new feature? (no)
      - If yes, how is the feature documented? (not applicable)

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zentol/flink 6464

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5332
commit 065e5420d9382b15fc32c2738c2ede33aee69d38
Author: zentol <chesnay@...>
Date:   2018-01-22T13:46:02Z

    [FLINK-6464][streaming] Stabilize default window operator names


> Metric name is not stable
> -------------------------
>                 Key: FLINK-6464
>                 URL: https://issues.apache.org/jira/browse/FLINK-6464
>             Project: Flink
>          Issue Type: Bug
>          Components: DataStream API, Metrics
>    Affects Versions: 1.2.0
>            Reporter: Andrey
>            Assignee: Chesnay Schepler
>            Priority: Critical
>             Fix For: 1.5.0, 1.4.1
> Currently according to the documentation (https://ci.apache.org/projects/flink/flink-docs-release-1.2/monitoring/metrics.html)
operator metrics constructed using the following pattern:
> <operator_name>, <subtask_index>
> For some operators, "operator_name" could contain default implementation of toString
method. For example:
> {code}
> TriggerWindow(TumblingProcessingTimeWindows(3000), ListStateDescriptor{serializer=org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer@c65792d4},
xxx.Trigger@665fe457, WindowedStream.apply(WindowedStream.java:521)) -> Sink: Unnamed
> {code}
> The part "@c65792d4" will be changed every time job is restarted/cancelled. 
> As a consequence it's not possible to store metrics for a long time.
> Expected:
> * ensure all operators return human readable, non-default names OR
> * change the way TriggerWindow generates it's name.

This message was sent by Atlassian JIRA

View raw message