beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-615) Add Support for Processing-Time Timers in FlinkRunner Window Operator
Date Wed, 28 Sep 2016 09:56:20 GMT

    [ https://issues.apache.org/jira/browse/BEAM-615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529125#comment-15529125
] 

ASF GitHub Bot commented on BEAM-615:
-------------------------------------

GitHub user aljoscha opened a pull request:

    https://github.com/apache/incubator-beam/pull/1021

    [BEAM-615] Add Support for Processing-Time Timers in FlinkRunner

    This adds support for processing-time timers that are checkpointed.
    
    There are no tests for this yet but I ran this minimal example successfully:
    ```java
    pipeline
        .apply(CountingInput.unbounded().withRate(5, Duration.standardSeconds(1)))
        .apply(
            Window.<Long>into(new GlobalWindows())
                .withAllowedLateness(Duration.ZERO)
                .triggering(Repeatedly.forever(AfterProcessingTime.pastFirstElementInPane().plusDelayOf(Duration.standardSeconds(1))))
                .discardingFiredPanes())
        .apply(ParDo.of(new DoFn<Long, KV<String, Long>>() {
          @DoFn.ProcessElement
          public void proc(ProcessContext ctx) {
            ctx.output(KV.of("key", ctx.element()));
          }
        }))
        .apply(Combine.<String, Long>perKey(new SerializableFunction<Iterable<Long>,
Long>() {
          @Override
          public Long apply(Iterable<Long> input) {
            System.out.println("GOT: " + Joiner.on(',').join(input));
            return 0L;
          }
        }));
    
    pipeline.run();
    ```
    
    R: @mxm for review

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/aljoscha/incubator-beam flink-processing-time-timers

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/1021.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1021
    
----
commit a1cc636c25561d6300a9a39c3a79d1dfd0eba37f
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2016-09-28T09:49:54Z

    [BEAM-615] Add Support for Processing-Time Timers in FlinkRunner Window Operator

----


> Add Support for Processing-Time Timers in FlinkRunner Window Operator
> ---------------------------------------------------------------------
>
>                 Key: BEAM-615
>                 URL: https://issues.apache.org/jira/browse/BEAM-615
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>    Affects Versions: 0.1.0-incubating, 0.2.0-incubating, 0.3.0-incubating
>            Reporter: Aljoscha Krettek
>            Assignee: Aljoscha Krettek
>
> The Flink runner never had support for processing-time timers, they are silently ignored
when a trigger tries to set one. This should be easy to add in {{WindowFnOperator}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message