flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aljoscha <...@git.apache.org>
Subject [GitHub] flink pull request: Stream API Refactoring
Date Fri, 02 Oct 2015 14:53:35 GMT
GitHub user aljoscha opened a pull request:


    Stream API Refactoring

    This is a WIP of the refactoring. I still want to add Javadocs and a new join operator
based on tagged union that uses the new windowing operators.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/aljoscha/flink api-rework

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1215
commit 560090348f8df93160501a55051a913625c7215e
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2015-09-30T13:05:13Z

    [FLINK-2666] Add timestamp extraction operator
    This adds a user function TimestampExtractor and an operator
    ExtractTimestampsOperator that can be used to extract timestamps and
    attach them to elements to do event-time windowing.
    Users can either use an AscendingTimestampExtractor that assumes that
    timestamps are monotonically increasing. (This allows it to derive the
    watermark very easily.) Or they use a TimestampExtractor, where they
    also have to provide the watermark.
    The ExtractTimestampOperator periodically (on the auto watermark
    interval) calls the extractor to get the current watermark and forwards
    This also adds an ITCase for this behaviour.

commit 5b843231fdc5dee8d4ceada02f3ff8c41daa0281
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2015-10-01T13:58:52Z

    Simplify Stream Java API Class Names
    KeyedDataStream -> KeyedStream
    KeyedWindowDataStream -> WindowedStream
    NonParallelWindowDataStream -> AllWindowedStream
    KeyedWindowFunction -> WindowFunction
    WindowFunction -> AllWindowFunction
    (along with rich functions and reduce function wrappers)
    WindowedStream.mapWindow -> WindowedStream.apply
    AllWindowedStream.mapWindow -> AllWindowedStream.apply
    Also renamed the tests to match the new names.

commit 187ed701f548666a5daaf244ee69d43032c39c6f
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2015-10-01T15:07:11Z

    Rename ConnectedDataStream to ConnectedStreams, Remove some operations
    The removed operations are tricky and some of them are not working
    correctly. For now, co-reduce, stream-cross and stream-join are
    I'm planning to add a new join implementation based on tagged union
    that uses the new windowing code.

commit 7892f321d0b0900a4331c7ee307a34778a8476c7
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2015-10-01T15:56:13Z

    Remove groupBy and GroupedDataStream
    Their functionality is subsumed by keyBy and KeyedStream

commit fd729f616c4386ddc72ecc817ea166df7a8f76aa
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2015-10-01T19:23:56Z

    Add Scala API for new Windowing
    This adds window/timeWindow to KeyedStream along with windowAll/timeWindowAll
    on DataStream.
    The added API classes are AllWindowedStream and WindowedStream.
    This also adds Translations tests similar to those for the Java API:
     - AllWindowTranslationTest.scala
     - WindowTranslationTest.scala

commit fcaa0fef2700730b768364861fc10e6bab628f47
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2015-10-02T14:48:13Z

    WIP on Javadoc


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message