beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaltay <>
Subject [GitHub] incubator-beam pull request #498: Python sdk merge from master
Date Mon, 20 Jun 2016 17:59:34 GMT
GitHub user aaltay opened a pull request:

    Python sdk merge from master

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
     - [ ] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [ ] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](

You can merge this pull request into a Git repository by running:

    $ git pull python-sdk

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #498
commit 9e797e3d3a6bda2ff42b8c3acf8fd60fec179aa4
Author: Thomas Groh <>
Date:   2016-05-20T22:18:11Z

    Execute NeedsRunner tests in the Direct Runner

commit ee1297e21a481fbea52475c0732526a0441d03cb
Author: Thomas Groh <>
Date:   2016-06-08T00:50:38Z

    Use a DirectExecutor for Watermark Callbacks
    This fixes a resource leak where the executor service is not properly
    shut down with the rest of the DirectRunner.

commit 82912196080b7ae79b39e355d10077ad8654eda6
Author: Kenn Knowles <>
Date:   2016-06-10T02:26:23Z

    This closes #392

commit de49d032730dd21691e6e4358fdcfef249aef46f
Author: Thomas Groh <>
Date:   2016-06-10T21:36:42Z

    Update Pipeline Execution Style in WindowedWordCountTest
    This sets the runner a Pipeline creation time rather than sending a
    (potentially rewritten) pipeline to a new runner instance.

commit a8a33b19933326c28522dee530974c96d4aef0cb
Author: Thomas Groh <>
Date:   2016-06-10T21:38:36Z

    Update Direct Module tests to explicitly set Pipeline

commit f73bd73caa5e8222946cfc20491fd2806edd1d2b
Author: Thomas Groh <>
Date:   2016-06-10T21:41:06Z

    Use TestPipeline#testingPipelineOptions in IO Tests

commit f2fb59c65119d5da56df5dd4e64fa1873c6ccbbb
Author: Thomas Groh <>
Date:   2016-06-10T21:43:10Z

    Move GcsUtil TextIO Tests to TextIOTest
    These tests are not a test of the DataflowRunner, nor any
    DataflowRunner specific behavior, so they should be part of TextIOTest

commit a2dbc41b406e35794b1856796c32593653dc903c
Author: Dan Halperin <>
Date:   2016-06-10T23:12:00Z

    [BEAM-334] DataflowPipelineRunner: bump environment major version

commit f0e266bbd1f5781f5280703876b9fd76c7836090
Author: Jesse Anderson <>
Date:   2016-05-16T20:58:08Z

    Changed Word Counts to use TypeDescriptors.

commit 50c0ed04784cff0454d89fcacab24e2a4f051edb
Author: Jesse Anderson <>
Date:   2016-05-16T21:09:18Z

    Updated complete examples to use TypeDescriptors.

commit 60964b611638178787106d422063a2cdfe6b9902
Author: Dan Halperin <>
Date:   2016-06-12T19:33:01Z

    Closes #447

commit dd07cba31ef1686495562914de13f1ca24ff9ce2
Author: Kenneth Knowles <>
Date:   2016-06-13T01:29:46Z

    Add success/failure counters to new PAssert mechanism

commit 99c596ebcfa481c043c3f26b8085ddb4f0867ef5
Author: manuzhang <>
Date:   2016-06-13T03:09:38Z

    [BEAM-336] update examples-java README

commit a2abc6a249cdc4e6000d1539df6c3b5cde8d39b0
Author: Maximilian Michels <>
Date:   2016-06-10T12:26:45Z

    [flink] fix potential NPE in ParDoWrapper

commit be05942da0f09a247e195d1d29513ae40e1a95e0
Author: Maximilian Michels <>
Date:   2016-06-13T12:57:31Z

    This closes #450

commit 51030829ef53001eb7db608fd56bcf1b4bea4249
Author: Scott Wegner <>
Date:   2016-06-10T19:52:43Z

    Port cleanupDaemonThreads fix to archetype module

commit fe5b8db6cfefc67e8d33d4673f94611c70a0df7e
Author: Luke Cwik <>
Date:   2016-06-13T16:48:09Z

    [BEAM-330] Port cleanupDaemonThreads fix to archetype module
    This closes #444.

commit a3feeefa2db3f759f88cacd8cb89268263ff955a
Author: Kenneth Knowles <>
Date:   2016-06-13T17:11:22Z

    This closes #448

commit dbddb9f3cda5b8052e5a0e5117a1d57508b229f9
Author: Kenneth Knowles <>
Date:   2016-06-13T18:04:25Z

    This closes #393

commit 9e0d7d650a04a63c7e9b392f938990cea90b108e
Author: Kenneth Knowles <>
Date:   2016-06-13T18:16:46Z

    This closes #449

commit 335202a033ced6f30f1b0e5df9da047241abc750
Author: Scott Wegner <>
Date:   2016-06-09T18:31:23Z

    Make example AddTimestampFn range deterministic
    The timestamps added in the WindowedWordCount example are currently
    based on when the bundles are executed, which makes the min/max bounds
    non-deterministic. This change makes the range  based on the
    construction time.

commit e1b305ea5d48e0f5f71ad0030c20c492b0f95c30
Author: bchambers <>
Date:   2016-06-13T18:21:31Z

    This closes #441

commit 874e199efbaa02e7299b2919b55a0c5f73287257
Author: Scott Wegner <>
Date:   2016-05-16T21:01:43Z

    Add configuration for Dataflow runner System.out/err

commit 915bc198dd8a616e6fe43b6d77f74619310e5b97
Author: Luke Cwik <>
Date:   2016-06-13T21:29:24Z

    [BEAM-327] Add configuration for Dataflow runner System.out/err
    This closes #426.

commit e60e3e6de85a022d4669333c544b0b86145d0d55
Author: Scott Wegner <>
Date:   2016-06-13T18:05:00Z

    Fix AutoComplete example streaming configuration

commit 5227624526334ef18dc2109978f309af2a7d3521
Author: bchambers <>
Date:   2016-06-14T00:17:27Z

    This closes #445

commit 0abeb068cd3fead73c664a00c0f01a71627658ef
Author: Dan Halperin <>
Date:   2016-06-14T00:34:58Z

    CompressedSourceTest: simplify
    We should use random.nextBytes(buff) instead of making the array in a loop.
    The code we now point to is the same as the for loop, so the test continues to pass.

commit f9a9214ddd688199ae4495bfff08d92b2fe91202
Author: Dan Halperin <>
Date:   2016-06-14T01:20:45Z

    Closes #456

commit 045b568f6be4b7b010d4fd4cfdd1536db943ce54
Author: Kenneth Knowles <>
Date:   2016-06-14T15:05:04Z

    Revert GBK-based PAssert
    This changed neglected the use of counters by the Dataflow runner,
    which is used to prevent tests for spuriously passing when
    a PCollection is empty.
    Obvious fixes for that revealed probable bugs in the in-process
    and Spark runner, as well as tests that happen to work with
    PAssert but are actually unsupported.
    A proper long-term fix is underway to address all of the above.
    In the meantime, this commit rolls back the changes to PAssert.

commit c8ad2e7dd5443ba40d126dca4cd3cb29b33103cf
Author: Kenneth Knowles <>
Date:   2016-06-14T15:52:03Z

    This closes #457


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message