beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From amitsela <>
Subject [GitHub] incubator-beam pull request #822: [BEAM-549] SparkRunner should support Beam...
Date Fri, 12 Aug 2016 22:00:00 GMT
GitHub user amitsela opened a pull request:

    [BEAM-549] SparkRunner should support Beam's KafkaIO instead of providing it's own.

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
     - [ ] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [ ] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](

You can merge this pull request into a Git repository by running:

    $ git pull BEAM-549

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #822
commit 960b4e9847df3d3fd5b9ab99bcfbeae6b2c24f01
Author: Sela <>
Date:   2016-08-12T21:40:09Z

    Spark Kafka io support via an adapting PTransform.

commit be567246ff88f9049d1aa206b7d7b556a3a8d565
Author: Sela <>
Date:   2016-08-12T21:41:34Z

    Expose relevant fields as public, they are final anyway.

commit 9414d760fcb1953f55eb99dfce78da56c464c15f
Author: Sela <>
Date:   2016-08-12T21:42:26Z

    Remove SparkRunner's implementation of KafkaIO.

commit 0f822ac80dfdca83cc9204e8784561d5f066b43e
Author: Sela <>
Date:   2016-08-12T21:43:07Z

    Dependency on Beam's KafkaIO.

commit 89f52131b044e6e99c96f9bb5741050233422ebc
Author: Sela <>
Date:   2016-08-12T21:44:23Z

    Translate reading of Kafka using Beam's KafkaIO, let the runner override with it's own

commit c1965659d5b477fe33c2ab396e9067ce3c2967a7
Author: Sela <>
Date:   2016-08-12T21:45:52Z

    Adapt tests.


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message