beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ib...@apache.org
Subject [beam] branch master updated (0851849 -> 9bceff5)
Date Fri, 03 Jan 2020 21:36:54 GMT
This is an automated email from the ASF dual-hosted git repository.

ibzib pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


    from 0851849  [BEAM-5605] Ensure that split calls are routed to the active bundle processor
for the bundle id. (#10489)
     new bf6577e  Import freezegun for Python time testing.
     new b329de6  Allow message stream to yield duplicates.
     new 4479865  [BEAM-8891] Create and submit Spark portable jar in Python.
     new 6f7b2b2  Refactor shared uber jar generation code into common subclass.
     new 4996c14  Make Spark REST URL a separate pipeline option.
     new 9bceff5  Merge pull request #10403 from ibzib/spark-jar

The 24863 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../beam/runners/spark/SparkPipelineRunner.java    |   4 +
 .../python/apache_beam/options/pipeline_options.py |  22 ++
 .../runners/portability/abstract_job_service.py    |  78 ++++++-
 .../portability/flink_uber_jar_job_server.py       |  71 +-----
 .../runners/portability/portable_runner.py         |  11 +-
 .../runners/portability/spark_runner.py            |  30 ++-
 .../portability/spark_uber_jar_job_server.py       | 255 +++++++++++++++++++++
 .../portability/spark_uber_jar_job_server_test.py  | 209 +++++++++++++++++
 sdks/python/setup.py                               |   1 +
 9 files changed, 593 insertions(+), 88 deletions(-)
 create mode 100644 sdks/python/apache_beam/runners/portability/spark_uber_jar_job_server.py
 create mode 100644 sdks/python/apache_beam/runners/portability/spark_uber_jar_job_server_test.py


Mime
View raw message