beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From echauc...@apache.org
Subject [beam] branch spark-runner_structured-streaming updated (8cdd143 -> f0522dc)
Date Wed, 26 Jun 2019 15:22:38 GMT
This is an automated email from the ASF dual-hosted git repository.

echauchot pushed a change to branch spark-runner_structured-streaming
in repository https://gitbox.apache.org/repos/asf/beam.git.


    from 8cdd143  Add TODO in Combine translations
     new 8a4372d  Update KVHelpers.extractKey() to deal with WindowedValue and update GBK
and CPK
     new 8d05d46  Fix comment about schemas
     new bba08b4  Implement reduce part of CombineGlobally translation with windowing
     new 4602f83  Output data after combine
     new 8f8bae4  Implement merge accumulators part of CombineGlobally translation with windowing
     new 9a269ef  Fix encoder in combine call
     new f0522dc  [to remove] temporary: revert extractKey while combinePerKey is not done
(so that it compiles)

The 7 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../batch/AggregatorCombinerGlobally.java          | 205 ++++++++++++++++++---
 .../batch/CombineGloballyTranslatorBatch.java      |  22 +--
 .../batch/CombinePerKeyTranslatorBatch.java        |  10 +-
 .../batch/GroupByKeyTranslatorBatch.java           |   4 +-
 .../translation/helpers/KVHelpers.java             |   5 +-
 .../translation/helpers/ReduceFnRunnerHelpers.java |  77 ++++++++
 6 files changed, 269 insertions(+), 54 deletions(-)
 create mode 100644 runners/spark/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/ReduceFnRunnerHelpers.java


Mime
View raw message