beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Halperin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-775) Remove Aggregators from the Java SDK
Date Tue, 02 May 2017 23:46:04 GMT

    [ https://issues.apache.org/jira/browse/BEAM-775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994041#comment-15994041
] 

Daniel Halperin commented on BEAM-775:
--------------------------------------

Remaining work:

* Rebuild DataflowWorker to stop using the three deprecated functions.
* Update the SDK with the new worker and remove the three deprecated functions.
* run a recursive grep on the code to check uses of the word Aggregator. E.g., 

{code}
[16:38:40 dhalperi@dhalperi:beam 6bf4ab7b0 ] grep -i aggregator -R . | grep -v "py:"  | grep
-v spark                           []
./runners/core-java/src/main/java/org/apache/beam/runners/core/DoFnRunners.java:      Object
aggregatorFactory,
./examples/java/src/main/java/org/apache/beam/examples/WindowedWordCount.java: * and using
a selected runner; defining DoFns; creating a custom aggregator;
./sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/DoFnTesterTest.java:   * A {@link
DoFn} that adds values to an aggregator and converts input to String in
./sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/LatestFnTest.java:  public void
testExtractOutputDefaultAggregator() {
./sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ApproximateUniqueTest.java:
* Tests for the ApproximateUnique aggregator transform.
./sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Aggregator.java: * An {@code
Aggregator<InputT>} enables monitoring of values of type {@code InputT},
./sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Aggregator.java:public interface
Aggregator<InputT, OutputT> {
./sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Latest.java: * <p>Example
2: track a latest computed value in an aggregator:
./sdks/java/core/src/main/java/org/apache/beam/sdk/util/SystemDoFnInternal.java: * <p>Currently,
the only effect of this is to mark any aggregators reported by an annotated
Binary file ./sdks/java/core/target/classes/org/apache/beam/sdk/transforms/Aggregator.class
matches
./sdks/java/maven-archetypes/examples/src/main/resources/archetype-resources/src/main/java/WindowedWordCount.java:
* and using a selected runner; defining DoFns; creating a custom aggregator;
./sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java/WindowedWordCount.java:
* and using a selected runner; defining DoFns; creating a custom aggregator;
{code}

> Remove Aggregators from the Java SDK
> ------------------------------------
>
>                 Key: BEAM-775
>                 URL: https://issues.apache.org/jira/browse/BEAM-775
>             Project: Beam
>          Issue Type: Sub-task
>          Components: sdk-java-core
>            Reporter: Ben Chambers
>            Assignee: Pablo Estrada
>              Labels: backward-incompatible
>             Fix For: First stable release
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message