beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ieme...@apache.org
Subject [beam] branch spark-runner_structured-streaming updated (2c5d37a -> 174595b)
Date Thu, 01 Aug 2019 15:33:06 GMT
This is an automated email from the ASF dual-hosted git repository.

iemejia pushed a change to branch spark-runner_structured-streaming
in repository https://gitbox.apache.org/repos/asf/beam.git.


 discard 2c5d37a  Fix hardcoded value
 discard b63892e  Use "sparkMaster" in local mode to obtain number of shuffle partitions + spotless apply
 discard 228b924  Improve Pardo translation performance: avoid calling a filter transform when there is only one output tag
 discard 446e221  Add a TODO on perf improvement of Pardo translation
 discard c088517  After testing performance and correctness, launch pipeline with dataset.foreach(). Make both test mode and production mode use foreach for uniformity. Move dataset print as a utility method
 discard 3d20031  Remove no more needed AggregatorCombinerPerKey (there is only AggregatorCombiner)
 discard 3e6a9e2  fixup! Add PipelineResults to Spark structured streaming.
 discard 9a1e4dc  Print number of leaf datasets
 discard b62a593  Add spark execution plans extended debug messages.
 discard 935e703  Update log4j configuration
 discard 797b40d  Add PipelineResults to Spark structured streaming.
    omit b9c8925  Make spotless happy
    omit 50b3a97  Added metrics sinks and tests
    omit df41a8b  Persist all output Dataset if there are multiple outputs in pipeline Enabled Use*Metrics tests
    omit a37817d  Add a test to check that CombineGlobally preserves windowing
    omit 170568b  Fix accumulators initialization in Combine that prevented CombineGlobally to work.
    omit 0fc97ee  Fix javadoc
    omit a3d1061  Add setEnableSparkMetricSinks() method
    omit f88db1b  Add missing dependencies to run Spark Structured Streaming Runner on Nexmark
    omit 12eb126  Add metrics support in DoFn
    omit 703e773  Ignore for now not working test testCombineGlobally
    omit 5bb44d6  Add a test that combine per key preserves windowing
    omit bea2c60  Clean groupByKeyTest
    omit 777be24  add comment in combine globally test
    omit 78270bf  Fixed immutable list bug
    omit 721b1d0  Fix javadoc of AggregatorCombiner
    omit ea959c6  Clean not more needed WindowingHelpers
    omit e84c0bb  Clean not more needed RowHelpers
    omit 2851c72  Clean no more needed KVHelpers
    omit f05e17c  Now that there is only Combine.PerKey translation, make only one Aggregator
    omit 18a2e2a  Remove CombineGlobally translation because it is less performant than the beam sdk one (key + combinePerKey.withHotkeyFanout)
    omit 6450bda  Remove the mapPartition that adds a key per partition because otherwise spark will reduce values per key instead of globally
    omit 11981e5  Fix bug in the window merging logic
    omit ddce346  Fix wrong encoder in combineGlobally GBK
    omit 3ae7e09  Fix case when a window does not merge into any other window
    omit e77e951  Apply a groupByKey avoids for some reason that the spark structured streaming fmwk casts data to Row which makes it impossible to deserialize without the coder shipped into the data. For performance reasons (avoid memory consumption and having to deserialize), we do not ship coder + data. Also add a mapparitions before GBK to avoid shuffling
    omit b1cf536  Revert extractKey while combinePerKey is not done (so that it compiles)
    omit 28067b2  Fix encoder in combine call
    omit a4af031  Implement merge accumulators part of CombineGlobally translation with windowing
    omit 3e49bf0  Output data after combine
    omit 233e532  Implement reduce part of CombineGlobally translation with windowing
    omit a9e7bf4  Fix comment about schemas
    omit 05cb7b8  Update KVHelpers.extractKey() to deal with WindowedValue and update GBK and CPK
    omit 1a85496  Add TODO in Combine translations
    omit 92bf293  Add a test that GBK preserves windowing
    omit 41de6da  Build fixup: enable UsesBoundedSplittableParDo category of tests Since the default overwrite is already in place and it is breaking only in one subcase
    omit ee91d24  Improve visibility of debug messages
    omit 6c9a3a8  re-enable reduceFnRunner timers for output
    omit 6bc9b17  Re-code GroupByKeyTranslatorBatch to conserve windowing instead of unwindowing/windowing(GlobalWindow): simplify code, use ReduceFnRunner to merge the windows
    omit ede9118  Add comment about checkpoint mark
    omit ab26efc  Update windowAssignTest
    omit 9d29d76  Put back batch/simpleSourceTest.testBoundedSource
    omit 9c3b009  Consider null object case on RowHelpers, fixes empty side inputs tests.
    omit ebc6ca9  Build fixup: hadoop-format is not mandataory to run ValidatesRunner tests
    omit cb1158b  Pass transform based doFnSchemaInformation in ParDo translation
    omit ff50759  Build fixup: enable UsesFailureMessage and UsesSchema categories on validatesRunner tests
    omit 1f4f0bc  Fixes ParDo not calling setup and not tearing down if exception on startBundle
    omit 1d52f35  Build fixup: Limit the number of partitions to make tests go 300% faster
    omit 82e28d1  Build fixup:  enable batch Validates Runner tests for Structured Streaming Runner
    omit 72f18c7  Apply Spotless
    omit a75e88a  Update javadoc
    omit 82ed0fc  implement source.stop
    omit e04d455  Ignore spark offsets (cf javadoc)
    omit 4b568da  Use PAssert in Spark Structured Streaming transform tests
    omit 9d6caa1  Rename SparkPipelineResult to SparkStructuredStreamingPipelineResult This is done to avoid an eventual collision with the one in SparkRunner. However this cannot happen at this moment because it is package private, so it is also done for consistency.
    omit 4f0e0e6  Add SparkStructuredStreamingPipelineOptions and SparkCommonPipelineOptions - SparkStructuredStreamingPipelineOptions was added to have the new   runner rely only on its specific options.
    omit 45781a8  Fix logging levels in Spark Structured Streaming translation
    omit 1d48ae5  Fix spotless issues after rebase
    omit 8602eaa  Pass doFnSchemaInformation to ParDo batch translation
    omit f36c3b8  Fix non-vendored imports from Spark Streaming Runner classes
    omit 07b01b1  Merge Spark Structured Streaming runner into main Spark module. Remove spark-structured-streaming module.Rename Runner to SparkStructuredStreamingRunner. Remove specific PipelineOotions and Registrar for Structured Streaming Runner
    omit 51630e2  Fix access level issues, typos and modernize code to Java 8 style
    omit df02413  Disable never ending test SimpleSourceTest.testUnboundedSource
    omit 85635ff  Apply spotless and fix spotbugs warnings
    omit 1bc4d75  Deal with checkpoint and offset based read
    omit 559e80f  Continue impl of offsets for streaming source
    omit b9af1b5  Clean streaming source
    omit 4ef8635  Clean unneeded 0 arg constructor in batch source
    omit e0ecd02  Specify checkpointLocation at the pipeline start
    omit a14698f  Add source streaming test
    omit c46ce22  Add transformators registry in PipelineTranslatorStreaming
    omit 10fac36  Add a TODO on spark output modes
    omit 9803c07  Implement first streaming source
    omit cb261f2  Add streaming source initialisation
    omit 8bd0f66  And unchecked warning suppression
    omit 9ff127d  Added TODO comment for ReshuffleTranslatorBatch
    omit 519c9b2  Added using CachedSideInputReader
    omit d629b1c  Don't use Reshuffle translation
    omit d705c22  Fix CheckStyle violations
    omit 2128544  Added SideInput support
    omit 8e029bd  Fix build wordcount:disable deps versions forcing, remove fail on warnings
    omit f7f3b54  Fix javadoc
    omit 07bceca  Implement WindowAssignTest
    omit dafe5c4  Implement WindowAssignTranslatorBatch
    omit e712187  Cleaning
    omit 976783c  [TO UPGRADE WITH THE 2 SPARK RUNNERS BEFORE MERGE] Change de wordcount build to test on new spark runner
    omit 2a8d92e  Fix encoder bug in combinePerkey
    omit 0bd6370  Add explanation about receiving a Row as input in the combiner
    omit 5d86507  Use more generic Row instead of GenericRowWithSchema
    omit c453b90  Fix combine. For unknown reason GenericRowWithSchema is used as input of combine so extract its content to be able to proceed
    omit ac7f271  Update test with Long
    omit 3ede119  Fix various type checking issues in Combine.Globally
    omit 571b3e9  Get back to classes in translators resolution because URNs cannot translate Combine.Globally
    omit b390509  Cleaning
    omit 0c48a94  Add CombineGlobally translation to avoid translating Combine.perKey as a composite transform based on Combine.PerKey (which uses low perf GBK)
    omit 7629b22  Introduce RowHelpers
    omit 11f6322  Add combinePerKey and CombineGlobally tests
    omit 9b3c604  Fix combiner using KV as input, use binary encoders in place of accumulatorEncoder and outputEncoder, use helpers, spotless
    omit 6da1c02  Introduce WindowingHelpers (and helpers package) and use it in Pardo, GBK and CombinePerKey
    omit 9407138  Improve type checking of Tuple2 encoder
    omit d1bafd1  First version of combinePerKey
    omit c7256da  Extract binary schema creation in a helper class
    omit 05a1a11  Fix getSideInputs
    omit 28d0be3  Generalize the use of SerializablePipelineOptions in place of (not serializable) PipelineOptions
    omit a33477f  Rename SparkDoFnFilterFunction to DoFnFilterFunction for consistency
    omit 3bbb0d1  Add a test for the most simple possible Combine
    omit d7104ee  Added "testTwoPardoInRow"
    omit 2b32d52  Fix for test elements container in GroupByKeyTest
    omit 8ede26e  Rename SparkOutputManager for consistency
    omit 106e54e  Fix kryo issue in GBK translator with a workaround
    omit 72bf4d3  Simplify logic of ParDo translator
    omit c8cefc8  Don't use deprecated sideInput.getWindowingStrategyInternal()
    omit 1f6c3b7  Rename SparkSideInputReader class and rename pruneOutput() to pruneOutputFilteredByTag()
    omit 8407488  Fixed Javadoc error
    omit 6efe901  Apply spotless
    omit fe00df0  Fix Encoders: create an Encoder for every manipulated type to avoid Spark fallback to genericRowWithSchema and cast to allow having Encoders with generic types such as WindowedValue<T> and get the type checking back
    omit 24a0f7d  Fail in case of having SideInouts or State/Timers
    omit d3c6e42  Add ComplexSourceTest
    omit e4948df  Remove no more needed putDatasetRaw
    omit 4f55612  Port latest changes of ReadSourceTranslatorBatch to ReadSourceTranslatorStreaming
    omit 55a6419  Fix type checking with Encoder of WindowedValue<T>
    omit 6eb2ff1  Add comments and TODO to GroupByKeyTranslatorBatch
    omit df89095  Add GroupByKeyTest
    omit 8e999c9  Clean
    omit cd58952  Address minor review notes
    omit 32bf4da  Add ParDoTest
    omit afc53ee  Clean
    omit bab98f1  Fix split bug
    omit 6b87e80  Remove bundleSize parameter and always use spark default parallelism
    omit 53ddaa0  Cleaning
    omit 152d858  Fix testMode output to comply with new binary schema
    omit 355fa9e  Fix errorprone
    omit 80aa7ea  Comment schema choices
    omit d9d424f  Serialize windowedValue to byte[] in source to be able to specify a binary dataset schema and deserialize windowedValue from Row to get a dataset<WindowedValue>
    omit 000381c  First attempt for ParDo primitive implementation
    omit 5067ca1  Add flatten test
    omit 21c426e  Enable gradle build scan
    omit 7c4ff5c  Enable test mode
    omit 33754a8  Put all transform translators Serializable
    omit b39024a  Simplify beam reader creation as it created once the source as already been partitioned
    omit b6a6912  Fix SourceTest
    omit 155a65b  Move SourceTest to same package as tested class
    omit 4d18207  Add serialization test
    omit 8ba03d6f Add SerializationDebugger
    omit ec13b5b  Fix serialization issues
    omit 0cfde2b  Clean unneeded fields in DatasetReader
    omit 3b868db  improve readability of options passing to the source
    omit 38b1514  Fix pipeline triggering: use a spark action instead of writing the dataset
    omit 6173a33  Deactivate deps resolution forcing that prevent using correct spark transitive dep
    omit 4f6d5b9  Refactor SourceTest to a UTest instaed of a main
    omit ab70e9e  Checkstyle and Findbugs
    omit b9acd02  Clean
    omit 2bc0d23  Add empty 0-arg constructor for mock source
    omit 0bdbf93  Add a dummy schema for reader
    omit 73d8c71  Apply spotless and fix  checkstyle
    omit c38d1d2  Use new PipelineOptionsSerializationUtils
    omit af3dfad  Add missing 0-arg public constructor
    omit 84f1963  Wire real SourceTransform and not mock and update the test
    omit 0c64b46  Refactor DatasetSource fields
    omit a96039e  Pass Beam Source and PipelineOptions to the spark DataSource as serialized strings
    omit 8bfa1a5  Move Source and translator mocks to a mock package.
    omit 5559890  Add ReadSourceTranslatorStreaming
    omit 2dca3c2  Clean
    omit 98d6e1e  Use raw Encoder<WindowedValue> also in regular ReadSourceTranslatorBatch
    omit 772efc4  Split batch and streaming sources and translators
    omit 0b9875f  Run pipeline in batch mode or in streaming mode
    omit a0243bf  Move DatasetSourceMock to proper batch mode
    omit 80b91bd  clean deps
    omit b5cc52a  Use raw WindowedValue so that spark Encoders could work (temporary)
    omit f99d1d1  fix mock, wire mock in translators and create a main test.
    omit 8a0343a  Add source mocks
    omit bc6b4e6  Experiment over using spark Catalog to pass in Beam Source through spark Table
    omit 6cfdf31  Improve type enforcement in ReadSourceTranslator
    omit aec130e  Improve exception flow
    omit 9b3a72e  start source instanciation
    omit 4b55ed6  Apply spotless
    omit 039976a  update TODO
    omit dfcfa6a  Implement read transform
    omit 6a9021d  Use Iterators.transform() to return Iterable
    omit 3491edf  Add primitive GroupByKeyTranslatorBatch implementation
    omit b045f0d  Add Flatten transformation translator
    omit 6ff5192  Create Datasets manipulation methods
    omit 7d7dcc6  Create PCollections manipulation methods
    omit e3ac83a  Add basic pipeline execution. Refactor translatePipeline() to return the translationContext on which we can run startPipeline()
    omit 52da1ea  Added SparkRunnerRegistrar
    omit d23f4fd  Add precise TODO for multiple TransformTranslator per transform URN
    omit 60bc61b  Post-pone batch qualifier in all classes names for readability
    omit 419211a  Add TODOs
    omit 3504543  Make codestyle and firebug happy
    omit af2e92b  apply spotless
    omit e4a647e  Move common translation context components to superclass
    omit cc80092  Move SparkTransformOverrides to correct package
    omit 1cd8dab  Improve javadocs
    omit b8e9f7a  Make transform translation clearer: renaming, comments
    omit 0621611  Refactoring: -move batch/streaming common translation visitor and utility methods to PipelineTranslator -rename batch dedicated classes to Batch* to differentiate with their streaming counterparts -Introduce TranslationContext for common batch/streaming components
    omit 2fa095a  Initialise BatchTranslationContext
    omit 77286bd  Organise methods in PipelineTranslator
    omit 72e7fb9  Renames: better differenciate pipeline translator for transform translator
    omit 73d37f2  Wire node translators with pipeline translator
    omit 4ecaa74  Add nodes translators structure
    omit e837be2  Add global pipeline translation structure
    omit 721fa2d  Start pipeline translation
    omit 4e8f89e  Add SparkPipelineOptions
    omit 56ad12e  Fix missing dep
    omit 474c0dc  Add an empty spark-structured-streaming runner project targeting spark 2.4.0
     add 4c328fe  [BEAM-7005] Add streaming GBK load test
     add 39661a5  Merge pull request #8355: [BEAM-7005] Add streaming GBK load test
     add 6182a40  [BEAM-7228] Add a Jenkins job running ValidatesRunner tests for Direct runner
     add 260677a  Merge pull request #8528: [BEAM-7228] Add a Jenkins job running ValidatesRunner tests for Direct runner
     add eb21fb8  [BEAM-7012] Support TestStream in portable Flink Runner
     add e55ab73  Merge pull request #8411: [BEAM-7012] Support TestStream in portable Flink Runner
     add e21cd54  Fix py3 "Map modified during iteration"
     add a3c99f7  Add and disable tests for optimized portable pipeline
     add c14b6e8  Add Python optimized test for Flink
     add a095321  Merge pull request #8488 from angoenka/py3_fix_iteration
     add 2d671a6  Moving to 2.14.0-SNAPSHOT on master branch.
     add 477a2a0  [BEAM-7143] adding withConsumerConfigUpdates
     add 1ac8e7a  Merge pull request #8398: [BEAM-7143] adding withConsumerConfigUpdates
     add 759ea22  Add Snippets for 3 new Beam Patterns 2 X FilePatterns 1 X SideInput Pattern
     add 311edfd  Merge pull request #8538 from rezarokni/Beam-Patterns
     add dae6e60  [BEAM-7257] adding withProducerConfigUpdates
     add 8154ec5  Merge pull request #8541: [BEAM-7257] adding withProducerConfigUpdates
     add 6643853  Update reviewer list for Py3 contributions.
     add ada26ba  Merge pull request #8529 from tvalentyn/patch-49
     add 7d8447e  [BEAM-7263] Deprecate set/getClientConfiguration in JdbcIO
     add 765fe3b  Merge pull request #8547 from iemejia/BEAM-7263-jdbc-deprecations
     add ac48aab  [BEAM-7265] Update Spark runner to use spark version 2.4.3
     add 6679b00  Merge pull request #8548: [BEAM-7265] Update Spark runner to use spark version 2.4.3
     add 40da292  [BEAM-7145] Make FlinkRunner compatible with Flink 1.8
     add 01e9878  Merge pull request #8540: [BEAM-7145] Make FlinkRunner compatible with Flink 1.8
     add c4dbe12  [BEAM-7260] UTF8 coder is breaking dataflow tests
     add da4bb49  Merge pull request #8545 from ihji/BEAM-7260
     add 8d59768  [BEAM-7267] Add python3*-dev to verify script
     add f310234  Merge pull request #8551 from angoenka/fix_verify_scipt
     add 2650c52  Downgrade missing coder error logs to info logs.
     add 117a2d6  Merge pull request #8533 from aaltay/errtinf
     add cc8bc6c  Revert "Merge pull request #8228: [BEAM-7008] adding UTF8 String coder to Java SDK ModelCoders"
     add 466038d  Revert "Merge pull request #8545 from ihji/BEAM-7260"
     add 0a1ec0c  Merge pull request #8544 from angoenka/revert_utf8
     add c02af60  [BEAM-4858] Increase tolerance on linear regression tests (#8556)
     add c9142ad2 [BEAM-6627] Generify ByteMonitor to accept any type, not just KV<byte[], byte[]>
     add d2f0631  [BEAM-6627] Add CountMonitor to IOIT metric utils
     add 59e68a2  [BEAM-6627] Add byte and element counting to TextIOIT
     add b8d7fe8  [BEAM-6627] Add byte and element counting to AvroIOIT
     add 74c0a47  [BEAM-6627] Add byte and element counting to ParquetIOIT
     add 535c7cf  [BEAM-6627] Add byte and element count metric to XmlIOIT
     add b022a1a  [BEAM-6627] Add byte and element count metric in TFRecordIOIT
     add a57c679  [BEAM-6627] Add byte and element counter metric to JdbcIOIT
     add bdefbef  [BEAM-6627] Add byte and item count metrics to MongoDBIOIT
     add 30aa39f  [BEAM-6627] Add byte and item count metrics to JdbcIOIT
     add a16a5b7  Merge pull request #8400: [BEAM-6627] Added byte and item counting metrics to integration tests
     add 0acb798  [BEAM-6627] Add item and byte counters to HadoopFormatIOIT
     add 9cc27c3  Merge pull request #8561: [BEAM-6627] Add item and byte counters to HadoopFormatIOIT
     add 6a36ad1  Updating python containers to beam-master-20190509
     add b780403  Merge pull request #8546 from angoenka/update_container_20190509
     add 7bc1f48  [BEAM-7269] remove StateSpec from hashCode of SimpleStateTag
     add b6f22ef  [BEAM-7269] remove StateSpec from hashCode of SimpleStateTag
     add 6c1d448  Small doc clean up
     add b4f1e37  [SQL] SpotlessApply for minor docs PR #8567
     add 1715194  Merge pull request #8567 from akedin/minor-docs-cleanup
     add 1e7ea0d  [BEAM-4046] decouple gradle project names and maven artifact ids
     add f198de0  [BEAM-4046] add project mappings and enable parsing of legacy commands
     add abece47  Merge pull request #8194: [BEAM-4046] decouple gradle project names and maven artifact ids
     add 460fbd3  [BEAM-7281] Update Python deps page to current release
     add dca4fe8  Merge pull request #8568: [BEAM-7281] Update Python deps page to current release
     add d618dc1  [BEAM-7253] test_with_jar_packages_invalid_file_name test fails on windows
     add 2527817  Merge pull request #8537: [BEAM-7253] test_with_jar_packages_invalid_file_name test fails on Windows
     add 9cb1c3c  [BEAM-7181][BEAM-7243] add streaming it tests python 3.6
     add 8d9abd0  Merge pull request #8550 from Juta/it-tests
     add 4629e82  [BEAM-562] Add DoFn.setup and DoFn.teardown to Python SDK (#7994)
     add 52be416  [BEAM-7282] portable Spark: don't fuse already optimized graph
     add 0cb53c1  Merge pull request #8569 from ibzib/dont-fuse
     add b0b2ac2  Beam community update blogpost (#8559)
     add c03bbec  [BEAM-7239] Do not close DataSource on Teardown on JdbcIO
     add 5c7ee60  Merge pull request #8565 from iemejia/beam-master
     add 0a5c700  [BEAM-7126] Remove LengthPrefixCoder from Flink's key Coder
     add 2b0e572  Merge pull request #8472: [BEAM-7126] Remove redundant key encoding for state backend
     add 126d8a0  Use a singleton PIPELINE_OPTION_PARAMETER in DoFnSignature.Parameter
     add eb302dd  Merge #8341: Use a singleton PIPELINE_OPTION_PARAMETER in DoFnSignature.Parameter
     add 2f6d730  [BEAM-7271] adding UTF8 String coder to Java SDK ModelCoder
     add d9430fc  Merge pull request #8557: [BEAM-7271] adding UTF8 String coder to Java SDK ModelCoder
     add 7e3559b  Fix trigger paths filters to execute precommit tests only if needed
     add a648016  Merge pull request #8483 from iemejia/fix-precommits-triggers
     add 0fdfaca  [BEAM-7028] Change ParDo load test according to the proposal, add a jenkins job for the load test
     add 3b7f085  Merge pull request #8539 from mwalenia/BEAM-7028-pardo-load-test
     add 891e52f  Add first version of Hazelcast Jet based Java Runner
     add 5674996  Merge pull request #8410: [BEAM-7305] Add first version of Hazelcast Jet based Java Runner
     add bfc2471  [website] Update Flink Runner page for version 1.8
     add ed6f68a  Merge pull request #8573: [website] Update Flink Runner page for version 1.8
     add fd627fc  Update references Dataflow to Beam in Flink Runner
     add 41dfd84  Add Coder components for BufferedElement Coder
     add 9b27720  Remove UnboundedSocketSource from Flink Runner
     add a975cc7  Merge pull request #8574: [cleanup] Remove UnboundedSocketSource from Flink Runner
     add b988307  [BEAM-6493] : Add cookbook and snippets examples in Kotlin (#8439)
     add 238659b  Merge pull request #8358: Beam Kata for Java and Python SDKs
     add efb1fa1  Fix project dependencies in pom.xml outputs
     add 690e898  Merge pull request #8577: Fix project dependencies in pom.xml outputs
     add a32c8db  [BEAM-4046] revert test infra to be compatible with pending PRs
     add 26583d6  fix syntax errors in groovy files
     add 3806d52  Merge pull request #8581 from adude3141/BEAM-4046-revert-test-infra
     add 9778f42  [BEAM-7251] Increase timeout for test BQ queries.
     add 36082de  Add missing javadoc to BigqueryClient.java
     add 31785cc  Merge pull request #8536 from udim/bq-test-client
     add c630217  Merge pull request #8425: [BEAM-7174] Add schema modification transforms
     add 32624ad  [BEAM-5865] Create optional auto-balancing sharding function for Flink
     add cc5c897  [BEAM-5865] Add unit tests and fix bug + changes from review
     add 9ce0d8e  [BEAM-5865] Improve FlinkPipelineOptions javadoc
     add 434f0ec  Merge pull request #8499: [BEAM-5865] Create optional auto-balancing sharding function for Flink
     add 98a4d2b  [BEAM-7121] Add deterministic proto coder (#8377)
     add 8b8fe76  [BEAM-7220] add JavaReadViaImpulse substitution to Java expansion service
     add a806c13  Merge pull request #8491 [BEAM-7220] Use JavaReadViaImpulse in Java expansion service
     add 0ad7f12  [BEAM-7311] merge internal commits to beam open source trunk to prepare for the security patch (#8582)
     add 4d54a33  [BEAM-7103] Adding AvroCoderTranslator for cross-language data transfer
     add c52935f  Merge pull request #8342: [BEAM-7103] Adding AvroCoderTranslator for cross-language data transfer
     add 568c3db  upgrade avatica to 1.14.0
     add e2c9571  upgrade to avatica 1.15.0
     add 7b1abc9  Merge pull request #8566 from vectorijk/upgrade-avatica
     add 8e41ab8  [BEAM-6138] Update java SDK to report user distribution tuple metrics over the FN API
     add 0294039  Merge pull request #8280 from ajamato/java_user_distribution
     add 163f781  Categorize tests from UsesTestStream to UsesTestStreamWithProcessingTime
     add f0b03b4  Merge pull request #8586: Categorize tests from UsesTestStream to UsesTestStreamWithProcessingTime
     add 7c92fe0  [BEAM-7154] Updating Go SDK errors (Part 3)
     add 4f9361b  Merge pull request #8560 from youngoli/beam7154-2
     add b253305  Create PAssert util function for late panes.
     add 1945762  [BEAM-7331] Create PAssert util function for late panes.
     add 931a0de  Simple fix: make use of Reify in Latest to replace duplicate code
     add 02ac871  Merge #8589: Simple fix: make use of Reify in Latest to replace duplicate code
     add 2fbb1ee  [BEAM-7287] Change JAVA_HOME to match new Jenkins agents (#8584)
     add 9fe0a03  Modified KinesisIO to use WatermarkPolicyFactory for watermark computation. Added tests
     add 288c95f  Merge pull request #8513: [BEAM-7240] Kinesis IO Watermark Computation Improvements
     add f3dc7dd  [website] Add design doc on portable representation of schemas
     add 07d8d5c  Merge pull request #8591: [website] Add design doc on Portable Beam Schemas
     add feaec61  [BEAM-7230] Add javadoc on provided poolable data sources on JdbcIO
     add b570842  Merge pull request #8595: [BEAM-7230] Add javadoc on provided poolable data sources on JdbcIO
     add 94a97fb  [BEAM-7283] Update aggregated javadoc versions during linking to point to javadoc.io
     add 8cc7c53  [BEAM-7283] Build javadocs using online links for all dependencies instead of a curated list.
     add ece60d8   [BEAM-7283] Update aggregated javadoc versions during linking to point to javadoc.io
     add 62f431f  Add Postcommit test suite for Python 3.7
     add adb6d0c  Merge pull request #8564 from fredo838/add-suites
     add 3cfcb41  Add retry strategy when TableRow or TableCell is null from BQ response.
     add 4691d85  Remove unused attemptNum
     add 38166bc  Merge pull request #8583 from boyuanzz/patch_test
     add d49a79c  [BEAM-6959] Go SDK: support integration tests on Flink
     add 1a09f79  run job server on an unused port
     add 11675bf  Merge pull request #8531 from ibzib/go-flink-subprocess
     add 951cee0  [BEAM-7339] Make input and checksum configurable for Python WordCountIT
     add 5e29d31  Merge pull request #8596 [BEAM-7339] Make input and checksum configurable for Python WordCountIT
     add d77474d  [BEAM-6908] Refactor Python performance test groovy file for easy configuration (#8518)
     add 8a3480f  [BEAM-7302] ensure artifactId set on pom also with unconfigured project
     add 548bbd4  Merge pull request #8594: [BEAM-7302] ensure artifactId set on pom also with unconfigured project
     add 3240ce0  Fix a typo in setting.gradle
     add 2ec133f  Merge pull request #8601 from tvalentyn/patch-50
     add 42ada03  [BEAM-7347] Pass dpb_service_zone flag to benchmarks to fix breakage
     add 15deadf  Merge pull request #8602 from markflyhigh/fix-perf
     add ddbdcd1  [BEAM-7339] Enable 1Gb input for Python wordcount benchmark
     add 21fece9  Merge pull request #8608 from markflyhigh/py-large-input
     add a1cb61a  Updates python SDK GCP client API in sync with the current Google internal definition. (#8554)
     add 38bef4a  [BEAM-7275] ParDoLifeCycleTest: collect lifecycle info for DoFn insta… (#8563)
     add 5687218  [BEAM-7349] Get back to "provided" scope for "kafka_clients"
     add bd6bada  Merge pull request #8600: [BEAM-7349] Get back to "provided" scope for "kafka_clients"
     add a06b7a6  [BEAM-7359] Fix static analysis issues for HadoopFormatIO (#8616)
     add 099fad8  [BEAM-7360] Fix static analysis issues for HCatalogIO (#8617)
     add b936299  Fix the ContainsKv assertion which at times didn't work with some JDKs
     add 4896d47  Merge pull request #8614: Fix the ContainsKv assertion that didn't work with some JDKs
     add 1782292  [BEAM-7154] Fixing small error bug.
     add bb046df  Merge pull request #8610 [BEAM-7154] Fixing small error bug.
     add dabf9c4  [BEAM-7367] Add Kafka clients jar as a Docker dependency
     add 6ab90e9  Merge pull request #8625: [BEAM-7367] Add Kafka clients jar as a Docker dependency
     add 411de45  [BEAM-6988] Implement a Python 3 version of getcallargs_forhints (#8590)
     add b7c23d0  Update README for beam_PerformanceTests_WordCountIT Jenkins job
     add 8bf45e9  Merge pull request #8609 from markflyhigh/update-doc
     add f05c612  Add various Jet Runner fixes & improvements
     add 7d62bb0  Add Jet-Runner to maven archetypes to facilitate easier running of examples
     add 5ccaa72  Merge pull request #8592: [BEAM-7305] Improve and extend Hazelcast Jet based Java Runner
     add 77eb66a  Reorg Beam SQL docs and added Calcite section
     add 7a65876  Merge pull request #8455: [BEAM-6916] Reorg Beam SQL docs and add Calcite section
     add eddc83a  [BEAM-6880] Remove deprecated Reference Runner code. (#8380)
     add 66eb4bf  [BEAM-7376] Update tox version used by gradle
     add 4f76268  Merge pull request #8631 from udim/datastore-version-pip-check
     add 2c9344d  [BEAM-7373] Migrating vet runner
     add 8da20c2  Merge pull request #8632 from youngoli/beam7373
     add cabbce6  [BEAM-7286] RedisIO support for INCRBY/DECRBY operations
     add 74899d6  Merge pull request: [BEAM-7286] RedisIO support for INCRBY/DECRBY operations
     add bec0f3e  [BEAM-7378] Fix Python PVR tests by adding a Kafka client dependency
     add 76a7247  Change scope of runtime dependency to runtime
     add 71759cb  Merge pull request #8633: [BEAM-7378] Fix Python PVR tests by adding a Kafka client dependency
     add 385f842  [BEAM-7378] Add trigger to Python Flink PVR tests for Kafka module
     add 3e976e6  Merge pull request #8634: [BEAM-7378] Add trigger to Python Flink PVR tests for Kafka module
     add ad27400  [BEAM-7135] Spark: close job bundle factory
     add 34d8e50  Merge pull request #8605 [BEAM-7135] Spark: close job bundle factory
     add 8b73a96  [BEAM-7230] Make PoolableDataSourceProvider a static singleton And simplifies all the inner method construction that was error-prone.
     add a37ba1a  Merge pull request #8635 from iemejia/BEAM-7230-jdbc-fix-pool-instantiation
     add fc36c9e  Update stability critical jobs status
     add 79125ae  Merge pull request #8630 from Ardagan/MonthDash
     add 27771ef  Add a passing test to postcommit suites.
     add 088d16c  Merge pull request #8627 from tvalentyn/patch-51
     add 2638883  [BEAM-6872] Add hook for user-defined JVM initialization in workers (#8104)
     add 8946e3e  [BEAM-6908] Enable WordCountIT benchmark on Py36+37
     add 033deea  Merge pull request #8628 from markflyhigh/py36-ws-benchmark
     add a6ae08a  [BEAM-7371] fix dependency check
     add e71af5d  Merge pull request #8615 [BEAM-7371] fix dependency check
     add ac4ff1d  Update .test-infra/jenkins/README
     add a71f305  Merge pull request #8643 from markflyhigh/update-readme
     add 766a765  [BEAM-6695] Latest PTransform for Python SDK (#8206)
     add 5e1c530  [BEAM-7364] Avoid possible signed integer overflow in hash.
     add 6b328be  Merge pull request #8593: [BEAM-7364] Avoid possible signed integer overflow in hash.
     add 938ee3e  [BEAM-7354] Starcgen fix when no identifiers specified. (#8611)
     add e754666  [BEAM-6888] Enable SpotBugs in JdbcIO
     add 67e0eaf  Merge pull request #8647 from iemejia/BEAM-6888-jdbc-enable-spotbugs
     add be8d10f  [BEAM-7366] Include the project's SourceSets in Spotless target
     add c8f3e88  Apply Spotless after fixing source directory detection
     add 809c335  Move StateContextsTest to let it execute as part of the tests
     add 38c62a8  Merge pull request #8624: [BEAM-7366] Include the project's SourceSets in Spotless target
     add 74d8727  [BEAM-5822] Vendor bytebuddy (#8357)
     add 7245eac  [BEAM-7365] Reduce the volume of test data in fastavro branch of test_dynamic_work_rebalancing_exhaustive to match the volume of avro branch.
     add 26a1635  Support sync_interval in TestAvro._write_data() to simplify test scenarios.
     add de08c89  Merge pull request #8646: [BEAM-7365] Reduces the volume of test data in fastavro branch to match the volume of avro branch
     add a7c9c01  [BEAM-6880] Removing python integration test for Java ULR.
     add 2b2d9e0  Merge pull request #8658 [BEAM-6880] Removing python integration test for Java ULR.
     add a9039ba  Make Jet Runner usable and documented
     add 9a7b60f  Merge pull request #8637: [BEAM-7305] Make Jet Runner usable and documented
     add ec70892  Make sfl4j runtime only for Samza Job Server too
     add a9800b6  Merge pull request #8642: Make sfl4j runtime only for Samza Job Server
     add b21ce5b  Merge pull request #8613: [BEAM-7356] Support generic field selections in Filter.
     add 6d75846  [BEAM-7393] Misc. improvements to Go error formatting.
     add 4e20419  Merge pull request #8655 from youngoli/go-errors
     add 8b379b4  [BEAM-7348] Support environment expiration
     add 8e0e83f  Merge pull request #8599: [BEAM-7348] Support environment expiration
     add 8d68bac  [BEAM-7023] WithKeys PTransform for Python SDK
     add f0ed4a9  Merge pull request #8663 from ttanay/beam-7023
     add 8d83576  [BEAM-7203] Signal Dataflow workers to use fastavro library on Python 3. (#8652)
     add bb92d46  [BEAM-7409] Fix minor issues with \ and = in release guide.
     add d810ab6  Merge pull request #8672 from lukecwik/release_guide
     add 4cf2830  [BEAM-6027] Fix slow downloads when reading from GCS (#8553)
     add a87ca3f  [BEAM-4046] Allow running gradlew from repo subdirs
     add 3eaab8b  Merge pull request #8656: [BEAM-4046] Allow running gradlew from repo subdirs
     add 0c50494  [BEAM-6380] Allow rewinding PipeStream
     add f2e9f64  Merge pull request #8661: [BEAM-6380] Allow rewinding PipeStream
     add 7840446  Reduce verbosity of setup.py sdist and futurize
     add b404755  Merge pull request #8657: Reduce verbosity of setup.py sdist and futurize
     add aef01da  [BEAM-7086] Refactoring repetitive error code.
     add 5556467  Merge pull request #8674 from youngoli/beam7086
     add ea32ab9  [BEAM-6284] Improve error message on waitUntilFinish. (#8629)
     add 97edda3  [BEAM-7131] cache executable stage output to prevent re-computation
     add 7775871  Spark portable runner: cache all re-used RDDs by default
     add 65f3c27  refactor to use pushDataset everywhere
     add 05b9dbe  Merge pull request #8558 [BEAM-7131] Spark: cache output to prevent re-computation
     add 4243fa2  Refine runners/reference/java dependencies and fix minor static analysis issues
     add f38f606  Merge pull request #8665: Refine runners/reference/java dependencies and fix minor static analysis issues
     add 51ea73f  [BEAM-7408] Make code example consistent with diagram
     add 3691bf7  Merge pull request #8673: [BEAM-7408] Make code example consistent with diagram
     add 5e161bb  Merge pull request #8670: [BEAM-7408] Make custom options example consistent across languages
     add 29c604d  Fix command on flink runner docs
     add aa8709e  Merge pull request #8678: [BEAM-7417] Fix incorrect command on flink runner docs
     add 7b851e6  Replace assertEquals(boolean, xyz) with assertBoolean(xyz) (#8037)
     add 72dd802  [BEAM-4046] Allow running gradlew from repo subdirs (windows)
     add fab12c7  Merge pull request #8676: [BEAM-4046] Allow running gradlew from repo subdirs (windows)
     add 7a5252e  [BEAM-6693] ApproximateUnique transforms for Python SDK (#8535)
     add 606b18b  [BEAM-7406] Include slf4j-jdk14 in Dataflow worker uber jar
     add 73c6c26  [BEAM-7422] Create new java worker in flink python test
     add dc26d2e  Merge pull request #8682 from angoenka/fix_expansion_task
     add 0f9f757  [BEAM-6429] side-input type coercion for multimaps
     add edfd894  Merge pull request #8654: [BEAM-6429] side-input type coercion for multimaps
     add 23714a3  Attempt to fix lots of URNs.
     add 34c1511  Fix spotless errors
     add c79f879  Update Python Dataflow precommit to use worker jar built locally
     add 064be0e  [BEAM-7387] Attempt to fix lots of URNs to follow common pattern.
     add a91f4f8  splitting support over FnAPI for Go SDK
     add 1809cf4  splitting support over FnAPI for Go SDK. updated
     add 7512579  Update datasource.go
     add 943e765  Merge pull request #8683: [BEAM-5791] splitting support over FnAPI for Go SDK
     add 5d93491  [BEAM-3934] Use try with resources to close the reader on JavaReadViaImpulse
     add 89c61b3  Merge pull request #8648: [BEAM-3934] Use try with resources to close the reader on JavaReadViaImpulse
     add 519e045  Fix small typo in vet.go
     add b1ab819  Merge pull request #8685 from lostluck/patch-4
     add 838c030  [BEAM-7383] Adding strict flag to runners to validate with vet runner
     add e0c7eb6  Merge pull request #8644 from youngoli/beam7383
     add 6054c37  Refactor default bundle size into a constant in TextIO
     add 9428cc8  Merge pull request #8692: Refactor default bundle size into a constant in TextIO
     add 9ed7ca2  [cleanup] Remove dead code from Flink Runner
     add 1da00e2  Merge pull request #8694: [cleanup] Remove dead code from Flink Runner
     add 42cfb31  [BEAM-7421] Add missing transform payload translator for Reshuffle
     add 8bf679c  Merge pull request #8693: [BEAM-7421] Add missing transform payload translator for Reshuffle
     add 75580db  [BEAM-7426] Add underscore support on Field lexer
     add 8304e94  Merge pull request #8689: [BEAM-7426] Add underscore support on Field lexer
     add 882eca8  [BEAM-7141] Add window & timestamp in timer callback method argument (#8408)
     add a7833bc  add error log when fail to begin staging artifact
     add dd0bf1a  Merge pull request #8700 [BEAM-7441] Log when failing to begin staging artifact.
     add 1ffe760  [BEAM-7436] Use coder for Flink's encoded key instead of falling back to Kryo
     add 266ab37  Merge pull request #8696: [BEAM-7436] Use coder for Flink's encoded key instead of falling back to Kryo
     add 2df702a  [BEAM-7114] Move dot pipeline graph renderer to runners-core-construction-java
     add 95297dd  Merge pull request #8649: [BEAM-7114] Move dot pipeline graph renderer to runners-core-construc…
     add 5f772bd  [SQL] Add support for `FLOAT64` to DataCatalogTableProvider
     add 5f83637  Merge pull request #8701 from akedin/dc-float64
     add 23115ce  [cleanup] Remove unncecessary Flink timestamp from UnboundedSourceWrapper
     add 9fc25de  Remove unused Accumulator in SparkGroupAlsoByWindowViaOutputBufferFn
     add fa96678  [BEAM-7448] Remove unneeded external windowedValue in collections for window merging as the windowedValue is already with the elements. Clean related functions and avoid calling so mappers.
     add cd8c6ec  Merge pull request #8711: [BEAM-7448] Remove redundant windowing information and unused accumulators on Spark runner
     add c552ce1  Remove usages of deprecated self.assertEquals in Python codebase
     add 5623d8b  Merge pull request #8612: [BEAM-7101] Remove usages of deprecated self.assertEquals in Python codebase
     add a9d7887  Close grpc channel after pulling messages from pubsub
     add d5d7a7b  Merge pull request #8708 from ostrokach/close_grcp_channel_pubsub
     add 1c26009  [BEAM-5354] Add side input nodes to their consumer's parent-scope.
     add 9647808  Merge pull request #8713 from lostluck/sideforms
     add 11a93ff  [BEAM-3204] Clean-up Coder to only contain FunctionSpec removing SdkFunctionSpec.
     add 1082584  [BEAM-3204] Clean-up Go code related to removing SdkFunctionSpec layer from Coder
     add 35d3d15  [BEAM-3204] Update Python SDK Coder to only use FunctionSpec instead of SdkFunctionSpec
     add a0388f0  fixup! Fix missed Python coder change
     add b6ceda1   [BEAM-3204] Clean-up Coder to only contain FunctionSpec removing its usage of SdkFunctionSpec
     add 07190ee  [BEAM-6769][BEAM-7327][BEAM-7382] add it test for writing and reading with bigqu… (#8621)
     add 9f4ce1c  Update thread count default comment in sdk_worker_main.py
     add 9dbf29c  Merge pull request #8707 from apache/charlesccychen-patch-1
     add 593ed6c  [BEAM-7221] don't test timers on portable Spark runner yet
     add 3455c9f  Merge pull request #8723: [BEAM-7221] don't test timers on portable Spark runner yet
     add de19c75  Merge pull request #7776: Make sure that BigQuery temp tables match the final output table.
     add 767b1a8  [BEAM-7021] added ToString transform on PythonSDK
     add 7943d5f  Merge pull request #8698 from mszb/BEAM-7021
     add 466a17d  Fix build break caused by #7776
     add b715c45  Merge pull request #8726 from apilloud/break
     add b74cb17  [SQL] Add array constructor test
     add e8666a4  Merge pull request #8727 from akedin/array-constructor
     add 5edc1e9  Add flavor text in 1st kata descriptions as suggested by JetBrains Educational Content Manager
     add d59cc7c  Update Stepik for JetBrains review
     add 8607140  Change Kata to Katas
     add 48c5cb2  Merge pull request #8623 from henryken/kata-flavor-text
     add d43b12d  [BEAM-7341] Elements produced by bounded GBK are always ON_TIME and ONLY_FIRING
     add 51c59b2  Merge pull request #8618: [BEAM-7341] Elements produced by bounded GBK are always ON_TIME and ONLY_FIRING
     add fe30366  [BEAM-7332] Add blog post announcing Beam Katas
     add 5f654cf  Merge pull request #8724 from henryken/BEAM-7332-kata-blog-post
     add 3f8ee2b  [BEAM-6623] Complete DataflowValidatesRunner test coverage for Python 3. (#8721)
     add 77cf84c  [Beam 7447] Add Snippets for Options pattern. (#8710)
     add c4c3fd8  [BEAM-7465] Upgrade Jackson to version 2.9.9
     add ecfb432  Merge pull request #8729: [BEAM-7465] Upgrade Jackson to version 2.9.9
     add 14bd7a6  Make metrics flush periodically
     add f4987e2  Rename pipeline option for parallelism
     add d6884d6  Make improvements in documentation
     add 52443d7  Add support for running "NeedsRunner" category of tests
     add 143efd8  Make cooperative ParDo execution an option, when possible
     add 41c865e  Remove previous edge optimization, it's not playing well with file-write sinks
     add 570ad98  Make ParDoLifecycleTest less restrictive
     add 25227c3  Make IMPULSE work in Jet Runner
     add ab6b65e  Merge pull request #8699: [BEAM-7305] Fix issues in and extend documentation for Hazelcast Jet Runner
     add b57c996  [BEAM-7442][BEAM-5650] Read sequentially from bounded sources in UnboundedSourceWrapper
     add 207f34e  Merge pull request #8715: [BEAM-7442][BEAM-5650] Read sequentially from bounded sources in UnboundedSourceWrapper
     add fb51440  [BEAM-7357] KinesisIO: fix too many checks that writing stream exists.
     add 14879ed  Merge pull request #8730: [BEAM-7357] KinesisIO: fix too many checks that writing stream exists.
     add d2dc5c3  Add Precommit it tests for Python 3.
     add c47d8d1  Merge pull request #8728: [BEAM-7459] Add Precommit IT test suites for Python 3
     add c661d18  Transform catalog for Python
     add 2d3bcdc  Merge pull request #8534: [BEAM-7464] Transform catalog for Python
     add b62b571  Downgrade verbose BeamFnMapTaskExecutor log messages
     add 2471e3e  Downgrade verbose BeamFnMapTaskExecutor log messages
     add 784d2c2  [BEAM-7397] Avoid String.format in state namespace key construction
     add c0de457  [BEAM-7397] Avoid String.format in state namespace key construction
     add 15d0341  [BEAM-5878] add tests for kwonly args in python 3 (#8505)
     add 8820f28  [BEAM-5709] BeamFnControlServiceTest - use latch instead of sleep
     add f2236e5  Merge pull request #8671 from youngoli/beam5709
     add 107849c  [BEAM-7412] shut down executor service in fn harness
     add 0b58c88  Merge pull request #8722 from ibzib/gcs-options
     add 265fb6b  Add enable_health_checker flag for Dataflow FnAPI worker
     add 088c011  Merge pull request #8681 from robinyqiu/resilience
     add ae60a72  [BEAM-6959] add beam_PostCommit_Go_VR_Flink job (#8606)
     add e21d28f  Fix float coder test on Windows.
     add 465784a  Merge pull request #8695 from robertwb/windows-float
     add 2959fcc  BEAM-7477 add is not empty helper test method (#8742)
     add f50e7d0  Beam Katas - Update Beam to v2.12.0, javadoc, pydoc to use 'current' (#8735)
     add 8f80722  [BEAM-5995] Added hot key params and method generating hot keys
     add 22d958a  [BEAM-5995] Add Python load tests that uses hot key
     add 63e0170  Merge pull request #8664 from kkucharc/add-hot-key
     add a189e31  Announcing Beam Summit Site (#8075)
     add 8bbdc8c  Prototype for wrapping python BoundedSoruce with SDF
     add 37326d3  [BEAM-7443] Create a BoundedSource -> SDF wrapper in Python SDK
     add fc7dedf  [BEAM-7479] dynamically allocate expansion port in portable runner tests
     add a0c90c7  Merge pull request #8746: [BEAM-7479] dynamically allocate expansion port in portable runner tests
     add 81813c0  Revert "[BEAM-5878] add tests for kwonly args in python 3 (#8505)"
     add 291f1e9  Merge pull request #8750: [BEAM-7481] Revert "[BEAM-5878] add tests for kwonly args in python 3"
     add 5082ef8  [BEAM-7362] Added a performance test for BigQueryIO write
     add 5d617ae  Merge pull request #8626: [BEAM-7362] BigQuery IO write performance test
     add 2839706  [BEAM-2943] Check existence of to-be-staged files in PipelineResources
     add 0cb4275  Merge pull request #8744: [BEAM-2943] Check existence of to-be-staged files in PipelineResources
     add 69f6951  Add support for full unboxing in Convert
     add 79d5671  Merge pull request #8651: Add support for full unboxing in Convert
     add 949c972  [BEAM-7487] Avoid throwing an exception in FlinkRunnerResult#cancel
     add ace6ce8  Merge pull request #8755: [BEAM-7487] Avoid throwing an exception in FlinkRunnerResult#cancel
     add 28bb357  Fixing a typo
     add e3926d8  Merge pull request #8749 from apache/aaltay-patch-1
     add 6d17908  [BEAM-3606] Remove guava-testlib from sdks/java/core main dependency set
     add 19a45bf  [BEAM-3606] Remove guava-testlib from sdks/java/core main dependency set
     add c3ea964  [BEAM-7175] Add Spark Portable ValidatesRunner Batch postcommit test
     add 3710eea  Merge pull request #8432: [BEAM-7175] Add Spark Portable ValidatesRunner Batch postcommit test
     add a69e76a  [BEAM-3606] Remove unused guava_testlib from dependency sets.
     add 0b96270  [BEAM-3606] Remove unused guava_testlib from dependency sets.
     add 1de0517  [BEAM-6620, BEAM-3606] Stop shading by default.
     add 44b042e  fixup! Fix sdks/java/container Kafka dependency
     add 163dbe0  fixup! Fixup JDBC gradle build to use shaded configuration for shaded project.
     add 2c95ac1  Fix Spark ValidatesRunner since the shadowTest classpath before didn't include testCompile dependencies as they were expected to be shaded into the test jar which hadoop-format wasn't doing.
     add 8232336  fixup! Fix publishing to use the correct configuration property, also ensure that the jar classifier when not shading is unspecified.
     add 2ede2b3  [BEAM-7388] Reify PTransform for Python SDK
     add 9fdf6f7  Merge pull request #8717 from ttanay/reify-python
     add 64c56d7  [BEAM-7357] Remove stream existance check
     add 85f51cc  Merge pull request #8758: [BEAM-7357] Remove stream existence check
     add 5b7f365  [BEAM-7351] increment ack deadline for flaky pubsub test
     add 4973c57  Merge pull request #8752 from Juta/flaky-it2
     add a012a3a  Update Beam site to use 2.13.0 as the latest release
     add b2b08d2  Mark old release archived
     add 366cdea  Merge pull request #8645 from angoenka/website_change_2_13
     add 28818e3  [BEAM-2857] Implementing WriteToFiles transform for fileio (Python) (#8394)
     add 5f26c5d  Update Flink artifact name in archetypes
     add a3a608d  Resolve Flink artifact name dynamically
     add 7c8530d  Merge pull request #8753: Update Flink artifact name in archetypes
     add e6104a2  Adding a Sequence Generator for SQL, and posting about SQL data sources (#8542)
     add e42028b  [BEAM-7437] BQ integration test for streaming inserts in streaming
     add 2b74933  Merge pull request #8748 from ttanay/it-streaming-inserts
     add 8097972  Update python containers to beam-master-20190605
     add 7c2eb17  Update python containers to beam-master-20190605
     add ae3a1f2  [BEAM-6356] Add the option to use TFRecord to store cache results using PCollection's PCoder (#8687)
     add e8ecea0  Use artifactId as archivesBaseName for vendor modules
     add 907d269  Merge pull request #8777: [BEAM-7493] Use artifactId as archivesBaseName for vendor modules
     add 6315d6f  Performs some simple clean-up for the tornadoes demo.
     add 3928a55  Merge pull request #8736: Perform some simple clean-up for the tornadoes demo
     add af9c5dc  Add explicit archivesBaseName to applyJavaNature and applyPortabilityNature
     add 9f46698  Merge pull request #8767: [BEAM-7493] Use specified archivesBaseName for project dependencies
     add 2305904  TfIdfTest should not be marked ValidatesRunner
     add d783082  : [BEAM-7490] TfIdfTest should not be marked ValidatesRunner from kennknowles/tfidf
     add 6ffd60e  [BEAM-7499] Skip test since it doesn't work on DirectRunner.
     add 67e3c34  Merge pull request #8779 from lukecwik/beam7499
     add e363d8c  Merge branch 'master' into beam3606
     add b26e622  [BEAM-6620, BEAM-3606] Stop shading by default.
     add 0f9d71a  Fix archivesBaseName method
     add fc8dc15  Merge pull request #8782: Fix archivesBaseName method
     add 97eb7d5  Fix python io developer guide document errors
     add 5319373  Merge pull request #8781: Fix python io developer guide document errors
     add c8631cb  Importing hamcrest dependency at matcher runtime
     add 37dc031  Merge pull request #8778 from pabloem/hamcrest-in-matcher
     add 706fd9c  Fix code block formatting in docstring. (#8756)
     add db82464  [BEAM-6952] fixes concatenated compressed files bug with python sdk (#8187)
     add 28c35be  Increase font and wording for click to expand link.
     add cc5b4da  Merge pull request #8773: [BEAM-7456] Increase font and wording for click to expand link
     add d633888  [BEAM-7492] [BEAM-7491] add Spark runner to Go SDK
     add 4ffdf1c  add badges for Go-on-Spark test
     add c63218d  Merge pull request #8768: [BEAM-7492] [BEAM-7491] add Spark runner to Go SDK
     add 6c37774  Lookup split requests on transform id rather than constant string.
     add 94967b3  Merge pull request #8771 Lookup split requests on transform id rather than constant string.
     add f3d2187  [BEAM-5664] Improve cancellation of portable pipelines
     add 9430d31  Merge pull request #8754: [BEAM-5664] Improve cancellation of portable pipelines
     add db13cfc  [BEAM-5995] Fix: cron job naming
     add 0ca5a1b  Merge pull request #8788: [BEAM-5995] Fix: cron Jenkins job naming
     add 00c0861  Using the default partitioner based on parallelism
     add 2e0e6a8  Merge pull request #8785: [BEAM-7413] Fixes bug in partitioning of GroupNonMergingWindowsFunctions
     add a9fa019  [BEAM-6199] Remove outdated Combine URNs
     add 5ee4cf4  [BEAM-6199] Remove outdated Combine URNs
     add 58be4e8  [BEAM-7507] Use TextFormat to escape workIds to a string rather than utf8
     add 99204ad  [BEAM-7507] Use TextFormat to escape workIds to a string rather than utf8
     add 0ee670e  Test
     add b2329fc  Adding synthetic pipeline changes.
     add 37b53df  Changes to restriction provider.
     add 7117e2e  Fixing some indents..
     add 5832fee  Fixing wrapping indent
     add 6a1371d  Fixing more indents
     add 26ae7a1  Some more
     add abc413a  Some more
     add afa0ffe  And almost done
     add 5d3dc85  Fixing some more things.
     add 8f8b0f2  Fixing OffsetRange, some more whitespace issues, and lint issues.
     add 9a039a1  Review changes.
     add 2154add  FIx for linter
     add 16f13c4  Some small fixes and splitting time between elements.
     add 479bf50  [BEAM-7342] Extend SyntheticPipeline map steps to be splittable.
     add 39d695f  [BEAM-7449] Add pages documenting common Beam pipeline patterns (with code samples)
     add 1d98301  Add work-in-progress drain and checkpoint to runner comparison
     add cd7f4e2  Merge pull request #8576: [BEAM-2888] Add the not-yet-fully-designed drain and checkpoint to runner comparison
     add 532eb97  Beam 2.13.0 Blog Post
     add d516914  Merge pull request #8667 from angoenka/blog_2_13
     add b24e82c  Merge pull request #8620: [BEAM-6673] Add schema support to BigQuery reads
     add c375e89  Skip flaky test
     add 79d478a  Merge pull request #8792 from pabloem/ff
     add 43b3e68  Fix CountingSource example stop_position bug
     add 38166b0  Merge pull request #8794: Fix CountingSource example stop_position bug
     add 13fcf78  [BEAM-6821] FileBasedSink improper paths (#8054)
     add cc9eec9  [BEAM-7512] Replace deprecated self.assertEquals with self.assertEqual
     add 6eef1ba  [BEAM-7512] Fix up some other deprecated usages of self.assertEquals
     add fc24deb  [BEAM-7512] Replace deprecated self.assertEquals with self.assertEqual
     add 89b08e1  [BEAM-7470] Update proto and all SDKs to make the logical data stream over the data plane identified solely by instruction id and transform id.
     add c85f0f5  [BEAM-7470] Update proto and all SDKs to make the logical data stream over the data plane identified solely by instruction id and transform id.
     add ce63125  BEAM-7475: Update programming guide with stateful and timer example (#8740)
     add 25a6874  [BEAM-6620] Remove references to DEFAULT_SHADOW_CLOSURE since it does nothing now.
     add 58634bc  [BEAM-6620] Remove references to DEFAULT_SHADOW_CLOSURE since it does nothing now.
     add b7c2512  Fixes the version number for concat compressed files fix
     add 0eec015  Merge pull request #8784:  [BEAM-6952]Fixes the version number for concat compressed files fix
     add f42d92e  Adding Python samples to the Stateful Processing post. (#8363)
     add d6301f59 fix minor typo
     add b6cb4f4  fix minor typo in BatchLoads.java
     add d445287  Update Python dependencies page for 2.13.0
     add f16b689  Merge pull request #8810: [BEAM-7524] Update Python dependencies page for 2.13.0
     add 3e0b352  [BEAM-7511] Fixes the bug in KafkaTable Initialization.
     add 550c1cc  [BEAM-7511] Fixes the bug in KafkaTable Initialization, returned the deleted methods back.
     add 93d9334  [BEAM-7511] Fixes the bug in KafkaTable Initialization, returned the deleted methods back.
     add b7b0868  Merge pull request #8797 from riazela/master
     add a8d0094  [BEAM-6693] replace mmh3 with default hash function (#8799)
     add 6cb1fd5  [website] Correct date in release blog post
     add bd81d3f  Merge pull request #8806 from mxm/release-blog-post
     add bae5db1  [BEAM-6865] refactor InMemoryBagUserStateFactory to standalone class
     add 2451601  Merge pull request #8802: [BEAM-6865] refactor InMemoryBagUserStateFactory to standalone class
     add 2e17f12  [BEAM-7526] Fix toBeamValue logic in BigQueryUtils
     add 01f14a0  Merge pull request #8814 from charithe/beam-7526
     add a8bdcc5  Remove the unnecessary enable_health_checker flag
     add 47ba869  Merge pull request #8783 from robinyqiu/resilience
     add 76a1d51  [BEAM-5995] add Jenkins job with GBK Python load tests (#8151)
     add 36df7b4  Use when/then style for Mockito stubs
     add e7aadf5  Use when/then style for Mockito stubs
     add d779459  [BEAM-7044] portable Spark: support stateful dofns
     add 4f22c54  Merge pull request #8812: [BEAM-7044] portable Spark: support stateful dofns
     add 37b76b6  Add aliases for machine_type and disk_type flags to match existing documentation (#8603)
     add 1aee9a8  Merge #8686: [BEAM-7399] Blog post on looping timers
     add 1f53d9e  Publish SDF progress information on root node.
     add b71cf58  Merge pull request #8816 Publish SDF progress information on root node.
     add 6af9567  Make the Go SDK more robust with respect to requested split points.
     add 0d61308  Merge pull request #8786 More robust Go SDK split point handling.
     add 1159dd6  [website] Add artifact staging document to design documents
     add 2b2b5d5  Merge pull request #8837: [website] Add artifact staging document to design documents
     add 2798d62  [BEAM-7510] Fixing fileio tests checking JSON serialization (flaky) (#8795)
     add c603563  [BEAM-7529] Add Sums.ofFloats() and Sums.ofDoubles()
     add d754094  Merge pull request #8820: [BEAM-7529] Add Sums.ofFloats() and Sums.ofDoubles()
     add 7421a17  Add side input and side output java katas
     add bb39cf0  Add side input and side output python katas
     add 2bddf9c  Modify the TestPipeline declaration to be 'final transient'
     add a5ed7c3  Add Create.of transform names in CoGroupByKey kata
     add c34f8a7  Improve the task description of ParDo OneToMany to describe OutputReceiver
     add ab40ce5  Update offset for some tasks
     add e2959b3  Merge branch 'master' into add-katas
     add 2ac75ce  Update the task descriptions: formatting, conventions, and styles
     add 4e53ea4  Update Beam version to v2.13.0
     add f19cf0e  Add BinaryCombineFn Lambda java kata
     add 29f0553  Add packages for all Java katas
     add ac6faff  Add composite transform Java kata
     add 64bd536  Add composite transform Python kata
     add 8adf0e3  Add 'public' modifier to early Task classes
     add a7e64d4  Rename Tests classes to TaskTest
     add 7a410ea  Add branching Java kata
     add 6f14fe0  Add branching Python kata
     add 3541d17  Add TextIO Read Java kata
     add 23ed747  Add the Kata descriptions for Branching and TextIO Read
     add d43a97b  Add TextIO ReadFromText Python kata
     add 58738a8  Add Kata description from Branching Python kata
     add 1a524aa  Add Built-in IOs Java kata
     add 8502d8e  Add Built-in IOs Python kata
     add 7ebe261  Add missing copyright for Python kata "Built-in IOs" test file
     add ef8665e  Add rat exclusion for Katas IO txt files
     add e60e173  Removed unused entry in Java kata study_project.xml
     add 3e3961b  Merge pull request #8804 from henryken/add-katas
     add af7ad3f  [BEAM-7407] Adds portable wordcount test for Python 3 with Flink runner. (#8745)
     add 999bc5a  Merge pull request #8552: [BEAM-7268] make sorter extension Hadoop-free
     add c9fb261  [BEAM-7221] portable Spark: timers
     add 66b090d  Merge pull request #8841: [BEAM-7221] Spark portable runner: support timers
     add 278c1e8  Add new proposal to design docs
     add 6f7415c  Merge pull request #8846: [website] Add Portable batch & streaming load tests in all sdks proposal to design docs
     add e9c6207  [BEAM-7368] Add method to prepare SDK harness image for jenkins jobs
     add 55e1624  [BEAM-7368] Add method to prepare Flink Job server's image for jenkins jobs
     add fd5c7a8  [BEAM-7368] Add cluster Flink creation and teardown steps
     add e5db98c  [BEAM-7368] Add GBK load test for Python + Flink
     add c83018c  [BEAM-7368] Specify a cron job for the load test
     add 0a6390c  [BEAM-7368] Apply code review suggestions
     add ba797dd  [BEAM-7368] Use post build script in Jenkins
     add 4c77324  [BEAM-7368] Allow defining number of slots per worker. Set to 1 by default.
     add b35efe8  [BEAM-7368] Added parameters that allow using different flink versions
     add 0e90336  [BEAM-7368] increase worker count to be equal to Flink's parallelism
     add db4ae61  [BEAM-7368] Bugfix: create proper amount of VMs for Flink Dataproc cluster
     add 9e31c94  [BEAM-7368] Remove deps to external spark config in Flink init action
     add fde7793  Merge pull request #8636: [BEAM-7368] Flink + Python + gbk load test
     add 54a730e  Documenting and refactoring fn api runner code (#8659)
     add 1362956  [BEAM-7541] provide runner & filesystem deps for IOIT in runtime
     add 0d9212e  Merge PR 8835
     add e933ddc  Adding helper functions, and documentation to fn_api_runner classes. (#8793)
     add 4e05a7f  Read YAML as binary stream in standard_coders_test.
     add 8c8ca66  Merge pull request: [BEAM-5270] Read YAML as binary stream in standard_coders_test
     add 681b8db  [BEAM-5315] Fix Py3 incompatibility in tfrecordio_test.
     add d0de455  Merge pull request #8852 from tvalentyn/patch-53
     add c7a592e  Remove Python batch support for Pub/Sub from docs
     add f49d1fd  Merge pull request #8840: [BEAM-7136] Remove Python batch support for Pub/Sub from docs
     add 06c89de  [BEAM-7445] Add search field to Beam website
     add 5200161  Merge pull request #8838: [BEAM-7445] Add search field to Beam website
     add 0e265f4  Fixed spaces for author in looping blog. Added twitter handle for rarokni
     add 460aba5  Merge pull request #8844: [BEAM-7399] Fix Looping Timer spacing
     add f9d476d  [BEAM-7033] Updates for reading with query using BigQuery Storage API
     add 9d0eaf1  Merge pull request #8829: [BEAM-7033] Updates for reading with query using BigQuery Storage API
     add e4008d1  [BEAM-6769] allow read None bytes
     add 82bf1ae  Merge pull request #8817: [BEAM-7530] allow read None bytes
     add 614dad7  [BEAM-7553] add portability to Spark webpage
     add 84e54af  Merge pull request #8854: [BEAM-7553] Add portable runner to Spark webpage
     add 3fdf691  [BEAM-7043] Add DynamoDBIO
     add 33d7cd0  Merge pull request #8390: [BEAM-7043] Add DynamoDBIO
     add 121df06  [BEAM-7533] Fix CoderRegistry for Float
     add 6d093f5  Merge pull request #8825: [BEAM-7533] Fix CoderRegistry for Float
     add f4cf92d  [BEAM-7126] Fix StateRequestHandler type variables to allow for proper key parameterization
     add 211c788  Merge pull request #8697: [BEAM-7126] Fix StateRequestHandler type variables to allow for proper key parameterization
     add c3a0fda  [BEAM-7402] Added a performance test for BigQuery IO read
     add 59ea050  Merge pull request #8675: [BEAM-7402] BigQuery IO read performance tests
     add 64abb60  [BEAM-7551] Checkpoint Flink's ImpulseSourceFunction
     add f2067fe  Merge pull request #8857: [BEAM-7551] Checkpoint Flink's ImpulseSourceFunction
     add d38a4da  Merge pull request #8858: [BEAM-7542] Fix faulty cast in BigQueryUtils
     add f5bc2fe  Adding a Link to The Doc for Cost Estimation
     add 2be7457  Adding a Link to The Doc for Cost Estimation
     add 5b3807b  [BEAM-7531] Remove and disallow KMS key setting
     add e46d0f8  [BEAM-7531] Remove and disallow KMS key setting
     add 55e3f16  Remove unused **kwargs arguments for various transforms.
     add 11aacd2  Remove unused **kwargs arguments for various transforms.
     add 9b99768  Reduce gradle build verbosity
     add 6b3b970  Merge pull request #8831: Reduce gradle build verbosity
     add b25be12  [BEAM-7432] Set input and output files for sdist
     add a71bfda  Merge pull request #8765: [BEAM-7432] Set input and output files for sdist
     add 9bbe6f5  [BEAM-7450] Support unbounded reads with HCatalogIO
     add 19804ac  Merge pull request #8718: [BEAM-7450] Support unbounded reads with HCatalogIO
     add 2d49e58  Add link to type hints design doc
     add 4d2eba2  Merge pull request #8865: [website] Add link to the type hints design doc for python 3
     add 1642655  [BEAM-7144] Fix for rescaling problem on Flink >= 1.6
     add 89c5abf  Merge pull request #8850: [BEAM-7144] Fix for rescaling problem on Flink >= 1.6
     add e432b96  [BEAM-6777] Let HealthzServlet respond actual health information of SDK harnesses (#8827)
     add 18100ca  Use code.jquery.com for jQuery CDN
     add ff89b2e  use https
     add 36f4f85  [website] Use code.jquery.com for jQuery CDN
     add bc2c286  [BEAM-7513] Implements row estimation for BigQuery.
     add 2de7d07  Merge pull request #8822 from riazela/bigquery-rowcount
     add 8f300f9  Fixes filesystem_test for Windows.
     add 7e633f3  Merge pull request #8869: [BEAM-7569] Fixes filesystem_test for Windows
     add 96934f2  Merge pull request #8836: Add Beam Katas to the website's "Learning Resources" page
     add e295aa8  Add Python 3.6, 3.7 to the list of supported version classifiers.
     add 3c61ab8  Merge pull request #8867 from tvalentyn/classifiers
     add 7b7c215  Revert "[BEAM-7513] Adding Row Count for Bigquery Table"
     add 35f99ba  Merge pull request #8882 from apache/revert-8822-bigquery-rowcount
     add 77456c1  [BEAM-7467] Add dependency classifier to published pom (#8868)
     add fe6891a  Fixing file naming for windows (#8870)
     add 944646e  [BEAM-7461] disalbe flaky tests due to flaky org.apache.beam.sdk.io.gcp.pubsub.TestPubsub.checkIfAnySubscriptionExists]
     add ce09b84  [BEAM-7461] Disalbe flaky tests
     add 83bc006  Use StateTags.ID_EQUIVALENCE when using comparing StateTags
     add a6c9c92  Use StateTags.ID_EQUIVALENCE when using comparing StateTags in WindmillStateCache
     add e7e13f9  [BEAM-5148] Add MongoDB IO to python sdk
     add acb1f56  Merge pull request #8826: [BEAM-5148] Implement MongoDB IO for Python SDK
     add e1f0783  support Datetime value conversion.
     add f50f4d5  Merge pull request #8376: [BEAM-7130]Support Datetime value conversion in convertAvroFieldStrict
     add 08a84e4  [BEAM-6674] Add schema support to JdbcIO read (#8725)
     add 408f744  [BEAM-7561] HdfsFileSystems.match can match a directory.
     add edd0e0c  [BEAM-7561] Validate correct directory matching in LocalFileSystemsTest
     add 5e5e7a5  Merge pull request #8849: [BEAM-7561] Enable HdfsFileSystems.match to match a directory
     add b0bf009  [BEAM-7543] ReduceByKey.combineBy must accept BinaryFunction<V, V, V>
     add 175a8d2  [BEAM-7543] deprecate old #combineBy(Sums.ofLongs(), TypeDescriptors.longs())
     add eb5a8c2  Merge pull request #8848: [BEAM-7543] ReduceByKey.combineBy must accept BinaryFunction<V, V, V>
     add 5c2b362  [BEAM-7579] Use bucket with default key in ITs
     add 4d33e4b  Address review comments
     add 25955c6  [BEAM-7579] Use bucket with default key in ITs
     add 15d7be3  [BEAM-7507] Avoid more String.format and ByteString.toStringUtf8 in hot paths
     add 29868a7  [BEAM-7507] Avoid more String.format and ByteString.toStringUtf8
     add 814713c  [BEAM-7467] Do not run gearpump pipeline remotely by default
     add 4f7d802  [BEAM-7467] Do not run gearpump pipeline remotely by default
     add db30622  [BEAM-6877] Fix trivial_inference for Python 3.x
     add 29807c8  Address review comments
     add 4dc1c0b  Allow more time for IT pipeline to run
     add 221e0a5  Merge pull request #8856: [BEAM-6877] Fix trivial_inference for Python 3.x
     add 441b6b1  [BEAM-6955] Use base version component of Beam Python SDK version when choosing Dataflow container image to use. (#8885)
     add c32acce  Add common code infrastructure for element-wise snippets
     add 3124531  Merge pull request #8811 from davidcavazos/element-wise-snippets
     add 6d21877  Moving to 2.15.0-SNAPSHOT on master branch.
     add e868b80  [BEAM-7513] Adding RowCount to BigQueryTable.
     add b2fd4e3  Merge pull request #8892 from riazela/bigquery-rowcount
     add 3cddc42  [BEAM-7603] Adding support for ValueProvider in Custom GCS Temp Location (#8833)
     add ea2675d  Add link to gRPC.io on website portability page
     add 0fd2cd3  Merge pull request #8909 [BEAM-7602] Add link to gRPC.io on website portability page
     add 6915114  [BEAM7597] Fix Typo on website
     add 94260de  Merge pull request #8911: [BEAM-7597] Fix Typo on website
     add 2bf6fde  [BEAM-5806] Update PubsubIO to be able to change the PubsubClientFactory
     add e9debbf  [BEAM-5806] Update PubsubIO to be able to change the PubsubClientFactory
     add fa37fc5  [BEAM-4046, BEAM-7307] revert PR 8581, migrate jenkins tests to use directory based names instead of artifact names (#8881)
     add c775eda  Make PubsubIO#withCoderAndParseFn public (#8879)
     add da79821  Add Python snippet for Keys, Values, KvSwap, and ToString transform
     add 778bf33  Merge pull request #8906 from davidcavazos/element-wise-simple
     add 1a39ecf  Add Python snippet for Filter transform
     add 4c1f28a  Merge pull request #8900 from davidcavazos/element-wise-filter
     add 03aa755  add compatibility check badges to README
     add df35d5c  address github comment
     add 0c04a02  Merge pull request #8791 from ylil93/master
     add 2b41800  Fixed typo and removed whitespace
     add 553c560  Merge pull request #8910: [BEAM-7601] Fix typo on community page
     add f1a6e06  Add a timeout to urlopen calls
     add 30c0dd8  Merge pull request #8925 from apache/aaltay-patch-1
     add ad88aa8  [Beam-6696] GroupIntoBatches transform for Python SDK (#8914)
     add 19b636d  Add Python snippet for FlatMap transform
     add 31342b0  Merge pull request #8901 from davidcavazos/element-wise-flat-map
     add b406dbc  Add README files on how to setup the project for both Java and Python Katas
     add 1d9d216  Merge pull request #8862: Beam Katas README to setup project
     add a2aa4b9  Add to docstring of advance_watermark_to_infinity
     add b4093ad  Merge pull request #8922 from ttanay/adv_watermark_docstring
     add ce4574f  Revert "Merge pull request #8561: [BEAM-6627] Add item and byte counters to HadoopFormatIOIT"
     add 3d2c969  Revert "Merge pull request #8400: [BEAM-6627] Added byte and item counting metrics to integration tests"
     add e0cbd2a  Merge pull request #8861: [BEAM-7431] revert expensive byte monitor operations from Core Operations and IO tests. Revert element counters too.
     add 690ab87  [BEAM-7307] Fix load tests failures due to mistakes in PR 8881
     add a229027  [BEAM-7307] Fix load tests failures due to mistake in previous PR
     add e3f3b57  Fix erros in py sdk io utils and add unit tests
     add abede33  Merge pull request #8887: Fix errors in py sdk io utils and add unit tests
     add fa96609  [BEAM-7572] ApproximateUnique.Globally and ApproximateUnique.PerKey classes are inaccessible
     add 0f862ba  [BEAM-7572] ApproximateUnique.Globally and ApproximateUnique.PerKey classes are inaccessible
     add 2d28efe  [BEAM-7598] Do not build Python tar file in run_integration_test.sh
     add 5b91cb4  Merge pull request #8916 from markflyhigh/fix-run-it-script
     add 08f600b  Add Python snippet for ParDo transform
     add 754a2ec  Merge pull request #8903 from davidcavazos/element-wise-pardo
     add fb8efe3  BEAM-7141: add key value timer callback (#8739)
     add 4416b56  [BEAM-7606] Fix JDBC time conversion tests
     add 7a08617  Merge pull request #8924: [BEAM-7606] Fix JDBC time conversion tests
     add e65c176  Update portable schema representation and java SchemaTranslation (#8853)
     add 8745d6e  [BEAM-7475] update wordcount example (#8803)
     add aa8b3d0  Implement splitAtFraction for the BigQuery Storage API
     add fd9ad83  Merge pull request #8832: [BEAM-7495] Add dynamic worker rebalancing to BigQuery Storage
     add 826b796  [BEAM-7013] Add HLL doc link to Beam website
     add 6143271  Merge pull request #8947 from robinyqiu/patch-2
     add 740d804  Drop experimental from python streaming in the doc
     add 350da8e  Merge pull request #8932 from apache/aaltay-patch-1
     add 137953b  [BEAM-4948, BEAM-6267, BEAM-5559, BEAM-7289] Update the version of guava to 26.0-jre for all our vendored artifacts containing guava
     add af93621  Merge pull request #8942: [BEAM-4948, BEAM-6267, BEAM-5559, BEAM-7289] Update the version of guava to 26.0-jre for vendored gRPC and guava
     add e858f49  [BEAM-4046, BEAM-7527] Fix benchmark with correct Gradle project
     add af0ca9b  Merge pull request #8919: [BEAM-4046, BEAM-7527] Fix benchmark with correct Gradle project
     add c44f352  Match Python 3 warning message in __init__.py with the one in setup.py.
     add 063091e  Merge pull request #8956 from tvalentyn/classifiers
     add e6cc73b  Spotless config update to include java files only under src directory
     add c39ba4a  Merge pull request #8954 from akedin/spotless-include-src
     add 1992cde  [BEAM-7548] fix flaky tests for ApproximateUnique (#8948)
     add 713fd6e  [BEAM-7326] add documentation bigquery data types
     add 0c0234b  Merge pull request #8873: [BEAM-7326] add documentation bigquery data types
     add 17bf89d  [BEAM-7586] Add Integration test for python mongodb io
     add 00c402d  Merge pull request #8895: [BEAM-7586] Add Integration test for python mongodb io
     add 3c43cc1  [BEAM-7547] Avoid WindmillStateCache cache hits for stale work.
     add 9cea449  [BEAM-7547] Avoid WindmillStateCache cache hits for stale work.
     add 0d5e711  Add Python snippet for Map transform
     add d1058a6  Merge pull request #8902: [BEAM-7389] Add Python snippet for Map transform
     add 6786afd  [BEAM-7548] Fix flaky tests for ApproximateUnique (#8960)
     add 43bb514  [BEAM-7589] Use only one KinesisProducer instance per JVM
     add b81025c  [BEAM-7589] Make KinesisIOIT compatible with all other ITs
     add 69f5b7ec Merge pull request #8955: [BEAM-7589] Use only one KinesisProducer instance per JVM
     add b6e45f4  [BEAM-7424] Retry HTTP 429 errors from GCS
     add 30fa5d8  Merge pull request #8933: [BEAM-7424] Retry HTTP 429 errors from GCS
     add a4fffee  [BEAM-6692] portable Spark: reshuffle translation
     add f22070b  Merge pull request #8966: [BEAM-6692] portable Spark: reshuffle translation
     add 54475b9  [BEAM-7640] Create amazon-web-services2 module and AwsOptions
     add 0d74293  Merge pull request #8963: [BEAM-7640] Create amazon-web-services2 module and AwsOptions
     add 16ceca5  [BEAM-7545] Adding RowCount to TextTable.
     add 3d576f7  Merge pull request #8951 from riazela/TextTableRowCount
     add b31f930  BEAM-3645 add ParallelBundleProcessor
     add 1f38467  BEAM-3645 reflect comments
     add b1e7644  BEAM-3645 add changes from review comments
     add f391aa4  BEAM-3645 add thread lock when generating process_bundle_id
     add 45d8a28  [BEAM-3645] add ParallelBundleManager
     add ef9f61b  Tiny typo fix
     add 05900a4  Merge pull request #8974 from youngoli/patch-7
     add d87c46d  [BEAM-7536] Fixed BQ dataset name in collecting Load Tests metrics
     add 6aceab4  Merge pull request #8834: [BEAM-7536] Fix BigQuery dataset name in collecting Load Tests metrics
     add 80c7d25  Update python containers to beam-master-20190605
     add fbd1f4c  Update python containers to beam-master-20190628
     add 88f0cef  [BEAM-7640] Rename the package name for amazon-web-services2 from aws to aws2
     add 3643f43  Merge pull request #8983: [BEAM-7640] Rename the package name for amazon-web-services2 from aws to aws2
     add 9f6fbf2  [BEAM-7504] Added top_count parameter
     add 884d1f2  [BEAM-7504] Create Combine Python Load Test Jenkins job
     add 7f83599  Merge pull request #8813: [BEAM-7504] Created Combine Python Load Test Jenkins job
     add 6a9d246  [BEAM-5315] improve test coverage bigquery special chars
     add b79f24c  Merge pull request #8878: [BEAM-5315] improve test coverage bigquery special chars
     add 3e4cbf2  [BEAM-4420] Allow connecting to zookeeper using external ip
     add b8a7edd  [BEAM-4420] Add KafkaIO integration test pipeline
     add 32e2e3e  Merge pull request #8986: [BEAM-4420] Add KafkaIO Integration Tests
     add 79878c3  [BEAM-7683] - fix withQueryFn when split is more than 0
     add cfecfb0  Merge pull request #9004: [BEAM-7683] MongoDBIO.withQueryFn ignores custom filter
     add 0859e88  [BEAM-7682] Fix Combine.GroupedValues javadoc code snippet
     add b2f0171  Merge pull request #8995: [BEAM-7682] Fix Combine.GroupedValues javadoc code snippet
     add 707be06  Add Jet Runner to the Get Started page
     add 73735cc  Merge pull request #8937: [BEAM-7305] Add Jet Runner to Get Started documentation page
     add 92eab27  [BEAM-7550] Reimplement Python ParDo load test according to the proposal
     add 2d5e493  Merge pull request #8847: [BEAM-7550] Missing pipeline parameters in ParDo Load Test
     add 279f2d4  [BEAM-5605] Update Beam Java SDK backlog to track latest changes in Beam Python SDK.
     add 82cfdb8  [BEAM-5605] Update Beam Java SDK backlog to track latest changes in Beam Python SDK.
     add 3772902  Fix RestrictionTracker docstring
     add 69bb363  Merge pull request #8999 from boyuanzz/doc_fix
     add ef4a51d  Consider Elasticsearch as one word in camelCase.
     add c2343c8  Merge pull request #9008 from RyanSkraba/BEAM-7698-rename-elasticsearch
     add 142b581  [BEAM-6480] Adds AvroIO sink for generic records. (#9005)
     add 0624ba6  unskip ReifyTest.test_window
     add 42ef04a  Merge pull request #8866 from ttanay/unskip-reify-test
     add 67ed7ac  [BEAM-6740] Add PTransformTranslator for Combine.Globally
     add 289867e  [BEAM-6740] Add extractAcummulatorCoder for Combine.Globally and fix tests
     add dbe59e4  [BEAM-7640] Change tests to use PayloadTranslator instead of unused toProto
     add 18e0ac8  [BEAM-6740] Refactor to remove duplicated code in CombineTranslation
     add 16e4fbc  [BEAM-6740] Add PTransformTranslator for Combine.Globally
     add 3fc4478  [BEAM-7437] Add streaming flag to BQ streaming inserts IT test
     add db92536  Change default timeout to 5 mins
     add db913ef  Merge pull request #8934: [BEAM-7437] Add streaming flag to BQ streaming inserts IT test
     add a00f753  [BEAM-7708] don't expect SQL shell bundled dependencies to be shadowed
     add 600099a  [BEAM-7708] don't expect SQL shell bundled dependencies to be shadowed
     add 4e1db10  [SQL][Doc] fix broken gradle command.
     add a2f8458  Merge pull request #9021 from amaliujia/patch-1
     add b2ef9ca  Added new example on how to create a custom unbounded streaming source
     add 66bd44f  Added new example on how to create a custom unbounded streaming source
     add cbac53b  Update code comments to improve readability in docs (#9024)
     add fab3ae0  [BEAM-7709] Re-use node for explicit flattens
     add 14d8c3d  [BEAM-7709] Re-use node for explicit flattens
     add a25c843  Reformat CamelCase function naming style to underscore style for synthetic_pipeline.
     add 3417c3a  fix lint
     add 5c0905c  Merge pull request #9016 from boyuanzz/fix_format
     add f8cd4a8  [BEAM-7653] Add PTransformTranslator for Combine.GroupedValues
     add f34c286  [BEAM-7653] Add PTransformTranslator for Combine.GroupedValues
     add eae055a  Update pyarrow version requirement in setup.py
     add 0b83651  Merge pull request #9027 from paulgc/master
     add f17d1db  BEAM-3645 add thread lock
     add 2a00f2e  [BEAM-3645] add thread lock for ParallelBundleManager
     add 265127f  [BEAM-7535] Created Jenkins job for BQ performance tests
     add fe36909  [BEAM-7535] Delete existing data if the table already exists
     add b4efbb2  Merge pull request #8874 from kamilwu/bq-perf-jenkins
     add 485a781  Update Python Dataflow runner to patch side input coders on the unified worker, too
     add 7d7f897  Update Python Dataflow runner to patch side input coders on the unified worker, too
     add 51c8873  [BEAM-7690] Port WordCountTest off DoFnTester
     add 4ba9989  [BEAM-7690] Port WordCountTest off DoFnTester
     add e2025ff  Use Beam's abstraction of pickler instead of dill in coder tests.
     add 3376417  Merge pull request #8975 Use Beam's abstraction of pickler instead of dill in coder tests.
     add 7d11bca  [BEAM-7668] Add ability to query a pipeline definition from a gRPC JobService
     add 05dba6a  Merge PR #8977
     add 5952ea0  [BEAM-7710] [website] remove outdated reference to KeyedCombineFn
     add c23cddb  Merge pull request #9017: [BEAM-7710] [website] remove outdated reference to KeyedCombineFn
     add ad6229c  [BEAM-7389] Add Python snippet for Partition transform (#8904)
     add 3c1acb0  Add Python snippet for WithTimestamps transform
     add 4169307  Merge pull request #8907 from davidcavazos/element-wise-with-timestamps
     add b72d848  [BEAM-7689] make a temporary directory unique for FileBaseSink
     add 5b68aae  Merge pull request #9013: [BEAM-7689] make a temporary directory unique for FileBaseSink
     add daa5ca1  [BEAM-5994] Fix condition which allows to publish metrics.
     add 8e0f2d8  Merge pull request #9035: [BEAM-5994] Fix condition which allows to publish metrics
     add 0740132  Update Beam Katas (Java) course on Stepik
     add f3babfe  Update Beam Katas (Python) course on Stepik
     add 9ade438  Update the ParDo OneToMany task description to follow latest convention
     add e8f8a60  Add Windowing Adding Timestamp using ParDo Java Kata
     add 713f5e4  Add Windowing Adding Timestamp using WithTimestamps Java Kata
     add 626037f  Add Fixed Time Window Java Kata
     add 2e9667c  Add kata description for the Adding Timestamp katas
     add 7e1f1e8  Rename 'util' package to 'org.apache.beam.learning.katas.util'
     add e2f6a5d  Add Triggers "Event Time Triggers" Java kata
     add 8e6f5cc  Add "DoFn Additional Parameters" Java kata
     add 521ddee  Move Built-in IOs Task into a package
     add 6d9e2a2  Add WithKeys Java kata
     add fe5c3c6  Change logging library to use log4j2 and add log4j2.xml in the util module
     add 2cd28fc  Add "Early Triggers" Java kata
     add 5eee523  Update WithKeys task description javadoc link to use 'current'
     add 0afe06b  Add "Window Accumulation Mode" Java kata
     add 8d25f7b  Add package for "Early Triggers" and "Window Accumulation Mode"
     add 5d8066f  Update study_project.xml
     add a18f0eb  Fix course syllabus on Stepik
     add 4789098  Fix the wrong file path for TextIO Read kata
     add 7e1fd00  Update the allowed lateness and discarding accumulation mode kata descriptions to include the links to javadoc
     add 6a5ddcf  Update course to Stepik
     add e2996a3  Reupload Python Katas lessons
     add 1c9799d  Merge pull request #8889 from henryken/add-windowing-katas
     add f8732f5  [BEAM-7503] Added iteration parameter to CoGBK test in Python
     add 39d4f89  [BEAM-7503] Created CoGBK Python Load Test Jenkins job
     add df04c07  [BEAM-3959] Added a flake F821 test
     add a199e6b  Merge pull request #8969 from kamilwu/cogbk-jenkins
     add 854489a  Remove unused NEWLINE in TextSink.TextWriter
     add f2a2f2a  Merge pull request #8989: Remove unused NEWLINE in TextSink.TextWriter
     add 25df6c9  [BEAM-7621] Null pointer exception when accessing null row fields in BeamSql
     add 10fd794  Merge pull request #8930: [BEAM-7621] Null pointer exception when accessing null row fields in BeamSql
     add 0b61d3e  Ensure that publishing vendored artifacts checks the contents of the jar before publishing.
     add 1912b6f  Merge pull request #9036 from lukecwik/vendor
     add b1fe777  [BEAM-4948, BEAM-6267, BEAM-5559, BEAM-7289] Fix shading of vendored guava to exclude classes from transitive dependencies which aren't needed at runtime.a
     add 0fce2b8  Merge pull request #9038 from lukecwik/vendor2
     add 2215b77  [BEAM-7729] Fixes the bug by checking the value first before parsing it.
     add 60f70bf  Merge pull request #9045 from riazela/BigQueryNullableBugFix
     add 6f8324c  [BEAM-7731] Adding helper function to handle if null else pattern for expression generation
     add d6044fa  Merge pull request #9047 [BEAM-7731] Adding helper function to handle if null else pattern for expression generatioin.
     add f7cbf88  [BEAM-7437] Raise RuntimeError for PY2 in BigqueryFullResultStreamingMatcher (#9044)
     add f34d835  Some trivial typos
     add 3f53bec  Merge pull request #9068 Some trivial typos
     add a353733  [BEAM-7737] Fix microbenchmark scripts compiled check (#9066)
     add 60ed617  [BEAM-7694]Fix error spelling in annotation of SparkTransformOverrides.
     add dce8f0d  update annotation for PTransformMatcher.
     add a89cc2a  Merge pull request #9006: [BEAM-7694] Fix error spelling in annotation of SparkTransformOverrides
     add 1e47cc0  [BEAM-7736] Free worker when work is skipped
     add 93be6ae  [BEAM-7736] Free worker when work is skipped
     add 3f3e93d  Done notifications for BigtableIO.Write
     add 5fb21e3  Merge pull request #7805: [BEAM-3061] Done notifications for BigtableIO.Write
     add 1d624b6  Add estimate_size() to source_test.LineSource
     add 92b283f  fix lint
     add fa1c239  Add _get_file_size and UT
     add 6fd2490  Merge pull request #8981 from boyuanzz/source_test
     add c773822  [BEAM-7546] Increasing environment cache to avoid chances of recreating the environment
     add d8eb89e  Merge pull request #9046 from angoenka/fix_portable_wordcount
     add de4dd5b  [BEAM-7656] Add sdk-worker-parallelism arg to flink job server shadow jar
     add b60aef1  Merge pull request #8967 from ibzib/parallelism
     add 2364886  Merge pull request #4 from apache/master
     add c27bef6  [BEAM-7666] Memory monitor change
     add beb62c1  [BEAM-7666] Adds the counter
     add 8001ad2  Merge pull request #8971 from dustin12/ThrottlePiping
     add 8d33dae  Edit of Looping Timer Blog to fix issue with Timer State
     add 708cea5  Merge pull request #9010: Edit of Looping Timer Blog to fix issue with Timer State
     add 41478d0  [BEAM-6675] Generate JDBC statement and preparedStatementSetter automatically when schema is available (#8962)
     add e02cb19  [BEAM-7715] Mark user facing APIs related to External transforms as @Experimental
     add 3363721  Merge pull request #9031: [BEAM-7715] Mark user facing APIs related to External transforms as @Experimental
     add 88c3f6b  [BEAM-7578] add py37 hdfs integration test (#8970)
     add a035703  [BEAM-7674] Combine batch and streaming BQ Streaming Insert ITs
     add 3bbec05  Merge pull request #9074 from ttanay/unify-bqsi-test
     add e821038  [BEAM-7665] Support TypeDefinition options in beam.Combine()
     add 126bb85  Merge branch 'master' of https://github.com/apache/beam into combine-typedef
     add aa3fe32  Revise the comments to only mention the TypeDefinition options.
     add b9f8b61  Merge pull request #8976 from htyleo/combine-typedef
     add ea82a89  Label the ParDos with their name
     add da45987  Merge pull request #9053 from happygiraffe/patch-1
     add b5f7a5d  [BEAM-7632] Update Python quickstart guide for Flink and Spark
     add 4a7ba2d  Merge pull request #8949 from ibzib/quickstart
     add f745702  Add Python snippet for Regex transform
     add db59a3d  Merge pull request #8905 from davidcavazos/element-wise-regex
     add d7bfb06  Fix stream position bug in BigQuery Storage stream source.
     add a0ec59b  Merge pull request #9059: [BEAM-7743] Fix stream position bug in BigQuery Storage stream source.
     add 885bd50  [BEAM-7557] - Migrate DynamoDBIO to AWS SDK for Java 2
     add ccfd31e  Merge pull request #9086: [BEAM-7557] Migrate DynamoDBIO to AWS SDK for Java 2
     add 066d11f  [BEAM-4420] Extract method for publishing already collected metrics
     add 6e6675c  [BEAM-4420] Collect & write metrics to BigQuery and console
     add adb281f  Merge pull request #9073: [BEAM-4420] KafkaIOIT metrics collection
     add cead81f  [BEAM-7530] Add it test to read None values from BigQuery (#8875)
     add 608a861  Adding logging and adding one more retry
     add 29daca2  Merge pull request #9084 from pabloem/bqlggn
     add 43d953d  [BEAM-7499] Fixup for tricky Reify testing issue (#9077)
     add 3b11a4c  [BEAM-7630] add ITs for writing and reading bytes from pubsub
     add 2b5e30d  Merge pull request #8985 from Juta/bytes
     add e851f6b  BEAM-3645 improve test cases
     add 38f8f3a  Merge pull request #9051 from Hannah-Jiang/python-ParallelBundleProcessor
     add eeee1de  [BEAM-7641] Collect xunit statistics for Py ITs (#8952)
     add c2f0d28  [BEAM-4948, BEAM-6267, BEAM-5559, BEAM-7289] Update the version of guava to 26.0-jre for all our vendored artifacts containing guava (#8899)
     add 76bc70d  [BEAM-7484] Metrics collection in BigQuery perf tests (#8766)
     add a223b09  [BEAM-7545] Reordering Beam Joins and check if the produced join is correct.
     add 8656d4c  Merge pull request #9040 from riazela/JoinReordering
     add 26358b8  [BEAM-7284] enabled to pickle MappingProxyType in order to pickle DataClass and added unittests for them
     add 719da2c  Merge pull request #9050 [BEAM-7284] Enabled pickling of python3 dataclasses
     add 7934826  Fix documentation on iterable coder spec.
     add 9e8a564  Add the ability to set the service account email for dataflow jobs
     add c503f54  Matching on filename, not directory for fileio
     add 90d8570  Lint fixup
     add f41e29f  Lint fixup
     add 265c780  Merge pull request #9097 from pabloem/fx-fio-win
     add e901a1e  Add retractions doc to design doc page.
     add d626584  Merge pull request #9102: [website] Add retractions doc to the design doc page
     add 8c8a902  Add last_attempted_record_start to python OffsetRangeTracker (#9058)
     add 8da5e09  [BEAM-7784] Fixup for Guava upgrade
     add 35c30fe  Merge pull request #9105: [BEAM-7784] Fixup for Guava upgrade
     add 648277a  Refactor portable JobService to allow better sharing of code.
     add b05554b  Simplify known runner parsing code.
     add e8a2de8  [BEAM-7722] Add a Python FlinkRunner that fetches and uses released artifacts.
     add c5f4334  Pull out generic java job server helpers from flink.
     add 9589835  Merge pull request #9043 [BEAM-7722] Python FlinkRunner fetching released artifacts.
     add e40cde7  Split Python 3 Postcommits into several jobs
     add 2bbf6eb  Merge pull request #9093: [BEAM-7714] [BEAM-7257] Split Python 3 postcommits into several Jenkins jobs.
     add 500c570  Skip DoFn params test in Python 2 on Windows
     add 11215fc  Merge pull request #9081 from davidcavazos/pardo_test_fix
     add 471be56  Add transform_name_mapping pipeline option for python sdk
     add 3115bab  Check streaming option when validate transform_name_mapping
     add 7fe54a0  Merge pull request #9072 from y1chi/transform_name_mapping
     add 31c8675  Revert incorrect doc on flink runner version.
     add 23a879f  Merge pull request #9110: Revert incorrect doc on flink runner version
     add 304882c  [BEAM-5191] Support for BigQuery clustering
     add 94a3c85  Merge pull request #8945: [BEAM-5191] Support for BigQuery clustering
     add 00383f9  Replace old badges for Python PostCommit test with split ones
     add 4c86b91  Merge pull request #9114 Update badges for Python PostCommit in README.md
     add 9ab0dae  [BEAM-7060] Support translation of native type variables.
     add 87ba0c0  [BEAM-7060] Translate the typing.Iterable type hint.
     add d2b5751  Minor fixes discovered in migration.
     add 21147f1  [BEAM-7060] Automated replace of typehints with typing.
     add f843044  Post-rewrite lint fixes.
     add f67f1dd  Manual fixes for over-agressive replace.
     add 0533f8f  More conservative typing module translation.
     add 3dd2888  Merge pull request #9109 [BEAM-7060] Migrate to native typing types where possible.
     add 6b33c0e  Relax pydot requirements.
     add c188582  Merge pull request #9113 Relax pydot requirements.
     add 10c51f5  Remove 4 empty spaces from PR template that mess up Python postcommit build status icons.
     add f6367cc  Merge pull request #9112 Fix PR template.
     add f7f0661  [BEAM-7679] Add randomness to ITs' BQ dataset name
     add 7ca8b76  Merge pull request #8991 [BEAM-7679] Add randomness to ITs' BQ dataset name
     add 3a0d9c4  [BEAM-7783] Adding BeamTableStatistics.
     add ff7a803  Merge pull request #9104 from riazela/BeamTableStatistics
     add b6f43f1  Default to PiplelineState.UNKNOWN when job state returned from v1beta3 cannot be recogninzed.
     add 7b9b7bf  fixup: Address review feedback.
     add ab80cc5  Merge pull request #9094 from tvalentyn/default_to_unknown
     add 0dee4e3  Update Python 3 entry in Python SDK roadmap.
     add 6518abf  Merge pull request #9123 from tvalentyn/patch-57
     add 8afee95  Revert "[BEAM-7060] Migrate to native typing types where possible."
     add 8bd3b50  Merge pull request #9122: [BEAM-7798] Revert "[BEAM-7060] Migrate to native typing types where possible."
     add c604768  [BEAM-7502] Create ParDo Python Load Test Jenkins job
     add 69e3b3e  [BEAM-7502] Renamed file with Python GBK Load Test job definition
     add 5d473b5  [BEAM-7502] Reduced number of iterations to 1 in Java ParDo job
     add 1640133  Merge pull request #9042: [BEAM-7502] Create ParDo Python Load Test Jenkins job
     add 8e9f980  Temporary workaround for [BEAM-7473] (#9023)
     add 75875de  Make SDFBoundedSource wrapper work with dynamic splitting (#8944)
     add 3a4d636  [BEAM-7775] Remove job_PerformanceTests_Spark
     add ed8a3f5  [BEAM-7775] Remove job_PerformanceTests_Dataflow
     add e9009cf  Merge pull request #9101 from lgajowy/BEAM-7775-remove-obsolete--performance-test-jobs
     add 5597086  Merge pull request #8943: Schema conversion cleanup
     add 7567bcf  [BEAM-7801] Fix wordcount_xlang postcommit.
     add de8c987  Merge pull request #9126: [BEAM-7801] Fix wordcount_xlang postcommit.
     add dc171ae  [BEAM-7781] Update from Go Version 1.10 to 1.12
     add 38cf8ad  [BEAM-7781] Use correct Go gradle tasks
     add 5ef5402  [BEAM-7781] Update to latest GoGradle
     add 3015444  [BEAM-7696]prepare files to stage also in local master of spark runner.
     add 2ae62ff  Merge pull request #9019: [BEAM-7696] Prepare files to stage also in local master of spark runner
     add 84ee4c6  [BEAM-7794] DynamoDBIOTest is blocking forever
     add 4777f09  Merge pull request #9121: [BEAM-7794] DynamoDBIOTest is blocking forever
     add 84ef8aa  [BEAM-7785] synchronize watermark update with bundle processing
     add 35e1cc3  Merge pull request #9108: [BEAM-7785] synchronize watermark update with bundle processing
     add 6867465  [BEAM-6783] byte[] breaks in BeamSQL codegen
     add f3a2f96  [SQL] Annotate BuiltinStringFunctions with Strict
     add 0e98899  Merge pull request #9119: [BEAM-6783] byte[] breaks in BeamSQL codegen
     add c2b600b  Add setFailOnNoMatchingTests(true)
     add 0f06f2e  Merge pull request #9120: Add setFailOnNoMatchingTests(true)
     add 5588d27  [BEAM-7726] Add tests for DataSource protocol
     add 26f921c  [BEAM-7726] Refactor to prepare for state backing.
     add 9c1589c  [BEAM-7726] Rename some SideInput code to State
     add 7688bcf  [BEAM-7726] Handle State Backed iterables
     add 78e95cc  [BEAM-7680] Removed flaky assertion and separated unit tests in synthetic source test (#9049)
     add c76c807  [BEAM-7650] Add docker system prune to inventory job
     add d510c2b  [BEAM-7650] Add docker system prune to inventory job
     add 27bb5bc  Revert "Revert "[BEAM-7060] Migrate to native typing types where possible.""
     add f357c48  [BEAM-7798] Unit test for BundleBasedRunner type hints.
     add d080028  [BEAM-7798] Typehints fix for bundle based runner.
     add 4935d0c  Merge pull request #9125 [BEAM-7798] [BEAM-7060] Native types with revert and fix.
     add 2f239e4  [BEAM-7747] Close the file handle owned by fastavro.write.Writer in _FastAvroSink.close().
     add 09fe498  Merge pull request #9111 [BEAM-7747] Propertly close file handle in FastAvroSink.
     add 082f5bc  [BEAM-7794] Ignore blocking DynamoDBIOtest class
     add 4672208  Merge pull request #9139: [BEAM-7794] Ignore blocking DynamoDBIOTest classes
     add 7cc8613  [BEAM-7661] Create Python coGBK load test job on Flink
     add edac4cb  [BEAM-7661] Generate the same number of hot keys from both sources
     add 5796931  Merge pull request #9127: [BEAM-7661] Create Python CoGBK load test job on Flink
     add 667bf85  [BEAM-7797]  Move upload_graph handling logic before template generation. (#9128)
     add 6967d23  Update the Dataflow API version to 0607
     add c8343eb  Merge pull request #9149 from rohdesamuel/update-versions
     add 7f94621  Configure console output collection for Python 3.6, 3.7 postcommit suites
     add 8050915  Merge pull request #9143: [BEAM-7714] Configure console output collection for Python 3.6, 3.7 postcommit suites.
     add ea024fc  Weaken overly-strict microbenchmark check_complied test. (#9100)
     add 9f933a6  Revert "[BEAM-7785] synchronize watermark update with bundle processing" (#9151)
     add 8ea6f41  [BEAM-7814] Wait for BQ query results in tests
     add 5c6f428  Merge pull request #9150 from udim/bq-test-client-wait
     add 090ad02  Add progress reporting to the BigQuery source
     add 4eb7610  Merge pull request #9079: [BEAM-7495] Add progress reporting to the BigQuery source
     add 28281a6  [BEAM-4181] Add readFiles transform to TfRecordIO.
     add c791104  Merge pull request #9088 from RyanSkraba/BEAM-4181-tfrecord-readfiles
     add 0ae24ea  Add hot key detection protos to Windmill
     add f03b6ba  Merge pull request #9155 from rohdesamuel/update-versions
     add 7a63558  [BEAM-7823] Implementing a Custom NonCumulativeCost Handler
     add 1c639d1  Merge pull request #9159 from riazela/CostMetadataHandler
     add 0008687  Update nexmark.md
     add a999e85  Merge pull request #9163: [website] Remove duplicate arguments of Nexmark Dataflow command
     add 82e419c  [Java] remove unused variable in DirectMetricsTest.java
     add 389f649  [BEAM-7755] adding support for nested rows and arrays in BigQuery to Beam row conversion (#9089)
     add 9f88443  Upgrade google-cloud-core and google-cloud-bigquery dependencies
     add 8117b58  Merge pull request #9162 from aaltay/gcc
     add 0650b9b  [BEAM-7641] Add xunit results to run_validatescontainer.sh
     add 9a3adc7  Merge pull request #9171: [BEAM-7641] Add xunit results to run_validatescontainer.sh
     add 7095832  [BEAM-7608] Honor DATASTORE_EMULATOR_HOST env var
     add 4eab256  Merge pull request #8921: [BEAM-7608] Honor DATASTORE_EMULATOR_HOST env var
     add 6cf3b11  [BEAM-6611] BigQuery file loads in Streaming for Python SDK (#8871)
     add 4c95d57  [BEAM-7662] Create Python Combine load test job on Flink
     add bf19854  Merge pull request #9115: [BEAM-7662] Create Python Combine load test job on Flink
     add 1cabb1a  Set job name in Jet Runner
     add 85bb004  Merge pull request #9177: [BEAM-7305] Set job name in Jet Runner
     add 45ef1f2  [BEAM-7830]: make spark default duration consistent with PipelineResult
     add d1e3e90  fixup! [BEAM-7830]: make spark default duration consistent with PipelineResult
     add 4e40b53  Merge pull request #9170: [BEAM-7830] Make spark default duration consistent with PipelineResult
     add 3d087be  [BEAM-7366] Restore Spotless check for Flink Runner
     add 8f6247e  Merge pull request #9176: [BEAM-7366] Restore Spotless check for Flink Runner
     add abe5a7e  [BEAM-7783] BeamIOSourceRel using custom BeamStatistics
     add e9e8731  Merge pull request #9161 from riazela/BeamTableStatistics
     add 206bf5f  [BEAM-7807] Forbid TestNG imports on Beam classes
     add 707f10b  [BEAM-7807] Update Beam tests to not import TestNG related classes
     add 0fcb9c6  Merge pull request #9141 from iemejia/BEAM-7807-ban-testng
     add f9e0969  [BEAM-7814] Make BigqueryMatcher wait for results. (#9154)
     add e813673  [BEAM-7623] Add support to select MAP with Row as values in Beam SQL
     add c80ca6b  Merge pull request #9181 from bmv126/map_with_row_as_value
     add f75c2fb  Provision datasources and dashboards from config.
     add 83bb3dd  [BEAM-7700] Java transform catalog (#9129)
     add 3ee9da9  [BEAM-7574] Use Map-based window combining
     add 6c1d115  [BEAM-7574] use Merging and NonMerging accumulating strategies
     add 945a606  [BEAM-7574] unify Combine.perKey and Combine.globally
     add 7be5dff  [BEAM-7574] Explode windows to key when needed
     add 1691935  [BEAM-7574] Optimize GC performance, finalize implementation
     add fb83467  Merge pull request #8946: [BEAM-7574] fix Combine performance for SparkRunner
     add bfdb169  Add arbitrary value matchers to MetricsResultMatchers.
     add d088904  [BEAM-6883] Improve long-running spark streaming test.
     add c787b5e  Merge pull request #9178: [BEAM-6883] Improve long-running spark streaming test.
     add c8164a4  Nitpicky grammar fix in programming guide
     add ccaf3fe  Merge pull request #9180 Nitpicky grammar fix in programming guide
     add 0e60dfa  Merge pull request #9183: [BEAM-7675] Unify Gradle test configuration across Py2 and Py3 (#9183)
     add c31a2b1  [BEAM-7837] Add adapter class for API incompatible changes in Spark 3
     add a60d550  Merge pull request #9191: [BEAM-7837] Add adapter class for API incompatible changes in Spark 3
     add c4c75bd  [BEAM-6202] Change retry policy for dataflow's get_job (#9182)
     add 80a99ed  Sets a default environment for Dataflow runner
     add 0f2c9dd  Merge pull request #9165: [BEAM-7824] Sets a default environment for Dataflow runner
     add 30989de  [BEAM-6877] trivial_inference: make remaining tests pass (#8893)
     add e378f13  [BEAM-7820] Add hot key detection to Batch Dataflow Worker (#9186)
     add 68acce8  [BEAM-2264] Reuse GCP credentials in GCS calls.
     add 786ea64  Merge pull request #9060: [BEAM-2264] Reuse GCP credentials in GCS calls.
     add 5f36ea1  Change retry policy for dataflow's list_messages
     add 2a18da4  Merge pull request #9196 from aaltay/m404
     add 5fb6609  [BEAM-7728] [SQL] Support ParquetTable (#9054)
     add 5a340c7  Fixed type on singleton side inputs sample
     add 60fab9b  Merge pull request #9195 from davidcavazos/filter-sample-fix
     add e8fad15  [BEAM-3645] Add multiplexing for python FnApiRunner (#8979)
     add 43a6430  [BEAM-7840] Provide MapTuple and FlatMapTuple for Python 3 users. (#9168)
     add 841610f  [BEAM-4948] Switches dataflow runner to use a container image with updated Guava dependency.
     add 380d5c9  Merge pull request #9205 [BEAM-4948] Switches Dataflow to container with updated Guava dependency.
     add 6cc59ac  [BEAM-6783] byte[] breaks in BeamSQL codegen
     add afeb935  Merge pull request #9169: [BEAM-6783] byte[] breaks in BeamSQL codegen
     add 172b563  [BEAM-7808] Add method to convert avro field to beam field
     add 866b8e7  Merge pull request #9142 from bmv126/Avro_field_to_beam_field
     add f8b3d3e  Add "--workerHarnessContainerImage=" to dataflow java11 legacy worker test target
     add 4684cb0  Merge pull request #9203 from boyuanzz/fix
     add 7c6241f  Moving to 2.16.0-SNAPSHOT on master branch.
     add e641e27  Update design-documents.md
     add e4e4590  Merge pull request #9207 from riazela/patch-2
     add 7472f81  [BEAM-7389] Add code examples for Filter page (#9184)
     add 06021ac  Add git scm extension to wipe workspace before cloning project
     add 2793a72  Merge pull request #9200: [BEAM-7788] Add git scm extension to wipe workspace before cloning project
     add 199618b  Use dateutil for more flexible parsing of RFC-3339 / ISO-8601 formatted strings
     add 4805f49  Pin python-dateutil to current major version
     add 2dc8609  Merge pull request #9147: Use dateutil for more flexible parsing of RFC-3339 formatted strings
     add 0285f29  [BEAM-7751] Add a CoGroupByKey load testing Jenkins job
     add b9f8590  Merge pull request #9087: [BEAM-7751] Add a CoGroupByKey load testing Jenkins job
     new b703f96  Add an empty spark-structured-streaming runner project targeting spark 2.4.0
     new 7f33be3  Fix missing dep
     new 44d3fab  Add SparkPipelineOptions
     new 23c42d3  Start pipeline translation
     new 4247c23  Add global pipeline translation structure
     new 3bb80f7  Add nodes translators structure
     new 684b735  Wire node translators with pipeline translator
     new c52ea32  Renames: better differenciate pipeline translator for transform translator
     new 070a6d7  Organise methods in PipelineTranslator
     new 2f0309b  Initialise BatchTranslationContext
     new c5f43ab  Refactoring: -move batch/streaming common translation visitor and utility methods to PipelineTranslator -rename batch dedicated classes to Batch* to differentiate with their streaming counterparts -Introduce TranslationContext for common batch/streaming components
     new 21cc0b5  Make transform translation clearer: renaming, comments
     new 30b0117  Improve javadocs
     new 9e035a0  Move SparkTransformOverrides to correct package
     new ee7d8ec  Move common translation context components to superclass
     new 22c99d2  apply spotless
     new c223c7e  Make codestyle and firebug happy
     new 6252c89  Add TODOs
     new 4a22ca4  Post-pone batch qualifier in all classes names for readability
     new 63e4b97  Add precise TODO for multiple TransformTranslator per transform URN
     new f80db00  Added SparkRunnerRegistrar
     new 663aeec  Add basic pipeline execution. Refactor translatePipeline() to return the translationContext on which we can run startPipeline()
     new 2f6ace4  Create PCollections manipulation methods
     new 3be3536  Create Datasets manipulation methods
     new e4afbb4  Add Flatten transformation translator
     new 58d9f0d  Add primitive GroupByKeyTranslatorBatch implementation
     new b88d12c  Use Iterators.transform() to return Iterable
     new 37357d5  Implement read transform
     new dd1bb28  update TODO
     new 0ee3697  Apply spotless
     new fd72ddd  start source instanciation
     new 6916dcc  Improve exception flow
     new fa8b8cc  Improve type enforcement in ReadSourceTranslator
     new ecd15ff  Experiment over using spark Catalog to pass in Beam Source through spark Table
     new 3fff9c7  Add source mocks
     new eaf06e8  fix mock, wire mock in translators and create a main test.
     new 32a0f31  Use raw WindowedValue so that spark Encoders could work (temporary)
     new 99d06c5  clean deps
     new c3d886f  Move DatasetSourceMock to proper batch mode
     new 46b6b75  Run pipeline in batch mode or in streaming mode
     new af423ad  Split batch and streaming sources and translators
     new 40d9059  Use raw Encoder<WindowedValue> also in regular ReadSourceTranslatorBatch
     new deee71f  Clean
     new ddbc0fc  Add ReadSourceTranslatorStreaming
     new 3668323  Move Source and translator mocks to a mock package.
     new 73eb96e  Pass Beam Source and PipelineOptions to the spark DataSource as serialized strings
     new 8288575  Refactor DatasetSource fields
     new 86f50a4  Wire real SourceTransform and not mock and update the test
     new e944d40  Add missing 0-arg public constructor
     new 5829530  Use new PipelineOptionsSerializationUtils
     new 59b56bc  Apply spotless and fix  checkstyle
     new 2424be8  Add a dummy schema for reader
     new 6338e78  Add empty 0-arg constructor for mock source
     new f53becb  Clean
     new 38e918a  Checkstyle and Findbugs
     new ab9acc2  Refactor SourceTest to a UTest instaed of a main
     new f9bdc19  Fix pipeline triggering: use a spark action instead of writing the dataset
     new 591aaf2  improve readability of options passing to the source
     new 0e9d7fa  Clean unneeded fields in DatasetReader
     new d99dcc3  Fix serialization issues
     new e28e3fd  Add SerializationDebugger
     new f369ddb  Add serialization test
     new 70b145a  Move SourceTest to same package as tested class
     new df8d0ef  Fix SourceTest
     new ab8b22e  Simplify beam reader creation as it created once the source as already been partitioned
     new ca02796  Put all transform translators Serializable
     new 6a683a3  Enable test mode
     new 1ef2566  Enable gradle build scan
     new 188e10d  Add flatten test
     new 739ad71  First attempt for ParDo primitive implementation
     new 6e00851  Serialize windowedValue to byte[] in source to be able to specify a binary dataset schema and deserialize windowedValue from Row to get a dataset<WindowedValue>
     new a0df1f9  Comment schema choices
     new ca5a2f8  Fix errorprone
     new f530a62  Fix testMode output to comply with new binary schema
     new d8b1531  Cleaning
     new 4f0b82d  Remove bundleSize parameter and always use spark default parallelism
     new c3e448c  Fix split bug
     new a98b158  Clean
     new 6005962  Add ParDoTest
     new b77a225  Address minor review notes
     new baf6885  Clean
     new c7116de  Add GroupByKeyTest
     new 409f3cb  Add comments and TODO to GroupByKeyTranslatorBatch
     new 54a88b6  Fix type checking with Encoder of WindowedValue<T>
     new a2ed2db  Port latest changes of ReadSourceTranslatorBatch to ReadSourceTranslatorStreaming
     new 58e9108  Remove no more needed putDatasetRaw
     new 951efec  Add ComplexSourceTest
     new 8f8fff6  Fail in case of having SideInouts or State/Timers
     new c805783  Fix Encoders: create an Encoder for every manipulated type to avoid Spark fallback to genericRowWithSchema and cast to allow having Encoders with generic types such as WindowedValue<T> and get the type checking back
     new 12a86ec  Apply spotless
     new fc9eed2  Fixed Javadoc error
     new dfac8e3  Rename SparkSideInputReader class and rename pruneOutput() to pruneOutputFilteredByTag()
     new 10b07ad  Don't use deprecated sideInput.getWindowingStrategyInternal()
     new 2bbd8c3  Simplify logic of ParDo translator
     new 424d39c  Fix kryo issue in GBK translator with a workaround
     new 898b299  Rename SparkOutputManager for consistency
     new 17b520b  Fix for test elements container in GroupByKeyTest
     new 873bc95  Added "testTwoPardoInRow"
     new 181e4c5  Add a test for the most simple possible Combine
     new c17ce6e  Rename SparkDoFnFilterFunction to DoFnFilterFunction for consistency
     new 3d2a05e  Generalize the use of SerializablePipelineOptions in place of (not serializable) PipelineOptions
     new 4a91a3c  Fix getSideInputs
     new 4f1676d  Extract binary schema creation in a helper class
     new 8429fb6  First version of combinePerKey
     new 788aefb  Improve type checking of Tuple2 encoder
     new f145076  Introduce WindowingHelpers (and helpers package) and use it in Pardo, GBK and CombinePerKey
     new ad8b109  Fix combiner using KV as input, use binary encoders in place of accumulatorEncoder and outputEncoder, use helpers, spotless
     new e0de5d4  Add combinePerKey and CombineGlobally tests
     new 6d1301c  Introduce RowHelpers
     new e870813  Add CombineGlobally translation to avoid translating Combine.perKey as a composite transform based on Combine.PerKey (which uses low perf GBK)
     new 4a1fd5b  Cleaning
     new f4a193c  Get back to classes in translators resolution because URNs cannot translate Combine.Globally
     new e11a72d  Fix various type checking issues in Combine.Globally
     new 143f09f  Update test with Long
     new ab06b8d  Fix combine. For unknown reason GenericRowWithSchema is used as input of combine so extract its content to be able to proceed
     new b8cf669  Use more generic Row instead of GenericRowWithSchema
     new 4f63965  Add explanation about receiving a Row as input in the combiner
     new bb210c4  Fix encoder bug in combinePerkey
     new 1f772db  Cleaning
     new f5a3a6a  Implement WindowAssignTranslatorBatch
     new 7e39e2a  Implement WindowAssignTest
     new 077a563  Fix javadoc
     new 5dd6884  Added SideInput support
     new dd3b0ef  Fix CheckStyle violations
     new 4d540fe  Don't use Reshuffle translation
     new d7b8ea2  Added using CachedSideInputReader
     new 2551488  Added TODO comment for ReshuffleTranslatorBatch
     new 62cbf48  And unchecked warning suppression
     new a476cb2  Add streaming source initialisation
     new 66c6090  Implement first streaming source
     new 3c9e3ad  Add a TODO on spark output modes
     new bac7b46  Add transformators registry in PipelineTranslatorStreaming
     new f446d14  Add source streaming test
     new fde0e46  Specify checkpointLocation at the pipeline start
     new e25571b  Clean unneeded 0 arg constructor in batch source
     new 041d8e3  Clean streaming source
     new 5499990  Continue impl of offsets for streaming source
     new dddbccf  Deal with checkpoint and offset based read
     new 4e87333  Apply spotless and fix spotbugs warnings
     new 2b37240  Disable never ending test SimpleSourceTest.testUnboundedSource
     new 470085a  Fix access level issues, typos and modernize code to Java 8 style
     new dac1ce9  Merge Spark Structured Streaming runner into main Spark module. Remove spark-structured-streaming module.Rename Runner to SparkStructuredStreamingRunner. Remove specific PipelineOotions and Registrar for Structured Streaming Runner
     new 696d17d  Fix non-vendored imports from Spark Streaming Runner classes
     new 6e0341b  Pass doFnSchemaInformation to ParDo batch translation
     new 0687f3c  Fix spotless issues after rebase
     new 6b83da4  Fix logging levels in Spark Structured Streaming translation
     new be96e01  Add SparkStructuredStreamingPipelineOptions and SparkCommonPipelineOptions - SparkStructuredStreamingPipelineOptions was added to have the new   runner rely only on its specific options.
     new 16b3eb0  Rename SparkPipelineResult to SparkStructuredStreamingPipelineResult This is done to avoid an eventual collision with the one in SparkRunner. However this cannot happen at this moment because it is package private, so it is also done for consistency.
     new 0d2dc3e  Use PAssert in Spark Structured Streaming transform tests
     new 035d261  Ignore spark offsets (cf javadoc)
     new ee67ef3  implement source.stop
     new aef79ae  Update javadoc
     new a69cc67  Apply Spotless
     new e2248ae  Enable batch Validates Runner tests for Structured Streaming Runner
     new 7aba40d  Limit the number of partitions to make tests go 300% faster
     new 01dc170  Fixes ParDo not calling setup and not tearing down if exception on startBundle
     new 246e8c3  Pass transform based doFnSchemaInformation in ParDo translation
     new 74316dc  Consider null object case on RowHelpers, fixes empty side inputs tests.
     new f460906  Put back batch/simpleSourceTest.testBoundedSource
     new 81d687c  Update windowAssignTest
     new b9c48ac  Add comment about checkpoint mark
     new 53fb97b  Re-code GroupByKeyTranslatorBatch to conserve windowing instead of unwindowing/windowing(GlobalWindow): simplify code, use ReduceFnRunner to merge the windows
     new 7d8ab5f  re-enable reduceFnRunner timers for output
     new edecedf  Improve visibility of debug messages
     new 02f690f  Add a test that GBK preserves windowing
     new 06f88a6  Add TODO in Combine translations
     new e7c9bed  Update KVHelpers.extractKey() to deal with WindowedValue and update GBK and CPK
     new 859b4ce  Fix comment about schemas
     new fcd021b  Implement reduce part of CombineGlobally translation with windowing
     new 2726c5a  Output data after combine
     new 80ecd5b  Implement merge accumulators part of CombineGlobally translation with windowing
     new c4c4d8d  Fix encoder in combine call
     new d47f721  Revert extractKey while combinePerKey is not done (so that it compiles)
     new c48421d  Apply a groupByKey avoids for some reason that the spark structured streaming fmwk casts data to Row which makes it impossible to deserialize without the coder shipped into the data. For performance reasons (avoid memory consumption and having to deserialize), we do not ship coder + data. Also add a mapparitions before GBK to avoid shuffling
     new 9f83191  Fix case when a window does not merge into any other window
     new ef54b3d  Fix wrong encoder in combineGlobally GBK
     new 537dfcc  Fix bug in the window merging logic
     new 007c937  Remove the mapPartition that adds a key per partition because otherwise spark will reduce values per key instead of globally
     new f060076  Remove CombineGlobally translation because it is less performant than the beam sdk one (key + combinePerKey.withHotkeyFanout)
     new ef47a2b  Now that there is only Combine.PerKey translation, make only one Aggregator
     new c4b0610  Clean no more needed KVHelpers
     new a80e8c4  Clean not more needed RowHelpers
     new 5840fa1  Clean not more needed WindowingHelpers
     new 3c4f55c  Fix javadoc of AggregatorCombiner
     new 1b17242  Fixed immutable list bug
     new 9a5d813  add comment in combine globally test
     new b8592f6  Clean groupByKeyTest
     new 8760699  Add a test that combine per key preserves windowing
     new 928d5ef  Ignore for now not working test testCombineGlobally
     new 3f72747  Add metrics support in DoFn
     new b27090b  Add missing dependencies to run Spark Structured Streaming Runner on Nexmark
     new 0252c70  Add setEnableSparkMetricSinks() method
     new efd3070  Fix javadoc
     new 4f27def  Fix accumulators initialization in Combine that prevented CombineGlobally to work.
     new e4d2e85  Add a test to check that CombineGlobally preserves windowing
     new dc5bcae  Persist all output Dataset if there are multiple outputs in pipeline Enabled Use*Metrics tests
     new 735e312  Added metrics sinks and tests
     new 9e81526  Make spotless happy
     new 7386c77  Add PipelineResults to Spark structured streaming.
     new 55539f5  Update log4j configuration
     new c98460a  Add spark execution plans extended debug messages.
     new dfe4f93  Print number of leaf datasets
     new 9d02370  fixup! Add PipelineResults to Spark structured streaming.
     new aed7062  Remove no more needed AggregatorCombinerPerKey (there is only AggregatorCombiner)
     new 9cc8f17  After testing performance and correctness, launch pipeline with dataset.foreach(). Make both test mode and production mode use foreach for uniformity. Move dataset print as a utility method
     new 4b36264  Add a TODO on perf improvement of Pardo translation
     new 66123bd  Improve Pardo translation performance: avoid calling a filter transform when there is only one output tag
     new 066fe45  Use "sparkMaster" in local mode to obtain number of shuffle partitions + spotless apply
     new 174595b  Fix hardcoded value

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (2c5d37a)
            \
             N -- N -- N   refs/heads/spark-runner_structured-streaming (174595b)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 209 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .github/PULL_REQUEST_TEMPLATE.md                   |    6 +-
 .gitignore                                         |    6 +-
 .test-infra/dataproc/create_flink_cluster.sh       |   14 +-
 .test-infra/dataproc/init-actions/flink.sh         |   24 +-
 .test-infra/jenkins/CommonJobProperties.groovy     |   12 +-
 .test-infra/jenkins/CommonTestProperties.groovy    |   10 +-
 .test-infra/jenkins/Infrastructure.groovy          |  120 +
 .test-infra/jenkins/LoadTestsBuilder.groovy        |   37 +-
 .test-infra/jenkins/NexmarkBuilder.groovy          |    2 +-
 .test-infra/jenkins/README.md                      |   50 +-
 .../dependency_check_report_generator_test.py      |   12 +-
 .../jenkins/dependency_check/generate_report.sh    |    4 +-
 .../dependency_check/version_comparer_test.py      |    2 +-
 .test-infra/jenkins/jira_utils/jira_manager.py     |    2 +-
 .../jenkins/jira_utils/jira_manager_test.py        |   14 +-
 .test-infra/jenkins/job_Inventory.groovy           |    3 +-
 .../jenkins/job_LoadTests_CoGBK_Java.groovy        |  209 ++
 .../job_LoadTests_Combine_Flink_Python.groovy      |  186 ++
 .../jenkins/job_LoadTests_Combine_Java.groovy      |   27 +-
 .../jenkins/job_LoadTests_Combine_Python.groovy    |  161 +
 .../jenkins/job_LoadTests_GBK_Flink_Python.groovy  |   87 +
 .test-infra/jenkins/job_LoadTests_GBK_Java.groovy  |  259 ++
 .../jenkins/job_LoadTests_GBK_Python.groovy        |  151 +
 .../job_LoadTests_GBK_Python_reiterate.groovy      |   97 +
 .test-infra/jenkins/job_LoadTests_Java.groovy      |  231 --
 .../jenkins/job_LoadTests_Java_Smoke.groovy        |   21 +-
 .../jenkins/job_LoadTests_ParDo_Java.groovy        |  181 ++
 .../jenkins/job_LoadTests_ParDo_Python.groovy      |  137 +
 .test-infra/jenkins/job_LoadTests_Python.groovy    |   80 -
 .../jenkins/job_LoadTests_Python_Smoke.groovy      |   70 +
 .../job_LoadTests_coGBK_Flink_Python.groovy        |  187 ++
 .../jenkins/job_LoadTests_coGBK_Python.groovy      |  167 ++
 .../job_PerformanceTests_BigQueryIO_Python.groovy  |  101 +
 .../jenkins/job_PerformanceTests_Dataflow.groovy   |   43 -
 .../job_PerformanceTests_FileBasedIO_IT.groovy     |    2 +-
 ...job_PerformanceTests_FileBasedIO_IT_HDFS.groovy |    2 +-
 .../job_PerformanceTests_HadoopFormat.groovy       |    2 +-
 .../jenkins/job_PerformanceTests_JDBC.groovy       |    2 +-
 .../job_PerformanceTests_MongoDBIO_IT.groovy       |    2 +-
 .../jenkins/job_PerformanceTests_Python.groovy     |  197 +-
 .../jenkins/job_PerformanceTests_Spark.groovy      |   46 -
 .../jenkins/job_Performancetests_Python35.groovy   |   63 -
 .../job_PostCommit_Go_ValidatesRunner_Flink.groovy |   38 +
 .../job_PostCommit_Go_ValidatesRunner_Spark.groovy |   38 +
 .../job_PostCommit_Java11_Dataflow_Examples.groovy |    2 +-
 ...mit_Java11_Dataflow_Portability_Examples.groovy |    2 +-
 ..._ValidatesRunner_PortabilityApi_Dataflow.groovy |    2 +-
 .../job_PostCommit_Java_Nexmark_Dataflow.groovy    |   16 +-
 .../job_PostCommit_Java_Nexmark_Direct.groovy      |   16 +-
 .../job_PostCommit_Java_Nexmark_Flink.groovy       |   16 +-
 .../job_PostCommit_Java_Nexmark_Spark.groovy       |    8 +-
 ...Java_PortableValidatesRunner_Flink_Batch.groovy |    2 +-
 ..._PortableValidatesRunner_Flink_Streaming.groovy |    2 +-
 ...Java_PortableValidatesRunner_Spark_Batch.groovy |   43 +
 ...job_PostCommit_Java_ValidatesRunner_Apex.groovy |    2 +-
 ...PostCommit_Java_ValidatesRunner_Dataflow.groovy |    2 +-
 ...unner_DataflowPortabilityExecutableStage.groovy |    2 +-
 ...mit_Java_ValidatesRunner_Dataflow_Java11.groovy |    2 +-
 ...b_PostCommit_Java_ValidatesRunner_Direct.groovy |   43 +
 ...ommit_Java_ValidatesRunner_Direct_Java11.groovy |   10 +-
 ...ob_PostCommit_Java_ValidatesRunner_Flink.groovy |    2 +-
 ...PostCommit_Java_ValidatesRunner_Gearpump.groovy |    2 +-
 ..._ValidatesRunner_PortabilityApi_Dataflow.groovy |    2 +-
 ...ob_PostCommit_Java_ValidatesRunner_Samza.groovy |    2 +-
 ...ob_PostCommit_Java_ValidatesRunner_Spark.groovy |    2 +-
 .test-infra/jenkins/job_PostCommit_Python2.groovy  |   44 +
 .test-infra/jenkins/job_PostCommit_Python35.groovy |   44 +
 .test-infra/jenkins/job_PostCommit_Python36.groovy |   44 +
 .test-infra/jenkins/job_PostCommit_Python37.groovy |   44 +
 .../jenkins/job_PostCommit_Python3_Verify.groovy   |   38 -
 .../job_PostCommit_Python_MongoDBIO_IT.groovy      |   38 +
 ...ommit_Python_ValidatesContainer_Dataflow.groovy |    4 +
 ...stCommit_Python_ValidatesRunner_Dataflow.groovy |   14 +-
 ..._PostCommit_Python_ValidatesRunner_Spark.groovy |   38 +
 .../jenkins/job_PostCommit_Python_Verify.groovy    |   40 -
 .../jenkins/job_PostCommit_Website_Publish.groovy  |    4 +-
 .../jenkins/job_PostCommit_Website_Test.groovy     |    2 +-
 .test-infra/jenkins/job_PreCommit_Go.groovy        |    1 -
 .../jenkins/job_PreCommit_Portable_Python.groovy   |    7 +-
 .test-infra/jenkins/job_PreCommit_Python.groovy    |    3 +-
 ...b_PreCommit_Python_ValidatesRunner_Flink.groovy |   27 +-
 .../jenkins/job_PreCommit_Website_Stage_GCS.groovy |    2 +-
 .../job_Release_Python_NightlySnapshot.groovy      |    4 +-
 .../kafka-cluster/03-zookeeper/30service.yml       |    1 +
 .test-infra/metrics/beamgrafana-deploy.yaml        |   20 +-
 .../metrics/build_and_publish_containers.sh        |   57 +
 .../dashboards/Post-Commits_status_dashboard.json  |  175 --
 .test-infra/metrics/dashboards/code_velocity.json  | 1167 --------
 .../metrics/dashboards/post-commit_tests.json      |  836 ------
 .../metrics/dashboards/pre-commit_tests.json       |  241 --
 .../metrics/dashboards/source_data_freshness.json  |  258 --
 .../dashboards/stability_critical_jobs_status.json |  332 ---
 .test-infra/metrics/docker-compose.yml             |   12 +-
 .test-infra/metrics/grafana/Dockerfile             |   25 +
 .../dashboards/Post-Commits_status_dashboard.json  |  175 ++
 .../metrics/grafana/dashboards/code_velocity.json  | 1128 +++++++
 .../grafana/dashboards/post-commit_tests.json      |  797 +++++
 .../grafana/dashboards/pre-commit_tests.json       |  435 +++
 .../grafana/dashboards/source_data_freshness.json  |  229 ++
 .../dashboards/stability_critical_jobs_status.json |  410 +++
 .../grafana/provisioning/dashboards/all.yaml       |   32 +
 .../provisioning/datasources/beampostgresql.yaml   |   35 +
 .test-infra/metrics/postgres/init.sql              |   18 +
 README.md                                          |    8 +-
 build.gradle                                       |  112 +-
 buildSrc/build.gradle                              |    2 +-
 .../org/apache/beam/gradle/BeamModulePlugin.groovy |  553 ++--
 .../org/apache/beam/gradle/GrpcVendoring.groovy    |   48 +-
 .../org/apache/beam/gradle/VendorJavaPlugin.groovy |   11 +-
 deprecation-warning.txt                            |   18 +
 examples/java/README.md                            |    5 +-
 examples/java/build.gradle                         |   60 +-
 .../apache/beam/examples/common/ExampleUtils.java  |    8 +-
 .../examples/common/WriteOneFilePerWindow.java     |    2 +-
 .../beam/examples/complete/AutoComplete.java       |    4 +-
 .../org/apache/beam/examples/complete/TfIdf.java   |    2 +-
 .../examples/complete/TopWikipediaSessions.java    |    2 +-
 .../beam/examples/complete/TrafficRoutes.java      |    2 +-
 .../beam/examples/complete/game/LeaderBoard.java   |    2 +-
 .../examples/complete/game/StatefulTeamScore.java  |    4 +-
 .../examples/complete/game/injector/Injector.java  |    2 +-
 .../complete/game/injector/InjectorUtils.java      |    2 +-
 .../game/injector/RetryHttpInitializerWrapper.java |    2 +-
 .../examples/complete/game/utils/WriteToText.java  |    2 +-
 .../beam/examples/cookbook/BigQueryTornadoes.java  |   46 +-
 .../apache/beam/examples/snippets/Snippets.java    |  206 +-
 .../kernel/SubProcessCommandLineArgs.java          |    2 +-
 .../subprocess/utils/CallingSubProcessUtils.java   |    2 +-
 .../beam/examples/DebuggingWordCountTest.java      |    2 +-
 .../apache/beam/examples/MinimalWordCountTest.java |    2 +-
 .../apache/beam/examples/WindowedWordCountIT.java  |    6 +-
 .../org/apache/beam/examples/WordCountTest.java    |   19 +-
 .../apache/beam/examples/complete/TfIdfTest.java   |    3 -
 .../examples/complete/TrafficMaxLaneFlowIT.java    |   42 +-
 .../beam/examples/complete/TrafficRoutesIT.java    |   38 +-
 .../examples/complete/game/LeaderBoardTest.java    |    2 +-
 .../beam/examples/complete/game/UserScoreTest.java |    2 +-
 .../examples/cookbook/BigQueryTornadoesTest.java   |    2 +-
 .../examples/cookbook/MaxPerKeyExamplesTest.java   |    2 +-
 .../beam/examples/cookbook/TriggerExampleTest.java |    4 +-
 .../subprocess/ExampleEchoPipelineTest.java        |    2 +-
 examples/kotlin/OWNERS                             |    5 -
 examples/kotlin/build.gradle                       |   60 +-
 .../beam/examples/kotlin/common/ExampleUtils.kt    |    8 +-
 .../kotlin/common/WriteOneFilePerWindow.kt         |    2 +-
 .../examples/kotlin/cookbook/BigQueryTornadoes.kt  |  195 ++
 .../kotlin/cookbook/CombinePerKeyExamples.kt       |  187 ++
 .../examples/kotlin/cookbook/DistinctExample.kt    |   94 +
 .../examples/kotlin/cookbook/FilterExamples.kt     |  229 ++
 .../beam/examples/kotlin/cookbook/JoinExamples.kt  |  174 ++
 .../examples/kotlin/cookbook/MaxPerKeyExamples.kt  |  155 +
 .../beam/examples/kotlin}/cookbook/README.md       |    0
 .../examples/kotlin/cookbook/TriggerExample.kt     |  524 ++++
 .../beam/examples/kotlin/snippets/Snippets.kt      |  382 +++
 .../examples/DebuggingWordCountTestKotlin.java     |    2 +-
 .../beam/examples/MinimalWordCountTestKotlin.java  |    2 +-
 .../beam/examples/WindowedWordCountITKotlin.kt     |    6 +-
 .../get-started/try-apache-beam-java.ipynb         |   12 +-
 gradle.properties                                  |    2 +-
 gradlew                                            |  184 +-
 gradlew.bat                                        |   93 +-
 gradlew => gradlew_orig                            |    0
 gradlew.bat => gradlew_orig.bat                    |    0
 learning/katas/README.md                           |   26 +
 learning/katas/java/.idea/study_project.xml        | 3151 ++++++++++++++++++++
 .../commontransforms/aggregation/count/Task.java   |   48 +
 .../Common Transforms/Aggregation/Count/task.html  |   29 +
 .../aggregation/count/TaskTest.java                |   46 +
 .../commontransforms/aggregation/max/Task.java     |   48 +
 .../Common Transforms/Aggregation/Max/task.html    |   29 +
 .../commontransforms/aggregation/max/TaskTest.java |   46 +
 .../commontransforms/aggregation/mean/Task.java    |   48 +
 .../Common Transforms/Aggregation/Mean/task.html   |   29 +
 .../aggregation/mean/TaskTest.java                 |   46 +
 .../commontransforms/aggregation/min/Task.java     |   48 +
 .../Common Transforms/Aggregation/Min/task.html    |   29 +
 .../commontransforms/aggregation/min/TaskTest.java |   46 +
 .../commontransforms/aggregation/sum/Task.java     |   48 +
 .../Common Transforms/Aggregation/Sum/task.html    |   29 +
 .../commontransforms/aggregation/sum/TaskTest.java |   46 +
 .../katas/commontransforms/filter/filter/Task.java |   49 +
 .../java/Common Transforms/Filter/Filter/task.html |   34 +
 .../commontransforms/filter/filter/TaskTest.java   |   46 +
 .../katas/commontransforms/filter/pardo/Task.java  |   59 +
 .../java/Common Transforms/Filter/ParDo/task.html  |   33 +
 .../commontransforms/filter/pardo/TaskTest.java    |   46 +
 .../katas/commontransforms/withkeys/Task.java      |   55 +
 .../Common Transforms/WithKeys/WithKeys/task.html  |   34 +
 .../katas/commontransforms/withkeys/TaskTest.java  |   57 +
 .../katas/coretransforms/branching/Task.java       |   57 +
 .../Core Transforms/Branching/Branching/task.html  |   35 +
 .../katas/coretransforms/branching/TaskTest.java   |   50 +
 .../katas/coretransforms/cogroupbykey/Task.java    |  100 +
 .../coretransforms/cogroupbykey/WordsAlphabet.java |   42 +
 .../CoGroupByKey/CoGroupByKey/task.html            |   45 +
 .../coretransforms/cogroupbykey/TaskTest.java      |   57 +
 .../combine/binarycombinefnlambda/Task.java        |   54 +
 .../Combine/BinaryCombineFn Lambda/task.html       |   43 +
 .../combine/binarycombinefnlambda/TaskTest.java    |   50 +
 .../combine/binarycombinefn/Task.java              |   64 +
 .../Combine/BinaryCombineFn/task.html              |   50 +
 .../combine/binarycombinefn/TaskTest.java          |   50 +
 .../coretransforms/combine/combineperkey/Task.java |   68 +
 .../Combine/Combine PerKey/task.html               |   48 +
 .../combine/combineperkey/TaskTest.java            |   53 +
 .../coretransforms/combine/combinefn/Task.java     |  108 +
 .../Core Transforms/Combine/CombineFn/task.html    |   52 +
 .../coretransforms/combine/combinefn/TaskTest.java |   46 +
 .../katas/coretransforms/combine/simple/Task.java  |   64 +
 .../Combine/Simple Function/task.html              |   49 +
 .../coretransforms/combine/simple/TaskTest.java    |   46 +
 .../katas/coretransforms/composite/Task.java       |   69 +
 .../Composite Transform/task.html                  |   51 +
 .../katas/coretransforms/composite/TaskTest.java   |   49 +
 .../coretransforms/dofnadditionalparams/Task.java  |   36 +
 .../DoFn Additional Parameters/task.html           |   52 +
 .../dofnadditionalparams/TaskTest.java             |   22 +
 .../katas/coretransforms/flatten/Task.java         |   60 +
 .../java/Core Transforms/Flatten/Flatten/task.html |   41 +
 .../katas/coretransforms/flatten/TaskTest.java     |   50 +
 .../katas/coretransforms/groupbykey/Task.java      |   60 +
 .../GroupByKey/GroupByKey/task.html                |   45 +
 .../katas/coretransforms/groupbykey/TaskTest.java  |   56 +
 .../coretransforms/map/flatmapelements/Task.java   |   54 +
 .../Core Transforms/Map/FlatMapElements/task.html  |   44 +
 .../map/flatmapelements/TaskTest.java              |   46 +
 .../katas/coretransforms/map/mapelements/Task.java |   53 +
 .../java/Core Transforms/Map/MapElements/task.html |   42 +
 .../coretransforms/map/mapelements/TaskTest.java   |   46 +
 .../coretransforms/map/pardoonetomany/Task.java    |   61 +
 .../Core Transforms/Map/ParDo OneToMany/task.html  |   38 +
 .../map/pardoonetomany/TaskTest.java               |   46 +
 .../katas/coretransforms/map/pardo/Task.java       |   57 +
 .../katas/java/Core Transforms/Map/ParDo/task.html |   42 +
 .../katas/coretransforms/map/pardo/TaskTest.java   |   46 +
 .../katas/coretransforms/partition/Task.java       |   62 +
 .../Core Transforms/Partition/Partition/task.html  |   48 +
 .../katas/coretransforms/partition/TaskTest.java   |   52 +
 .../katas/coretransforms/sideinput/Person.java     |   79 +
 .../katas/coretransforms/sideinput/Task.java       |   94 +
 .../Side Input/Side Input/task.html                |   54 +
 .../katas/coretransforms/sideinput/TaskTest.java   |   75 +
 .../katas/coretransforms/sideoutput/Task.java      |   70 +
 .../Side Output/Side Output/task.html              |   44 +
 .../katas/coretransforms/sideoutput/TaskTest.java  |   54 +
 .../learning/katas/examples/wordcount/Task.java    |   76 +
 .../java/Examples/Word Count/Word Count/task.html  |   36 +
 .../katas/examples/wordcount/TaskTest.java         |   56 +
 .../beam/learning/katas/io/builtinios/Task.java    |   36 +
 .../java/IO/Built-in IOs/Built-in IOs/task.html    |   29 +
 .../learning/katas/io/builtinios/TaskTest.java     |   22 +
 .../katas/java/IO/TextIO/TextIO Read/countries.txt |   10 +
 .../beam/learning/katas/io/textio/read/Task.java   |   53 +
 .../katas/java/IO/TextIO/TextIO Read/task.html     |   47 +
 .../learning/katas/io/textio/read/TaskTest.java    |   57 +
 .../beam/learning/katas/intro/hello/Task.java      |   45 +
 .../Introduction/Hello Beam/Hello Beam/task.html   |   53 +
 .../beam/learning/katas/intro/hello/TaskTest.java  |   43 +
 learning/katas/java/README.md                      |   31 +
 .../triggers/earlytriggers/GenerateEvent.java      |   42 +
 .../katas/triggers/earlytriggers/Task.java         |   64 +
 .../Early Triggers/Early Triggers/task.html        |   59 +
 .../katas/triggers/earlytriggers/TaskTest.java     |   67 +
 .../triggers/eventtimetriggers/GenerateEvent.java  |   42 +
 .../katas/triggers/eventtimetriggers/Task.java     |   60 +
 .../Event Time Triggers/task.html                  |   78 +
 .../katas/triggers/eventtimetriggers/TaskTest.java |   70 +
 .../triggers/windowaccummode/GenerateEvent.java    |   42 +
 .../katas/triggers/windowaccummode/Task.java       |   64 +
 .../Window Accumulation Mode/task.html             |   63 +
 .../katas/triggers/windowaccummode/TaskTest.java   |   67 +
 .../windowing/addingtimestamp/pardo/Event.java     |   92 +
 .../windowing/addingtimestamp/pardo/Task.java      |   66 +
 .../Windowing/Adding Timestamp/ParDo/task.html     |   48 +
 .../windowing/addingtimestamp/pardo/TaskTest.java  |   85 +
 .../addingtimestamp/withtimestamps/Event.java      |   92 +
 .../addingtimestamp/withtimestamps/Task.java       |   58 +
 .../Adding Timestamp/WithTimestamps/task.html      |   42 +
 .../addingtimestamp/withtimestamps/TaskTest.java   |   85 +
 .../learning/katas/windowing/fixedwindow/Task.java |   70 +
 .../Fixed Time Window/Fixed Time Window/task.html  |   61 +
 .../katas/windowing/fixedwindow/TaskTest.java      |   86 +
 .../katas/windowing/fixedwindow/WindowedEvent.java |   64 +
 learning/katas/java/build.gradle                   |  118 +
 .../katas/java/gradle/wrapper/gradle-wrapper.jar   |  Bin 0 -> 54413 bytes
 .../java/gradle/wrapper/gradle-wrapper.properties  |   23 +
 learning/katas/java/gradlew                        |  190 ++
 gradlew.bat => learning/katas/java/gradlew.bat     |    0
 learning/katas/java/settings.gradle                |   43 +
 learning/katas/java/util/resources/log4j2.xml      |   31 +
 .../org/apache/beam/learning/katas/util/Log.java   |   81 +
 .../beam/learning/katas/util/ContainsKvs.java      |   60 +
 .../apache/beam/learning/katas/util/KvMatcher.java |   56 +
 learning/katas/python/.idea/study_project.xml      | 2317 ++++++++++++++
 .../Common Transforms/Aggregation/Count/task.html  |   29 +
 .../Common Transforms/Aggregation/Count/task.py    |   27 +
 .../Common Transforms/Aggregation/Count/tests.py   |   33 +
 .../Aggregation/Largest/task.html                  |   29 +
 .../Common Transforms/Aggregation/Largest/task.py  |   27 +
 .../Common Transforms/Aggregation/Largest/tests.py |   33 +
 .../Common Transforms/Aggregation/Mean/task.html   |   29 +
 .../Common Transforms/Aggregation/Mean/task.py     |   27 +
 .../Common Transforms/Aggregation/Mean/tests.py    |   33 +
 .../Aggregation/Smallest/task.html                 |   29 +
 .../Common Transforms/Aggregation/Smallest/task.py |   27 +
 .../Aggregation/Smallest/tests.py                  |   33 +
 .../Common Transforms/Aggregation/Sum/task.html    |   30 +
 .../Common Transforms/Aggregation/Sum/task.py      |   27 +
 .../Common Transforms/Aggregation/Sum/tests.py     |   33 +
 .../Common Transforms/Filter/Filter/task.html      |   34 +
 .../python/Common Transforms/Filter/Filter/task.py |   27 +
 .../Common Transforms/Filter/Filter/tests.py       |   44 +
 .../Common Transforms/Filter/ParDo/task.html       |   31 +
 .../python/Common Transforms/Filter/ParDo/task.py  |   35 +
 .../python/Common Transforms/Filter/ParDo/tests.py |   33 +
 .../Core Transforms/Branching/Branching/task.html  |   35 +
 .../Core Transforms/Branching/Branching/task.py    |   31 +
 .../Core Transforms/Branching/Branching/tests.py   |   43 +
 .../CoGroupByKey/CoGroupByKey/task.html            |   42 +
 .../CoGroupByKey/CoGroupByKey/task.py              |   57 +
 .../CoGroupByKey/CoGroupByKey/tests.py             |   37 +
 .../Combine/Combine PerKey/task.html               |   48 +
 .../Core Transforms/Combine/Combine PerKey/task.py |   32 +
 .../Combine/Combine PerKey/tests.py                |   49 +
 .../Core Transforms/Combine/CombineFn/task.html    |   52 +
 .../Core Transforms/Combine/CombineFn/task.py      |   46 +
 .../Core Transforms/Combine/CombineFn/tests.py     |   44 +
 .../Combine/Simple Function/task.html              |   47 +
 .../Combine/Simple Function/task.py                |   37 +
 .../Combine/Simple Function/tests.py               |   44 +
 .../Composite Transform/task.html                  |   49 +
 .../Composite Transform/task.py                    |   37 +
 .../Composite Transform/tests.py                   |   44 +
 .../Core Transforms/Flatten/Flatten/task.html      |   40 +
 .../python/Core Transforms/Flatten/Flatten/task.py |   33 +
 .../Core Transforms/Flatten/Flatten/tests.py       |   44 +
 .../GroupByKey/GroupByKey/task.html                |   44 +
 .../Core Transforms/GroupByKey/GroupByKey/task.py  |   28 +
 .../Core Transforms/GroupByKey/GroupByKey/tests.py |   35 +
 .../python/Core Transforms/Map/FlatMap/task.html   |   43 +
 .../python/Core Transforms/Map/FlatMap/task.py     |   27 +
 .../python/Core Transforms/Map/FlatMap/tests.py    |   44 +
 .../katas/python/Core Transforms/Map/Map/task.html |   39 +
 .../katas/python/Core Transforms/Map/Map/task.py   |   27 +
 .../katas/python/Core Transforms/Map/Map/tests.py  |   44 +
 .../Core Transforms/Map/ParDo OneToMany/task.html  |   43 +
 .../Core Transforms/Map/ParDo OneToMany/task.py    |   34 +
 .../Core Transforms/Map/ParDo OneToMany/tests.py   |   55 +
 .../python/Core Transforms/Map/ParDo/task.html     |   45 +
 .../katas/python/Core Transforms/Map/ParDo/task.py |   34 +
 .../python/Core Transforms/Map/ParDo/tests.py      |   55 +
 .../Core Transforms/Partition/Partition/task.html  |   47 +
 .../Core Transforms/Partition/Partition/task.py    |   38 +
 .../Core Transforms/Partition/Partition/tests.py   |   53 +
 .../Side Input/Side Input/task.html                |   53 +
 .../Core Transforms/Side Input/Side Input/task.py  |   69 +
 .../Core Transforms/Side Input/Side Input/tests.py |   70 +
 .../Side Output/Side Output/task.html              |   44 +
 .../Side Output/Side Output/task.py                |   45 +
 .../Side Output/Side Output/tests.py               |   65 +
 .../Examples/Word Count/Word Count/task.html       |   36 +
 .../python/Examples/Word Count/Word Count/task.py  |   36 +
 .../python/Examples/Word Count/Word Count/tests.py |   39 +
 .../python/IO/Built-in IOs/Built-in IOs/task.html  |   29 +
 .../python/IO/Built-in IOs/Built-in IOs/task.py    |   22 +
 .../python/IO/Built-in IOs/Built-in IOs/tests.py   |   16 +
 .../python/IO/TextIO/ReadFromText/countries.txt    |   10 +
 .../katas/python/IO/TextIO/ReadFromText/task.html  |   45 +
 .../katas/python/IO/TextIO/ReadFromText/task.py    |   29 +
 .../katas/python/IO/TextIO/ReadFromText/tests.py   |   55 +
 .../Introduction/Hello Beam/Hello Beam/task.html   |   53 +
 .../Introduction/Hello Beam/Hello Beam/task.py     |   26 +
 .../Introduction/Hello Beam/Hello Beam/tests.py    |   41 +
 learning/katas/python/README.md                    |   32 +
 learning/katas/python/log_elements.py              |   37 +
 learning/katas/python/requirements.txt             |   18 +
 learning/katas/python/test_helper.py               |  240 ++
 model/fn-execution/build.gradle                    |    4 +-
 .../fn-execution/src/main/proto/beam_fn_api.proto  |   69 +-
 model/job-management/build.gradle                  |    4 +-
 .../src/main/proto/beam_job_api.proto              |   16 +
 .../pipeline/src/main/proto/beam_runner_api.proto  |   88 +-
 model/pipeline/src/main/proto/schema.proto         |   85 +
 ownership/PYTHON_DEPENDENCY_OWNERS.yaml            |    3 +
 project-mappings                                   |  112 +
 release/build.gradle                               |   14 +-
 .../python-release/python_release_automation.sh    |   16 +-
 .../src/main/scripts/build_release_candidate.sh    |    2 +-
 release/src/main/scripts/run_rc_validation.sh      |   12 +-
 release/src/main/scripts/verify_release_build.sh   |    4 +
 runners/apex/build.gradle                          |   42 +-
 .../org/apache/beam/runners/apex/ApexRunner.java   |    6 +-
 .../beam/runners/apex/ApexRunnerRegistrar.java     |    2 +-
 .../apache/beam/runners/apex/ApexYarnLauncher.java |    8 +-
 .../translation/FlattenPCollectionTranslator.java  |    4 +-
 .../runners/apex/translation/ParDoTranslator.java  |    4 +-
 .../apex/translation/TranslationContext.java       |    4 +-
 .../operators/ApexGroupByKeyOperator.java          |    4 +-
 .../translation/operators/ApexParDoOperator.java   |    8 +-
 .../operators/ApexProcessFnOperator.java           |    4 +-
 .../operators/ApexReadUnboundedInputOperator.java  |    2 +-
 .../translation/operators/ApexTimerInternals.java  |    8 +-
 .../apex/translation/utils/ApexStateInternals.java |    4 +-
 .../apex/translation/utils/ApexStreamTuple.java    |    2 +-
 .../translation/utils/CoderAdapterStreamCodec.java |    2 +-
 .../apache/beam/runners/apex/ApexRunnerTest.java   |    2 +-
 .../beam/runners/apex/examples/WordCountTest.java  |    2 +-
 .../translation/ApexGroupByKeyOperatorTest.java    |    2 +-
 .../FlattenPCollectionTranslatorTest.java          |    4 +-
 .../apex/translation/GroupByKeyTranslatorTest.java |    4 +-
 .../apex/translation/ParDoTranslatorTest.java      |    4 +-
 .../translation/ReadUnboundTranslatorTest.java     |   10 +-
 .../apex/translation/SideInputTranslationTest.java |    2 +-
 .../operators/ApexTimerInternalsTest.java          |    2 +-
 runners/core-construction-java/build.gradle        |   35 +-
 .../core/construction/ArtifactServiceStager.java   |   14 +-
 .../core/construction/AvroCoderRegistrar.java      |   40 +
 .../core/construction/AvroCoderTranslator.java     |   44 +
 .../beam/runners/core/construction/BeamUrns.java   |    2 +-
 .../core/construction/CoderTranslation.java        |   43 +-
 .../core/construction/CoderTranslators.java        |    2 +-
 .../core/construction/CombineTranslation.java      |  235 +-
 .../CreatePCollectionViewTranslation.java          |    4 +-
 .../construction/DeduplicatedFlattenFactory.java   |    2 +-
 .../DefaultExpansionServiceClientFactory.java      |    2 +-
 .../core/construction/DisplayDataTranslation.java  |    4 +-
 .../construction/EmptyFlattenAsCreateFactory.java  |    2 +-
 .../runners/core/construction/Environments.java    |   10 +-
 .../construction/ExecutableStageTranslation.java   |   10 +-
 .../beam/runners/core/construction/External.java   |   10 +-
 .../core/construction/ExternalTranslation.java     |    8 +-
 .../core/construction/JavaReadViaImpulse.java      |   17 +-
 .../core/construction/ModelCoderRegistrar.java     |   12 +-
 .../runners/core/construction/ModelCoders.java     |   16 +-
 .../construction/PCollectionViewTranslation.java   |    4 +-
 .../core/construction/PTransformMatchers.java      |    2 +-
 .../core/construction/PTransformReplacements.java  |    4 +-
 .../core/construction/PTransformTranslation.java   |   12 +-
 .../core/construction/ParDoTranslation.java        |   21 +-
 .../construction/PipelineOptionsTranslation.java   |   10 +-
 .../core/construction/PipelineResources.java       |   23 +-
 .../core/construction/PipelineTranslation.java     |    6 +-
 .../runners/core/construction/PrimitiveCreate.java |    2 +-
 .../runners/core/construction/ReadTranslation.java |   12 +-
 .../core/construction/RehydratedComponents.java    |    8 +-
 .../core/construction/ReplacementOutputs.java      |    6 +-
 .../core/construction/ReshuffleTranslation.java    |   58 +
 .../core/construction/SchemaTranslation.java       |  196 +-
 .../runners/core/construction/SdkComponents.java   |   10 +-
 .../runners/core/construction/SplittableParDo.java |    6 +-
 .../construction/SplittableParDoNaiveBounded.java  |    4 +-
 .../core/construction/TestStreamTranslation.java   |    6 +-
 .../runners/core/construction/TransformInputs.java |    4 +-
 .../core/construction/TriggerTranslation.java      |    4 +-
 .../UnboundedReadFromBoundedSource.java            |   10 +-
 .../core/construction/WindowIntoTranslation.java   |    6 +-
 .../construction/WindowingStrategyTranslation.java |    8 +-
 .../core/construction/WriteFilesTranslation.java   |   15 +-
 .../construction/expansion/ExpansionServer.java    |    6 +-
 .../construction/expansion/ExpansionService.java   |   26 +-
 .../core/construction/graph/FusedPipeline.java     |    2 +-
 .../graph/GreedyPCollectionFusers.java             |    8 +-
 .../construction/graph/GreedyPipelineFuser.java    |   14 +-
 .../core/construction/graph/GreedyStageFuser.java  |    4 +-
 .../graph/ImmutableExecutableStage.java            |    2 +-
 .../runners/core/construction/graph/Networks.java  |   31 +-
 .../construction/graph/OutputDeduplicator.java     |    6 +-
 .../core/construction/graph/PipelineValidator.java |   12 +-
 .../core/construction/graph/ProtoOverrides.java    |    2 +-
 .../core/construction/graph/QueryablePipeline.java |   18 +-
 .../construction/graph/SideInputReference.java     |    2 +-
 .../construction/renderer/PipelineDotRenderer.java |  126 +
 .../renderer/PortablePipelineDotRenderer.java      |  113 +
 .../core/construction/renderer/package-info.java   |   20 +
 .../construction/ArtifactServiceStagerTest.java    |   12 +-
 .../core/construction/CoderTranslationTest.java    |   11 +-
 .../core/construction/CombineTranslationTest.java  |   28 +-
 .../runners/core/construction/CommonCoderTest.java |   14 +-
 .../CreatePCollectionViewTranslationTest.java      |    2 +-
 .../runners/core/construction/ExternalTest.java    |   12 +-
 .../construction/ForwardingPTransformTest.java     |    2 +-
 .../InMemoryArtifactStagerService.java             |    8 +-
 .../core/construction/JavaReadViaImpulseTest.java  |    2 +-
 .../runners/core/construction/ModelCodersTest.java |   13 +-
 .../construction/PCollectionTranslationTest.java   |    2 +-
 .../core/construction/PTransformMatchersTest.java  |    4 +-
 .../construction/PTransformReplacementsTest.java   |    2 +-
 .../construction/PTransformTranslationTest.java    |    2 +-
 .../core/construction/ParDoTranslationTest.java    |    2 +-
 .../PipelineOptionsTranslationTest.java            |    8 +-
 .../core/construction/PipelineResourcesTest.java   |   15 +-
 .../core/construction/PipelineTranslationTest.java |    2 +-
 .../core/construction/ReadTranslationTest.java     |    2 +-
 .../core/construction/ReplacementOutputsTest.java  |    4 +-
 .../construction/ReshuffleTranslationTest.java     |   41 +
 .../core/construction/SchemaTranslationTest.java   |   89 +
 .../construction/TestStreamTranslationTest.java    |    2 +-
 .../core/construction/TriggerTranslationTest.java  |    2 +-
 .../UnboundedReadFromBoundedSourceTest.java        |    6 +-
 .../construction/WindowIntoTranslationTest.java    |    6 +-
 .../WindowingStrategyTranslationTest.java          |    2 +-
 .../construction/WriteFilesTranslationTest.java    |    2 +-
 .../expansion/ExpansionServiceTest.java            |   10 +-
 .../expansion/TestExpansionService.java            |    6 +-
 .../construction/graph/ExecutableStageMatcher.java |    2 +-
 .../construction/graph/ExecutableStageTest.java    |    2 +-
 .../core/construction/graph/FusedPipelineTest.java |    4 +-
 .../graph/GreedyPipelineFuserTest.java             |    2 +-
 .../construction/graph/GreedyStageFuserTest.java   |    4 +-
 .../core/construction/graph/NetworksTest.java      |   20 +-
 .../construction/graph/OutputDeduplicatorTest.java |    4 +-
 .../construction/graph/ProtoOverridesTest.java     |    9 +-
 .../construction/graph/QueryablePipelineTest.java  |    4 +-
 .../renderer/PipelineDotRendererTest.java          |   97 +
 .../renderer/PortablePipelineDotRendererTest.java  |   68 +
 runners/core-java/build.gradle                     |   30 +-
 .../apache/beam/runners/core/ActiveWindowSet.java  |    2 +-
 .../beam/runners/core/GlobalCombineFnRunners.java  |    2 +-
 .../runners/core/GroupByKeyViaGroupByKeyOnly.java  |    2 +-
 .../core/InMemoryMultimapSideInputView.java        |    6 +-
 .../beam/runners/core/InMemoryStateInternals.java  |    6 +-
 .../beam/runners/core/InMemoryTimerInternals.java  |   12 +-
 .../beam/runners/core/KeyedWorkItemCoder.java      |    2 +-
 .../apache/beam/runners/core/KeyedWorkItems.java   |    4 +-
 .../runners/core/LateDataDroppingDoFnRunner.java   |    4 +-
 .../apache/beam/runners/core/LateDataUtils.java    |    4 +-
 .../beam/runners/core/MergingActiveWindowSet.java  |   10 +-
 .../runners/core/NonMergingActiveWindowSet.java    |    4 +-
 .../beam/runners/core/NullSideInputReader.java     |    2 +-
 ...TimeBoundedSplittableProcessElementInvoker.java |    8 +-
 .../apache/beam/runners/core/PaneInfoTracker.java  |    4 +-
 .../beam/runners/core/PeekingReiterator.java       |    4 +-
 .../apache/beam/runners/core/ProcessFnRunner.java  |    4 +-
 .../beam/runners/core/ReduceFnContextFactory.java  |    6 +-
 .../apache/beam/runners/core/ReduceFnRunner.java   |   10 +-
 .../apache/beam/runners/core/SideInputHandler.java |    2 +-
 .../apache/beam/runners/core/SimpleDoFnRunner.java |   10 +-
 .../core/SimplePushbackSideInputDoFnRunner.java    |    4 +-
 .../core/SplittableParDoViaKeyedWorkItems.java     |    4 +-
 .../core/SplittableProcessElementInvoker.java      |    2 +-
 .../apache/beam/runners/core/StateNamespaces.java  |   20 +-
 .../org/apache/beam/runners/core/StateTable.java   |    6 +-
 .../org/apache/beam/runners/core/StateTags.java    |    9 +-
 .../apache/beam/runners/core/SystemReduceFn.java   |    2 +-
 .../apache/beam/runners/core/TimerInternals.java   |    2 +-
 .../apache/beam/runners/core/WatermarkHold.java    |    4 +-
 .../runners/core/metrics/DefaultMetricResults.java |    2 +-
 .../core/metrics/ExecutionStateSampler.java        |    6 +-
 .../core/metrics/ExecutionStateTracker.java        |    6 +-
 .../beam/runners/core/metrics/MetricUpdates.java   |    2 +-
 .../runners/core/metrics/MetricsContainerImpl.java |   54 +-
 .../core/metrics/MetricsContainerStepMap.java      |    2 +-
 .../beam/runners/core/metrics/MetricsMap.java      |    4 +-
 .../beam/runners/core/metrics/MetricsPusher.java   |    4 +-
 .../runners/core/metrics/MetricsTranslation.java   |    6 +-
 .../core/metrics/MonitoringInfoConstants.java      |    2 +
 .../core/metrics/MonitoringInfoMetricName.java     |    6 +-
 .../runners/core/metrics/SimpleExecutionState.java |    2 +-
 .../core/metrics/SimpleMonitoringInfoBuilder.java  |   30 +-
 .../core/metrics/SpecMonitoringInfoValidator.java  |    2 +-
 .../core/triggers/AfterAllStateMachine.java        |    6 +-
 .../AfterDelayFromFirstElementStateMachine.java    |    2 +-
 .../core/triggers/AfterEachStateMachine.java       |    6 +-
 .../core/triggers/AfterFirstStateMachine.java      |    6 +-
 ...fterSynchronizedProcessingTimeStateMachine.java |    2 +-
 .../core/triggers/AfterWatermarkStateMachine.java  |    4 +-
 .../triggers/ExecutableTriggerStateMachine.java    |    4 +-
 .../runners/core/triggers/FinishedTriggersSet.java |    2 +-
 .../core/triggers/OrFinallyStateMachine.java       |    2 +-
 .../runners/core/triggers/TriggerStateMachine.java |    2 +-
 .../TriggerStateMachineContextFactory.java         |    6 +-
 .../core/triggers/TriggerStateMachineRunner.java   |    6 +-
 .../core/InMemoryMultimapSideInputViewTest.java    |    2 +-
 .../beam/runners/core/KeyedWorkItemCoderTest.java  |    2 +-
 .../core/LateDataDroppingDoFnRunnerTest.java       |    4 +-
 .../runners/core/MergingActiveWindowSetTest.java   |    4 +-
 ...BoundedSplittableProcessElementInvokerTest.java |    2 +-
 .../beam/runners/core/ReduceFnRunnerTest.java      |    2 +-
 .../apache/beam/runners/core/ReduceFnTester.java   |   10 +-
 .../beam/runners/core/SideInputHandlerTest.java    |    2 +-
 .../beam/runners/core/SimpleDoFnRunnerTest.java    |    4 +-
 .../SimplePushbackSideInputDoFnRunnerTest.java     |    2 +-
 .../runners/core/SplittableParDoProcessFnTest.java |    2 +-
 .../beam/runners/core/StateInternalsTest.java      |    2 +-
 .../org/apache/beam/runners/core/StateTagTest.java |   20 +-
 .../beam/runners/core/StatefulDoFnRunnerTest.java  |    4 +-
 .../apache/beam/runners/core/WindowMatchers.java   |    2 +-
 .../beam/runners/core/WindowMatchersTest.java      |    2 +-
 .../core/metrics/MetricsContainerImplTest.java     |   56 +
 .../core/metrics/MetricsTranslationTest.java       |    4 +-
 .../metrics/SimpleMonitoringInfoBuilderTest.java   |   31 +-
 .../metrics/SpecMonitoringInfoValidatorTest.java   |   15 +-
 .../core/triggers/StubTriggerStateMachine.java     |    2 +-
 .../core/triggers/TriggerStateMachineTester.java   |    8 +-
 .../core/triggers/TriggerStateMachinesTest.java    |    2 +-
 runners/direct-java/build.gradle                   |  104 +-
 .../direct/BoundedReadEvaluatorFactory.java        |   12 +-
 .../beam/runners/direct/CloningBundleFactory.java  |    2 +-
 .../beam/runners/direct/CommittedResult.java       |    2 +-
 .../direct/CopyOnAccessInMemoryStateInternals.java |    6 +-
 .../apache/beam/runners/direct/DirectGraph.java    |    2 +-
 .../beam/runners/direct/DirectGraphVisitor.java    |    8 +-
 .../beam/runners/direct/DirectGroupByKey.java      |    6 +-
 .../direct/DirectGroupByKeyOverrideFactory.java    |    2 +-
 .../apache/beam/runners/direct/DirectMetrics.java  |    2 +-
 .../beam/runners/direct/DirectRegistrar.java       |    2 +-
 .../apache/beam/runners/direct/DirectRunner.java   |   14 +-
 .../runners/direct/DirectTransformExecutor.java    |    4 +-
 .../beam/runners/direct/DoFnLifecycleManager.java  |   10 +-
 .../beam/runners/direct/EvaluationContext.java     |   12 +-
 .../direct/ExecutorServiceParallelExecutor.java    |   16 +-
 .../runners/direct/FlattenEvaluatorFactory.java    |    2 +-
 .../direct/GroupAlsoByWindowEvaluatorFactory.java  |    4 +-
 .../direct/GroupByKeyOnlyEvaluatorFactory.java     |    4 +-
 .../direct/ImmutabilityCheckingBundleFactory.java  |    6 +-
 .../runners/direct/ImmutableListBundleFactory.java |    8 +-
 .../runners/direct/ImpulseEvaluatorFactory.java    |    4 +-
 .../runners/direct/KeyedPValueTrackingVisitor.java |    4 +-
 .../beam/runners/direct/MultiStepCombine.java      |   12 +-
 .../apache/beam/runners/direct/ParDoEvaluator.java |    4 +-
 .../beam/runners/direct/ParDoEvaluatorFactory.java |    6 +-
 .../runners/direct/ParDoMultiOverrideFactory.java  |    7 +-
 .../beam/runners/direct/QuiescenceDriver.java      |    4 +-
 .../beam/runners/direct/RootProviderRegistry.java  |    4 +-
 .../beam/runners/direct/SideInputContainer.java    |   18 +-
 .../SplittableProcessElementsEvaluatorFactory.java |    8 +-
 .../direct/StatefulParDoEvaluatorFactory.java      |   10 +-
 .../org/apache/beam/runners/direct/StepAndKey.java |    2 +-
 .../beam/runners/direct/StepTransformResult.java   |    2 +-
 .../runners/direct/TestStreamEvaluatorFactory.java |    8 +-
 .../runners/direct/TransformEvaluatorRegistry.java |    6 +-
 .../runners/direct/TransformExecutorServices.java  |    2 +-
 .../runners/direct/UnboundedReadDeduplicator.java  |    6 +-
 .../direct/UnboundedReadEvaluatorFactory.java      |    6 +-
 .../beam/runners/direct/ViewEvaluatorFactory.java  |    2 +-
 .../beam/runners/direct/ViewOverrideFactory.java   |    3 +-
 .../runners/direct/WatermarkCallbackExecutor.java  |   11 +-
 .../beam/runners/direct/WatermarkManager.java      |   32 +-
 .../runners/direct/WindowEvaluatorFactory.java     |    2 +-
 .../runners/direct/WriteWithShardingFactory.java   |    6 +-
 .../runners/direct/portable/BundleFactory.java     |   47 -
 .../BundleFactoryOutputReceiverFactory.java        |   68 -
 .../runners/direct/portable/BundleProcessor.java   |   33 -
 .../runners/direct/portable/CommittedBundle.java   |   86 -
 .../runners/direct/portable/CommittedResult.java   |   63 -
 .../direct/portable/CompletionCallback.java        |   42 -
 .../CopyOnAccessInMemoryStateInternals.java        |  463 ---
 .../runners/direct/portable/DirectGroupByKey.java  |  133 -
 .../runners/direct/portable/DirectMetrics.java     |  307 --
 .../direct/portable/DirectStateAndTimers.java      |   70 -
 .../direct/portable/DirectTimerInternals.java      |  105 -
 .../direct/portable/DirectTransformExecutor.java   |  151 -
 .../direct/portable/DisplayDataValidator.java      |   67 -
 .../direct/portable/EmptyInputProvider.java        |   38 -
 .../runners/direct/portable/EvaluationContext.java |  333 ---
 ...valuationContextStepStateAndTimersProvider.java |   39 -
 .../direct/portable/ExecutorServiceFactory.java    |   30 -
 .../portable/ExecutorServiceParallelExecutor.java  |  392 ---
 .../direct/portable/FlattenEvaluatorFactory.java   |   79 -
 .../GroupAlsoByWindowEvaluatorFactory.java         |  282 --
 .../portable/GroupByKeyOnlyEvaluatorFactory.java   |  157 -
 .../portable/ImmutableListBundleFactory.java       |  171 --
 .../direct/portable/ImpulseEvaluatorFactory.java   |  110 -
 .../direct/portable/PCollectionViewWindow.java     |   66 -
 .../direct/portable/PCollectionViewWriter.java     |   34 -
 .../portable/PassthroughTransformEvaluator.java    |   47 -
 .../runners/direct/portable/PipelineExecutor.java  |   61 -
 .../runners/direct/portable/PortableGraph.java     |   73 -
 .../runners/direct/portable/QuiescenceDriver.java  |  328 --
 .../runners/direct/portable/ReferenceRunner.java   |  559 ----
 .../portable/RemoteStageEvaluatorFactory.java      |   95 -
 .../runners/direct/portable/RootInputProvider.java |   44 -
 .../direct/portable/RootProviderRegistry.java      |   68 -
 .../beam/runners/direct/portable/SourceShard.java  |   33 -
 .../SplittableRemoteStageEvaluatorFactory.java     |  174 --
 .../beam/runners/direct/portable/StepAndKey.java   |   67 -
 .../direct/portable/StepStateAndTimers.java        |   34 -
 .../direct/portable/StepTransformResult.java       |  135 -
 .../direct/portable/TransformEvaluator.java        |   45 -
 .../direct/portable/TransformEvaluatorFactory.java |   58 -
 .../portable/TransformEvaluatorRegistry.java       |  126 -
 .../runners/direct/portable/TransformExecutor.java |   21 -
 .../direct/portable/TransformExecutorFactory.java  |   29 -
 .../direct/portable/TransformExecutorService.java  |   38 -
 .../direct/portable/TransformExecutorServices.java |  180 --
 .../runners/direct/portable/TransformResult.java   |  101 -
 .../runners/direct/portable/UncommittedBundle.java |   56 -
 .../direct/portable/WatermarkCallbackExecutor.java |  171 --
 .../direct/portable/WindowEvaluatorFactory.java    |  123 -
 .../artifact/LocalArtifactStagingLocation.java     |  116 -
 .../LocalFileSystemArtifactRetrievalService.java   |  119 -
 .../LocalFileSystemArtifactStagerService.java      |  282 --
 .../UnsupportedArtifactRetrievalService.java       |   42 -
 .../direct/portable/artifact/package-info.java     |   20 -
 .../runners/direct/portable/job/PreparingJob.java  |   59 -
 .../portable/job/ReferenceRunnerJobServer.java     |  164 -
 .../portable/job/ReferenceRunnerJobService.java    |  308 --
 .../runners/direct/portable/job/package-info.java  |   23 -
 .../beam/runners/direct/portable/package-info.java |   25 -
 .../direct/BoundedReadEvaluatorFactoryTest.java    |    4 +-
 .../runners/direct/CloningBundleFactoryTest.java   |    4 +-
 .../beam/runners/direct/CommittedResultTest.java   |    4 +-
 .../CopyOnAccessInMemoryStateInternalsTest.java    |    2 +-
 .../runners/direct/DirectGraphVisitorTest.java     |    2 +-
 .../beam/runners/direct/DirectMetricsTest.java     |    3 +-
 .../beam/runners/direct/DirectRegistrarTest.java   |    4 +-
 .../runners/direct/DirectRunnerApiSurfaceTest.java |    4 +-
 .../beam/runners/direct/DirectRunnerTest.java      |    4 +-
 .../direct/DirectTransformExecutorTest.java        |    6 +-
 .../runners/direct/DoFnLifecycleManagerTest.java   |    2 +-
 .../runners/direct/DoFnLifecycleManagersTest.java  |    2 +-
 .../beam/runners/direct/EvaluationContextTest.java |    4 +-
 .../ExecutorServiceParallelExecutorTest.java       |    4 +-
 .../direct/FlattenEvaluatorFactoryTest.java        |    2 +-
 .../direct/GroupByKeyEvaluatorFactoryTest.java     |    6 +-
 .../direct/GroupByKeyOnlyEvaluatorFactoryTest.java |    6 +-
 .../direct/ImmutableListBundleFactoryTest.java     |    2 +-
 .../direct/ImpulseEvaluatorFactoryTest.java        |    2 +-
 .../org/apache/beam/runners/direct/MockClock.java  |    2 +-
 .../beam/runners/direct/ParDoEvaluatorTest.java    |    6 +-
 .../runners/direct/SideInputContainerTest.java     |    6 +-
 .../direct/StatefulParDoEvaluatorFactoryTest.java  |    4 +-
 .../direct/TestStreamEvaluatorFactoryTest.java     |    2 +-
 .../direct/TransformExecutorServicesTest.java      |    2 +-
 .../direct/UnboundedReadDeduplicatorTest.java      |    8 +-
 .../direct/UnboundedReadEvaluatorFactoryTest.java  |   14 +-
 .../runners/direct/ViewEvaluatorFactoryTest.java   |    2 +-
 .../beam/runners/direct/WatermarkManagerTest.java  |    6 +-
 .../runners/direct/WindowEvaluatorFactoryTest.java |    6 +-
 .../direct/WriteWithShardingFactoryTest.java       |    2 +-
 .../BundleFactoryOutputReceiverFactoryTest.java    |  262 --
 .../direct/portable/CommittedResultTest.java       |  129 -
 .../CopyOnAccessInMemoryStateInternalsTest.java    |  591 ----
 .../runners/direct/portable/DirectMetricsTest.java |  232 --
 .../direct/portable/DirectTimerInternalsTest.java  |  132 -
 .../portable/DirectTransformExecutorTest.java      |  330 --
 .../direct/portable/EvaluationContextTest.java     |  315 --
 .../direct/portable/ExecutableGraphBuilder.java    |   81 -
 .../portable/FlattenEvaluatorFactoryTest.java      |  122 -
 .../GroupByKeyOnlyEvaluatorFactoryTest.java        |  247 --
 .../portable/ImmutableListBundleFactoryTest.java   |  234 --
 .../portable/ImpulseEvaluatorFactoryTest.java      |  130 -
 .../beam/runners/direct/portable/MockClock.java    |   65 -
 .../runners/direct/portable/PortableGraphTest.java |  176 --
 .../direct/portable/ReferenceRunnerTest.java       |  213 --
 .../portable/RemoteStageEvaluatorFactoryTest.java  |  238 --
 .../direct/portable/StepTransformResultTest.java   |   95 -
 .../portable/TransformExecutorServicesTest.java    |  143 -
 .../portable/WatermarkCallbackExecutorTest.java    |  136 -
 .../portable/WindowEvaluatorFactoryTest.java       |  314 --
 .../artifact/LocalArtifactStagingLocationTest.java |  122 -
 ...ocalFileSystemArtifactRetrievalServiceTest.java |  240 --
 .../LocalFileSystemArtifactStagerServiceTest.java  |  313 --
 .../UnsupportedArtifactRetrievalServiceTest.java   |  150 -
 .../job/ReferenceRunnerJobServiceTest.java         |  150 -
 runners/extensions-java/metrics/build.gradle       |   12 +-
 runners/flink/1.5/build.gradle                     |   34 +
 .../{1.7 => 1.5}/job-server-container/build.gradle |    0
 runners/flink/1.5/job-server/build.gradle          |   31 +
 .../translation/types/CoderTypeSerializer.java     |  208 ++
 .../translation/types/EncodedValueSerializer.java  |    0
 .../FlinkBroadcastStateInternalsTest.java          |    0
 .../flink/streaming/FlinkStateInternalsTest.java   |    0
 .../translation/types/CoderTypeSerializerTest.java |    0
 runners/flink/1.6/build.gradle                     |    9 +-
 runners/flink/1.6/job-server/build.gradle          |    1 +
 runners/flink/1.7/build.gradle                     |    9 +-
 runners/flink/1.7/job-server/build.gradle          |    1 +
 runners/flink/1.8/build.gradle                     |   34 +
 .../{1.7 => 1.8}/job-server-container/build.gradle |    0
 runners/flink/1.8/job-server/build.gradle          |   31 +
 .../translation/types/CoderTypeSerializer.java     |  165 +
 .../translation/types/EncodedValueSerializer.java  |  121 +
 .../FlinkBroadcastStateInternalsTest.java          |   96 +
 .../flink/streaming/FlinkStateInternalsTest.java   |  151 +
 .../translation/types/CoderTypeSerializerTest.java |   73 +
 runners/flink/build.gradle                         |   31 -
 runners/flink/flink_runner.gradle                  |  107 +-
 runners/flink/job-server-container/build.gradle    |   24 -
 .../flink_job_server_container.gradle              |    2 +-
 runners/flink/job-server/build.gradle              |   30 -
 runners/flink/job-server/flink_job_server.gradle   |   38 +-
 .../runners/flink/CreateStreamingFlinkView.java    |    2 +-
 .../FlinkBatchPortablePipelineTranslator.java      |   18 +-
 .../flink/FlinkBatchTransformTranslators.java      |    8 +-
 .../flink/FlinkBatchTranslationContext.java        |    2 +-
 .../runners/flink/FlinkExecutionEnvironments.java  |   14 +-
 .../apache/beam/runners/flink/FlinkJobInvoker.java |    4 +-
 .../flink/FlinkPipelineExecutionEnvironment.java   |    6 +-
 .../beam/runners/flink/FlinkPipelineOptions.java   |   12 +
 .../org/apache/beam/runners/flink/FlinkRunner.java |    4 +-
 .../beam/runners/flink/FlinkRunnerRegistrar.java   |    2 +-
 .../beam/runners/flink/FlinkRunnerResult.java      |    6 +-
 .../flink/FlinkStreamingPipelineTranslator.java    |  195 +-
 .../FlinkStreamingPortablePipelineTranslator.java  |   66 +-
 .../flink/FlinkStreamingTransformTranslators.java  |   19 +-
 .../flink/FlinkStreamingTranslationContext.java    |    9 +-
 .../runners/flink/FlinkTransformOverrides.java     |    7 +-
 .../flink/metrics/FlinkMetricContainer.java        |    2 +-
 .../functions/AbstractFlinkCombineRunner.java      |    2 +-
 .../translation/functions/FlinkAssignContext.java  |    2 +-
 .../FlinkDefaultExecutableStageContext.java        |    4 +-
 .../translation/functions/FlinkDoFnFunction.java   |    2 +-
 .../functions/FlinkExecutableStageFunction.java    |  215 +-
 .../functions/FlinkSideInputReader.java            |    4 +-
 .../functions/FlinkStatefulDoFnFunction.java       |    2 +-
 .../FlinkStreamingSideInputHandlerFactory.java     |    6 +-
 .../functions/HashingFlinkCombineRunner.java       |    2 +-
 .../functions/ImpulseSourceFunction.java           |   33 +-
 ...CountingFlinkExecutableStageContextFactory.java |    4 +-
 .../functions/SideInputInitializer.java            |    2 +-
 .../functions/SortingFlinkCombineRunner.java       |    4 +-
 .../translation/types/CoderTypeInformation.java    |    7 +-
 .../translation/types/CoderTypeSerializer.java     |  208 --
 .../types/InspectableByteArrayOutputStream.java    |   32 -
 .../translation/wrappers/DataInputViewWrapper.java |   10 +-
 .../wrappers/DataOutputViewWrapper.java            |    8 +-
 .../translation/wrappers/ImpulseInputFormat.java   |    2 +-
 .../wrappers/streaming/DoFnOperator.java           |   30 +-
 .../streaming/ExecutableStageDoFnOperator.java     |   69 +-
 .../wrappers/streaming/FlinkKeyUtils.java          |   55 +-
 .../streaming/KeyedPushedBackElementsHandler.java  |   38 +-
 .../streaming/KvToByteBufferKeySelector.java       |    6 +-
 .../NonKeyedPushedBackElementsHandler.java         |    2 +-
 .../streaming/SingletonKeyedWorkItemCoder.java     |    2 +-
 .../wrappers/streaming/SplittableDoFnOperator.java |    5 +-
 .../wrappers/streaming/WindowDoFnOperator.java     |    3 +-
 .../wrappers/streaming/io/TestStreamSource.java    |   11 +-
 .../streaming/io/UnboundedSocketSource.java        |  261 --
 .../streaming/io/UnboundedSourceWrapper.java       |   51 +-
 .../streaming/stableinput/BufferedElements.java    |    4 +-
 .../stableinput/KeyedBufferingElementsHandler.java |   45 +-
 .../NonKeyedBufferingElementsHandler.java          |    2 +-
 .../streaming/state/FlinkStateInternals.java       |  126 +-
 .../state/KeyGroupCheckpointedOperator.java        |   34 -
 .../streaming/state/KeyGroupRestoringOperator.java |   31 -
 .../runners/flink/FlinkJobServerDriverTest.java    |    2 +-
 .../FlinkPipelineExecutionEnvironmentTest.java     |   49 +-
 .../beam/runners/flink/FlinkRunnerResultTest.java  |   50 +
 .../beam/runners/flink/FlinkSavepointTest.java     |  111 +-
 .../FlinkStreamingPipelineTranslatorTest.java      |  123 +
 .../beam/runners/flink/FlinkTestPipeline.java      |    4 +-
 .../beam/runners/flink/PortableExecutionTest.java  |    6 +-
 .../runners/flink/PortableStateExecutionTest.java  |    4 +-
 .../runners/flink/PortableTimersExecutionTest.java |    4 +-
 .../beam/runners/flink/ReadSourcePortableTest.java |    6 +-
 .../runners/flink/ReadSourceStreamingTest.java     |    2 +-
 .../apache/beam/runners/flink/ReadSourceTest.java  |    2 +-
 .../flink/metrics/FlinkMetricContainerTest.java    |    2 +-
 .../flink/streaming/GroupByNullKeyTest.java        |    2 +-
 .../flink/streaming/TopWikipediaSessionsTest.java  |    2 +-
 .../FlinkDefaultExecutableStageContextTest.java    |    2 +-
 .../FlinkExecutableStageFunctionTest.java          |    4 +-
 .../functions/ImpulseSourceFunctionTest.java       |   83 +-
 ...tingFlinkExecutableStageContextFactoryTest.java |    2 +-
 .../wrappers/streaming/DoFnOperatorTest.java       |   10 +-
 .../streaming/ExecutableStageDoFnOperatorTest.java |   63 +-
 .../wrappers/streaming/StreamRecordStripper.java   |    4 +-
 .../wrappers/streaming/WindowDoFnOperatorTest.java |    2 +-
 .../streaming/io/UnboundedSourceWrapperTest.java   |   82 +-
 .../stableinput/BufferedElementsTest.java          |    2 +-
 runners/gearpump/build.gradle                      |   40 +-
 .../runners/gearpump/GearpumpPipelineOptions.java  |    2 +-
 .../runners/gearpump/GearpumpRunnerRegistrar.java  |    2 +-
 .../translators/CreateStreamingGearpumpView.java   |    2 +-
 .../translators/FlattenPCollectionsTranslator.java |    2 +-
 .../translators/GearpumpPipelineTranslator.java    |    2 +-
 .../gearpump/translators/GroupByKeyTranslator.java |    4 +-
 .../gearpump/translators/TranslationContext.java   |    4 +-
 .../translators/WindowAssignTranslator.java        |    2 +-
 .../translators/functions/DoFnFunction.java        |    2 +-
 .../translators/utils/TranslatorUtils.java         |    2 +-
 .../gearpump/GearpumpRunnerRegistrarTest.java      |    2 +-
 .../beam/runners/gearpump/PipelineOptionsTest.java |    2 +-
 .../translators/GroupByKeyTranslatorTest.java      |    4 +-
 .../translators/WindowAssignTranslatorTest.java    |    2 +-
 .../translators/io/GearpumpSourceTest.java         |    2 +-
 .../gearpump/translators/io/ValueSoureTest.java    |    4 +-
 .../translators/utils/TranslatorUtilsTest.java     |    2 +-
 runners/google-cloud-dataflow-java/build.gradle    |  166 +-
 .../examples-streaming/build.gradle                |   24 +-
 .../examples/build.gradle                          |   38 +-
 .../dataflow/BatchStatefulParDoOverrides.java      |    2 +-
 .../beam/runners/dataflow/BatchViewOverrides.java  |   32 +-
 .../beam/runners/dataflow/DataflowClient.java      |    2 +-
 .../beam/runners/dataflow/DataflowMetrics.java     |    6 +-
 .../dataflow/DataflowPTransformMatchers.java       |    2 +-
 .../beam/runners/dataflow/DataflowPipelineJob.java |  239 +-
 .../dataflow/DataflowPipelineRegistrar.java        |    2 +-
 .../dataflow/DataflowPipelineTranslator.java       |   16 +-
 .../beam/runners/dataflow/DataflowRunner.java      |   46 +-
 .../beam/runners/dataflow/DataflowRunnerInfo.java  |    4 +-
 .../dataflow/PrimitiveParDoSingleFactory.java      |    6 +-
 .../beam/runners/dataflow/TestDataflowRunner.java  |    8 +-
 .../runners/dataflow/internal/CustomSources.java   |    4 +-
 .../beam/runners/dataflow/internal/IsmFormat.java  |   12 +-
 .../options/DataflowWorkerLoggingOptions.java      |    2 +-
 .../beam/runners/dataflow/util/CloudKnownType.java |    2 +-
 .../beam/runners/dataflow/util/CloudObject.java    |    2 +-
 .../dataflow/util/CloudObjectTranslators.java      |    4 +-
 .../beam/runners/dataflow/util/CloudObjects.java   |    8 +-
 .../runners/dataflow/util/DataflowTemplateJob.java |    2 +-
 .../runners/dataflow/util/DataflowTransport.java   |    2 +-
 ...DefaultCoderCloudObjectTranslatorRegistrar.java |   12 +-
 .../beam/runners/dataflow/util/GcsStager.java      |    6 +-
 .../beam/runners/dataflow/util/MonitoringUtil.java |    6 +-
 .../runners/dataflow/util/OutputReference.java     |    2 +-
 .../beam/runners/dataflow/util/PackageUtil.java    |   18 +-
 .../runners/dataflow/util/RandomAccessData.java    |   10 +-
 .../util/SchemaCoderCloudObjectTranslator.java     |    6 +-
 .../SerializableCoderCloudObjectTranslator.java    |    2 +-
 .../beam/runners/dataflow/util/TimeUtil.java       |    2 +-
 .../dataflow/BatchStatefulParDoOverridesTest.java  |    2 +-
 .../runners/dataflow/BatchViewOverridesTest.java   |    4 +-
 .../beam/runners/dataflow/DataflowMetricsTest.java |    6 +-
 .../runners/dataflow/DataflowPipelineJobTest.java  |   46 +-
 .../dataflow/DataflowPipelineRegistrarTest.java    |    4 +-
 .../dataflow/DataflowPipelineTranslatorTest.java   |    8 +-
 .../beam/runners/dataflow/DataflowRunnerTest.java  |   36 +-
 .../dataflow/PrimitiveParDoSingleFactoryTest.java  |    2 +-
 .../runners/dataflow/TestDataflowRunnerTest.java   |    6 +-
 .../options/DataflowPipelineOptionsTest.java       |    2 +-
 .../options/DataflowWorkerLoggingOptionsTest.java  |    2 +-
 .../runners/dataflow/util/CloudObjectsTest.java    |    4 +-
 .../runners/dataflow/util/PackageUtilTest.java     |   12 +-
 .../dataflow/util/RandomAccessDataTest.java        |    2 +-
 .../beam/runners/dataflow/util/StructsTest.java    |    4 +-
 .../google-cloud-dataflow-java/worker/build.gradle |   38 +-
 .../worker/legacy-worker/build.gradle              |   92 +-
 .../worker/ApplianceShuffleEntryReader.java        |    2 +-
 .../worker/AssignWindowsParDoFnFactory.java        |    4 +-
 .../runners/dataflow/worker/AvroByteReader.java    |    2 +-
 .../dataflow/worker/AvroByteReaderFactory.java     |    2 +-
 .../beam/runners/dataflow/worker/AvroByteSink.java |    2 +-
 .../dataflow/worker/AvroByteSinkFactory.java       |    2 +-
 .../dataflow/worker/BatchDataflowWorker.java       |   21 +-
 .../dataflow/worker/BatchModeExecutionContext.java |   10 +-
 .../worker/BeamFnMapTaskExecutorFactory.java       |   30 +-
 .../runners/dataflow/worker/ByteStringCoder.java   |    4 +-
 .../worker/ChunkingShuffleBatchReader.java         |    2 +-
 .../dataflow/worker/CombineValuesFnFactory.java    |    2 +-
 .../beam/runners/dataflow/worker/ConcatReader.java |    6 +-
 .../worker/ContextActivationObserverRegistry.java  |    6 +-
 .../dataflow/worker/CounterShortIdCache.java       |    6 +-
 .../CreateIsmShardKeyAndSortKeyDoFnFactory.java    |    4 +-
 .../runners/dataflow/worker/DataflowApiUtils.java  |    4 +-
 .../worker/DataflowBatchWorkerHarness.java         |    9 +-
 .../worker/DataflowElementExecutionTracker.java    |   10 +-
 .../dataflow/worker/DataflowExecutionContext.java  |    6 +-
 .../dataflow/worker/DataflowExecutionStateKey.java |    4 +-
 .../worker/DataflowExecutionStateRegistry.java     |    4 +-
 .../worker/DataflowMapTaskExecutorFactory.java     |    2 +-
 .../dataflow/worker/DataflowOperationContext.java  |    6 +-
 .../dataflow/worker/DataflowOutputCounter.java     |    2 +-
 .../dataflow/worker/DataflowProcessFnRunner.java   |    4 +-
 .../dataflow/worker/DataflowRunnerHarness.java     |    4 +-
 .../worker/DataflowSideInputReadCounter.java       |    2 +-
 .../dataflow/worker/DataflowSystemMetrics.java     |    5 +-
 .../worker/DataflowWorkProgressUpdater.java        |   50 +-
 .../dataflow/worker/DataflowWorkUnitClient.java    |    8 +-
 .../worker/DataflowWorkerHarnessHelper.java        |    2 +-
 .../dataflow/worker/DefaultParDoFnFactory.java     |    2 +-
 .../runners/dataflow/worker/ExperimentContext.java |    4 +-
 ...FetchAndFilterStreamingSideInputsOperation.java |    2 +-
 .../beam/runners/dataflow/worker/Filepatterns.java |    2 +-
 .../dataflow/worker/FnApiWindowMappingFn.java      |   24 +-
 .../worker/GroupAlsoByWindowParDoFnFactory.java    |    6 +-
 .../dataflow/worker/GroupAlsoByWindowsParDoFn.java |    2 +-
 .../dataflow/worker/GroupingShuffleReader.java     |    4 +-
 .../worker/GroupingShuffleReaderFactory.java       |    4 +-
 ...ingShuffleReaderWithFaultyBytesReadCounter.java |    2 +-
 .../runners/dataflow/worker/InMemoryReader.java    |    8 +-
 .../dataflow/worker/InMemoryReaderFactory.java     |    2 +-
 .../worker/IntrinsicMapTaskExecutorFactory.java    |   10 +-
 .../beam/runners/dataflow/worker/IsmReader.java    |    2 +-
 .../runners/dataflow/worker/IsmReaderFactory.java  |    6 +-
 .../runners/dataflow/worker/IsmReaderImpl.java     |   30 +-
 .../dataflow/worker/IsmSideInputReader.java        |   40 +-
 .../beam/runners/dataflow/worker/IsmSink.java      |   12 +-
 .../runners/dataflow/worker/IsmSinkFactory.java    |    6 +-
 .../worker/LazilyInitializedSideInputReader.java   |    4 +-
 .../worker/MetricTrackingWindmillServerStub.java   |    4 +-
 .../dataflow/worker/MetricsContainerRegistry.java  |    2 +-
 ...nmentContextActivationObserverRegistration.java |    2 +-
 .../beam/runners/dataflow/worker/OrderedCode.java  |    4 +-
 .../worker/PairWithConstantKeyDoFnFactory.java     |    2 +-
 .../dataflow/worker/PartialGroupByKeyParDoFns.java |    6 +-
 .../dataflow/worker/PartitioningShuffleReader.java |    2 +-
 .../worker/PartitioningShuffleReaderFactory.java   |    4 +-
 .../beam/runners/dataflow/worker/PubsubReader.java |    4 +-
 .../beam/runners/dataflow/worker/PubsubSink.java   |    4 +-
 .../beam/runners/dataflow/worker/ReaderCache.java  |   12 +-
 .../runners/dataflow/worker/ReaderRegistry.java    |    8 +-
 .../ReifyTimestampAndWindowsParDoFnFactory.java    |    2 +-
 ...HarnessCoderCloudObjectTranslatorRegistrar.java |    2 +-
 .../dataflow/worker/SdkHarnessRegistries.java      |   19 +-
 .../dataflow/worker/SdkHarnessRegistry.java        |    3 +
 .../beam/runners/dataflow/worker/ShuffleSink.java  |    4 +-
 .../dataflow/worker/ShuffleSinkFactory.java        |    4 +-
 .../runners/dataflow/worker/SimpleParDoFn.java     |    8 +-
 .../beam/runners/dataflow/worker/SinkRegistry.java |    6 +-
 .../dataflow/worker/SizeReportingSinkWrapper.java  |    2 +-
 .../worker/SourceOperationExecutorFactory.java     |    2 +-
 .../beam/runners/dataflow/worker/StateFetcher.java |   14 +-
 .../dataflow/worker/StreamingDataflowWorker.java   |   94 +-
 .../worker/StreamingGroupAlsoByWindowsDoFns.java   |    2 +-
 .../StreamingKeyedWorkItemSideInputDoFnRunner.java |    6 +-
 .../worker/StreamingModeExecutionContext.java      |   21 +-
 .../StreamingPCollectionViewWriterDoFnFactory.java |    4 +-
 .../StreamingPCollectionViewWriterParDoFn.java     |    4 +-
 .../dataflow/worker/StreamingSideInputFetcher.java |   10 +-
 .../worker/StreamingStepMetricsContainer.java      |   20 +-
 .../worker/ToIsmRecordForMultimapDoFnFactory.java  |    4 +-
 .../dataflow/worker/UngroupedShuffleReader.java    |    2 +-
 .../worker/UngroupedShuffleReaderFactory.java      |    2 +-
 .../dataflow/worker/UngroupedWindmillReader.java   |    4 +-
 .../dataflow/worker/UserParDoFnFactory.java        |    6 +-
 .../runners/dataflow/worker/ValuesDoFnFactory.java |    2 +-
 .../beam/runners/dataflow/worker/Weighers.java     |    2 +-
 .../dataflow/worker/WindmillKeyedWorkItem.java     |    8 +-
 .../dataflow/worker/WindmillNamespacePrefix.java   |    2 +-
 .../worker/WindmillReaderIteratorBase.java         |    2 +-
 .../beam/runners/dataflow/worker/WindmillSink.java |    4 +-
 .../dataflow/worker/WindmillStateCache.java        |   85 +-
 .../dataflow/worker/WindmillStateInternals.java    |   14 +-
 .../dataflow/worker/WindmillStateReader.java       |   27 +-
 .../runners/dataflow/worker/WindmillTimeUtils.java |    2 +-
 .../dataflow/worker/WindmillTimerInternals.java    |   14 +-
 .../dataflow/worker/WindowingWindmillReader.java   |    4 +-
 .../dataflow/worker/WorkItemStatusClient.java      |   10 +-
 .../runners/dataflow/worker/WorkUnitClient.java    |    2 +-
 .../dataflow/worker/WorkerCustomSources.java       |   12 +-
 .../worker/WorkerUncaughtExceptionHandler.java     |    2 +-
 .../runners/dataflow/worker/apiary/Apiary.java     |    2 +-
 .../FixMultiOutputInfosOnParDoInstructions.java    |    2 +-
 .../runners/dataflow/worker/counters/Counter.java  |    2 +-
 .../dataflow/worker/counters/CounterFactory.java   |   10 +-
 .../dataflow/worker/counters/CounterName.java      |    6 +-
 .../dataflow/worker/counters/CounterSet.java       |    4 +-
 .../counters/DataflowCounterUpdateExtractor.java   |    2 +-
 .../dataflow/worker/fn/BeamFnControlService.java   |    2 +-
 .../worker/fn/control/BeamFnMapTaskExecutor.java   |   12 +-
 .../control/DataflowSideInputHandlerFactory.java   |    4 +-
 ...piMonitoringInfoToCounterUpdateTransformer.java |    2 +-
 ...ecMonitoringInfoToCounterUpdateTransformer.java |    2 +-
 .../control/RegisterAndProcessBundleOperation.java |   16 +-
 .../worker/fn/data/BeamFnDataGrpcService.java      |    6 +-
 .../fn/data/RemoteGrpcPortReadOperation.java       |   15 +-
 .../fn/data/RemoteGrpcPortWriteOperation.java      |   25 +-
 .../worker/fn/logging/BeamFnLoggingService.java    |    6 +-
 .../fn/stream/ServerStreamObserverFactory.java     |    6 +-
 .../graph/CloneAmbiguousFlattensFunction.java      |    6 +-
 .../graph/CreateExecutableStageNodeFunction.java   |   28 +-
 .../graph/CreateRegisterFnOperationFunction.java   |   27 +-
 .../graph/DeduceFlattenLocationsFunction.java      |   10 +-
 .../worker/graph/DeduceNodeLocationsFunction.java  |    4 +-
 .../beam/runners/dataflow/worker/graph/Edges.java  |    2 +-
 ...nsertFetchAndFilterStreamingSideInputNodes.java |   12 +-
 .../worker/graph/LengthPrefixUnknownCoders.java    |   11 +-
 .../worker/graph/MapTaskToNetworkFunction.java     |    6 +-
 .../runners/dataflow/worker/graph/Networks.java    |   16 +-
 .../beam/runners/dataflow/worker/graph/Nodes.java  |   19 +-
 .../worker/graph/RegisterNodeFunction.java         |   34 +-
 .../graph/RemoveFlattenInstructionsFunction.java   |    8 +-
 .../graph/ReplacePgbkWithPrecombineFunction.java   |    2 +-
 .../logging/DataflowWorkerLoggingHandler.java      |    6 +-
 .../logging/DataflowWorkerLoggingInitializer.java  |    6 +-
 .../JulHandlerPrintStreamAdapterFactory.java       |    2 +-
 .../dataflow/worker/profiler/ScopedProfiler.java   |    2 +-
 .../dataflow/worker/status/BaseStatusServlet.java  |    2 +-
 .../dataflow/worker/status/DebugCapture.java       |    2 +-
 .../dataflow/worker/status/HealthzServlet.java     |   17 +-
 .../dataflow/worker/status/HeapzServlet.java       |    2 +-
 .../dataflow/worker/status/ThreadzServlet.java     |    2 +-
 .../dataflow/worker/status/WorkerStatusPages.java  |   12 +-
 .../util/BatchGroupAlsoByWindowAndCombineFn.java   |    6 +-
 .../util/BatchGroupAlsoByWindowViaIteratorsFn.java |   10 +-
 .../worker/util/BatchGroupAlsoByWindowsDoFns.java  |    2 +-
 .../dataflow/worker/util/MemoryMonitor.java        |   14 +-
 .../dataflow/worker/util/ScalableBloomFilter.java  |   14 +-
 .../dataflow/worker/util/TimerOrElement.java       |    4 +-
 .../dataflow/worker/util/ValueInEmptyWindows.java  |    2 +-
 .../worker/util/common/ForwardingReiterator.java   |    2 +-
 .../common/worker/BatchingShuffleEntryReader.java  |    4 +-
 .../common/worker/ByteArrayShufflePosition.java    |    6 +-
 .../common/worker/CachingShuffleBatchReader.java   |   12 +-
 .../util/common/worker/FlattenOperation.java       |    2 +-
 .../worker/GroupingShuffleEntryIterator.java       |    4 +-
 .../common/worker/GroupingShuffleRangeTracker.java |    4 +-
 .../worker/util/common/worker/GroupingTables.java  |    2 +-
 .../worker/util/common/worker/MapTaskExecutor.java |    4 +-
 .../worker/util/common/worker/ParDoOperation.java  |    2 +-
 .../common/worker/ProgressTrackingReiterator.java  |    2 +-
 .../worker/util/common/worker/ReadOperation.java   |    6 +-
 .../util/common/worker/ShuffleReadCounter.java     |    2 +-
 .../worker/util/common/worker/WorkExecutor.java    |    2 +-
 .../util/common/worker/WorkProgressUpdater.java    |    6 +-
 .../worker/util/common/worker/WriteOperation.java  |    2 +-
 .../worker/windmill/DirectStreamObserver.java      |    4 +-
 .../windmill/ForwardingClientResponseObserver.java |    6 +-
 .../worker/windmill/GrpcWindmillServer.java        |   46 +-
 .../worker/windmill/StreamObserverFactory.java     |    4 +-
 .../worker/windmill/WindmillServerBase.java        |    2 +-
 .../worker/windmill/WindmillServerStub.java        |    2 +-
 .../dataflow/harness/test/TestExecutors.java       |    2 +-
 .../runners/dataflow/harness/test/TestStreams.java |    4 +-
 .../dataflow/worker/BatchDataflowWorkerTest.java   |    4 +-
 .../worker/CombineValuesFnFactoryTest.java         |    6 +-
 ...CreateIsmShardKeyAndSortKeyDoFnFactoryTest.java |    4 +-
 .../DataflowElementExecutionTrackerTest.java       |    2 +-
 .../worker/DataflowExecutionStateTrackerTest.java  |    2 +-
 .../runners/dataflow/worker/DataflowMatchers.java  |    2 +-
 .../worker/DataflowOperationContextTest.java       |    2 +-
 .../worker/DataflowWorkProgressUpdaterTest.java    |   61 +
 .../worker/DataflowWorkUnitClientTest.java         |    6 +-
 .../worker/DataflowWorkerHarnessHelperTest.java    |    2 +-
 .../dataflow/worker/DefaultParDoFnFactoryTest.java |    2 +-
 .../dataflow/worker/ExperimentContextTest.java     |    2 +-
 .../dataflow/worker/FakeWindmillServer.java        |    4 +-
 .../dataflow/worker/FnApiWindowMappingFnTest.java  |    2 +-
 .../GroupAlsoByWindowParDoFnFactoryTest.java       |    2 +-
 .../dataflow/worker/GroupingShuffleReaderTest.java |    2 +-
 .../IntrinsicMapTaskExecutorFactoryTest.java       |   10 +-
 .../worker/IntrinsicMapTaskExecutorTest.java       |    2 +-
 .../runners/dataflow/worker/IsmFormatTest.java     |    2 +-
 .../dataflow/worker/IsmReaderFactoryTest.java      |    8 +-
 .../runners/dataflow/worker/IsmReaderTest.java     |   20 +-
 .../dataflow/worker/IsmSideInputReaderTest.java    |   32 +-
 .../beam/runners/dataflow/worker/IsmSinkTest.java  |    2 +-
 .../LazilyInitializedSideInputReaderTest.java      |    2 +-
 .../dataflow/worker/LogRecordMatcherTest.java      |    2 +-
 .../beam/runners/dataflow/worker/LogSaverTest.java |    2 +-
 .../runners/dataflow/worker/OrderedCodeTest.java   |    4 +-
 .../worker/PairWithConstantKeyDoFnFactoryTest.java |    2 +-
 .../worker/PartialGroupByKeyParDoFnsTest.java      |    6 +-
 .../worker/PartitioningShuffleReaderTest.java      |    2 +-
 .../runners/dataflow/worker/PubsubReaderTest.java  |    2 +-
 .../runners/dataflow/worker/PubsubSinkTest.java    |    2 +-
 .../runners/dataflow/worker/ReaderCacheTest.java   |    4 +-
 ...ReifyTimestampAndWindowsParDoFnFactoryTest.java |    4 +-
 ...essCoderCloudObjectTranslatorRegistrarTest.java |    4 +-
 .../runners/dataflow/worker/ShuffleSinkTest.java   |    2 +-
 .../runners/dataflow/worker/SimpleParDoFnTest.java |    6 +-
 .../runners/dataflow/worker/StateFetcherTest.java  |    8 +-
 .../worker/StreamingDataflowWorkerTest.java        |   29 +-
 .../worker/StreamingGroupAlsoByWindowFnsTest.java  |    2 +-
 ...reamingGroupAlsoByWindowsReshuffleDoFnTest.java |    2 +-
 ...eamingKeyedWorkItemSideInputDoFnRunnerTest.java |    4 +-
 .../worker/StreamingModeExecutionContextTest.java  |    4 +-
 ...eamingPCollectionViewWriterDoFnFactoryTest.java |    2 +-
 .../worker/StreamingSideInputDoFnRunnerTest.java   |    4 +-
 .../worker/StreamingSideInputFetcherTest.java      |    6 +-
 .../runners/dataflow/worker/TestShuffleReader.java |    2 +-
 .../ToIsmRecordForMultimapDoFnFactoryTest.java     |    4 +-
 .../worker/UngroupedShuffleReaderTest.java         |    2 +-
 .../dataflow/worker/UserParDoFnFactoryTest.java    |    2 +-
 .../dataflow/worker/ValuesDoFnFactoryTest.java     |    2 +-
 .../dataflow/worker/WindmillKeyedWorkItemTest.java |    2 +-
 .../worker/WindmillReaderIteratorBaseTest.java     |    2 +-
 .../dataflow/worker/WindmillStateCacheTest.java    |  141 +-
 .../worker/WindmillStateInternalsTest.java         |   49 +-
 .../dataflow/worker/WindmillStateReaderTest.java   |    4 +-
 .../worker/WindmillTimerInternalsTest.java         |    2 +-
 .../dataflow/worker/WorkItemStatusClientTest.java  |    2 +-
 .../WorkerCustomSourcesSplitOnlySourceTest.java    |    2 +-
 .../dataflow/worker/WorkerCustomSourcesTest.java   |   12 +-
 .../worker/WorkerPipelineOptionsFactoryTest.java   |    2 +-
 ...FixMultiOutputInfosOnParDoInstructionsTest.java |    2 +-
 .../worker/counters/CounterFactoryTest.java        |    2 +-
 .../dataflow/worker/counters/CounterTest.java      |   34 +-
 .../worker/fn/BeamFnControlServiceTest.java        |   43 +-
 .../fn/control/BeamFnMapTaskExecutorTest.java      |    8 +-
 .../DataflowSideInputHandlerFactoryTest.java       |    4 +-
 .../RegisterAndProcessBundleOperationTest.java     |    8 +-
 .../worker/fn/control/TimerReceiverTest.java       |   34 +-
 .../worker/fn/data/BeamFnDataGrpcServiceTest.java  |   50 +-
 .../fn/data/RemoteGrpcPortReadOperationTest.java   |   15 +-
 .../fn/data/RemoteGrpcPortWriteOperationTest.java  |   14 +-
 .../fn/logging/BeamFnLoggingServiceTest.java       |   14 +-
 .../fn/stream/ServerStreamObserverFactoryTest.java |    4 +-
 .../graph/CloneAmbiguousFlattensFunctionTest.java  |   13 +-
 .../CreateRegisterFnOperationFunctionTest.java     |   44 +-
 .../graph/DeduceFlattenLocationsFunctionTest.java  |   16 +-
 .../graph/DeduceNodeLocationsFunctionTest.java     |   18 +-
 ...tFetchAndFilterStreamingSideInputNodesTest.java |   18 +-
 .../graph/LengthPrefixUnknownCodersTest.java       |   21 +-
 .../worker/graph/MapTaskToNetworkFunctionTest.java |    6 +-
 .../dataflow/worker/graph/NetworksTest.java        |   10 +-
 .../runners/dataflow/worker/graph/NodesTest.java   |    4 +-
 .../RemoveFlattenInstructionsFunctionTest.java     |   17 +-
 .../ReplacePgbkWithPrecombineFunctionTest.java     |    8 +-
 .../logging/DataflowWorkerLoggingHandlerTest.java  |    4 +-
 .../DataflowWorkerLoggingInitializerTest.java      |    2 +-
 .../JulHandlerPrintStreamAdapterFactoryTest.java   |    2 +-
 .../worker/profiler/ScopedProfilerTest.java        |    2 +-
 .../dataflow/worker/status/ThreadzServletTest.java |    2 +-
 .../worker/status/WorkerStatusPagesTest.java       |   20 +-
 .../worker/util/CounterHamcrestMatchers.java       |    4 +-
 .../util/DataflowCounterUpdateExtractorTest.java   |    2 +-
 .../worker/util/GroupAlsoByWindowProperties.java   |   10 +-
 .../dataflow/worker/util/ListOutputManager.java    |    4 +-
 .../util/common/worker/ExecutorTestUtils.java      |    2 +-
 .../worker/GroupingShuffleEntryIteratorTest.java   |    4 +-
 .../util/common/worker/MapTaskExecutorTest.java    |    2 +-
 .../util/common/worker/ReadOperationTest.java      |    2 +-
 .../util/common/worker/TestOutputReceiver.java     |    2 +-
 .../worker/windmill/GrpcWindmillServerTest.java    |   14 +-
 .../worker/windmill/build.gradle                   |    5 +-
 .../worker/windmill/src/main/proto/windmill.proto  |   11 +
 runners/java-fn-execution/build.gradle             |   32 +-
 .../apache/beam/runners/fnexecution/FnService.java |    6 +-
 .../GrpcContextHeaderAccessorProvider.java         |   16 +-
 .../beam/runners/fnexecution/GrpcFnServer.java     |    4 +-
 .../fnexecution/InProcessServerFactory.java        |    8 +-
 .../beam/runners/fnexecution/ServerFactory.java    |   24 +-
 .../BeamFileSystemArtifactRetrievalService.java    |   28 +-
 .../BeamFileSystemArtifactStagingService.java      |   18 +-
 .../control/DefaultJobBundleFactory.java           |  249 +-
 .../fnexecution/control/FnApiControlClient.java    |    6 +-
 .../control/FnApiControlClientPoolService.java     |    6 +-
 .../fnexecution/control/MapControlClientPool.java  |    2 +-
 .../control/ProcessBundleDescriptors.java          |   85 +-
 .../fnexecution/control/SdkHarnessClient.java      |   37 +-
 .../SingleEnvironmentInstanceJobBundleFactory.java |   15 +-
 .../fnexecution/control/TimerReceiverFactory.java  |   90 +
 .../runners/fnexecution/data/GrpcDataService.java  |   12 +-
 .../fnexecution/data/RemoteInputDestination.java   |    6 +-
 .../fnexecution/environment/DockerCommand.java     |    4 +-
 .../environment/DockerEnvironmentFactory.java      |    8 +-
 .../environment/EmbeddedEnvironmentFactory.java    |    2 +-
 .../environment/ExternalEnvironmentFactory.java    |    2 +-
 .../environment/ProcessEnvironmentFactory.java     |    4 +-
 .../fnexecution/environment/ProcessManager.java    |    6 +-
 .../jobsubmission/InMemoryJobService.java          |   33 +-
 .../fnexecution/jobsubmission/JobInvocation.java   |   56 +-
 .../fnexecution/jobsubmission/JobInvoker.java      |    8 +-
 .../fnexecution/jobsubmission/JobPreparation.java  |    2 +-
 .../fnexecution/jobsubmission/JobServerDriver.java |    2 +-
 .../fnexecution/logging/GrpcLoggingService.java    |    4 +-
 .../runners/fnexecution/provisioning/JobInfo.java  |    2 +-
 .../provisioning/StaticGrpcProvisionService.java   |    2 +-
 .../splittabledofn/SDFFeederViaStateAndTimers.java |   10 +-
 .../fnexecution/state/GrpcStateService.java        |    6 +-
 .../state/InMemoryBagUserStateFactory.java         |  118 +
 .../fnexecution/state/StateRequestHandlers.java    |   16 +-
 .../translation/BatchSideInputHandlerFactory.java  |    8 +-
 .../translation/PipelineTranslatorUtils.java       |   68 +-
 .../wire/LengthPrefixUnknownCoders.java            |   11 +-
 .../beam/runners/fnexecution/wire/WireCoders.java  |    2 +-
 .../runners/fnexecution/EmbeddedSdkHarness.java    |    2 +-
 .../GrpcContextHeaderAccessorProviderTest.java     |   22 +-
 .../runners/fnexecution/ServerFactoryTest.java     |   16 +-
 .../BeamFileSystemArtifactServicesTest.java        |   18 +-
 .../control/DefaultJobBundleFactoryTest.java       |  131 +-
 .../control/FnApiControlClientPoolServiceTest.java |    4 +-
 .../control/FnApiControlClientTest.java            |    2 +-
 .../fnexecution/control/RemoteExecutionTest.java   |  206 +-
 .../fnexecution/control/SdkHarnessClientTest.java  |   67 +-
 ...gleEnvironmentInstanceJobBundleFactoryTest.java |    2 +-
 .../fnexecution/data/GrpcDataServiceTest.java      |   24 +-
 .../data/RemoteInputDestinationTest.java           |   14 +-
 .../fnexecution/environment/DockerCommandTest.java |    4 +-
 .../jobsubmission/InMemoryJobServiceTest.java      |   74 +-
 .../jobsubmission/JobInvocationTest.java           |  192 ++
 .../logging/GrpcLoggingServiceTest.java            |    6 +-
 .../StaticGrpcProvisionServiceTest.java            |   10 +-
 .../fnexecution/state/GrpcStateServiceTest.java    |    4 +-
 .../translation/PipelineTranslatorUtilsTest.java   |    4 +-
 .../wire/LengthPrefixUnknownCodersTest.java        |    2 +-
 runners/jet-experimental/build.gradle              |  106 +
 .../org/apache/beam/runners/jet/DAGBuilder.java    |  226 ++
 .../runners/jet/FailedRunningPipelineResults.java  |   91 +
 .../apache/beam/runners/jet/JetGraphVisitor.java   |  100 +
 .../beam/runners/jet/JetPipelineOptions.java       |   68 +
 .../apache/beam/runners/jet/JetPipelineResult.java |  132 +
 .../org/apache/beam/runners/jet/JetRunner.java     |  207 ++
 .../beam/runners/jet/JetRunnerRegistrar.java       |   54 +
 .../beam/runners/jet/JetTransformTranslator.java   |   33 +
 .../beam/runners/jet/JetTransformTranslators.java  |  431 +++
 .../beam/runners/jet/JetTranslationContext.java    |   52 +
 .../java/org/apache/beam/runners/jet/Utils.java    |  292 ++
 .../beam/runners/jet/metrics/AbstractMetric.java   |   35 +
 .../beam/runners/jet/metrics/CounterImpl.java      |   56 +
 .../beam/runners/jet/metrics/DistributionImpl.java |   51 +
 .../apache/beam/runners/jet/metrics/GaugeImpl.java |   42 +
 .../beam/runners/jet/metrics/JetMetricResults.java |  212 ++
 .../runners/jet/metrics/JetMetricsContainer.java   |  138 +
 .../beam/runners/jet/metrics/package-info.java     |   20 +
 .../org/apache/beam/runners/jet/package-info.java  |   20 +
 .../runners/jet/processors/AbstractParDoP.java     |  521 ++++
 .../beam/runners/jet/processors/AssignWindowP.java |  130 +
 .../runners/jet/processors/BoundedSourceP.java     |  215 ++
 .../beam/runners/jet/processors/FlattenP.java      |   87 +
 .../beam/runners/jet/processors/ImpulseP.java      |  107 +
 .../apache/beam/runners/jet/processors/ParDoP.java |  191 ++
 .../runners/jet/processors/StatefulParDoP.java     |  302 ++
 .../runners/jet/processors/UnboundedSourceP.java   |  281 ++
 .../apache/beam/runners/jet/processors/ViewP.java  |  130 +
 .../beam/runners/jet/processors/WindowGroupP.java  |  339 +++
 .../beam/runners/jet/processors/package-info.java  |   20 +
 .../beam/runners/jet/JetTestRunnerRegistrar.java   |   54 +
 .../org/apache/beam/runners/jet/TestJetRunner.java |  147 +
 .../org/apache/beam/runners/jet/TestStreamP.java   |  101 +
 runners/local-java/build.gradle                    |   15 +-
 runners/reference/java/build.gradle                |   24 +-
 .../beam/runners/reference/CloseableResource.java  |    4 +-
 .../runners/reference/ExternalWorkerService.java   |    4 +-
 .../reference/JobServicePipelineResult.java        |    4 +-
 .../beam/runners/reference/PortableRunner.java     |   12 +-
 .../runners/reference/PortableRunnerRegistrar.java |    4 +-
 .../runners/reference/testing/TestJobService.java  |    2 +-
 .../testing/TestPortablePipelineOptions.java       |    2 +-
 .../runners/reference/CloseableResourceTest.java   |    6 +-
 .../beam/runners/reference/PortableRunnerTest.java |    6 +-
 runners/reference/job-server/build.gradle          |   54 -
 runners/samza/build.gradle                         |   64 +-
 runners/samza/job-server/build.gradle              |    8 +-
 .../beam/runners/samza/SamzaExecutionContext.java  |    2 +-
 .../beam/runners/samza/SamzaJobServerDriver.java   |   80 +-
 .../samza/SamzaPipelineLifeCycleListener.java      |    2 +-
 .../samza/SamzaPipelineOptionsValidator.java       |    2 +-
 .../beam/runners/samza/SamzaPipelineResult.java    |   16 +-
 .../beam/runners/samza/SamzaPipelineRunner.java    |    4 +-
 .../samza/SamzaPortablePipelineOptions.java        |   36 +
 .../org/apache/beam/runners/samza/SamzaRunner.java |   15 +-
 .../beam/runners/samza/SamzaRunnerRegistrar.java   |    2 +-
 .../runners/samza/adapter/BoundedSourceSystem.java |    2 +-
 .../samza/adapter/UnboundedSourceSystem.java       |    4 +-
 .../apache/beam/runners/samza/runtime/DoFnOp.java  |    2 +-
 .../runtime/DoFnRunnerWithKeyedInternals.java      |    2 +-
 .../beam/runners/samza/runtime/KeyedInternals.java |    2 +-
 .../runners/samza/runtime/SamzaAssignContext.java  |    2 +-
 .../runners/samza/runtime/SamzaDoFnRunners.java    |    2 +-
 .../samza/runtime/SamzaStoreStateInternals.java    |    6 +-
 .../runners/samza/translation/ConfigBuilder.java   |    6 +-
 .../runners/samza/translation/ConfigContext.java   |    2 +-
 .../translation/FlattenPCollectionsTranslator.java |    4 +-
 .../samza/translation/GroupByKeyTranslator.java    |    2 +-
 .../translation/ParDoBoundMultiTranslator.java     |    2 +-
 .../translation/PortableTranslationContext.java    |    2 +-
 .../runners/samza/translation/ReadTranslator.java  |    2 +-
 .../samza/translation/SamzaPipelineTranslator.java |    4 +-
 .../SamzaPortablePipelineTranslator.java           |    2 +-
 .../SamzaPublishViewTransformOverride.java         |    2 +-
 .../samza/translation/SamzaTransformOverrides.java |    2 +-
 .../samza/translation/TranslationContext.java      |    2 +-
 .../samza/translation/WindowAssignTranslator.java  |    2 +-
 .../runners/samza/util/PipelineDotRenderer.java    |  128 -
 .../samza/util/PortablePipelineDotRenderer.java    |  124 -
 .../samza/util/SamzaPipelineTranslatorUtils.java   |    4 +-
 .../samza/adapter/BoundedSourceSystemTest.java     |    2 +-
 .../runtime/SamzaStoreStateInternalsTest.java      |    6 +-
 .../samza/translation/ConfigGeneratorTest.java     |    4 +-
 runners/spark/build.gradle                         |   66 +-
 runners/spark/job-server/build.gradle              |   18 +-
 .../apache/beam/runners/spark/SparkJobInvoker.java |    4 +-
 .../runners/spark/SparkNativePipelineVisitor.java  |    4 +-
 .../beam/runners/spark/SparkPipelineOptions.java   |   34 +-
 .../beam/runners/spark/SparkPipelineResult.java    |    8 +-
 .../beam/runners/spark/SparkPipelineRunner.java    |   14 +-
 .../org/apache/beam/runners/spark/SparkRunner.java |    6 +-
 .../beam/runners/spark/SparkRunnerRegistrar.java   |    2 +-
 .../runners/spark/SparkTransformOverrides.java     |    4 +-
 .../apache/beam/runners/spark/TestSparkRunner.java |    4 +-
 .../spark/aggregators/AggregatorsAccumulator.java  |    4 +-
 .../spark/aggregators/NamedAggregators.java        |    4 +-
 .../beam/runners/spark/coders/CoderHelpers.java    |   43 +-
 .../spark/coders/SparkRunnerKryoRegistrator.java   |    6 +-
 .../apache/beam/runners/spark/io/CreateStream.java |    4 +-
 .../beam/runners/spark/io/MicrobatchSource.java    |   12 +-
 .../beam/runners/spark/io/SourceDStream.java       |    2 +-
 .../apache/beam/runners/spark/io/SourceRDD.java    |    6 +-
 .../runners/spark/io/SparkUnboundedSource.java     |    2 +-
 .../runners/spark/metrics/MetricsAccumulator.java  |    4 +-
 .../runners/spark/metrics/SparkBeamMetric.java     |    4 +-
 .../runners/spark/metrics/WithMetricsSupport.java  |   16 +-
 .../SparkGroupAlsoByWindowViaWindowSet.java        |   21 +-
 .../spark/stateful/SparkStateInternals.java        |    4 +-
 .../spark/stateful/SparkTimerInternals.java        |    6 +-
 .../runners/spark/stateful/StateSpecFunctions.java |    8 +-
 .../aggregators/AggregatorsAccumulator.java        |    2 +-
 .../aggregators/NamedAggregators.java              |    4 +-
 .../metrics/MetricsAccumulator.java                |    2 +-
 .../metrics/SparkBeamMetric.java                   |    4 +-
 .../metrics/WithMetricsSupport.java                |   16 +-
 .../translation/SparkTransformOverrides.java       |    2 +-
 .../translation/TranslationContext.java            |    2 +-
 .../translation/batch/AggregatorCombiner.java      |    4 +-
 .../translation/batch/DatasetSourceBatch.java      |    2 +-
 .../translation/batch/DoFnFunction.java            |    8 +-
 .../translation/batch/FlattenTranslatorBatch.java  |    2 +-
 .../batch/GroupByKeyTranslatorBatch.java           |    2 +-
 .../translation/batch/ParDoTranslatorBatch.java    |    2 +-
 .../translation/batch/ProcessContext.java          |    2 +-
 .../batch/functions/SparkSideInputReader.java      |    4 +-
 .../translation/helpers/WindowingHelpers.java      |    2 +-
 .../streaming/DatasetSourceStreaming.java          |    2 +-
 .../translation/utils/CachedSideInputReader.java   |    2 +-
 .../translation/utils/SideInputStorage.java        |    4 +-
 .../runners/spark/translation/BoundedDataset.java  |    2 +-
 .../spark/translation/EvaluationContext.java       |    4 +-
 .../spark/translation/GroupCombineFunctions.java   |  151 +-
 .../GroupNonMergingWindowsFunctions.java           |  226 +-
 .../spark/translation/MultiDoFnFunction.java       |    8 +-
 .../spark/translation/SparkAbstractCombineFn.java  |  124 -
 .../spark/translation/SparkAssignWindowFn.java     |    2 +-
 .../SparkBatchPortablePipelineTranslator.java      |  220 +-
 .../runners/spark/translation/SparkCombineFn.java  |  833 ++++++
 .../translation/SparkExecutableStageFunction.java  |  203 +-
 .../spark/translation/SparkGlobalCombineFn.java    |  261 --
 .../SparkGroupAlsoByWindowViaOutputBufferFn.java   |   13 +-
 .../spark/translation/SparkKeyedCombineFn.java     |  278 --
 .../spark/translation/SparkProcessContext.java     |    4 +-
 .../spark/translation/SparkTranslationContext.java |   24 +-
 .../spark/translation/TransformTranslator.java     |   84 +-
 .../spark/translation/TranslationUtils.java        |   57 +-
 .../translation/ValueAndCoderLazySerializable.java |    4 +-
 .../SparkRunnerStreamingContextFactory.java        |    2 +-
 .../streaming/StreamingTransformTranslator.java    |   17 +-
 .../streaming/WatermarkSyncedDStream.java          |    6 +-
 .../apache/beam/runners/spark/util/ByteArray.java  |    2 +-
 .../runners/spark/util/CachedSideInputReader.java  |    2 +-
 .../runners/spark/util/GlobalWatermarkHolder.java  |   12 +-
 .../beam/runners/spark/util/SideInputStorage.java  |    4 +-
 .../beam/runners/spark/util/SparkCompat.java       |   65 +
 .../runners/spark/util/SparkSideInputReader.java   |    4 +-
 .../runners/spark/ProvidedSparkContextTest.java    |    4 +-
 .../runners/spark/SparkPortableExecutionTest.java  |  132 +-
 .../runners/spark/SparkRunnerDebuggerTest.java     |    4 +-
 .../runners/spark/SparkRunnerRegistrarTest.java    |    4 +-
 .../spark/TestSparkPipelineOptionsRegistrar.java   |    2 +-
 .../aggregators/metrics/sink/InMemoryMetrics.java  |    2 +-
 .../metrics/sink/SparkMetricsSinkTest.java         |    4 +-
 .../beam/runners/spark/io/AvroPipelineTest.java    |    4 +-
 .../beam/runners/spark/io/NumShardsTest.java       |    6 +-
 .../spark/io/ReaderToIteratorAdapterTest.java      |    2 +-
 .../aggregators/metrics/sink/InMemoryMetrics.java  |    2 +-
 .../metrics/sink/SparkMetricsSinkTest.java         |    4 +-
 .../translation/batch/GroupByKeyTest.java          |    2 +-
 .../GroupNonMergingWindowsFunctionsTest.java       |  124 +-
 .../spark/translation/SparkCombineFnTest.java      |  298 ++
 .../SparkExecutableStageFunctionTest.java          |    5 +-
 .../ResumeFromCheckpointStreamingTest.java         |   10 +-
 .../streaming/SparkCoGroupByKeyStreamingTest.java  |    2 +-
 .../streaming/StreamingSourceMetricsTest.java      |   29 +-
 sdks/go/README.md                                  |    2 +-
 sdks/go/cmd/starcgen/starcgen.go                   |    6 +-
 sdks/go/container/build.gradle                     |    6 +-
 sdks/go/examples/build.gradle                      |    6 +-
 sdks/go/pkg/beam/combine.go                        |   28 +-
 sdks/go/pkg/beam/combine_test.go                   |   57 +
 sdks/go/pkg/beam/core/funcx/fn.go                  |   10 +-
 sdks/go/pkg/beam/core/funcx/signature.go           |   15 +-
 sdks/go/pkg/beam/core/graph/bind.go                |   64 +-
 sdks/go/pkg/beam/core/graph/coder/coder.go         |   13 +-
 sdks/go/pkg/beam/core/graph/coder/registry.go      |    7 +-
 sdks/go/pkg/beam/core/graph/coder/varint.go        |    2 +-
 sdks/go/pkg/beam/core/graph/edge.go                |   46 +-
 sdks/go/pkg/beam/core/graph/fn.go                  |   16 +-
 sdks/go/pkg/beam/core/graph/graph.go               |    7 +-
 sdks/go/pkg/beam/core/runtime/coderx/float.go      |    5 +-
 sdks/go/pkg/beam/core/runtime/coderx/varint.go     |    9 +-
 sdks/go/pkg/beam/core/runtime/exec/coder.go        |    3 +-
 sdks/go/pkg/beam/core/runtime/exec/cogbk.go        |    4 +-
 sdks/go/pkg/beam/core/runtime/exec/combine.go      |   23 +-
 sdks/go/pkg/beam/core/runtime/exec/combine_test.go |    5 +-
 sdks/go/pkg/beam/core/runtime/exec/data.go         |   28 +-
 sdks/go/pkg/beam/core/runtime/exec/datasink.go     |    3 +-
 sdks/go/pkg/beam/core/runtime/exec/datasource.go   |  326 +-
 .../pkg/beam/core/runtime/exec/datasource_test.go  |  248 ++
 sdks/go/pkg/beam/core/runtime/exec/fn.go           |   15 +-
 .../pkg/beam/core/runtime/exec/fullvalue_test.go   |   10 +
 sdks/go/pkg/beam/core/runtime/exec/input.go        |    3 +-
 sdks/go/pkg/beam/core/runtime/exec/pardo.go        |   13 +-
 sdks/go/pkg/beam/core/runtime/exec/plan.go         |   32 +-
 sdks/go/pkg/beam/core/runtime/exec/sideinput.go    |   21 +-
 sdks/go/pkg/beam/core/runtime/exec/translate.go    |   59 +-
 sdks/go/pkg/beam/core/runtime/exec/unit_test.go    |   56 +-
 sdks/go/pkg/beam/core/runtime/exec/util.go         |    5 +-
 sdks/go/pkg/beam/core/runtime/graphx/coder.go      |   83 +-
 sdks/go/pkg/beam/core/runtime/graphx/dataflow.go   |   27 +-
 sdks/go/pkg/beam/core/runtime/graphx/serialize.go  |  149 +-
 sdks/go/pkg/beam/core/runtime/graphx/translate.go  |   21 +-
 .../pkg/beam/core/runtime/graphx/translate_test.go |  130 +-
 sdks/go/pkg/beam/core/runtime/harness/datamgr.go   |   36 +-
 .../pkg/beam/core/runtime/harness/datamgr_test.go  |    8 +-
 sdks/go/pkg/beam/core/runtime/harness/harness.go   |   40 +-
 sdks/go/pkg/beam/core/runtime/harness/logging.go   |    3 +-
 sdks/go/pkg/beam/core/runtime/harness/session.go   |   13 +-
 sdks/go/pkg/beam/core/runtime/harness/statemgr.go  |   84 +-
 sdks/go/pkg/beam/core/runtime/pipelinex/replace.go |    3 +-
 sdks/go/pkg/beam/core/runtime/symbols.go           |    4 +-
 sdks/go/pkg/beam/core/typex/fulltype.go            |   12 +-
 sdks/go/pkg/beam/core/util/dot/dot.go              |   20 +-
 sdks/go/pkg/beam/core/util/hooks/hooks.go          |   12 +-
 sdks/go/pkg/beam/core/util/ioutilx/read.go         |    3 +-
 sdks/go/pkg/beam/core/util/protox/any.go           |    9 +-
 sdks/go/pkg/beam/core/util/protox/base64.go        |    4 +-
 sdks/go/pkg/beam/core/util/reflectx/call.go        |    4 +-
 sdks/go/pkg/beam/core/util/reflectx/json.go        |    5 +-
 sdks/go/pkg/beam/core/util/symtab/symtab.go        |   15 +-
 sdks/go/pkg/beam/internal/errors/errors.go         |   17 +-
 sdks/go/pkg/beam/internal/errors/errors_test.go    |    4 +-
 .../beam/model/fnexecution_v1/beam_fn_api.pb.go    |  801 ++---
 .../jobmanagement_v1/beam_expansion_api.pb.go      |   45 +-
 .../beam/model/pipeline_v1/beam_runner_api.pb.go   | 1295 ++++----
 sdks/go/pkg/beam/options/jobopts/options.go        |    4 +
 sdks/go/pkg/beam/runners/dataflow/dataflow.go      |   34 +-
 .../pkg/beam/runners/dataflow/dataflowlib/job.go   |   16 +-
 sdks/go/pkg/beam/runners/direct/buffer.go          |    2 +-
 sdks/go/pkg/beam/runners/direct/direct.go          |   10 +
 sdks/go/pkg/beam/runners/session/session.go        |    4 +-
 sdks/go/pkg/beam/runners/spark/spark.go            |   34 +
 sdks/go/pkg/beam/runners/universal/universal.go    |   13 +
 .../pkg/beam/runners/vet/testpipeline/functions.go |   52 +
 .../beam/runners/vet/testpipeline/testpipeline.go  |   84 +
 .../runners/vet/testpipeline/testpipeline.shims.go |  190 ++
 sdks/go/pkg/beam/runners/vet/vet.go                |  598 ++++
 sdks/go/pkg/beam/runners/vet/vet_test.go           |   65 +
 sdks/go/pkg/beam/transforms/stats/stats.shims.go   |  143 +-
 sdks/go/pkg/beam/x/beamx/run.go                    |    1 +
 sdks/go/test/build.gradle                          |   38 +-
 sdks/go/test/integration/primitives/pardo.go       |    3 +-
 sdks/go/test/run_integration_tests.sh              |  133 +-
 sdks/java/bom/build.gradle                         |   10 +-
 .../src/main/resources/beam/checkstyle.xml         |    8 +
 .../src/main/resources/beam/spotbugs-filter.xml    |   38 +-
 .../src/main/resources/beam/suppressions.xml       |    7 +-
 sdks/java/container/build.gradle                   |   12 +-
 sdks/java/core/build.gradle                        |    9 +-
 .../parser/generated/FieldSpecifierNotation.g4     |    2 +-
 .../main/java/org/apache/beam/sdk/Pipeline.java    |   43 +-
 .../java/org/apache/beam/sdk/PipelineRunner.java   |    2 +-
 .../java/org/apache/beam/sdk/coders/AvroCoder.java |    4 +-
 .../apache/beam/sdk/coders/BigDecimalCoder.java    |    2 +-
 .../apache/beam/sdk/coders/BigIntegerCoder.java    |    2 +-
 .../org/apache/beam/sdk/coders/ByteArrayCoder.java |    2 +-
 .../java/org/apache/beam/sdk/coders/Coder.java     |   12 +-
 .../org/apache/beam/sdk/coders/CoderProviders.java |    4 +-
 .../org/apache/beam/sdk/coders/CoderRegistry.java  |   23 +-
 .../org/apache/beam/sdk/coders/DefaultCoder.java   |    2 +-
 .../org/apache/beam/sdk/coders/DelegateCoder.java  |    4 +-
 .../apache/beam/sdk/coders/IterableLikeCoder.java  |    2 +-
 .../apache/beam/sdk/coders/LengthPrefixCoder.java  |    6 +-
 .../java/org/apache/beam/sdk/coders/MapCoder.java  |    2 +-
 .../org/apache/beam/sdk/coders/NullableCoder.java  |    4 +-
 .../java/org/apache/beam/sdk/coders/RowCoder.java  |    4 +-
 .../apache/beam/sdk/coders/RowCoderGenerator.java  |    6 +-
 .../apache/beam/sdk/coders/SerializableCoder.java  |    2 +-
 .../apache/beam/sdk/coders/ShardedKeyCoder.java    |    2 +-
 .../org/apache/beam/sdk/coders/SnappyCoder.java    |    2 +-
 .../apache/beam/sdk/coders/StringUtf8Coder.java    |    4 +-
 .../beam/sdk/coders/StructuralByteArray.java       |    2 +-
 .../sdk/expansion/ExternalTransformRegistrar.java  |    2 +
 .../apache/beam/sdk/harness/JvmInitializer.java    |   56 +
 .../org/apache/beam/sdk/harness/package-info.java  |   24 +
 .../main/java/org/apache/beam/sdk/io/AvroIO.java   |   37 +-
 .../java/org/apache/beam/sdk/io/AvroSource.java    |   10 +-
 .../java/org/apache/beam/sdk/io/AvroUtils.java     |    6 +-
 .../sdk/io/BoundedReadFromUnboundedSource.java     |    2 +-
 .../org/apache/beam/sdk/io/CompressedSource.java   |    4 +-
 .../java/org/apache/beam/sdk/io/Compression.java   |    6 +-
 .../beam/sdk/io/ConstantAvroDestination.java       |   11 +-
 .../org/apache/beam/sdk/io/CountingSource.java     |    6 +-
 .../apache/beam/sdk/io/DefaultFilenamePolicy.java  |    8 +-
 .../beam/sdk/io/DynamicAvroDestinations.java       |    2 +-
 .../beam/sdk/io/DynamicFileDestinations.java       |    2 +-
 .../java/org/apache/beam/sdk/io/FileBasedSink.java |   46 +-
 .../org/apache/beam/sdk/io/FileBasedSource.java    |   10 +-
 .../main/java/org/apache/beam/sdk/io/FileIO.java   |   94 +-
 .../java/org/apache/beam/sdk/io/FileSystems.java   |   35 +-
 .../org/apache/beam/sdk/io/GenerateSequence.java   |   10 +-
 .../org/apache/beam/sdk/io/LocalFileSystem.java    |   14 +-
 .../beam/sdk/io/LocalFileSystemRegistrar.java      |    2 +-
 .../org/apache/beam/sdk/io/LocalResourceId.java    |    6 +-
 .../org/apache/beam/sdk/io/OffsetBasedSource.java  |    2 +-
 .../beam/sdk/io/SerializableAvroCodecFactory.java  |    4 +-
 .../org/apache/beam/sdk/io/ShardingFunction.java   |   31 +
 .../main/java/org/apache/beam/sdk/io/Source.java   |    2 +-
 .../java/org/apache/beam/sdk/io/TFRecordIO.java    |   53 +-
 .../main/java/org/apache/beam/sdk/io/TextIO.java   |   18 +-
 .../apache/beam/sdk/io/TextRowCountEstimator.java  |  219 ++
 .../main/java/org/apache/beam/sdk/io/TextSink.java |    1 -
 .../java/org/apache/beam/sdk/io/TextSource.java    |    4 +-
 .../java/org/apache/beam/sdk/io/WriteFiles.java    |  104 +-
 .../org/apache/beam/sdk/io/WriteFilesResult.java   |    2 +-
 .../apache/beam/sdk/io/fs/ResourceIdTester.java    |   68 +-
 .../java/org/apache/beam/sdk/io/range/ByteKey.java |    2 +-
 .../org/apache/beam/sdk/io/range/ByteKeyRange.java |   12 +-
 .../beam/sdk/io/range/ByteKeyRangeTracker.java     |    4 +-
 .../org/apache/beam/sdk/io/range/OffsetRange.java  |    2 +-
 .../beam/sdk/io/range/OffsetRangeTracker.java      |    4 +-
 .../apache/beam/sdk/metrics/MetricFiltering.java   |    2 +-
 .../org/apache/beam/sdk/metrics/MetricName.java    |    4 +-
 .../apache/beam/sdk/metrics/MetricNameFilter.java  |    2 +-
 .../beam/sdk/metrics/MetricQueryResults.java       |    2 +-
 .../org/apache/beam/sdk/metrics/MetricsFilter.java |    2 +-
 .../org/apache/beam/sdk/metrics/SourceMetrics.java |    2 +-
 .../options/DefaultPipelineOptionsRegistrar.java   |    2 +-
 .../beam/sdk/options/ExperimentalOptions.java      |    2 +-
 .../options/ManualDockerEnvironmentOptions.java    |    2 +-
 .../apache/beam/sdk/options/PipelineOptions.java   |    2 +-
 .../beam/sdk/options/PipelineOptionsFactory.java   |   90 +-
 .../beam/sdk/options/PipelineOptionsReflector.java |    6 +-
 .../beam/sdk/options/PipelineOptionsValidator.java |   12 +-
 .../beam/sdk/options/PortablePipelineOptions.java  |    6 +
 .../beam/sdk/options/ProxyInvocationHandler.java   |   22 +-
 .../apache/beam/sdk/options/SdkHarnessOptions.java |    2 +-
 .../org/apache/beam/sdk/options/ValueProvider.java |    5 +-
 .../apache/beam/sdk/options/ValueProviders.java    |    2 +-
 .../apache/beam/sdk/runners/PTransformMatcher.java |    2 +-
 .../beam/sdk/runners/TransformHierarchy.java       |   14 +-
 .../apache/beam/sdk/schemas/AutoValueSchema.java   |    2 +-
 .../beam/sdk/schemas/FieldAccessDescriptor.java    |   74 +-
 .../beam/sdk/schemas/FieldTypeDescriptors.java     |    6 +-
 .../sdk/schemas/FieldValueTypeInformation.java     |    2 +-
 .../beam/sdk/schemas/FromRowUsingCreator.java      |    8 +-
 .../apache/beam/sdk/schemas/JavaBeanSchema.java    |    2 +-
 .../apache/beam/sdk/schemas/JavaFieldSchema.java   |    2 +-
 .../org/apache/beam/sdk/schemas/LogicalTypes.java  |    2 +-
 .../java/org/apache/beam/sdk/schemas/Schema.java   |   18 +-
 .../apache/beam/sdk/schemas/SchemaRegistry.java    |    8 +-
 .../schemas/SchemaUserTypeConstructorCreator.java  |    2 +-
 .../sdk/schemas/annotations/DefaultSchema.java     |    6 +-
 .../parser/FieldAccessDescriptorParser.java        |    4 +-
 .../beam/sdk/schemas/transforms/AddFields.java     |  434 +++
 .../apache/beam/sdk/schemas/transforms/Cast.java   |    6 +-
 .../beam/sdk/schemas/transforms/CoGroup.java       |    6 +-
 .../beam/sdk/schemas/transforms/Convert.java       |   43 +-
 .../beam/sdk/schemas/transforms/DropFields.java    |  149 +
 .../apache/beam/sdk/schemas/transforms/Filter.java |  162 +-
 .../beam/sdk/schemas/transforms/RenameFields.java  |  187 ++
 .../sdk/schemas/transforms/SchemaAggregateFn.java  |    2 +-
 .../apache/beam/sdk/schemas/transforms/Unnest.java |    2 +-
 .../beam/sdk/schemas/utils/AutoValueUtils.java     |    4 +-
 .../beam/sdk/schemas/utils/AvroByteBuddyUtils.java |    2 +-
 .../apache/beam/sdk/schemas/utils/AvroUtils.java   |   40 +-
 .../beam/sdk/schemas/utils/ByteBuddyUtils.java     |    6 +-
 .../beam/sdk/schemas/utils/ConvertHelpers.java     |    6 +-
 .../beam/sdk/schemas/utils/JavaBeanUtils.java      |    2 +-
 .../apache/beam/sdk/schemas/utils/POJOUtils.java   |    2 +-
 .../beam/sdk/schemas/utils/ReflectUtils.java       |    6 +-
 .../beam/sdk/schemas/utils/SchemaZipFold.java      |    2 +-
 .../beam/sdk/schemas/utils/SelectHelpers.java      |    8 +-
 .../sdk/schemas/utils/StaticSchemaInference.java   |    4 +-
 .../java/org/apache/beam/sdk/state/StateSpecs.java |    2 +-
 .../org/apache/beam/sdk/testing/Annotations.java   |    4 +-
 .../apache/beam/sdk/testing/CoderProperties.java   |   12 +-
 .../beam/sdk/testing/FileChecksumMatcher.java      |   10 +-
 .../beam/sdk/testing/MatcherDeserializer.java      |    2 +-
 .../apache/beam/sdk/testing/MatcherSerializer.java |    2 +-
 .../java/org/apache/beam/sdk/testing/PAssert.java  |   46 +-
 .../apache/beam/sdk/testing/PaneExtractors.java    |   18 +
 .../beam/sdk/testing/SerializableMatchers.java     |    2 +-
 .../apache/beam/sdk/testing/SourceTestUtils.java   |    6 +-
 .../org/apache/beam/sdk/testing/StaticWindows.java |    6 +-
 .../apache/beam/sdk/testing/SuccessOrFailure.java  |    4 +-
 .../org/apache/beam/sdk/testing/TestPipeline.java  |   14 +-
 .../org/apache/beam/sdk/testing/TestStream.java    |    8 +-
 .../java/org/apache/beam/sdk/testing/UsesKms.java  |   26 +
 .../apache/beam/sdk/testing/WindowFnTestUtils.java |    6 +-
 .../apache/beam/sdk/testing/WindowSupplier.java    |    6 +-
 .../beam/sdk/transforms/ApproximateQuantiles.java  |    8 +-
 .../beam/sdk/transforms/ApproximateUnique.java     |   17 +-
 .../org/apache/beam/sdk/transforms/Combine.java    |   19 +-
 .../apache/beam/sdk/transforms/CombineFnBase.java  |    2 +-
 .../org/apache/beam/sdk/transforms/CombineFns.java |   14 +-
 .../org/apache/beam/sdk/transforms/Contextful.java |    2 +-
 .../org/apache/beam/sdk/transforms/Create.java     |   12 +-
 .../java/org/apache/beam/sdk/transforms/DoFn.java  |   71 +
 .../beam/sdk/transforms/DoFnOutputReceivers.java   |    4 +-
 .../beam/sdk/transforms/DoFnSchemaInformation.java |    2 +-
 .../org/apache/beam/sdk/transforms/DoFnTester.java |    8 +-
 .../sdk/transforms/ExternalTransformBuilder.java   |    2 +
 .../beam/sdk/transforms/FlatMapElements.java       |    4 +-
 .../beam/sdk/transforms/GroupIntoBatches.java      |    6 +-
 .../org/apache/beam/sdk/transforms/Latest.java     |   43 +-
 .../apache/beam/sdk/transforms/MapElements.java    |    6 +-
 .../java/org/apache/beam/sdk/transforms/Mean.java  |    2 +-
 .../java/org/apache/beam/sdk/transforms/ParDo.java |    4 +-
 .../apache/beam/sdk/transforms/Requirements.java   |    2 +-
 .../org/apache/beam/sdk/transforms/Reshuffle.java  |    2 +-
 .../org/apache/beam/sdk/transforms/Sample.java     |    2 +-
 .../org/apache/beam/sdk/transforms/ToString.java   |    2 +-
 .../java/org/apache/beam/sdk/transforms/Top.java   |    4 +-
 .../java/org/apache/beam/sdk/transforms/Wait.java  |    4 +-
 .../java/org/apache/beam/sdk/transforms/Watch.java |   26 +-
 .../apache/beam/sdk/transforms/WithFailures.java   |    2 +-
 .../org/apache/beam/sdk/transforms/WithKeys.java   |    2 +-
 .../apache/beam/sdk/transforms/WithTimestamps.java |    2 +-
 .../beam/sdk/transforms/display/DisplayData.java   |   14 +-
 .../beam/sdk/transforms/join/CoGbkResult.java      |    8 +-
 .../sdk/transforms/join/KeyedPCollectionTuple.java |    2 +-
 .../reflect/ByteBuddyDoFnInvokerFactory.java       |    4 +-
 .../reflect/ByteBuddyOnTimerInvokerFactory.java    |   12 +-
 .../beam/sdk/transforms/reflect/DoFnSignature.java |    6 +-
 .../sdk/transforms/reflect/DoFnSignatures.java     |   12 +-
 .../reflect/StableInvokerNamingStrategy.java       |    2 +-
 .../sdk/transforms/splittabledofn/Backlog.java     |   90 -
 .../sdk/transforms/splittabledofn/Backlogs.java    |   58 -
 .../splittabledofn/ByteKeyRangeTracker.java        |   27 +-
 .../splittabledofn/OffsetRangeTracker.java         |   17 +-
 .../beam/sdk/transforms/splittabledofn/Sizes.java  |   57 +
 .../beam/sdk/transforms/windowing/AfterAll.java    |    4 +-
 .../beam/sdk/transforms/windowing/AfterEach.java   |    4 +-
 .../beam/sdk/transforms/windowing/AfterFirst.java  |    4 +-
 .../transforms/windowing/AfterProcessingTime.java  |    2 +-
 .../windowing/AfterSynchronizedProcessingTime.java |    2 +-
 .../sdk/transforms/windowing/AfterWatermark.java   |    4 +-
 .../sdk/transforms/windowing/OrFinallyTrigger.java |    2 +-
 .../beam/sdk/transforms/windowing/PaneInfo.java    |    8 +-
 .../beam/sdk/transforms/windowing/Sessions.java    |    6 +
 .../transforms/windowing/TimestampCombiner.java    |    6 +-
 .../beam/sdk/transforms/windowing/Trigger.java     |    2 +-
 .../beam/sdk/transforms/windowing/Window.java      |    4 +-
 .../java/org/apache/beam/sdk/util/ApiSurface.java  |   32 +-
 .../org/apache/beam/sdk/util/AppliedCombineFn.java |    2 +-
 .../apache/beam/sdk/util/BucketingFunction.java    |    2 +-
 .../util/BufferedElementCountingOutputStream.java  |    2 +-
 .../java/org/apache/beam/sdk/util/ClassPath.java   |   36 +-
 .../java/org/apache/beam/sdk/util/CoderUtils.java  |    4 +-
 .../apache/beam/sdk/util/ExplicitShardedFile.java  |    8 +-
 .../sdk/util/FilePatternMatchingShardedFile.java   |   10 +-
 .../org/apache/beam/sdk/util/FluentBackoff.java    |    4 +-
 .../org/apache/beam/sdk/util/InstanceBuilder.java  |    6 +-
 .../org/apache/beam/sdk/util/MovingFunction.java   |    2 +-
 .../java/org/apache/beam/sdk/util/NameUtils.java   |    6 +-
 .../apache/beam/sdk/util/NumberedShardedFile.java  |   14 +-
 .../java/org/apache/beam/sdk/util/ReleaseInfo.java |    2 +-
 .../apache/beam/sdk/util/RowJsonDeserializer.java  |    2 +-
 .../apache/beam/sdk/util/RowJsonValidation.java    |    2 +-
 .../apache/beam/sdk/util/SerializableUtils.java    |    2 +-
 .../java/org/apache/beam/sdk/util/StringUtils.java |    2 +-
 .../apache/beam/sdk/util/UnownedInputStream.java   |    2 +-
 .../apache/beam/sdk/util/UnownedOutputStream.java  |    2 +-
 .../org/apache/beam/sdk/util/WindowedValue.java    |   63 +-
 .../java/org/apache/beam/sdk/util/ZipFiles.java    |   16 +-
 .../beam/sdk/util/common/ReflectHelpers.java       |   55 +-
 .../main/java/org/apache/beam/sdk/values/KV.java   |    2 +-
 .../org/apache/beam/sdk/values/PCollection.java    |    4 +-
 .../apache/beam/sdk/values/PCollectionList.java    |    4 +-
 .../apache/beam/sdk/values/PCollectionTuple.java   |    2 +-
 .../apache/beam/sdk/values/PCollectionViews.java   |   10 +-
 .../org/apache/beam/sdk/values/PValueBase.java     |    2 +-
 .../main/java/org/apache/beam/sdk/values/Row.java  |   10 +-
 .../org/apache/beam/sdk/values/RowWithGetters.java |    4 +-
 .../org/apache/beam/sdk/values/TaggedPValue.java   |    2 +-
 .../apache/beam/sdk/values/TimestampedValue.java   |    2 +-
 .../java/org/apache/beam/sdk/values/TupleTag.java  |    4 +-
 .../org/apache/beam/sdk/values/TupleTagList.java   |    4 +-
 .../org/apache/beam/sdk/values/TypeDescriptor.java |   10 +-
 .../org/apache/beam/sdk/values/TypeParameter.java  |    2 +-
 .../beam/sdk/values/ValueInSingleWindow.java       |    2 +-
 .../apache/beam/sdk/values/ValueWithRecordId.java  |    2 +-
 .../apache/beam/sdk/values/WindowingStrategy.java  |    4 +-
 .../java/org/apache/beam/sdk/PipelineTest.java     |    4 +-
 .../apache/beam/sdk/coders/AvroCoderTestPojo.java  |    2 +-
 .../beam/sdk/coders/BigDecimalCoderTest.java       |    2 +-
 .../beam/sdk/coders/BigIntegerCoderTest.java       |    2 +-
 .../apache/beam/sdk/coders/CoderRegistryTest.java  |    6 +-
 .../apache/beam/sdk/coders/DefaultCoderTest.java   |    2 +-
 .../apache/beam/sdk/coders/DelegateCoderTest.java  |    4 +-
 .../apache/beam/sdk/coders/DurationCoderTest.java  |    2 +-
 .../org/apache/beam/sdk/coders/FloatCoderTest.java |   92 +
 .../apache/beam/sdk/coders/InstantCoderTest.java   |    4 +-
 .../beam/sdk/coders/LengthPrefixCoderTest.java     |    2 +-
 .../org/apache/beam/sdk/coders/MapCoderTest.java   |    2 +-
 .../apache/beam/sdk/coders/NullableCoderTest.java  |    4 +-
 .../beam/sdk/coders/PrintBase64Encodings.java      |    4 +-
 .../org/apache/beam/sdk/coders/RowCoderTest.java   |    2 +-
 .../beam/sdk/coders/StructuralByteArrayTest.java   |    2 +-
 .../beam/sdk/coders/StructuredCoderTest.java       |    2 +-
 .../java/org/apache/beam/sdk/io/AvroIOTest.java    |  181 +-
 .../org/apache/beam/sdk/io/AvroSourceTest.java     |    2 +-
 .../apache/beam/sdk/io/CompressedSourceTest.java   |   10 +-
 .../org/apache/beam/sdk/io/FileBasedSinkTest.java  |    4 +-
 .../java/org/apache/beam/sdk/io/FileIOTest.java    |    2 +-
 .../org/apache/beam/sdk/io/FileSystemsTest.java    |    8 +-
 .../beam/sdk/io/LocalFileSystemRegistrarTest.java  |    2 +-
 .../apache/beam/sdk/io/LocalFileSystemTest.java    |   31 +-
 .../org/apache/beam/sdk/io/TFRecordIOTest.java     |   80 +-
 .../org/apache/beam/sdk/io/TextIOReadTest.java     |   10 +-
 .../org/apache/beam/sdk/io/TextIOWriteTest.java    |   20 +-
 .../beam/sdk/io/TextRowCountEstimatorTest.java     |  116 +
 .../org/apache/beam/sdk/io/WriteFilesTest.java     |   90 +-
 .../io/range/ByteKeyRangeEstimateFractionTest.java |    2 +-
 .../io/range/ByteKeyRangeInterpolateKeyTest.java   |    2 +-
 .../apache/beam/sdk/io/range/ByteKeyRangeTest.java |    2 +-
 .../beam/sdk/metrics/MetricResultsMatchers.java    |  158 +-
 .../sdk/options/PipelineOptionsFactoryTest.java    |   16 +-
 .../sdk/options/PipelineOptionsReflectorTest.java  |    2 +-
 .../beam/sdk/options/PipelineOptionsTest.java      |    2 +-
 .../sdk/options/ProxyInvocationHandlerTest.java    |    8 +-
 .../beam/sdk/options/SdkHarnessOptionsTest.java    |    2 +-
 .../apache/beam/sdk/options/ValueProviderTest.java |    2 +-
 .../beam/sdk/options/ValueProvidersTest.java       |    2 +-
 .../beam/sdk/runners/TransformHierarchyTest.java   |    2 +-
 .../beam/sdk/schemas/AutoValueSchemaTest.java      |    5 +-
 .../apache/beam/sdk/schemas/AvroSchemaTest.java    |    4 +-
 .../sdk/schemas/FieldAccessDescriptorTest.java     |    4 +-
 .../beam/sdk/schemas/JavaBeanSchemaTest.java       |   19 +-
 .../beam/sdk/schemas/JavaFieldSchemaTest.java      |   17 +-
 .../beam/sdk/schemas/SchemaRegistryTest.java       |    2 +-
 .../beam/sdk/schemas/transforms/AddFieldsTest.java |  356 +++
 .../beam/sdk/schemas/transforms/CastTest.java      |    2 +-
 .../sdk/schemas/transforms/CastValidatorTest.java  |    4 +-
 .../beam/sdk/schemas/transforms/CoGroupTest.java   |    4 +-
 .../beam/sdk/schemas/transforms/ConvertTest.java   |   30 +-
 .../sdk/schemas/transforms/DropFieldsTest.java     |  161 +
 .../beam/sdk/schemas/transforms/FilterTest.java    |  106 +-
 .../beam/sdk/schemas/transforms/GroupTest.java     |    4 +-
 .../beam/sdk/schemas/transforms/JoinTest.java      |    2 +-
 .../beam/sdk/schemas/transforms/JoinTestUtils.java |    2 +-
 .../sdk/schemas/transforms/RenameFieldsTest.java   |  261 ++
 .../beam/sdk/schemas/transforms/SelectTest.java    |    4 +-
 .../beam/sdk/schemas/utils/AvroGenerators.java     |    6 +-
 .../beam/sdk/schemas/utils/AvroUtilsTest.java      |   40 +-
 .../beam/sdk/schemas/utils/JavaBeanUtilsTest.java  |    8 +-
 .../beam/sdk/schemas/utils/POJOUtilsTest.java      |    8 +-
 .../beam/sdk/schemas/utils/SchemaZipFoldTest.java  |    2 +-
 .../beam/sdk/schemas/utils/SelectHelpersTest.java  |   25 +-
 .../apache/beam/sdk/state/StateContextsTest.java   |    0
 .../beam/sdk/testing/CoderPropertiesTest.java      |    2 +-
 .../beam/sdk/testing/CombineFnTesterTest.java      |    2 +-
 .../beam/sdk/testing/FileChecksumMatcherTest.java  |    2 +-
 .../beam/sdk/testing/GatherAllPanesTest.java       |    2 +-
 .../sdk/testing/InterceptingUrlClassLoader.java    |    6 +-
 .../org/apache/beam/sdk/testing/PAssertTest.java   |    7 +-
 .../beam/sdk/testing/PCollectionViewTesting.java   |    2 +-
 .../beam/sdk/testing/PaneExtractorsTest.java       |   35 +-
 .../beam/sdk/testing/SerializableMatchersTest.java |    2 +-
 .../beam/sdk/testing/SourceTestUtilsTest.java      |    2 +-
 .../apache/beam/sdk/testing/StaticWindowsTest.java |    2 +-
 .../beam/sdk/testing/WindowSupplierTest.java       |    2 +-
 .../sdk/transforms/ApproximateQuantilesTest.java   |    2 +-
 .../beam/sdk/transforms/ApproximateUniqueTest.java |    4 +-
 .../apache/beam/sdk/transforms/CombineFnsTest.java |    4 +-
 .../apache/beam/sdk/transforms/CombineTest.java    |   12 +-
 .../org/apache/beam/sdk/transforms/CreateTest.java |    4 +-
 .../apache/beam/sdk/transforms/DistinctTest.java   |   11 +-
 .../apache/beam/sdk/transforms/DoFnTesterTest.java |    2 +-
 .../beam/sdk/transforms/FlatMapElementsTest.java   |    4 +-
 .../apache/beam/sdk/transforms/FlattenTest.java    |    2 +-
 .../apache/beam/sdk/transforms/GroupByKeyTest.java |    2 +-
 .../beam/sdk/transforms/GroupIntoBatchesTest.java  |    2 +-
 .../apache/beam/sdk/transforms/LatestFnTest.java   |    2 +-
 .../beam/sdk/transforms/MapElementsTest.java       |    2 +-
 .../org/apache/beam/sdk/transforms/MaxTest.java    |    2 +-
 .../org/apache/beam/sdk/transforms/MeanTest.java   |    2 +-
 .../org/apache/beam/sdk/transforms/MinTest.java    |    2 +-
 .../beam/sdk/transforms/ParDoLifecycleTest.java    |  177 +-
 .../beam/sdk/transforms/ParDoSchemaTest.java       |    2 +-
 .../org/apache/beam/sdk/transforms/ParDoTest.java  |   16 +-
 .../apache/beam/sdk/transforms/ReshuffleTest.java  |    2 +-
 .../org/apache/beam/sdk/transforms/SampleTest.java |    6 +-
 .../beam/sdk/transforms/SplittableDoFnTest.java    |    4 +-
 .../org/apache/beam/sdk/transforms/SumTest.java    |    2 +-
 .../org/apache/beam/sdk/transforms/ViewTest.java   |    4 +-
 .../org/apache/beam/sdk/transforms/WaitTest.java   |   18 +-
 .../org/apache/beam/sdk/transforms/WatchTest.java  |   18 +-
 .../beam/sdk/transforms/WithFailuresTest.java      |    2 +-
 .../transforms/display/DisplayDataEvaluator.java   |    2 +-
 .../transforms/display/DisplayDataMatchers.java    |    2 +-
 .../sdk/transforms/display/DisplayDataTest.java    |    8 +-
 .../sdk/transforms/join/CoGbkResultCoderTest.java  |    2 +-
 .../beam/sdk/transforms/join/CoGroupByKeyTest.java |    4 +-
 .../beam/sdk/transforms/join/UnionCoderTest.java   |    2 +-
 .../reflect/DoFnSignaturesSplittableDoFnTest.java  |    2 +-
 .../splittabledofn/ByteKeyRangeTrackerTest.java    |   17 +-
 .../splittabledofn/OffsetRangeTrackerTest.java     |   13 +-
 .../sdk/transforms/windowing/GlobalWindowTest.java |    4 +-
 .../transforms/windowing/IntervalWindowTest.java   |    2 +-
 .../sdk/transforms/windowing/SessionsTest.java     |    2 +-
 .../beam/sdk/transforms/windowing/StubTrigger.java |    2 +-
 .../beam/sdk/transforms/windowing/WindowTest.java  |    2 +-
 .../sdk/transforms/windowing/WindowingTest.java    |    2 +-
 .../org/apache/beam/sdk/util/ApiSurfaceTest.java   |    8 +-
 .../BufferedElementCountingOutputStreamTest.java   |    4 +-
 .../apache/beam/sdk/util/CombineFnUtilTest.java    |    2 +-
 .../sdk/util/ExposedByteArrayInputStreamTest.java  |    2 +-
 .../sdk/util/ExposedByteArrayOutputStreamTest.java |    2 +-
 .../util/FilePatternMatchingShardedFileTest.java   |    2 +-
 .../beam/sdk/util/MutationDetectorsTest.java       |    6 +-
 .../beam/sdk/util/NumberedShardedFileTest.java     |    2 +-
 .../beam/sdk/util/SerializableUtilsTest.java       |    4 +-
 .../apache/beam/sdk/util/WindowedValueTest.java    |   24 +-
 .../org/apache/beam/sdk/util/ZipFilesTest.java     |   10 +-
 .../beam/sdk/util/common/ReflectHelpersTest.java   |   41 +
 .../java/org/apache/beam/sdk/values/KVTest.java    |    2 +-
 .../beam/sdk/values/PCollectionListTest.java       |    2 +-
 .../beam/sdk/values/PCollectionTupleTest.java      |    2 +-
 .../java/org/apache/beam/sdk/values/RowTest.java   |   11 +-
 .../org/apache/beam/sdk/values/TupleTagTest.java   |    4 +-
 .../apache/beam/sdk/values/TypeDescriptorTest.java |    2 +-
 ....sdk.util.common.ReflectHelpersTest$FakeService |   19 +
 sdks/java/extensions/euphoria/README.md            |    2 +-
 sdks/java/extensions/euphoria/build.gradle         |   10 +-
 .../euphoria/core/client/operator/CountByKey.java  |    2 +-
 .../euphoria/core/client/operator/Distinct.java    |    2 +-
 .../euphoria/core/client/operator/FlatMap.java     |    2 +-
 .../euphoria/core/client/operator/ReduceByKey.java |  307 +-
 .../core/client/operator/ReduceWindow.java         |  294 +-
 .../euphoria/core/client/operator/Union.java       |    2 +-
 .../core/client/operator/WindowBuilder.java        |    2 +-
 .../core/client/util/PCollectionLists.java         |    2 +-
 .../extensions/euphoria/core/client/util/Sums.java |   58 +-
 .../core/translate/AbstractJoinTranslator.java     |    2 +-
 .../translate/BroadcastHashJoinTranslator.java     |    6 +-
 .../translate/CompositeOperatorTranslator.java     |    2 +-
 .../euphoria/core/translate/OperatorTransform.java |    2 +-
 .../core/translate/ReduceByKeyTranslator.java      |   85 +-
 .../provider/GenericTranslatorProvider.java        |    2 +-
 .../core/client/operator/ReduceByKeyTest.java      |   99 +-
 .../euphoria/core/client/util/SumsTest.java        |   52 +
 .../core/docs/DocumentationExamplesTest.java       |    2 +-
 .../euphoria/core/testkit/ReduceByKeyTest.java     |   49 +-
 .../core/testkit/accumulators/NanosecondTimer.java |    2 +-
 .../google-cloud-platform-core/build.gradle        |   43 +-
 .../sdk/extensions/gcp/options/GcpOptions.java     |   21 +-
 .../gcp/options/GcpPipelineOptionsRegistrar.java   |    2 +-
 .../sdk/extensions/gcp/options/GcsOptions.java     |    4 +-
 .../sdk/extensions/gcp/storage/GcsFileSystem.java  |   18 +-
 .../gcp/storage/GcsFileSystemRegistrar.java        |    4 +-
 .../extensions/gcp/storage/GcsPathValidator.java   |    2 +-
 .../sdk/extensions/gcp/storage/GcsResourceId.java  |    6 +-
 .../beam/sdk/extensions/gcp/util/GcsUtil.java      |   12 +-
 .../beam/sdk/extensions/gcp/util/Transport.java    |    2 +-
 .../sdk/extensions/gcp/util/gcsfs/GcsPath.java     |    4 +-
 .../sdk/extensions/gcp/GcpCoreApiSurfaceTest.java  |    2 +-
 .../sdk/extensions/gcp/options/GcpOptionsTest.java |   18 +-
 .../gcp/storage/GcsFileSystemRegistrarTest.java    |    2 +-
 .../extensions/gcp/storage/GcsFileSystemTest.java  |    4 +-
 .../extensions/gcp/util/CustomHttpErrorsTest.java  |    8 +-
 .../beam/sdk/extensions/gcp/util/GcsUtilIT.java    |   12 +-
 .../beam/sdk/extensions/gcp/util/GcsUtilTest.java  |    4 +-
 sdks/java/extensions/jackson/build.gradle          |   12 +-
 .../beam/sdk/extensions/jackson/AsJsons.java       |    2 +-
 .../beam/sdk/extensions/jackson/ParseJsons.java    |    2 +-
 .../extensions/jackson/JacksonTransformsTest.java  |    2 +-
 sdks/java/extensions/join-library/build.gradle     |    6 +-
 .../beam/sdk/extensions/joinlibrary/Join.java      |    2 +-
 sdks/java/extensions/kryo/build.gradle             |    8 +-
 .../sdk/extensions/kryo/KryoCoderProvider.java     |    2 +-
 sdks/java/extensions/protobuf/build.gradle         |    8 +-
 .../sdk/extensions/protobuf/ByteStringCoder.java   |    2 +-
 .../beam/sdk/extensions/protobuf/ProtoCoder.java   |    6 +-
 .../protobuf/ProtobufCoderProviderRegistrar.java   |    2 +-
 .../beam/sdk/extensions/protobuf/ProtobufUtil.java |    2 +-
 .../extensions/protobuf/ByteStringCoderTest.java   |    2 +-
 .../sdk/extensions/protobuf/ProtoCoderTest.java    |    2 +-
 .../sdk/extensions/protobuf/ProtobufUtilTest.java  |    4 +-
 sdks/java/extensions/sketching/build.gradle        |   24 +-
 .../extensions/sketching/ApproximateDistinct.java  |    2 +-
 .../extensions/sketching/SketchFrequencies.java    |    2 +-
 sdks/java/extensions/sorter/build.gradle           |    6 +-
 .../extensions/sorter/BufferedExternalSorter.java  |   24 +-
 .../beam/sdk/extensions/sorter/ExternalSorter.java |  198 +-
 .../extensions/sorter/HadoopExternalSorter.java    |  199 ++
 .../beam/sdk/extensions/sorter/InMemorySorter.java |    6 +-
 .../extensions/sorter/NativeExternalSorter.java    |   77 +
 .../sdk/extensions/sorter/NativeFileSorter.java    |  283 ++
 .../extensions/sorter/ExternalSorterBenchmark.java |   62 +
 .../sdk/extensions/sorter/ExternalSorterTest.java  |   56 +-
 .../sdk/extensions/sorter/SorterTestUtils.java     |    2 +-
 sdks/java/extensions/sql/build.gradle              |   36 +-
 sdks/java/extensions/sql/datacatalog/build.gradle  |   16 +-
 .../sql/example/BeamSqlDataCatalogExample.java     |    4 +-
 .../datacatalog/DataCatalogTableProvider.java      |    2 +-
 .../sql/meta/provider/datacatalog/SchemaUtils.java |    5 +-
 .../sql/meta/provider/datacatalog/TableUtils.java  |    2 +-
 sdks/java/extensions/sql/hcatalog/build.gradle     |    8 +-
 .../meta/provider/hcatalog/DatabaseProvider.java   |    2 +-
 .../provider/hcatalog/BeamSqlHiveSchemaTest.java   |    2 +-
 sdks/java/extensions/sql/jdbc/build.gradle         |   10 +-
 .../sdk/extensions/sql/jdbc/BeamSqlLineIT.java     |    7 +-
 sdks/java/extensions/sql/shell/build.gradle        |   10 +-
 .../apache/beam/sdk/extensions/sql/BeamSqlCli.java |    7 +-
 .../beam/sdk/extensions/sql/BeamSqlTable.java      |    7 +
 .../beam/sdk/extensions/sql/SqlTransform.java      |    6 +-
 .../sdk/extensions/sql/example/BeamSqlExample.java |    2 +-
 .../extensions/sql/example/BeamSqlPojoExample.java |    2 +-
 .../sdk/extensions/sql/impl/BeamCalciteSchema.java |    5 +-
 .../sql/impl/BeamCalciteSchemaFactory.java         |   14 +-
 .../sdk/extensions/sql/impl/BeamCalciteTable.java  |   46 +-
 .../beam/sdk/extensions/sql/impl/BeamSqlEnv.java   |   45 +-
 .../sql/impl/BeamSqlPipelineOptionsRegistrar.java  |    2 +-
 .../extensions/sql/impl/BeamTableStatistics.java   |   93 +
 .../extensions/sql/impl/CalciteQueryPlanner.java   |   74 +-
 .../sdk/extensions/sql/impl/JdbcConnection.java    |   11 +-
 .../beam/sdk/extensions/sql/impl/JdbcDriver.java   |    6 +-
 .../extensions/sql/impl/ScalarFunctionImpl.java    |   67 +-
 .../sql/impl/UdfImplReflectiveFunctionBase.java    |    2 +-
 .../sql/impl/parser/SqlColumnDeclaration.java      |    2 +-
 .../sql/impl/parser/SqlCreateExternalTable.java    |    2 +-
 .../extensions/sql/impl/parser/SqlDropObject.java  |    2 +-
 .../extensions/sql/impl/planner/BeamRuleSets.java  |    9 +-
 .../sql/impl/rel/BeamAggregationRel.java           |    4 +-
 .../sdk/extensions/sql/impl/rel/BeamCalcRel.java   |  235 +-
 .../sql/impl/rel/BeamEnumerableConverter.java      |    2 +-
 .../sdk/extensions/sql/impl/rel/BeamIOSinkRel.java |    2 +-
 .../extensions/sql/impl/rel/BeamIOSourceRel.java   |   42 +-
 .../sdk/extensions/sql/impl/rel/BeamJoinRel.java   |   37 +-
 .../sql/impl/rel/BeamSetOperatorRelBase.java       |    2 +-
 .../sdk/extensions/sql/impl/rel/BeamSortRel.java   |   23 +-
 .../extensions/sql/impl/rel/BeamUncollectRel.java  |    2 +-
 .../sdk/extensions/sql/impl/rel/BeamUnnestRel.java |    2 +-
 .../sdk/extensions/sql/impl/rel/BeamValuesRel.java |    4 +-
 .../sql/impl/rule/BeamJoinAssociateRule.java       |   52 +
 .../sql/impl/rule/BeamJoinPushThroughJoinRule.java |   70 +
 .../sql/impl/rule/JoinRelOptRuleCall.java          |  105 +
 .../impl/transform/BeamBuiltinAggregations.java    |    2 +-
 .../impl/transform/BeamSetOperatorsTransforms.java |    2 +-
 .../sql/impl/transform/agg/CovarianceFn.java       |    2 +-
 .../sql/impl/udf/BuiltinStringFunctions.java       |   57 +-
 .../sql/impl/utils/BigDecimalConverter.java        |    2 +-
 .../extensions/sql/impl/utils/CalciteUtils.java    |    6 +-
 .../sql/meta/provider/ReadOnlyTableProvider.java   |    2 +-
 .../sql/meta/provider/bigquery/BigQueryTable.java  |   41 +-
 .../provider/bigquery/BigQueryTableProvider.java   |    2 +-
 .../sql/meta/provider/kafka/BeamKafkaTable.java    |    8 +-
 .../parquet/GenericRecordReadConverter.java        |   64 +
 .../sql/meta/provider/parquet/ParquetTable.java    |   60 +
 .../provider/parquet/ParquetTableProvider.java     |   52 +
 .../sql/meta/provider/parquet/package-info.java    |   20 +
 .../provider/seqgen/GenerateSequenceTable.java     |   69 +
 .../seqgen/GenerateSequenceTableProvider.java      |   51 +
 .../sql/meta/provider/seqgen/package-info.java     |   20 +
 .../sql/meta/provider/test/TestBoundedTable.java   |    7 +
 .../sql/meta/provider/test/TestTableProvider.java  |   10 +-
 .../sql/meta/provider/test/TestTableUtils.java     |    2 +-
 .../sql/meta/provider/test/TestUnboundedTable.java |   14 +
 .../sql/meta/provider/text/TextTable.java          |   35 +
 .../sql/meta/provider/text/TextTableProvider.java  |    8 +-
 .../sql/meta/store/InMemoryMetaStore.java          |    2 +-
 .../sdk/extensions/sql/BeamComplexTypeTest.java    |  328 +-
 .../sdk/extensions/sql/BeamSqlDslFilterTest.java   |   22 +
 .../sdk/extensions/sql/BeamSqlDslProjectTest.java  |   14 +
 .../sql/BeamSqlDslSqlStdOperatorsTest.java         |    8 +-
 .../sdk/extensions/sql/BeamSqlDslUdfUdafTest.java  |    2 +-
 .../beam/sdk/extensions/sql/BeamSqlMapTest.java    |    2 +-
 .../extensions/sql/BeamSqlMultipleSchemasTest.java |    2 +-
 .../sdk/extensions/sql/PubsubToBigqueryIT.java     |    4 +-
 .../apache/beam/sdk/extensions/sql/TestUtils.java  |    2 +-
 .../sdk/extensions/sql/impl/BeamSqlEnvTest.java    |    7 +-
 .../sdk/extensions/sql/impl/JdbcDriverTest.java    |   33 +-
 .../extensions/sql/impl/parser/BeamDDLTest.java    |    7 +-
 .../sdk/extensions/sql/impl/rel/BaseRelTest.java   |    6 +-
 .../sql/impl/rel/BeamIOSourceRelTest.java          |  128 +
 .../rel/BeamJoinRelUnboundedVsBoundedTest.java     |    2 +-
 .../sql/impl/rule/JoinReorderingTest.java          |  463 +++
 ...BeamSqlBuiltinFunctionsIntegrationTestBase.java |   13 +-
 .../provider/bigquery/BigQueryReadWriteIT.java     |    2 +-
 .../meta/provider/bigquery/BigQueryRowCountIT.java |  160 +
 .../meta/provider/bigquery/BigQueryTestTable.java  |   45 +
 .../bigquery/BigQueryTestTableProvider.java        |   71 +
 .../provider/kafka/KafkaTableProviderTest.java     |    2 +-
 .../provider/parquet/GenericRecordToRowTest.java   |   79 +
 .../provider/parquet/ParquetTableReadTest.java     |   90 +
 .../sql/meta/provider/pubsub/PubsubJsonIT.java     |   11 +-
 .../provider/pubsub/PubsubMessageToRowTest.java    |    6 +-
 .../meta/provider/text/TextTableProviderTest.java  |    2 +-
 .../beam/sdk/extensions/sql/utils/RowAsserts.java  |    2 +-
 .../sql/src/test/resources/users.parquet           |  Bin 0 -> 615 bytes
 sdks/java/fn-execution/build.gradle                |   14 +-
 .../org/apache/beam/sdk/fn/JvmInitializers.java    |   49 +
 .../beam/sdk/fn/channel/ManagedChannelFactory.java |   18 +-
 .../beam/sdk/fn/channel/SocketAddressFactory.java  |    6 +-
 .../data/BeamFnDataBufferingOutboundObserver.java  |   14 +-
 .../sdk/fn/data/BeamFnDataGrpcMultiplexer.java     |   20 +-
 .../sdk/fn/data/BeamFnDataInboundObserver.java     |    4 +-
 .../apache/beam/sdk/fn/data/LogicalEndpoint.java   |   10 +-
 .../beam/sdk/fn/data/RemoteGrpcPortRead.java       |    8 +-
 .../beam/sdk/fn/data/RemoteGrpcPortWrite.java      |    8 +-
 .../sdk/fn/splittabledofn/RestrictionTrackers.java |   41 +-
 .../sdk/fn/stream/BufferingStreamObserver.java     |    8 +-
 .../org/apache/beam/sdk/fn/stream/DataStreams.java |    8 +-
 .../beam/sdk/fn/stream/DirectStreamObserver.java   |    4 +-
 .../stream/ForwardingClientResponseObserver.java   |    6 +-
 .../sdk/fn/stream/OutboundObserverFactory.java     |    4 +-
 .../sdk/fn/stream/SynchronizedStreamObserver.java  |    2 +-
 .../fn/test/InProcessManagedChannelFactory.java    |    4 +-
 .../org/apache/beam/sdk/fn/test/TestExecutors.java |    2 +-
 .../org/apache/beam/sdk/fn/test/TestStreams.java   |    4 +-
 .../sdk/fn/windowing/EncodedBoundedWindow.java     |    4 +-
 .../apache/beam/sdk/fn/JvmInitializersTest.java    |   78 +
 .../sdk/fn/channel/ManagedChannelFactoryTest.java  |    6 +-
 .../sdk/fn/channel/SocketAddressFactoryTest.java   |    2 +-
 .../BeamFnDataBufferingOutboundObserverTest.java   |   14 +-
 .../sdk/fn/data/BeamFnDataGrpcMultiplexerTest.java |   16 +-
 .../beam/sdk/fn/data/RemoteGrpcPortReadTest.java   |    2 +-
 .../beam/sdk/fn/data/RemoteGrpcPortWriteTest.java  |    2 +-
 .../fn/splittabledofn/RestrictionTrackersTest.java |   52 +-
 .../sdk/fn/stream/BufferingStreamObserverTest.java |    4 +-
 .../apache/beam/sdk/fn/stream/DataStreamsTest.java |   10 +-
 .../sdk/fn/stream/DirectStreamObserverTest.java    |    4 +-
 .../ForwardingClientResponseObserverTest.java      |    6 +-
 .../sdk/fn/stream/OutboundObserverFactoryTest.java |    4 +-
 .../sdk/fn/windowing/EncodedBoundedWindowTest.java |    2 +-
 sdks/java/harness/build.gradle                     |   21 +-
 .../beam/fn/harness/AssignWindowsRunner.java       |    8 +-
 .../beam/fn/harness/BeamFnDataReadRunner.java      |   23 +-
 .../beam/fn/harness/BeamFnDataWriteRunner.java     |   24 +-
 .../beam/fn/harness/BoundedSourceRunner.java       |    6 +-
 .../org/apache/beam/fn/harness/CombineRunners.java |    6 +-
 .../fn/harness/DoFnPTransformRunnerFactory.java    |   14 +-
 .../org/apache/beam/fn/harness/FlattenRunner.java  |    4 +-
 .../apache/beam/fn/harness/FnApiDoFnRunner.java    |   10 +-
 .../java/org/apache/beam/fn/harness/FnHarness.java |   65 +-
 .../org/apache/beam/fn/harness/MapFnRunners.java   |    4 +-
 .../beam/fn/harness/PrecombineGroupingTable.java   |    6 +-
 .../harness/SplittableProcessElementsRunner.java   |   12 +-
 .../beam/fn/harness/WindowMappingFnRunner.java     |    2 +-
 .../beam/fn/harness/WindowMergingFnRunner.java     |    4 +-
 .../harness/control/AddHarnessIdInterceptor.java   |   10 +-
 .../fn/harness/control/BeamFnControlClient.java    |    8 +-
 .../fn/harness/control/ProcessBundleHandler.java   |   24 +-
 .../beam/fn/harness/control/RegisterHandler.java   |    2 +-
 .../beam/fn/harness/data/BeamFnDataGrpcClient.java |   10 +-
 .../harness/data/MultiplexingFnDataReceiver.java   |    2 +-
 .../harness/data/PCollectionConsumerRegistry.java  |    4 +-
 .../fn/harness/data/QueueingBeamFnDataClient.java  |    8 +-
 .../fn/harness/logging/BeamFnLoggingClient.java    |   18 +-
 .../apache/beam/fn/harness/state/BagUserState.java |    6 +-
 .../harness/state/BeamFnStateGrpcClientCache.java  |    4 +-
 .../beam/fn/harness/state/FnApiStateAccessor.java  |   10 +-
 .../state/LazyCachingIteratorToIterable.java       |    2 +-
 .../beam/fn/harness/state/MultimapSideInput.java   |    2 +-
 .../fn/harness/state/StateFetchingIterators.java   |    4 +-
 .../stream/HarnessStreamObserverFactories.java     |    2 +-
 .../beam/fn/harness/AssignWindowsRunnerTest.java   |    4 +-
 .../beam/fn/harness/BeamFnDataReadRunnerTest.java  |   27 +-
 .../beam/fn/harness/BeamFnDataWriteRunnerTest.java |   27 +-
 .../beam/fn/harness/BoundedSourceRunnerTest.java   |   12 +-
 .../apache/beam/fn/harness/CombineRunnersTest.java |    2 +-
 .../apache/beam/fn/harness/FlattenRunnerTest.java  |    2 +-
 .../beam/fn/harness/FnApiDoFnRunnerTest.java       |    8 +-
 .../org/apache/beam/fn/harness/FnHarnessTest.java  |   69 +-
 .../apache/beam/fn/harness/MapFnRunnersTest.java   |    4 +-
 .../beam/fn/harness/WindowMergingFnRunnerTest.java |    6 +-
 .../harness/control/BeamFnControlClientTest.java   |   12 +-
 .../harness/control/ProcessBundleHandlerTest.java  |   10 +-
 .../fn/harness/control/RegisterHandlerTest.java    |   14 +-
 .../fn/harness/data/BeamFnDataGrpcClientTest.java  |   38 +-
 .../data/BeamFnDataInboundObserverTest.java        |    7 +-
 .../data/MultiplexingFnDataReceiverTest.java       |    2 +-
 .../harness/data/QueueingBeamFnDataClientTest.java |   40 +-
 .../harness/logging/BeamFnLoggingClientTest.java   |   18 +-
 .../beam/fn/harness/state/BagUserStateTest.java    |    6 +-
 .../state/BeamFnStateGrpcClientCacheTest.java      |   18 +-
 .../fn/harness/state/FakeBeamFnStateClient.java    |    2 +-
 .../state/LazyCachingIteratorToIterableTest.java   |    4 +-
 .../fn/harness/state/MultimapSideInputTest.java    |    6 +-
 .../harness/state/StateFetchingIteratorsTest.java  |    4 +-
 .../stream/HarnessStreamObserverFactoriesTest.java |    4 +-
 sdks/java/io/amazon-web-services/build.gradle      |   43 +-
 .../sdk/io/aws/dynamodb/AttributeValueCoder.java   |  166 ++
 .../AttributeValueCoderProviderRegistrar.java      |   37 +
 .../sdk/io/aws/dynamodb/AwsClientsProvider.java    |   34 +
 .../sdk/io/aws/dynamodb/BasicDynamoDBProvider.java |   75 +
 .../beam/sdk/io/aws/dynamodb/DynamoDBIO.java       |  536 ++++
 .../beam/sdk/io/aws/dynamodb/package-info.java     |   19 +
 .../apache/beam/sdk/io/aws/options/AwsModule.java  |    2 +-
 .../aws/options/AwsPipelineOptionsRegistrar.java   |    2 +-
 .../io/aws/s3/DefaultS3ClientBuilderFactory.java   |    2 +-
 .../apache/beam/sdk/io/aws/s3/S3FileSystem.java    |   30 +-
 .../beam/sdk/io/aws/s3/S3FileSystemRegistrar.java  |    4 +-
 .../io/aws/s3/S3ReadableSeekableByteChannel.java   |    4 +-
 .../apache/beam/sdk/io/aws/s3/S3ResourceId.java    |   10 +-
 .../beam/sdk/io/aws/s3/S3WritableByteChannel.java  |    6 +-
 .../beam/sdk/io/aws/sns/BasicSnsProvider.java      |    2 +-
 .../beam/sdk/io/aws/sns/PublishResultCoder.java    |    2 +-
 .../sdk/io/aws/sns/SnsCoderProviderRegistrar.java  |    2 +-
 .../java/org/apache/beam/sdk/io/aws/sns/SnsIO.java |    6 +-
 .../beam/sdk/io/aws/sqs/SqsCheckpointMark.java     |    4 +-
 .../java/org/apache/beam/sdk/io/aws/sqs/SqsIO.java |    2 +-
 .../beam/sdk/io/aws/sqs/SqsUnboundedSource.java    |    4 +-
 .../io/aws/dynamodb/AttributeValueCoderTest.java   |  211 ++
 .../io/aws/dynamodb/AwsClientsProviderMock.java    |   46 +
 .../beam/sdk/io/aws/dynamodb/DynamoDBIOTest.java   |  215 ++
 .../sdk/io/aws/dynamodb/DynamoDBIOTestHelper.java  |  168 ++
 .../beam/sdk/io/aws/s3/MatchResultMatcher.java     |    6 +-
 .../beam/sdk/io/aws/s3/S3FileSystemTest.java       |    4 +-
 .../org/apache/beam/sdk/io/aws/sns/SnsIOTest.java  |    2 +-
 sdks/java/io/amazon-web-services2/build.gradle     |   52 +
 .../sdk/io/aws2/dynamodb/AttributeValueCoder.java  |  158 +
 .../aws2/dynamodb/BasicDynamoDbClientProvider.java |   57 +
 .../beam/sdk/io/aws2/dynamodb/DynamoDBIO.java      |  532 ++++
 .../io/aws2/dynamodb/DynamoDbClientProvider.java   |   31 +
 .../beam/sdk/io/aws2/dynamodb/package-info.java    |   19 +
 .../apache/beam/sdk/io/aws2/options/AwsModule.java |  197 ++
 .../beam/sdk/io/aws2/options/AwsOptions.java       |  101 +
 .../aws2/options/AwsPipelineOptionsRegistrar.java  |   33 +
 .../beam/sdk/io/aws2/options/package-info.java     |   22 +
 .../io/aws2/dynamodb/AttributeValueCoderTest.java  |  208 ++
 .../beam/sdk/io/aws2/dynamodb/DynamoDBIOTest.java  |  313 ++
 .../sdk/io/aws2/dynamodb/DynamoDBIOTestHelper.java |  216 ++
 .../aws2/dynamodb/DynamoDbClientProviderMock.java  |   39 +
 .../beam/sdk/io/aws2/options/AwsModuleTest.java    |  119 +
 sdks/java/io/amqp/build.gradle                     |   10 +-
 .../java/org/apache/beam/sdk/io/amqp/AmqpIO.java   |    4 +-
 .../apache/beam/sdk/io/amqp/AmqpMessageCoder.java  |    2 +-
 .../io/amqp/AmqpMessageCoderProviderRegistrar.java |    2 +-
 .../beam/sdk/io/amqp/AmqpMessageCoderTest.java     |    2 +-
 sdks/java/io/cassandra/build.gradle                |   32 +-
 .../apache/beam/sdk/io/cassandra/CassandraIO.java  |   10 +-
 .../beam/sdk/io/cassandra/CassandraIOTest.java     |    6 +-
 sdks/java/io/clickhouse/build.gradle               |    8 +-
 .../beam/sdk/io/clickhouse/ClickHouseIO.java       |    6 +-
 .../beam/sdk/io/clickhouse/ClickHouseWriter.java   |    4 +-
 .../beam/sdk/io/clickhouse/BaseClickHouseTest.java |    2 +-
 .../beam/sdk/io/clickhouse/TableSchemaTest.java    |    2 +-
 sdks/java/io/common/build.gradle                   |    4 +-
 .../beam/sdk/io/common/DatabaseTestHelper.java     |   18 +
 .../org/apache/beam/sdk/io/common/HashingFn.java   |    6 +-
 .../org/apache/beam/sdk/io/common/IOITHelper.java  |    6 +-
 .../org/apache/beam/sdk/io/common/TestRow.java     |    2 +-
 .../elasticsearch-tests-2/build.gradle             |   16 +-
 .../sdk/io/elasticsearch/ElasticsearchIOIT.java    |    6 +-
 .../sdk/io/elasticsearch/ElasticsearchIOTest.java  |    2 +-
 .../elasticsearch-tests-5/build.gradle             |   14 +-
 .../sdk/io/elasticsearch/ElasticsearchIOIT.java    |    8 +-
 .../elasticsearch-tests-6/build.gradle             |   14 +-
 .../sdk/io/elasticsearch/ElasticsearchIOIT.java    |    8 +-
 .../elasticsearch-tests-common/build.gradle        |   12 +-
 .../io/elasticsearch/ElasticSearchIOTestUtils.java |  262 --
 .../io/elasticsearch/ElasticsearchIOITCommon.java  |    2 +-
 .../elasticsearch/ElasticsearchIOTestCommon.java   |   58 +-
 .../io/elasticsearch/ElasticsearchIOTestUtils.java |  262 ++
 sdks/java/io/elasticsearch/build.gradle            |   20 +-
 .../beam/sdk/io/elasticsearch/ElasticsearchIO.java |   10 +-
 sdks/java/io/file-based-io-tests/build.gradle      |   18 +-
 .../beam/sdk/io/common/FileBasedIOITHelper.java    |    4 +-
 .../java/org/apache/beam/sdk/io/xml/XmlIOIT.java   |    2 +-
 sdks/java/io/google-cloud-platform/build.gradle    |  113 +-
 .../beam/sdk/io/gcp/bigquery/BatchLoads.java       |   27 +-
 .../sdk/io/gcp/bigquery/BigQueryAvroUtils.java     |   16 +-
 .../bigquery/BigQueryCoderProviderRegistrar.java   |    2 +-
 .../beam/sdk/io/gcp/bigquery/BigQueryHelpers.java  |   27 +-
 .../beam/sdk/io/gcp/bigquery/BigQueryIO.java       |  197 +-
 .../sdk/io/gcp/bigquery/BigQueryQuerySource.java   |  105 +-
 .../io/gcp/bigquery/BigQueryQuerySourceDef.java    |  162 +
 .../bigquery/BigQuerySchemaRetrievalException.java |   25 +
 .../beam/sdk/io/gcp/bigquery/BigQueryServices.java |   19 +-
 .../sdk/io/gcp/bigquery/BigQueryServicesImpl.java  |   38 +-
 .../sdk/io/gcp/bigquery/BigQuerySourceBase.java    |   12 +-
 .../sdk/io/gcp/bigquery/BigQuerySourceDef.java     |   52 +
 .../gcp/bigquery/BigQueryStorageQuerySource.java   |    2 +-
 .../io/gcp/bigquery/BigQueryStorageSourceBase.java |   26 +-
 .../gcp/bigquery/BigQueryStorageStreamSource.java  |  195 +-
 .../gcp/bigquery/BigQueryStorageTableSource.java   |    6 +-
 .../sdk/io/gcp/bigquery/BigQueryTableSource.java   |   63 +-
 .../io/gcp/bigquery/BigQueryTableSourceDef.java    |  104 +
 .../beam/sdk/io/gcp/bigquery/BigQueryUtils.java    |  188 +-
 .../beam/sdk/io/gcp/bigquery/CreateTables.java     |   23 +-
 .../sdk/io/gcp/bigquery/DynamicDestinations.java   |   13 +-
 .../gcp/bigquery/DynamicDestinationsHelpers.java   |  142 +-
 .../sdk/io/gcp/bigquery/InsertRetryPolicy.java     |    2 +-
 .../io/gcp/bigquery/PassThroughThenCleanup.java    |    2 +-
 .../beam/sdk/io/gcp/bigquery/PrepareWrite.java     |    2 +-
 .../beam/sdk/io/gcp/bigquery/StreamingWriteFn.java |    4 +-
 .../beam/sdk/io/gcp/bigquery/TableDestination.java |   68 +-
 .../io/gcp/bigquery/TableDestinationCoderV2.java   |    7 +-
 .../io/gcp/bigquery/TableDestinationCoderV3.java   |   64 +
 .../sdk/io/gcp/bigquery/TableRowInfoCoder.java     |    2 +-
 .../beam/sdk/io/gcp/bigquery/TableRowWriter.java   |    4 +-
 .../beam/sdk/io/gcp/bigquery/TagWithUniqueIds.java |    2 +-
 .../beam/sdk/io/gcp/bigquery/TestBigQuery.java     |    2 +-
 .../sdk/io/gcp/bigquery/WriteBundlesToFiles.java   |    6 +-
 .../beam/sdk/io/gcp/bigquery/WritePartition.java   |    4 +-
 .../beam/sdk/io/gcp/bigquery/WriteRename.java      |    6 +-
 .../beam/sdk/io/gcp/bigquery/WriteResult.java      |    4 +-
 .../beam/sdk/io/gcp/bigquery/WriteTables.java      |   27 +-
 .../beam/sdk/io/gcp/bigtable/BigtableConfig.java   |    6 +-
 .../beam/sdk/io/gcp/bigtable/BigtableIO.java       |  272 +-
 .../sdk/io/gcp/bigtable/BigtableServiceImpl.java   |   15 +-
 .../sdk/io/gcp/bigtable/BigtableWriteResult.java   |   37 +
 .../io/gcp/bigtable/BigtableWriteResultCoder.java  |   55 +
 .../bigtable/VendoredListenableFutureAdapter.java  |    2 +-
 .../gcp/common/GcpIoPipelineOptionsRegistrar.java  |    2 +-
 .../sdk/io/gcp/datastore/AdaptiveThrottler.java    |    2 +-
 .../beam/sdk/io/gcp/datastore/DatastoreV1.java     |   16 +-
 .../beam/sdk/io/gcp/pubsub/PubsubClient.java       |   10 +-
 .../gcp/pubsub/PubsubCoderProviderRegistrar.java   |    2 +-
 .../beam/sdk/io/gcp/pubsub/PubsubGrpcClient.java   |    8 +-
 .../apache/beam/sdk/io/gcp/pubsub/PubsubIO.java    |   65 +-
 .../beam/sdk/io/gcp/pubsub/PubsubJsonClient.java   |    8 +-
 .../beam/sdk/io/gcp/pubsub/PubsubMessage.java      |    2 +-
 .../gcp/pubsub/PubsubMessagePayloadOnlyCoder.java  |    2 +-
 .../beam/sdk/io/gcp/pubsub/PubsubTestClient.java   |    6 +-
 .../sdk/io/gcp/pubsub/PubsubUnboundedSink.java     |    6 +-
 .../sdk/io/gcp/pubsub/PubsubUnboundedSource.java   |   12 +-
 .../beam/sdk/io/gcp/pubsub/TestPubsubSignal.java   |    8 +-
 .../beam/sdk/io/gcp/spanner/BatchSpannerRead.java  |    2 +-
 .../sdk/io/gcp/spanner/MutationCellCounter.java    |    2 +-
 .../beam/sdk/io/gcp/spanner/MutationGroup.java     |    4 +-
 .../beam/sdk/io/gcp/spanner/MutationUtils.java     |    2 +-
 .../beam/sdk/io/gcp/spanner/NaiveSpannerRead.java  |    2 +-
 .../beam/sdk/io/gcp/spanner/OrderedCode.java       |   10 +-
 .../beam/sdk/io/gcp/spanner/SpannerConfig.java     |    4 +-
 .../apache/beam/sdk/io/gcp/spanner/SpannerIO.java  |   12 +-
 .../beam/sdk/io/gcp/spanner/SpannerSchema.java     |   12 +-
 .../sdk/io/gcp/spanner/SpannerWriteResult.java     |    2 +-
 .../beam/sdk/io/gcp/testing/BigqueryClient.java    |   15 +-
 .../beam/sdk/io/gcp/testing/BigqueryMatcher.java   |   10 +-
 .../cloud/spanner/FakeBatchTransactionId.java      |    2 +-
 .../apache/beam/sdk/io/gcp/GcpApiSurfaceTest.java  |    2 +-
 .../sdk/io/gcp/bigquery/BigQueryAvroUtilsTest.java |    6 +-
 .../sdk/io/gcp/bigquery/BigQueryHelpersTest.java   |    4 +-
 .../beam/sdk/io/gcp/bigquery/BigQueryIOReadIT.java |    4 +-
 .../sdk/io/gcp/bigquery/BigQueryIOReadTest.java    |  154 +-
 .../io/gcp/bigquery/BigQueryIOStorageQueryIT.java  |    2 +-
 .../gcp/bigquery/BigQueryIOStorageQueryTest.java   |   29 +-
 .../io/gcp/bigquery/BigQueryIOStorageReadIT.java   |    2 +-
 .../io/gcp/bigquery/BigQueryIOStorageReadTest.java |  568 +++-
 .../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java   |  132 +-
 .../beam/sdk/io/gcp/bigquery/BigQueryKmsKeyIT.java |    5 +-
 .../io/gcp/bigquery/BigQueryServicesImplTest.java  |    6 +-
 .../BigQueryTimePartitioningClusteringIT.java      |  228 ++
 .../sdk/io/gcp/bigquery/BigQueryToTableIT.java     |    4 +-
 .../sdk/io/gcp/bigquery/BigQueryUtilsTest.java     |  229 +-
 .../sdk/io/gcp/bigquery/FakeBigQueryServices.java  |   24 +-
 .../sdk/io/gcp/bigquery/FakeDatasetService.java    |    8 +-
 .../beam/sdk/io/gcp/bigquery/FakeJobService.java   |   26 +-
 .../sdk/io/gcp/bigquery/InsertRetryPolicyTest.java |    2 +-
 .../sdk/io/gcp/bigtable/BigtableConfigTest.java    |    4 +-
 .../beam/sdk/io/gcp/bigtable/BigtableIOTest.java   |  166 +-
 .../io/gcp/bigtable/BigtableServiceImplTest.java   |    2 +-
 .../beam/sdk/io/gcp/bigtable/BigtableWriteIT.java  |    2 +-
 .../beam/sdk/io/gcp/pubsub/PubsubClientTest.java   |    2 +-
 .../sdk/io/gcp/pubsub/PubsubGrpcClientTest.java    |    6 +-
 .../beam/sdk/io/gcp/pubsub/PubsubIOTest.java       |   40 +-
 .../sdk/io/gcp/pubsub/PubsubJsonClientTest.java    |    4 +-
 .../beam/sdk/io/gcp/pubsub/PubsubReadIT.java       |    2 +-
 .../sdk/io/gcp/pubsub/PubsubTestClientTest.java    |    6 +-
 .../sdk/io/gcp/pubsub/PubsubUnboundedSinkTest.java |    6 +-
 .../io/gcp/pubsub/PubsubUnboundedSourceTest.java   |    2 +-
 .../sdk/io/gcp/spanner/MutationKeyEncoderTest.java |    2 +-
 .../beam/sdk/io/gcp/spanner/OrderedCodeTest.java   |   10 +-
 .../sdk/io/gcp/spanner/SpannerIOWriteTest.java     |    6 +-
 .../beam/sdk/io/gcp/spanner/SpannerWriteIT.java    |    6 +-
 .../beam/sdk/io/gcp/storage/GcsKmsKeyIT.java       |   31 +-
 .../sdk/io/gcp/testing/BigqueryMatcherTest.java    |    2 +-
 sdks/java/io/hadoop-common/build.gradle            |    2 +-
 sdks/java/io/hadoop-file-system/build.gradle       |   17 +-
 .../apache/beam/sdk/io/hdfs/HadoopFileSystem.java  |   26 +-
 .../beam/sdk/io/hdfs/HadoopFileSystemOptions.java  |    8 +-
 .../io/hdfs/HadoopFileSystemOptionsRegistrar.java  |    2 +-
 .../sdk/io/hdfs/HadoopFileSystemRegistrar.java     |    4 +-
 .../apache/beam/sdk/io/hdfs/HadoopResourceId.java  |    4 +-
 .../hdfs/HadoopFileSystemOptionsRegistrarTest.java |    2 +-
 .../sdk/io/hdfs/HadoopFileSystemOptionsTest.java   |    6 +-
 .../sdk/io/hdfs/HadoopFileSystemRegistrarTest.java |    6 +-
 .../beam/sdk/io/hdfs/HadoopFileSystemTest.java     |   27 +-
 sdks/java/io/hadoop-format/build.gradle            |   32 +-
 .../sdk/io/hadoop/format/HDFSSynchronization.java  |    4 +-
 .../beam/sdk/io/hadoop/format/HadoopFormatIO.java  |  132 +-
 .../beam/sdk/io/hadoop/format/HadoopFormats.java   |    6 +-
 .../format/ConfigurableEmployeeInputFormat.java    |    2 +-
 .../apache/beam/sdk/io/hadoop/format/Employee.java |   10 +-
 .../sdk/io/hadoop/format/EmployeeInputFormat.java  |    2 +-
 .../io/hadoop/format/HDFSSynchronizationTest.java  |    7 +-
 .../hadoop/format/HadoopFormatIOCassandraTest.java |    2 -
 .../hadoop/format/HadoopFormatIOElasticTest.java   |    8 +-
 .../sdk/io/hadoop/format/HadoopFormatIOIT.java     |    2 +-
 .../io/hadoop/format/HadoopFormatIOReadTest.java   |   18 +-
 .../format/HadoopFormatIOSequenceFileTest.java     |   18 +-
 .../format/ReuseObjectsEmployeeInputFormat.java    |    4 +-
 .../sdk/io/hadoop/format/TestEmployeeDataSet.java  |    4 +-
 .../sdk/io/hadoop/format/TestRowDBWritable.java    |    5 +-
 sdks/java/io/hbase/build.gradle                    |   16 +-
 .../sdk/io/hbase/HBaseCoderProviderRegistrar.java  |    2 +-
 .../java/org/apache/beam/sdk/io/hbase/HBaseIO.java |    4 +-
 sdks/java/io/hcatalog/build.gradle                 |   14 +-
 .../org/apache/beam/sdk/io/hcatalog/HCatToRow.java |    2 +-
 .../beam/sdk/io/hcatalog/HCatalogBeamSchema.java   |    6 +-
 .../apache/beam/sdk/io/hcatalog/HCatalogIO.java    |  101 +-
 .../apache/beam/sdk/io/hcatalog/HCatalogUtils.java |   87 +
 .../beam/sdk/io/hcatalog/PartitionPollerFn.java    |   56 +
 .../beam/sdk/io/hcatalog/PartitionReaderFn.java    |  111 +
 .../apache/beam/sdk/io/hcatalog/SchemaUtils.java   |    2 +-
 .../sdk/io/hcatalog/HCatalogBeamSchemaTest.java    |   13 +-
 .../apache/beam/sdk/io/hcatalog/HCatalogIOIT.java  |    2 +-
 .../beam/sdk/io/hcatalog/HCatalogIOTest.java       |   95 +-
 sdks/java/io/jdbc/build.gradle                     |   22 +-
 .../sdk/io/jdbc/BeamSchemaInferenceException.java  |   25 +
 .../java/org/apache/beam/sdk/io/jdbc/JdbcIO.java   |  442 ++-
 .../java/org/apache/beam/sdk/io/jdbc/JdbcUtil.java |  235 ++
 .../org/apache/beam/sdk/io/jdbc/LogicalTypes.java  |  241 ++
 .../org/apache/beam/sdk/io/jdbc/SchemaUtil.java    |  411 +++
 .../org/apache/beam/sdk/io/jdbc/JdbcIOTest.java    |  429 +++
 .../org/apache/beam/sdk/io/jdbc/JdbcUtilTest.java  |   44 +
 .../org/apache/beam/sdk/io/jdbc/RowWithSchema.java |   44 +
 .../apache/beam/sdk/io/jdbc/SchemaUtilTest.java    |  345 +++
 sdks/java/io/jms/build.gradle                      |   12 +-
 .../java/org/apache/beam/sdk/io/jms/JmsIO.java     |    4 +-
 .../java/org/apache/beam/sdk/io/jms/JmsIOTest.java |    2 +-
 sdks/java/io/kafka/build.gradle                    |   28 +-
 .../org/apache/beam/sdk/io/kafka/ConsumerSpEL.java |    6 +-
 .../CustomTimestampPolicyWithLimitedDelay.java     |    2 +-
 .../beam/sdk/io/kafka/KafkaCheckpointMark.java     |    2 +-
 .../beam/sdk/io/kafka/KafkaExactlyOnceSink.java    |   26 +-
 .../java/org/apache/beam/sdk/io/kafka/KafkaIO.java |   89 +-
 .../org/apache/beam/sdk/io/kafka/KafkaRecord.java  |    2 +-
 .../beam/sdk/io/kafka/KafkaUnboundedReader.java    |   10 +-
 .../beam/sdk/io/kafka/KafkaUnboundedSource.java    |    6 +-
 .../org/apache/beam/sdk/io/kafka/ProducerSpEL.java |    2 +-
 .../beam/sdk/io/kafka/TimestampPolicyFactory.java  |    2 +-
 .../CustomTimestampPolicyWithLimitedDelayTest.java |    2 +-
 .../beam/sdk/io/kafka/KafkaIOExternalTest.java     |   14 +-
 .../org/apache/beam/sdk/io/kafka/KafkaIOIT.java    |  209 ++
 .../org/apache/beam/sdk/io/kafka/KafkaIOTest.java  |   19 +-
 sdks/java/io/kinesis/build.gradle                  |   36 +-
 .../beam/sdk/io/kinesis/BasicKinesisProvider.java  |    2 +-
 .../sdk/io/kinesis/DynamicCheckpointGenerator.java |    2 +-
 .../org/apache/beam/sdk/io/kinesis/KinesisIO.java  |  253 +-
 .../apache/beam/sdk/io/kinesis/KinesisReader.java  |   23 +-
 .../sdk/io/kinesis/KinesisReaderCheckpoint.java    |    6 +-
 .../apache/beam/sdk/io/kinesis/KinesisSource.java  |   11 +-
 .../beam/sdk/io/kinesis/KinesisWatermark.java      |  116 -
 .../apache/beam/sdk/io/kinesis/RecordFilter.java   |    2 +-
 .../beam/sdk/io/kinesis/ShardCheckpoint.java       |    4 +-
 .../beam/sdk/io/kinesis/ShardReadersPool.java      |   33 +-
 .../beam/sdk/io/kinesis/ShardRecordsIterator.java  |   26 +-
 .../sdk/io/kinesis/SimplifiedKinesisClient.java    |    4 +-
 .../apache/beam/sdk/io/kinesis/StartingPoint.java  |    2 +-
 .../sdk/io/kinesis/StartingPointShardsFinder.java  |    2 +-
 .../sdk/io/kinesis/StaticCheckpointGenerator.java  |    2 +-
 .../beam/sdk/io/kinesis/WatermarkParameters.java   |   98 +
 .../beam/sdk/io/kinesis/WatermarkPolicy.java       |   29 +
 .../sdk/io/kinesis/WatermarkPolicyFactory.java     |  152 +
 .../beam/sdk/io/kinesis/AmazonKinesisMock.java     |    4 +-
 .../io/kinesis/DynamicCheckpointGeneratorTest.java |   30 +-
 .../apache/beam/sdk/io/kinesis/KinesisIOIT.java    |  124 +-
 .../beam/sdk/io/kinesis/KinesisMockReadTest.java   |    5 +-
 .../beam/sdk/io/kinesis/KinesisMockWriteTest.java  |   38 +-
 .../beam/sdk/io/kinesis/KinesisProducerMock.java   |   39 +-
 .../io/kinesis/KinesisReaderCheckpointTest.java    |    2 +-
 .../beam/sdk/io/kinesis/KinesisReaderTest.java     |   41 +-
 .../beam/sdk/io/kinesis/KinesisServiceMock.java    |    2 +-
 .../beam/sdk/io/kinesis/KinesisTestOptions.java    |   12 +
 .../beam/sdk/io/kinesis/KinesisWatermarkTest.java  |  138 -
 .../beam/sdk/io/kinesis/RecordFilterTest.java      |   14 +-
 .../beam/sdk/io/kinesis/ShardCheckpointTest.java   |    5 +-
 .../beam/sdk/io/kinesis/ShardReadersPoolTest.java  |   65 +-
 .../sdk/io/kinesis/ShardRecordsIteratorTest.java   |   16 +-
 .../io/kinesis/SimplifiedKinesisClientTest.java    |   50 +-
 .../io/kinesis/StartingPointShardsFinderTest.java  |   40 +-
 .../beam/sdk/io/kinesis/WatermarkPolicyTest.java   |  152 +
 sdks/java/io/kudu/build.gradle                     |   14 +-
 .../java/org/apache/beam/sdk/io/kudu/KuduIO.java   |    8 +-
 .../apache/beam/sdk/io/kudu/KuduServiceImpl.java   |    4 +-
 .../org/apache/beam/sdk/io/kudu/KuduTestUtils.java |    2 +-
 sdks/java/io/mongodb/build.gradle                  |   16 +-
 .../org/apache/beam/sdk/io/mongodb/FindQuery.java  |    2 +-
 .../beam/sdk/io/mongodb/MongoDbGridFSIO.java       |    4 +-
 .../org/apache/beam/sdk/io/mongodb/MongoDbIO.java  |   13 +-
 .../apache/beam/sdk/io/mongodb/MongoDBIOIT.java    |    2 +-
 .../apache/beam/sdk/io/mongodb/MongoDbIOTest.java  |    2 +-
 sdks/java/io/mqtt/build.gradle                     |   16 +-
 .../java/org/apache/beam/sdk/io/mqtt/MqttIO.java   |    4 +-
 sdks/java/io/parquet/build.gradle                  |   22 +-
 .../org/apache/beam/sdk/io/parquet/ParquetIO.java  |    2 +-
 sdks/java/io/rabbitmq/build.gradle                 |   13 +-
 .../apache/beam/sdk/io/rabbitmq/RabbitMqIO.java    |    2 +-
 sdks/java/io/redis/build.gradle                    |   10 +-
 .../sdk/io/redis/RedisConnectionConfiguration.java |    2 +-
 .../java/org/apache/beam/sdk/io/redis/RedisIO.java |   32 +-
 .../org/apache/beam/sdk/io/redis/RedisIOTest.java  |   31 +
 sdks/java/io/solr/build.gradle                     |   14 +-
 .../beam/sdk/io/solr/AuthorizedSolrClient.java     |    2 +-
 .../java/org/apache/beam/sdk/io/solr/SolrIO.java   |   10 +-
 .../org/apache/beam/sdk/io/solr/SolrIOTest.java    |    2 +-
 sdks/java/io/synthetic/build.gradle                |   16 +-
 .../sdk/io/synthetic/SyntheticBoundedSource.java   |    4 +-
 .../beam/sdk/io/synthetic/SyntheticOptions.java    |    6 +-
 .../sdk/io/synthetic/SyntheticSourceOptions.java   |    4 +-
 .../beam/sdk/io/synthetic/SyntheticStep.java       |   10 +-
 .../sdk/io/synthetic/delay/SyntheticDelay.java     |    6 +-
 .../beam/sdk/io/synthetic/SyntheticStepTest.java   |    2 +-
 sdks/java/io/tika/build.gradle                     |   10 +-
 .../org/apache/beam/sdk/io/tika/ParseResult.java   |    8 +-
 .../java/org/apache/beam/sdk/io/tika/TikaIO.java   |    4 +-
 sdks/java/io/xml/build.gradle                      |   14 +-
 .../java/org/apache/beam/sdk/io/xml/JAXBCoder.java |    2 +-
 .../java/org/apache/beam/sdk/io/xml/XmlIO.java     |    4 +-
 .../org/apache/beam/sdk/io/xml/JAXBCoderTest.java  |    2 +-
 .../java/org/apache/beam/sdk/io/xml/XmlIOTest.java |    2 +-
 .../org/apache/beam/sdk/io/xml/XmlSourceTest.java  |    2 +-
 sdks/java/javadoc/apiclient-docs/package-list      |   34 -
 sdks/java/javadoc/avro-docs/package-list           |   30 -
 sdks/java/javadoc/bq-docs/package-list             |    2 -
 sdks/java/javadoc/build.gradle                     |   31 +-
 sdks/java/javadoc/datastore-docs/package-list      |    2 -
 sdks/java/javadoc/guava-docs/package-list          |   15 -
 sdks/java/javadoc/hamcrest-docs/package-list       |   10 -
 .../javadoc/jackson-annotations-docs/package-list  |    1 -
 .../javadoc/jackson-databind-docs/package-list     |   20 -
 sdks/java/javadoc/joda-docs/package-list           |    7 -
 sdks/java/javadoc/junit-docs/package-list          |    7 -
 sdks/java/javadoc/oauth-docs/package-list          |   11 -
 sdks/java/maven-archetypes/examples/build.gradle   |    9 +-
 .../src/main/resources/archetype-resources/pom.xml |   19 +-
 sdks/java/maven-archetypes/starter/build.gradle    |    4 +-
 sdks/java/testing/load-tests/build.gradle          |   39 +-
 .../beam/sdk/loadtests/CoGroupByKeyLoadTest.java   |    2 +-
 .../apache/beam/sdk/loadtests/CombineLoadTest.java |    4 +-
 .../beam/sdk/loadtests/GroupByKeyLoadTest.java     |    2 +-
 .../org/apache/beam/sdk/loadtests/LoadTest.java    |    8 +-
 .../apache/beam/sdk/loadtests/LoadTestResult.java  |    2 +-
 .../apache/beam/sdk/loadtests/ParDoLoadTest.java   |   62 +-
 .../beam/sdk/loadtests/SyntheticDataPublisher.java |    2 +-
 sdks/java/testing/nexmark/build.gradle             |   68 +-
 .../java/org/apache/beam/sdk/nexmark/Main.java     |    4 +-
 .../apache/beam/sdk/nexmark/NexmarkLauncher.java   |   12 +-
 .../org/apache/beam/sdk/nexmark/NexmarkUtils.java  |   10 +-
 .../org/apache/beam/sdk/nexmark/model/Auction.java |    2 +-
 .../apache/beam/sdk/nexmark/model/AuctionBid.java  |    2 +-
 .../beam/sdk/nexmark/model/BidsPerSession.java     |    2 +-
 .../beam/sdk/nexmark/model/CategoryPrice.java      |    2 +-
 .../org/apache/beam/sdk/nexmark/model/Done.java    |    2 +-
 .../org/apache/beam/sdk/nexmark/model/Event.java   |    2 +-
 .../beam/sdk/nexmark/model/IdNameReserve.java      |    2 +-
 .../org/apache/beam/sdk/nexmark/model/Person.java  |    2 +-
 .../apache/beam/sdk/nexmark/model/SellerPrice.java |    2 +-
 .../sdk/nexmark/queries/BoundedSideInputJoin.java  |    2 +-
 .../beam/sdk/nexmark/queries/Query3Model.java      |    4 +-
 .../apache/beam/sdk/nexmark/queries/Query6.java    |    2 +-
 .../beam/sdk/nexmark/queries/Query8Model.java      |    4 +-
 .../sdk/nexmark/queries/SessionSideInputJoin.java  |    2 +-
 .../nexmark/queries/SessionSideInputJoinModel.java |    6 +-
 .../beam/sdk/nexmark/queries/WinningBids.java      |    2 +-
 .../queries/sql/SqlBoundedSideInputJoin.java       |    2 +-
 .../beam/sdk/nexmark/queries/sql/SqlQuery5.java    |    2 +-
 .../sdk/nexmark/sources/generator/Generator.java   |    2 +-
 .../sources/generator/GeneratorCheckpoint.java     |    2 +-
 .../apache/beam/sdk/nexmark/NexmarkUtilsTest.java  |   10 +-
 .../beam/sdk/nexmark/model/sql/RowSizeTest.java    |    2 +-
 .../nexmark/queries/BoundedSideInputJoinTest.java  |    2 +-
 .../nexmark/queries/SessionSideInputJoinTest.java  |    2 +-
 .../queries/sql/SqlBoundedSideInputJoinTest.java   |    2 +-
 .../sdk/nexmark/queries/sql/SqlQuery2Test.java     |    2 +-
 .../sdk/nexmark/queries/sql/SqlQuery3Test.java     |    2 +-
 .../sdk/nexmark/queries/sql/SqlQuery5Test.java     |    2 +-
 .../sdk/nexmark/queries/sql/SqlQuery7Test.java     |    2 +-
 sdks/java/testing/test-utils/build.gradle          |   23 +-
 .../apache/beam/sdk/testutils/NamedTestResult.java |    2 +-
 .../beam/sdk/testutils/metrics/IOITMetrics.java    |   19 +-
 .../beam/sdk/testutils/metrics/MetricsReader.java  |    6 +-
 .../publishing/BigQueryResultsPublisherTest.java   |    2 +-
 sdks/python/apache_beam/__init__.py                |    4 +-
 sdks/python/apache_beam/coders/avro_coder.py       |   99 +
 sdks/python/apache_beam/coders/avro_coder_test.py  |   71 +
 sdks/python/apache_beam/coders/coder_impl.py       |    7 +
 sdks/python/apache_beam/coders/coders.py           |   50 +-
 sdks/python/apache_beam/coders/coders_test.py      |   38 +-
 .../apache_beam/coders/coders_test_common.py       |    6 +-
 sdks/python/apache_beam/coders/observable_test.py  |    6 +-
 .../apache_beam/coders/standard_coders_test.py     |   27 +-
 sdks/python/apache_beam/coders/stream_test.py      |   46 +-
 .../apache_beam/examples/complete/estimate_pi.py   |    6 +-
 .../examples/complete/game/game_stats_it_test.py   |    6 -
 .../examples/complete/game/leader_board_it_test.py |    6 -
 .../examples/cookbook/group_with_coder.py          |    4 +-
 .../apache_beam/examples/snippets/snippets.py      |   96 +-
 .../apache_beam/examples/snippets/snippets_test.py |    9 +-
 .../snippets/transforms}/__init__.py               |    0
 .../snippets/transforms/element_wise}/__init__.py  |    0
 .../snippets/transforms/element_wise/filter.py     |  182 ++
 .../transforms/element_wise/filter_test.py         |   80 +
 .../snippets/transforms/element_wise/flat_map.py   |  221 ++
 .../transforms/element_wise/flat_map_test.py       |   87 +
 .../snippets/transforms/element_wise/keys.py       |   42 +
 .../snippets/transforms/element_wise/keys_test.py  |   55 +
 .../snippets/transforms/element_wise/kvswap.py     |   42 +
 .../transforms/element_wise/kvswap_test.py         |   55 +
 .../snippets/transforms/element_wise/map.py        |  203 ++
 .../snippets/transforms/element_wise/map_test.py   |   85 +
 .../snippets/transforms/element_wise/pardo.py      |   83 +
 .../snippets/transforms/element_wise/pardo_test.py |   82 +
 .../snippets/transforms/element_wise/partition.py  |  136 +
 .../transforms/element_wise/partition_test.py      |   84 +
 .../snippets/transforms/element_wise/regex.py      |  161 +
 .../snippets/transforms/element_wise/regex_test.py |  115 +
 .../snippets/transforms/element_wise/to_string.py  |   42 +
 .../transforms/element_wise/to_string_test.py      |   55 +
 .../snippets/transforms/element_wise/values.py     |   42 +
 .../transforms/element_wise/values_test.py         |   55 +
 .../transforms/element_wise/with_timestamps.py     |  106 +
 .../element_wise/with_timestamps_test.py           |   95 +
 sdks/python/apache_beam/examples/snippets/util.py  |   56 +
 .../apache_beam/examples/snippets/util_test.py     |   67 +
 .../examples/streaming_wordcount_it_test.py        |   14 +-
 .../apache_beam/examples/wordcount_it_test.py      |   33 +-
 .../python/apache_beam/examples/wordcount_xlang.py |    3 +-
 sdks/python/apache_beam/internal/gcp/auth.py       |  149 +-
 .../apache_beam/internal/gcp/json_value_test.py    |   36 +-
 .../apache_beam/internal/http_client_test.py       |   24 +-
 sdks/python/apache_beam/internal/module_test.py    |   10 +
 sdks/python/apache_beam/internal/pickler.py        |    7 +
 sdks/python/apache_beam/internal/pickler_test.py   |   43 +-
 sdks/python/apache_beam/internal/util_test.py      |   20 +-
 sdks/python/apache_beam/io/__init__.py             |    2 +-
 sdks/python/apache_beam/io/avroio.py               |    1 +
 sdks/python/apache_beam/io/avroio_test.py          |   56 +-
 sdks/python/apache_beam/io/concat_source_test.py   |   78 +-
 .../apache_beam/io/external/generate_sequence.py   |    2 +
 sdks/python/apache_beam/io/external/kafka.py       |    4 +
 sdks/python/apache_beam/io/filebasedsink.py        |    3 +-
 sdks/python/apache_beam/io/filebasedsource_test.py |   34 +-
 sdks/python/apache_beam/io/fileio.py               |  558 +++-
 sdks/python/apache_beam/io/fileio_test.py          |  370 ++-
 sdks/python/apache_beam/io/filesystem.py           |   34 +-
 sdks/python/apache_beam/io/filesystem_test.py      |   73 +
 sdks/python/apache_beam/io/filesystemio.py         |   50 +-
 sdks/python/apache_beam/io/filesystemio_test.py    |   51 +-
 .../io/gcp/big_query_query_to_table_it_test.py     |    6 -
 sdks/python/apache_beam/io/gcp/bigquery.py         |   47 +-
 .../apache_beam/io/gcp/bigquery_file_loads.py      |  162 +-
 .../apache_beam/io/gcp/bigquery_file_loads_test.py |  102 +-
 .../apache_beam/io/gcp/bigquery_read_it_test.py    |  213 ++
 .../apache_beam/io/gcp/bigquery_read_perf_test.py  |  147 +
 sdks/python/apache_beam/io/gcp/bigquery_test.py    |   75 +-
 sdks/python/apache_beam/io/gcp/bigquery_tools.py   |   12 +
 .../apache_beam/io/gcp/bigquery_tools_test.py      |    6 +-
 .../apache_beam/io/gcp/bigquery_write_it_test.py   |  274 ++
 .../apache_beam/io/gcp/bigquery_write_perf_test.py |  115 +
 .../apache_beam/io/gcp/datastore/v1new/helper.py   |    6 +-
 sdks/python/apache_beam/io/gcp/gcsio.py            |   38 +-
 .../apache_beam/io/gcp/gcsio_integration_test.py   |    2 +-
 sdks/python/apache_beam/io/gcp/gcsio_test.py       |    2 +-
 .../internal/clients/storage/storage_v1_client.py  |    2 +-
 .../apache_beam/io/gcp/pubsub_integration_test.py  |   34 +-
 sdks/python/apache_beam/io/gcp/pubsub_test.py      |   35 +-
 .../apache_beam/io/gcp/tests/bigquery_matcher.py   |  102 +-
 .../io/gcp/tests/bigquery_matcher_test.py          |   25 +-
 .../apache_beam/io/gcp/tests/pubsub_matcher.py     |    2 +-
 .../io/gcp/tests/pubsub_matcher_test.py            |    6 +-
 sdks/python/apache_beam/io/gcp/tests/utils.py      |    5 +-
 .../python/apache_beam/io/hadoopfilesystem_test.py |    4 +-
 .../io/hdfs_integration_test/Dockerfile            |    3 +-
 .../hdfs_integration_test/hdfs_integration_test.sh |   10 +-
 sdks/python/apache_beam/io/iobase.py               |  269 +-
 sdks/python/apache_beam/io/iobase_test.py          |  158 +
 sdks/python/apache_beam/io/localfilesystem_test.py |    4 +-
 sdks/python/apache_beam/io/mongodbio.py            |  370 +++
 sdks/python/apache_beam/io/mongodbio_it_test.py    |   92 +
 sdks/python/apache_beam/io/mongodbio_test.py       |  215 ++
 sdks/python/apache_beam/io/parquetio_test.py       |    6 +-
 sdks/python/apache_beam/io/range_trackers.py       |   37 +-
 sdks/python/apache_beam/io/range_trackers_test.py  |   27 +-
 sdks/python/apache_beam/io/restriction_trackers.py |    5 +-
 .../apache_beam/io/source_test_utils_test.py       |   12 +-
 sdks/python/apache_beam/io/sources_test.py         |   35 +-
 sdks/python/apache_beam/io/tfrecordio_test.py      |    4 +-
 sdks/python/apache_beam/io/utils.py                |   11 +-
 sdks/python/apache_beam/io/utils_test.py           |   66 +
 .../python/apache_beam/options/pipeline_options.py |   21 +-
 .../apache_beam/options/pipeline_options_test.py   |   25 +
 .../options/pipeline_options_validator.py          |   15 +
 .../options/pipeline_options_validator_test.py     |   33 +
 sdks/python/apache_beam/pipeline.py                |   16 +-
 sdks/python/apache_beam/pipeline_test.py           |   81 +-
 sdks/python/apache_beam/pvalue.py                  |   16 +-
 sdks/python/apache_beam/runners/common.pxd         |    6 +
 sdks/python/apache_beam/runners/common.py          |   87 +-
 sdks/python/apache_beam/runners/common_test.py     |   59 +
 .../dataflow_exercise_metrics_pipeline_test.py     |    6 -
 .../runners/dataflow/dataflow_runner.py            |   56 +-
 .../runners/dataflow/dataflow_runner_test.py       |   25 +
 .../runners/dataflow/internal/apiclient.py         |   66 +-
 .../runners/dataflow/internal/apiclient_test.py    |   82 +-
 .../clients/dataflow/dataflow_v1b3_client.py       |  218 ++
 .../clients/dataflow/dataflow_v1b3_messages.py     |  182 +-
 .../apache_beam/runners/dataflow/internal/names.py |    4 +-
 .../runners/dataflow/ptransform_overrides.py       |   17 +
 .../apache_beam/runners/direct/direct_runner.py    |   13 +-
 .../runners/direct/direct_runner_test.py           |    8 +
 .../runners/direct/helper_transforms.py            |    7 +-
 .../runners/direct/transform_evaluator.py          |   45 +-
 .../runners/interactive/cache_manager.py           |   88 +-
 .../runners/interactive/cache_manager_test.py      |   36 +-
 .../runners/interactive/interactive_runner.py      |    9 +-
 .../python/apache_beam/runners/pipeline_context.py |    7 +-
 .../runners/portability/flink_runner.py            |   69 +
 .../runners/portability/flink_runner_test.py       |   14 +-
 .../runners/portability/fn_api_runner.py           | 1048 ++++---
 .../runners/portability/fn_api_runner_test.py      |  190 +-
 .../portability/fn_api_runner_transforms.py        |   73 +-
 .../portability/java_reference_runner_test.py      |  166 --
 .../apache_beam/runners/portability/job_server.py  |  290 +-
 .../runners/portability/local_job_service.py       |   21 +-
 .../runners/portability/portable_runner.py         |   68 +-
 .../runners/portability/portable_runner_test.py    |   10 +
 .../runners/portability/spark_runner_test.py       |   20 +-
 .../apache_beam/runners/portability/stager.py      |    2 +-
 .../apache_beam/runners/portability/stager_test.py |    2 +-
 sdks/python/apache_beam/runners/runner.py          |   64 +-
 .../apache_beam/runners/worker/bundle_processor.py |  120 +-
 .../apache_beam/runners/worker/data_plane.py       |   80 +-
 .../apache_beam/runners/worker/data_plane_test.py  |   53 +-
 .../apache_beam/runners/worker/operations.pxd      |    1 +
 .../apache_beam/runners/worker/operations.py       |   14 +
 .../apache_beam/runners/worker/sdk_worker.py       |   17 +-
 .../apache_beam/runners/worker/sdk_worker_main.py  |    7 +-
 .../runners/worker/sdk_worker_main_test.py         |    2 +-
 .../apache_beam/testing/load_tests/build.gradle    |    5 +-
 .../testing/load_tests/co_group_by_key_test.py     |   33 +-
 .../apache_beam/testing/load_tests/combine_test.py |   20 +-
 .../testing/load_tests/group_by_key_test.py        |    4 +-
 .../apache_beam/testing/load_tests/load_test.py    |   34 +-
 .../testing/load_tests/load_test_metrics_utils.py  |   39 +-
 .../apache_beam/testing/load_tests/pardo_test.py   |   70 +-
 .../testing/load_tests/sideinput_test.py           |    4 +-
 .../apache_beam/testing/synthetic_pipeline.py      |  260 +-
 .../apache_beam/testing/synthetic_pipeline_test.py |  127 +-
 sdks/python/apache_beam/testing/test_stream.py     |    2 +-
 sdks/python/apache_beam/testing/util.py            |   32 +
 sdks/python/apache_beam/testing/util_test.py       |   11 +
 .../apache_beam/tools/coders_microbenchmark.py     |   32 +-
 .../apache_beam/tools/microbenchmarks_test.py      |   20 +
 sdks/python/apache_beam/tools/utils.py             |    3 +-
 sdks/python/apache_beam/transforms/__init__.py     |    1 +
 sdks/python/apache_beam/transforms/combiners.py    |   96 +-
 .../apache_beam/transforms/combiners_test.py       |   91 +-
 sdks/python/apache_beam/transforms/core.py         |  232 +-
 .../dataflow_distribution_counter_test.py          |   24 +-
 .../apache_beam/transforms/dofn_lifecycle_test.py  |   89 +
 sdks/python/apache_beam/transforms/external.py     |    7 +
 sdks/python/apache_beam/transforms/ptransform.py   |   11 +-
 .../apache_beam/transforms/ptransform_test.py      |  142 +-
 sdks/python/apache_beam/transforms/stats.py        |  236 ++
 sdks/python/apache_beam/transforms/stats_test.py   |  349 +++
 sdks/python/apache_beam/transforms/trigger.py      |   14 +-
 sdks/python/apache_beam/transforms/trigger_test.py |    4 +-
 .../apache_beam/transforms/userstate_test.py       |   49 +-
 sdks/python/apache_beam/transforms/util.py         |  230 +-
 sdks/python/apache_beam/transforms/util_test.py    |  213 +-
 sdks/python/apache_beam/transforms/window.py       |   16 +-
 sdks/python/apache_beam/typehints/decorators.py    |   62 +-
 .../typehints/native_type_compatibility.py         |   62 +-
 .../typehints/native_type_compatibility_test.py    |   16 +-
 sdks/python/apache_beam/typehints/opcodes.py       |  113 +-
 .../apache_beam/typehints/trivial_inference.py     |  177 +-
 .../typehints/trivial_inference_test.py            |  107 +-
 .../apache_beam/typehints/typed_pipeline_test.py   |   22 +-
 sdks/python/apache_beam/typehints/typehints.py     |   26 +-
 .../python/apache_beam/typehints/typehints_test.py |   87 +-
 sdks/python/apache_beam/utils/retry.py             |    9 +
 sdks/python/apache_beam/utils/timestamp.py         |   30 +-
 sdks/python/apache_beam/utils/timestamp_test.py    |    9 +
 sdks/python/apache_beam/utils/windowed_value.py    |   16 +-
 sdks/python/apache_beam/version.py                 |    2 +-
 sdks/python/build.gradle                           |  412 +--
 sdks/python/container/build.gradle                 |    8 +-
 sdks/python/container/py3/build.gradle             |    4 +-
 sdks/python/container/run_validatescontainer.sh    |    7 +-
 sdks/python/gen_protos.py                          |    3 +-
 sdks/python/scripts/run_dependency_check.sh        |    3 +-
 sdks/python/scripts/run_integration_test.sh        |   33 +-
 sdks/python/scripts/run_pylint.sh                  |    3 +-
 sdks/python/scripts/run_snapshot_publish.sh        |    2 +-
 sdks/python/setup.py                               |   19 +-
 sdks/python/test-suites/dataflow/build.gradle      |   47 -
 sdks/python/test-suites/dataflow/py2/build.gradle  |  116 +
 sdks/python/test-suites/dataflow/py35/build.gradle |   35 +-
 sdks/python/test-suites/dataflow/py36/build.gradle |   36 +-
 sdks/python/test-suites/dataflow/py37/build.gradle |  126 +
 sdks/python/test-suites/direct/py2/build.gradle    |  114 +
 sdks/python/test-suites/direct/py35/build.gradle   |    5 +-
 sdks/python/test-suites/direct/py36/build.gradle   |    6 +-
 sdks/python/test-suites/direct/py37/build.gradle   |   63 +
 sdks/python/test-suites/portable/py2/build.gradle  |  235 ++
 sdks/python/test-suites/portable/py35/build.gradle |   32 +
 sdks/python/test-suites/tox/py2/build.gradle       |   59 +
 settings.gradle                                    |  349 +--
 vendor/bytebuddy-1_9_3/build.gradle                |   34 +
 vendor/grpc-1_13_1/build.gradle                    |   36 -
 vendor/grpc-1_21_0/build.gradle                    |   36 +
 vendor/guava-20_0/build.gradle                     |   35 -
 vendor/guava-26_0-jre/build.gradle                 |   42 +
 vendor/sdks-java-extensions-protobuf/build.gradle  |   26 +-
 website/README.md                                  |    4 +-
 website/_config.yml                                |    2 +-
 website/_config_test.yml                           |    2 +-
 website/build.gradle                               |    2 +-
 website/src/.htaccess                              |    2 +-
 website/src/_data/authors.yml                      |   12 +
 website/src/_data/capability-matrix.yml            |  217 ++
 website/src/_includes/capability-matrix.md         |    9 +-
 website/src/_includes/head.html                    |    2 +-
 website/src/_includes/header.html                  |   16 +
 .../src/_includes/section-menu/documentation.html  |  133 +-
 website/src/_includes/section-menu/runners.html    |    1 +
 website/src/_includes/section-menu/sdks.html       |   29 +-
 .../src/_posts/2017-02-13-stateful-processing.md   |  110 +-
 website/src/_posts/2018-01-09-beam-a-look-back.md  |    2 +-
 website/src/_posts/2019-03-18-beam-summit-site.md  |   38 +
 .../2019-05-01-adding-data-sources-to-sql.md       |  202 ++
 .../_posts/2019-05-11-beam-summit-europe-2019.md   |   88 +
 website/src/_posts/2019-05-22-beam-2.13.0.md       |   77 +
 website/src/_posts/2019-05-30-beam-kata-release.md |   59 +
 website/src/_posts/2019-06-11-looping-timers.md    |  349 +++
 website/src/_sass/_global.sass                     |    2 +-
 website/src/_sass/capability-matrix.scss           |    2 +-
 website/src/community/in-person.md                 |    6 +-
 website/src/contribute/design-documents.md         |    7 +
 website/src/contribute/ptransform-style-guide.md   |    4 +-
 website/src/contribute/release-guide.md            |   24 +-
 website/src/contribute/runner-guide.md             |    4 +-
 .../documentation/dsls/sql/aggregate-functions.md  |   34 -
 .../dsls/sql/calcite/aggregate-functions.md        |   37 +
 .../documentation/dsls/sql/calcite/data-types.md   |   48 +
 .../dsls/sql/calcite/lexical-structure.md          | 1050 +++++++
 .../src/documentation/dsls/sql/calcite/overview.md |   88 +
 .../documentation/dsls/sql/calcite/query-syntax.md |  716 +++++
 .../dsls/sql/calcite/scalar-functions.md           |  137 +
 .../dsls/sql/create-external-table.md              |  364 ---
 website/src/documentation/dsls/sql/data-types.md   |   46 -
 .../dsls/sql/extensions/create-external-table.md   |  365 +++
 .../src/documentation/dsls/sql/extensions/joins.md |   73 +
 .../src/documentation/dsls/sql/extensions/set.md   |   56 +
 .../dsls/sql/extensions/user-defined-functions.md  |  128 +
 .../sql/extensions/windowing-and-triggering.md     |   67 +
 website/src/documentation/dsls/sql/joins.md        |   76 -
 .../documentation/dsls/sql/lexical-structure.md    | 1048 -------
 website/src/documentation/dsls/sql/overview.md     |   37 +-
 .../src/documentation/dsls/sql/scalar-functions.md |  134 -
 website/src/documentation/dsls/sql/select.md       |  715 -----
 website/src/documentation/dsls/sql/set.md          |   55 -
 website/src/documentation/dsls/sql/shell.md        |   10 +-
 .../dsls/sql/user-defined-functions.md             |  124 -
 .../dsls/sql/windowing-and-triggering.md           |   66 -
 website/src/documentation/index.md                 |    3 +-
 .../documentation/io/built-in-google-bigquery.md   |   51 +-
 website/src/documentation/io/built-in-hadoop.md    |   16 +-
 website/src/documentation/io/built-in.md           |    1 -
 .../documentation/patterns/custom-io-patterns.md   |   42 +
 .../patterns/file-processing-patterns.md           |  107 +
 website/src/documentation/patterns/overview.md     |   45 +
 .../patterns/pipeline-option-patterns.md           |   47 +
 .../documentation/patterns/side-input-patterns.md  |   48 +
 website/src/documentation/programming-guide.md     |  122 +-
 .../documentation/resources/learning-resources.md  |   27 +
 website/src/documentation/runners/flink.md         |   31 +-
 website/src/documentation/runners/jet.md           |  197 ++
 website/src/documentation/runners/spark.md         |  132 +-
 website/src/documentation/sdks/euphoria.md         |    4 +-
 .../src/documentation/sdks/java-dependencies.md    |   17 +-
 website/src/documentation/sdks/nexmark.md          |   45 +-
 .../src/documentation/sdks/python-dependencies.md  |   93 +-
 website/src/documentation/sdks/python-streaming.md |    2 +-
 .../java/aggregation/approximatequantiles.md       |   43 +
 .../java/aggregation/approximateunique.md          |   40 +
 .../transforms/java/aggregation/cogroupbykey.md    |   73 +
 .../transforms/java/aggregation/combine.md         |   82 +
 .../java/aggregation/combinewithcontext.md         |   37 +
 .../transforms/java/aggregation/count.md           |   50 +
 .../transforms/java/aggregation/distinct.md        |   43 +
 .../transforms/java/aggregation/groupbykey.md      |   50 +
 .../java/aggregation/groupintobatches.md           |   42 +
 .../transforms/java/aggregation/latest.md          |   52 +
 .../transforms/java/aggregation/max.md             |   56 +
 .../transforms/java/aggregation/mean.md            |   58 +
 .../transforms/java/aggregation/min.md             |   42 +
 .../transforms/java/aggregation/sample.md          |   40 +
 .../transforms/java/aggregation/sum.md             |   51 +
 .../transforms/java/aggregation/top.md             |   39 +
 .../transforms/java/element-wise/filter.md         |   62 +
 .../java/element-wise/flatmapelements.md           |   40 +
 .../transforms/java/element-wise/keys.md           |   43 +
 .../transforms/java/element-wise/kvswap.md         |   43 +
 .../transforms/java/element-wise/mapelements.md    |   63 +
 .../transforms/java/element-wise/pardo.md          |  152 +
 .../transforms/java/element-wise/partition.md      |   62 +
 .../transforms/java/element-wise/regex.md          |   36 +
 .../transforms/java/element-wise/reify.md          |   39 +
 .../transforms/java/element-wise/tostring.md       |   37 +
 .../transforms/java/element-wise/values.md         |   44 +
 .../transforms/java/element-wise/withkeys.md       |   55 +
 .../transforms/java/element-wise/withtimestamps.md |   36 +
 website/src/documentation/transforms/java/index.md |   81 +
 .../documentation/transforms/java/other/create.md  |   40 +
 .../documentation/transforms/java/other/flatten.md |   67 +
 .../documentation/transforms/java/other/passert.md |   61 +
 .../documentation/transforms/java/other/view.md    |   37 +
 .../documentation/transforms/java/other/window.md  |   40 +
 .../python/aggregation/approximatequantiles.md     |   26 +
 .../python/aggregation/approximateunique.md        |   25 +
 .../transforms/python/aggregation/cogroupbykey.md  |   45 +
 .../python/aggregation/combineglobally.md          |   43 +
 .../python/aggregation/combinewithcontext.md       |   26 +
 .../transforms/python/aggregation/count.md         |   36 +
 .../transforms/python/aggregation/distinct.md      |   37 +
 .../transforms/python/aggregation/groupbykey.md    |   41 +
 .../python/aggregation/groupintobatches.md         |   26 +
 .../transforms/python/aggregation/latest.md        |   26 +
 .../transforms/python/aggregation/max.md           |   26 +
 .../transforms/python/aggregation/mean.md          |   39 +
 .../transforms/python/aggregation/min.md           |   26 +
 .../transforms/python/aggregation/sample.md        |   38 +
 .../transforms/python/aggregation/sum.md           |   26 +
 .../transforms/python/aggregation/top.md           |   38 +
 .../transforms/python/element-wise/filter.md       |  195 ++
 .../transforms/python/element-wise/flatmap.md      |   42 +
 .../transforms/python/element-wise/keys.md         |   39 +
 .../transforms/python/element-wise/kvswap.md       |   39 +
 .../transforms/python/element-wise/map.md          |   42 +
 .../transforms/python/element-wise/pardo.md        |   44 +
 .../transforms/python/element-wise/partition.md    |   50 +
 .../transforms/python/element-wise/regex.md        |   28 +
 .../transforms/python/element-wise/reify.md        |   28 +
 .../transforms/python/element-wise/tostring.md     |   27 +
 .../transforms/python/element-wise/values.md       |   37 +
 .../transforms/python/element-wise/withkeys.md     |   27 +
 .../python/element-wise/withtimestamps.md          |   28 +
 .../src/documentation/transforms/python/index.md   |   86 +
 .../transforms/python/other/create.md              |   38 +
 .../transforms/python/other/flatten.md             |   43 +
 .../transforms/python/other/passert.md             |   25 +
 .../transforms/python/other/reshuffle.md           |   41 +
 .../documentation/transforms/python/other/view.md  |   25 +
 .../transforms/python/other/windowinto.md          |   41 +
 website/src/get-started/beam-overview.md           |    1 +
 website/src/get-started/downloads.md               |   25 +-
 website/src/get-started/quickstart-java.md         |   45 +-
 website/src/get-started/quickstart-py.md           |    9 +-
 website/src/get-started/wordcount-example.md       |   38 +-
 .../blog/beam-kata/beam-kata-intellij-edu-1.png    |  Bin 0 -> 52004 bytes
 .../blog/beam-kata/beam-kata-intellij-edu-2.png    |  Bin 0 -> 148295 bytes
 .../blog/beam-kata/beam-kata-pycharm-edu-1.png     |  Bin 0 -> 44655 bytes
 .../blog/beam-kata/beam-kata-pycharm-edu-2.png     |  Bin 0 -> 164407 bytes
 website/src/images/logos/runners/jet.png           |  Bin 0 -> 9128 bytes
 website/src/js/section-nav.js                      |    6 +-
 website/src/roadmap/portability.md                 |   33 +-
 website/src/roadmap/python-sdk.md                  |   10 +-
 2935 files changed, 86951 insertions(+), 36364 deletions(-)
 create mode 100644 .test-infra/jenkins/Infrastructure.groovy
 create mode 100644 .test-infra/jenkins/job_LoadTests_CoGBK_Java.groovy
 create mode 100644 .test-infra/jenkins/job_LoadTests_Combine_Flink_Python.groovy
 create mode 100644 .test-infra/jenkins/job_LoadTests_Combine_Python.groovy
 create mode 100644 .test-infra/jenkins/job_LoadTests_GBK_Flink_Python.groovy
 create mode 100644 .test-infra/jenkins/job_LoadTests_GBK_Java.groovy
 create mode 100644 .test-infra/jenkins/job_LoadTests_GBK_Python.groovy
 create mode 100644 .test-infra/jenkins/job_LoadTests_GBK_Python_reiterate.groovy
 delete mode 100644 .test-infra/jenkins/job_LoadTests_Java.groovy
 create mode 100644 .test-infra/jenkins/job_LoadTests_ParDo_Java.groovy
 create mode 100644 .test-infra/jenkins/job_LoadTests_ParDo_Python.groovy
 delete mode 100644 .test-infra/jenkins/job_LoadTests_Python.groovy
 create mode 100644 .test-infra/jenkins/job_LoadTests_Python_Smoke.groovy
 create mode 100644 .test-infra/jenkins/job_LoadTests_coGBK_Flink_Python.groovy
 create mode 100644 .test-infra/jenkins/job_LoadTests_coGBK_Python.groovy
 create mode 100644 .test-infra/jenkins/job_PerformanceTests_BigQueryIO_Python.groovy
 delete mode 100644 .test-infra/jenkins/job_PerformanceTests_Dataflow.groovy
 delete mode 100644 .test-infra/jenkins/job_PerformanceTests_Spark.groovy
 delete mode 100644 .test-infra/jenkins/job_Performancetests_Python35.groovy
 create mode 100644 .test-infra/jenkins/job_PostCommit_Go_ValidatesRunner_Flink.groovy
 create mode 100644 .test-infra/jenkins/job_PostCommit_Go_ValidatesRunner_Spark.groovy
 create mode 100644 .test-infra/jenkins/job_PostCommit_Java_PortableValidatesRunner_Spark_Batch.groovy
 create mode 100644 .test-infra/jenkins/job_PostCommit_Java_ValidatesRunner_Direct.groovy
 create mode 100644 .test-infra/jenkins/job_PostCommit_Python2.groovy
 create mode 100644 .test-infra/jenkins/job_PostCommit_Python35.groovy
 create mode 100644 .test-infra/jenkins/job_PostCommit_Python36.groovy
 create mode 100644 .test-infra/jenkins/job_PostCommit_Python37.groovy
 delete mode 100644 .test-infra/jenkins/job_PostCommit_Python3_Verify.groovy
 create mode 100644 .test-infra/jenkins/job_PostCommit_Python_MongoDBIO_IT.groovy
 create mode 100644 .test-infra/jenkins/job_PostCommit_Python_ValidatesRunner_Spark.groovy
 delete mode 100644 .test-infra/jenkins/job_PostCommit_Python_Verify.groovy
 create mode 100755 .test-infra/metrics/build_and_publish_containers.sh
 delete mode 100644 .test-infra/metrics/dashboards/Post-Commits_status_dashboard.json
 delete mode 100644 .test-infra/metrics/dashboards/code_velocity.json
 delete mode 100644 .test-infra/metrics/dashboards/post-commit_tests.json
 delete mode 100644 .test-infra/metrics/dashboards/pre-commit_tests.json
 delete mode 100644 .test-infra/metrics/dashboards/source_data_freshness.json
 delete mode 100644 .test-infra/metrics/dashboards/stability_critical_jobs_status.json
 create mode 100644 .test-infra/metrics/grafana/Dockerfile
 create mode 100644 .test-infra/metrics/grafana/dashboards/Post-Commits_status_dashboard.json
 create mode 100644 .test-infra/metrics/grafana/dashboards/code_velocity.json
 create mode 100644 .test-infra/metrics/grafana/dashboards/post-commit_tests.json
 create mode 100644 .test-infra/metrics/grafana/dashboards/pre-commit_tests.json
 create mode 100644 .test-infra/metrics/grafana/dashboards/source_data_freshness.json
 create mode 100644 .test-infra/metrics/grafana/dashboards/stability_critical_jobs_status.json
 create mode 100644 .test-infra/metrics/grafana/provisioning/dashboards/all.yaml
 create mode 100644 .test-infra/metrics/grafana/provisioning/datasources/beampostgresql.yaml
 create mode 100644 .test-infra/metrics/postgres/init.sql
 create mode 100644 deprecation-warning.txt
 delete mode 100644 examples/kotlin/OWNERS
 create mode 100644 examples/kotlin/src/main/java/org/apache/beam/examples/kotlin/cookbook/BigQueryTornadoes.kt
 create mode 100644 examples/kotlin/src/main/java/org/apache/beam/examples/kotlin/cookbook/CombinePerKeyExamples.kt
 create mode 100644 examples/kotlin/src/main/java/org/apache/beam/examples/kotlin/cookbook/DistinctExample.kt
 create mode 100644 examples/kotlin/src/main/java/org/apache/beam/examples/kotlin/cookbook/FilterExamples.kt
 create mode 100644 examples/kotlin/src/main/java/org/apache/beam/examples/kotlin/cookbook/JoinExamples.kt
 create mode 100644 examples/kotlin/src/main/java/org/apache/beam/examples/kotlin/cookbook/MaxPerKeyExamples.kt
 copy examples/{java/src/main/java/org/apache/beam/examples => kotlin/src/main/java/org/apache/beam/examples/kotlin}/cookbook/README.md (100%)
 create mode 100644 examples/kotlin/src/main/java/org/apache/beam/examples/kotlin/cookbook/TriggerExample.kt
 create mode 100644 examples/kotlin/src/main/java/org/apache/beam/examples/kotlin/snippets/Snippets.kt
 copy gradlew => gradlew_orig (100%)
 copy gradlew.bat => gradlew_orig.bat (100%)
 create mode 100644 learning/katas/README.md
 create mode 100644 learning/katas/java/.idea/study_project.xml
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Count/src/org/apache/beam/learning/katas/commontransforms/aggregation/count/Task.java
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Count/task.html
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Count/test/org/apache/beam/learning/katas/commontransforms/aggregation/count/TaskTest.java
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Max/src/org/apache/beam/learning/katas/commontransforms/aggregation/max/Task.java
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Max/task.html
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Max/test/org/apache/beam/learning/katas/commontransforms/aggregation/max/TaskTest.java
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Mean/src/org/apache/beam/learning/katas/commontransforms/aggregation/mean/Task.java
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Mean/task.html
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Mean/test/org/apache/beam/learning/katas/commontransforms/aggregation/mean/TaskTest.java
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Min/src/org/apache/beam/learning/katas/commontransforms/aggregation/min/Task.java
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Min/task.html
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Min/test/org/apache/beam/learning/katas/commontransforms/aggregation/min/TaskTest.java
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Sum/src/org/apache/beam/learning/katas/commontransforms/aggregation/sum/Task.java
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Sum/task.html
 create mode 100644 learning/katas/java/Common Transforms/Aggregation/Sum/test/org/apache/beam/learning/katas/commontransforms/aggregation/sum/TaskTest.java
 create mode 100644 learning/katas/java/Common Transforms/Filter/Filter/src/org/apache/beam/learning/katas/commontransforms/filter/filter/Task.java
 create mode 100644 learning/katas/java/Common Transforms/Filter/Filter/task.html
 create mode 100644 learning/katas/java/Common Transforms/Filter/Filter/test/org/apache/beam/learning/katas/commontransforms/filter/filter/TaskTest.java
 create mode 100644 learning/katas/java/Common Transforms/Filter/ParDo/src/org/apache/beam/learning/katas/commontransforms/filter/pardo/Task.java
 create mode 100644 learning/katas/java/Common Transforms/Filter/ParDo/task.html
 create mode 100644 learning/katas/java/Common Transforms/Filter/ParDo/test/org/apache/beam/learning/katas/commontransforms/filter/pardo/TaskTest.java
 create mode 100644 learning/katas/java/Common Transforms/WithKeys/WithKeys/src/org/apache/beam/learning/katas/commontransforms/withkeys/Task.java
 create mode 100644 learning/katas/java/Common Transforms/WithKeys/WithKeys/task.html
 create mode 100644 learning/katas/java/Common Transforms/WithKeys/WithKeys/test/org/apache/beam/learning/katas/commontransforms/withkeys/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Branching/Branching/src/org/apache/beam/learning/katas/coretransforms/branching/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Branching/Branching/task.html
 create mode 100644 learning/katas/java/Core Transforms/Branching/Branching/test/org/apache/beam/learning/katas/coretransforms/branching/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/CoGroupByKey/CoGroupByKey/src/org/apache/beam/learning/katas/coretransforms/cogroupbykey/Task.java
 create mode 100644 learning/katas/java/Core Transforms/CoGroupByKey/CoGroupByKey/src/org/apache/beam/learning/katas/coretransforms/cogroupbykey/WordsAlphabet.java
 create mode 100644 learning/katas/java/Core Transforms/CoGroupByKey/CoGroupByKey/task.html
 create mode 100644 learning/katas/java/Core Transforms/CoGroupByKey/CoGroupByKey/test/org/apache/beam/learning/katas/coretransforms/cogroupbykey/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Combine/BinaryCombineFn Lambda/src/org/apache/beam/learning/katas/coretransforms/combine/binarycombinefnlambda/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Combine/BinaryCombineFn Lambda/task.html
 create mode 100644 learning/katas/java/Core Transforms/Combine/BinaryCombineFn Lambda/test/org/apache/beam/learning/katas/coretransforms/combine/binarycombinefnlambda/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Combine/BinaryCombineFn/src/org/apache/beam/learning/katas/coretransforms/combine/binarycombinefn/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Combine/BinaryCombineFn/task.html
 create mode 100644 learning/katas/java/Core Transforms/Combine/BinaryCombineFn/test/org/apache/beam/learning/katas/coretransforms/combine/binarycombinefn/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Combine/Combine PerKey/src/org/apache/beam/learning/katas/coretransforms/combine/combineperkey/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Combine/Combine PerKey/task.html
 create mode 100644 learning/katas/java/Core Transforms/Combine/Combine PerKey/test/org/apache/beam/learning/katas/coretransforms/combine/combineperkey/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Combine/CombineFn/src/org/apache/beam/learning/katas/coretransforms/combine/combinefn/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Combine/CombineFn/task.html
 create mode 100644 learning/katas/java/Core Transforms/Combine/CombineFn/test/org/apache/beam/learning/katas/coretransforms/combine/combinefn/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Combine/Simple Function/src/org/apache/beam/learning/katas/coretransforms/combine/simple/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Combine/Simple Function/task.html
 create mode 100644 learning/katas/java/Core Transforms/Combine/Simple Function/test/org/apache/beam/learning/katas/coretransforms/combine/simple/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Composite Transform/Composite Transform/src/org/apache/beam/learning/katas/coretransforms/composite/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Composite Transform/Composite Transform/task.html
 create mode 100644 learning/katas/java/Core Transforms/Composite Transform/Composite Transform/test/org/apache/beam/learning/katas/coretransforms/composite/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/DoFn Additional Parameters/DoFn Additional Parameters/src/org/apache/beam/learning/katas/coretransforms/dofnadditionalparams/Task.java
 create mode 100644 learning/katas/java/Core Transforms/DoFn Additional Parameters/DoFn Additional Parameters/task.html
 create mode 100644 learning/katas/java/Core Transforms/DoFn Additional Parameters/DoFn Additional Parameters/test/org/apache/beam/learning/katas/coretransforms/dofnadditionalparams/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Flatten/Flatten/src/org/apache/beam/learning/katas/coretransforms/flatten/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Flatten/Flatten/task.html
 create mode 100644 learning/katas/java/Core Transforms/Flatten/Flatten/test/org/apache/beam/learning/katas/coretransforms/flatten/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/GroupByKey/GroupByKey/src/org/apache/beam/learning/katas/coretransforms/groupbykey/Task.java
 create mode 100644 learning/katas/java/Core Transforms/GroupByKey/GroupByKey/task.html
 create mode 100644 learning/katas/java/Core Transforms/GroupByKey/GroupByKey/test/org/apache/beam/learning/katas/coretransforms/groupbykey/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Map/FlatMapElements/src/org/apache/beam/learning/katas/coretransforms/map/flatmapelements/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Map/FlatMapElements/task.html
 create mode 100644 learning/katas/java/Core Transforms/Map/FlatMapElements/test/org/apache/beam/learning/katas/coretransforms/map/flatmapelements/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Map/MapElements/src/org/apache/beam/learning/katas/coretransforms/map/mapelements/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Map/MapElements/task.html
 create mode 100644 learning/katas/java/Core Transforms/Map/MapElements/test/org/apache/beam/learning/katas/coretransforms/map/mapelements/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Map/ParDo OneToMany/src/org/apache/beam/learning/katas/coretransforms/map/pardoonetomany/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Map/ParDo OneToMany/task.html
 create mode 100644 learning/katas/java/Core Transforms/Map/ParDo OneToMany/test/org/apache/beam/learning/katas/coretransforms/map/pardoonetomany/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Map/ParDo/src/org/apache/beam/learning/katas/coretransforms/map/pardo/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Map/ParDo/task.html
 create mode 100644 learning/katas/java/Core Transforms/Map/ParDo/test/org/apache/beam/learning/katas/coretransforms/map/pardo/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Partition/Partition/src/org/apache/beam/learning/katas/coretransforms/partition/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Partition/Partition/task.html
 create mode 100644 learning/katas/java/Core Transforms/Partition/Partition/test/org/apache/beam/learning/katas/coretransforms/partition/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Side Input/Side Input/src/org/apache/beam/learning/katas/coretransforms/sideinput/Person.java
 create mode 100644 learning/katas/java/Core Transforms/Side Input/Side Input/src/org/apache/beam/learning/katas/coretransforms/sideinput/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Side Input/Side Input/task.html
 create mode 100644 learning/katas/java/Core Transforms/Side Input/Side Input/test/org/apache/beam/learning/katas/coretransforms/sideinput/TaskTest.java
 create mode 100644 learning/katas/java/Core Transforms/Side Output/Side Output/src/org/apache/beam/learning/katas/coretransforms/sideoutput/Task.java
 create mode 100644 learning/katas/java/Core Transforms/Side Output/Side Output/task.html
 create mode 100644 learning/katas/java/Core Transforms/Side Output/Side Output/test/org/apache/beam/learning/katas/coretransforms/sideoutput/TaskTest.java
 create mode 100644 learning/katas/java/Examples/Word Count/Word Count/src/org/apache/beam/learning/katas/examples/wordcount/Task.java
 create mode 100644 learning/katas/java/Examples/Word Count/Word Count/task.html
 create mode 100644 learning/katas/java/Examples/Word Count/Word Count/test/org/apache/beam/learning/katas/examples/wordcount/TaskTest.java
 create mode 100644 learning/katas/java/IO/Built-in IOs/Built-in IOs/src/org/apache/beam/learning/katas/io/builtinios/Task.java
 create mode 100644 learning/katas/java/IO/Built-in IOs/Built-in IOs/task.html
 create mode 100644 learning/katas/java/IO/Built-in IOs/Built-in IOs/test/org/apache/beam/learning/katas/io/builtinios/TaskTest.java
 create mode 100644 learning/katas/java/IO/TextIO/TextIO Read/countries.txt
 create mode 100644 learning/katas/java/IO/TextIO/TextIO Read/src/org/apache/beam/learning/katas/io/textio/read/Task.java
 create mode 100644 learning/katas/java/IO/TextIO/TextIO Read/task.html
 create mode 100644 learning/katas/java/IO/TextIO/TextIO Read/test/org/apache/beam/learning/katas/io/textio/read/TaskTest.java
 create mode 100644 learning/katas/java/Introduction/Hello Beam/Hello Beam/src/org/apache/beam/learning/katas/intro/hello/Task.java
 create mode 100644 learning/katas/java/Introduction/Hello Beam/Hello Beam/task.html
 create mode 100644 learning/katas/java/Introduction/Hello Beam/Hello Beam/test/org/apache/beam/learning/katas/intro/hello/TaskTest.java
 create mode 100644 learning/katas/java/README.md
 create mode 100644 learning/katas/java/Triggers/Early Triggers/Early Triggers/src/org/apache/beam/learning/katas/triggers/earlytriggers/GenerateEvent.java
 create mode 100644 learning/katas/java/Triggers/Early Triggers/Early Triggers/src/org/apache/beam/learning/katas/triggers/earlytriggers/Task.java
 create mode 100644 learning/katas/java/Triggers/Early Triggers/Early Triggers/task.html
 create mode 100644 learning/katas/java/Triggers/Early Triggers/Early Triggers/test/org/apache/beam/learning/katas/triggers/earlytriggers/TaskTest.java
 create mode 100644 learning/katas/java/Triggers/Event Time Triggers/Event Time Triggers/src/org/apache/beam/learning/katas/triggers/eventtimetriggers/GenerateEvent.java
 create mode 100644 learning/katas/java/Triggers/Event Time Triggers/Event Time Triggers/src/org/apache/beam/learning/katas/triggers/eventtimetriggers/Task.java
 create mode 100644 learning/katas/java/Triggers/Event Time Triggers/Event Time Triggers/task.html
 create mode 100644 learning/katas/java/Triggers/Event Time Triggers/Event Time Triggers/test/org/apache/beam/learning/katas/triggers/eventtimetriggers/TaskTest.java
 create mode 100644 learning/katas/java/Triggers/Window Accumulation Mode/Window Accumulation Mode/src/org/apache/beam/learning/katas/triggers/windowaccummode/GenerateEvent.java
 create mode 100644 learning/katas/java/Triggers/Window Accumulation Mode/Window Accumulation Mode/src/org/apache/beam/learning/katas/triggers/windowaccummode/Task.java
 create mode 100644 learning/katas/java/Triggers/Window Accumulation Mode/Window Accumulation Mode/task.html
 create mode 100644 learning/katas/java/Triggers/Window Accumulation Mode/Window Accumulation Mode/test/org/apache/beam/learning/katas/triggers/windowaccummode/TaskTest.java
 create mode 100644 learning/katas/java/Windowing/Adding Timestamp/ParDo/src/org/apache/beam/learning/katas/windowing/addingtimestamp/pardo/Event.java
 create mode 100644 learning/katas/java/Windowing/Adding Timestamp/ParDo/src/org/apache/beam/learning/katas/windowing/addingtimestamp/pardo/Task.java
 create mode 100644 learning/katas/java/Windowing/Adding Timestamp/ParDo/task.html
 create mode 100644 learning/katas/java/Windowing/Adding Timestamp/ParDo/test/org/apache/beam/learning/katas/windowing/addingtimestamp/pardo/TaskTest.java
 create mode 100644 learning/katas/java/Windowing/Adding Timestamp/WithTimestamps/src/org/apache/beam/learning/katas/windowing/addingtimestamp/withtimestamps/Event.java
 create mode 100644 learning/katas/java/Windowing/Adding Timestamp/WithTimestamps/src/org/apache/beam/learning/katas/windowing/addingtimestamp/withtimestamps/Task.java
 create mode 100644 learning/katas/java/Windowing/Adding Timestamp/WithTimestamps/task.html
 create mode 100644 learning/katas/java/Windowing/Adding Timestamp/WithTimestamps/test/org/apache/beam/learning/katas/windowing/addingtimestamp/withtimestamps/TaskTest.java
 create mode 100644 learning/katas/java/Windowing/Fixed Time Window/Fixed Time Window/src/org/apache/beam/learning/katas/windowing/fixedwindow/Task.java
 create mode 100644 learning/katas/java/Windowing/Fixed Time Window/Fixed Time Window/task.html
 create mode 100644 learning/katas/java/Windowing/Fixed Time Window/Fixed Time Window/test/org/apache/beam/learning/katas/windowing/fixedwindow/TaskTest.java
 create mode 100644 learning/katas/java/Windowing/Fixed Time Window/Fixed Time Window/test/org/apache/beam/learning/katas/windowing/fixedwindow/WindowedEvent.java
 create mode 100644 learning/katas/java/build.gradle
 create mode 100644 learning/katas/java/gradle/wrapper/gradle-wrapper.jar
 create mode 100644 learning/katas/java/gradle/wrapper/gradle-wrapper.properties
 create mode 100755 learning/katas/java/gradlew
 copy gradlew.bat => learning/katas/java/gradlew.bat (100%)
 create mode 100644 learning/katas/java/settings.gradle
 create mode 100644 learning/katas/java/util/resources/log4j2.xml
 create mode 100644 learning/katas/java/util/src/org/apache/beam/learning/katas/util/Log.java
 create mode 100644 learning/katas/java/util/test/org/apache/beam/learning/katas/util/ContainsKvs.java
 create mode 100644 learning/katas/java/util/test/org/apache/beam/learning/katas/util/KvMatcher.java
 create mode 100644 learning/katas/python/.idea/study_project.xml
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Count/task.html
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Count/task.py
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Count/tests.py
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Largest/task.html
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Largest/task.py
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Largest/tests.py
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Mean/task.html
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Mean/task.py
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Mean/tests.py
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Smallest/task.html
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Smallest/task.py
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Smallest/tests.py
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Sum/task.html
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Sum/task.py
 create mode 100644 learning/katas/python/Common Transforms/Aggregation/Sum/tests.py
 create mode 100644 learning/katas/python/Common Transforms/Filter/Filter/task.html
 create mode 100644 learning/katas/python/Common Transforms/Filter/Filter/task.py
 create mode 100644 learning/katas/python/Common Transforms/Filter/Filter/tests.py
 create mode 100644 learning/katas/python/Common Transforms/Filter/ParDo/task.html
 create mode 100644 learning/katas/python/Common Transforms/Filter/ParDo/task.py
 create mode 100644 learning/katas/python/Common Transforms/Filter/ParDo/tests.py
 create mode 100644 learning/katas/python/Core Transforms/Branching/Branching/task.html
 create mode 100644 learning/katas/python/Core Transforms/Branching/Branching/task.py
 create mode 100644 learning/katas/python/Core Transforms/Branching/Branching/tests.py
 create mode 100644 learning/katas/python/Core Transforms/CoGroupByKey/CoGroupByKey/task.html
 create mode 100644 learning/katas/python/Core Transforms/CoGroupByKey/CoGroupByKey/task.py
 create mode 100644 learning/katas/python/Core Transforms/CoGroupByKey/CoGroupByKey/tests.py
 create mode 100644 learning/katas/python/Core Transforms/Combine/Combine PerKey/task.html
 create mode 100644 learning/katas/python/Core Transforms/Combine/Combine PerKey/task.py
 create mode 100644 learning/katas/python/Core Transforms/Combine/Combine PerKey/tests.py
 create mode 100644 learning/katas/python/Core Transforms/Combine/CombineFn/task.html
 create mode 100644 learning/katas/python/Core Transforms/Combine/CombineFn/task.py
 create mode 100644 learning/katas/python/Core Transforms/Combine/CombineFn/tests.py
 create mode 100644 learning/katas/python/Core Transforms/Combine/Simple Function/task.html
 create mode 100644 learning/katas/python/Core Transforms/Combine/Simple Function/task.py
 create mode 100644 learning/katas/python/Core Transforms/Combine/Simple Function/tests.py
 create mode 100644 learning/katas/python/Core Transforms/Composite Transform/Composite Transform/task.html
 create mode 100644 learning/katas/python/Core Transforms/Composite Transform/Composite Transform/task.py
 create mode 100644 learning/katas/python/Core Transforms/Composite Transform/Composite Transform/tests.py
 create mode 100644 learning/katas/python/Core Transforms/Flatten/Flatten/task.html
 create mode 100644 learning/katas/python/Core Transforms/Flatten/Flatten/task.py
 create mode 100644 learning/katas/python/Core Transforms/Flatten/Flatten/tests.py
 create mode 100644 learning/katas/python/Core Transforms/GroupByKey/GroupByKey/task.html
 create mode 100644 learning/katas/python/Core Transforms/GroupByKey/GroupByKey/task.py
 create mode 100644 learning/katas/python/Core Transforms/GroupByKey/GroupByKey/tests.py
 create mode 100644 learning/katas/python/Core Transforms/Map/FlatMap/task.html
 create mode 100644 learning/katas/python/Core Transforms/Map/FlatMap/task.py
 create mode 100644 learning/katas/python/Core Transforms/Map/FlatMap/tests.py
 create mode 100644 learning/katas/python/Core Transforms/Map/Map/task.html
 create mode 100644 learning/katas/python/Core Transforms/Map/Map/task.py
 create mode 100644 learning/katas/python/Core Transforms/Map/Map/tests.py
 create mode 100644 learning/katas/python/Core Transforms/Map/ParDo OneToMany/task.html
 create mode 100644 learning/katas/python/Core Transforms/Map/ParDo OneToMany/task.py
 create mode 100644 learning/katas/python/Core Transforms/Map/ParDo OneToMany/tests.py
 create mode 100644 learning/katas/python/Core Transforms/Map/ParDo/task.html
 create mode 100644 learning/katas/python/Core Transforms/Map/ParDo/task.py
 create mode 100644 learning/katas/python/Core Transforms/Map/ParDo/tests.py
 create mode 100644 learning/katas/python/Core Transforms/Partition/Partition/task.html
 create mode 100644 learning/katas/python/Core Transforms/Partition/Partition/task.py
 create mode 100644 learning/katas/python/Core Transforms/Partition/Partition/tests.py
 create mode 100644 learning/katas/python/Core Transforms/Side Input/Side Input/task.html
 create mode 100644 learning/katas/python/Core Transforms/Side Input/Side Input/task.py
 create mode 100644 learning/katas/python/Core Transforms/Side Input/Side Input/tests.py
 create mode 100644 learning/katas/python/Core Transforms/Side Output/Side Output/task.html
 create mode 100644 learning/katas/python/Core Transforms/Side Output/Side Output/task.py
 create mode 100644 learning/katas/python/Core Transforms/Side Output/Side Output/tests.py
 create mode 100644 learning/katas/python/Examples/Word Count/Word Count/task.html
 create mode 100644 learning/katas/python/Examples/Word Count/Word Count/task.py
 create mode 100644 learning/katas/python/Examples/Word Count/Word Count/tests.py
 create mode 100644 learning/katas/python/IO/Built-in IOs/Built-in IOs/task.html
 create mode 100644 learning/katas/python/IO/Built-in IOs/Built-in IOs/task.py
 create mode 100644 learning/katas/python/IO/Built-in IOs/Built-in IOs/tests.py
 create mode 100644 learning/katas/python/IO/TextIO/ReadFromText/countries.txt
 create mode 100644 learning/katas/python/IO/TextIO/ReadFromText/task.html
 create mode 100644 learning/katas/python/IO/TextIO/ReadFromText/task.py
 create mode 100644 learning/katas/python/IO/TextIO/ReadFromText/tests.py
 create mode 100644 learning/katas/python/Introduction/Hello Beam/Hello Beam/task.html
 create mode 100644 learning/katas/python/Introduction/Hello Beam/Hello Beam/task.py
 create mode 100644 learning/katas/python/Introduction/Hello Beam/Hello Beam/tests.py
 create mode 100644 learning/katas/python/README.md
 create mode 100644 learning/katas/python/log_elements.py
 create mode 100644 learning/katas/python/requirements.txt
 create mode 100644 learning/katas/python/test_helper.py
 create mode 100644 model/pipeline/src/main/proto/schema.proto
 create mode 100644 project-mappings
 create mode 100644 runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/AvroCoderRegistrar.java
 create mode 100644 runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/AvroCoderTranslator.java
 create mode 100644 runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/ReshuffleTranslation.java
 create mode 100644 runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/renderer/PipelineDotRenderer.java
 create mode 100644 runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/renderer/PortablePipelineDotRenderer.java
 create mode 100644 runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/renderer/package-info.java
 create mode 100644 runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/ReshuffleTranslationTest.java
 create mode 100644 runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/SchemaTranslationTest.java
 create mode 100644 runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/renderer/PipelineDotRendererTest.java
 create mode 100644 runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/renderer/PortablePipelineDotRendererTest.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/BundleFactory.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/BundleFactoryOutputReceiverFactory.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/BundleProcessor.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/CommittedBundle.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/CommittedResult.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/CompletionCallback.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/CopyOnAccessInMemoryStateInternals.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/DirectGroupByKey.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/DirectMetrics.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/DirectStateAndTimers.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/DirectTimerInternals.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/DirectTransformExecutor.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/DisplayDataValidator.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/EmptyInputProvider.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/EvaluationContext.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/EvaluationContextStepStateAndTimersProvider.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/ExecutorServiceFactory.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/ExecutorServiceParallelExecutor.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/FlattenEvaluatorFactory.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/GroupAlsoByWindowEvaluatorFactory.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/GroupByKeyOnlyEvaluatorFactory.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/ImmutableListBundleFactory.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/ImpulseEvaluatorFactory.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/PCollectionViewWindow.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/PCollectionViewWriter.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/PassthroughTransformEvaluator.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/PipelineExecutor.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/PortableGraph.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/QuiescenceDriver.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/ReferenceRunner.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/RemoteStageEvaluatorFactory.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/RootInputProvider.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/RootProviderRegistry.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/SourceShard.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/SplittableRemoteStageEvaluatorFactory.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/StepAndKey.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/StepStateAndTimers.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/StepTransformResult.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/TransformEvaluator.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/TransformEvaluatorFactory.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/TransformEvaluatorRegistry.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/TransformExecutor.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/TransformExecutorFactory.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/TransformExecutorService.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/TransformExecutorServices.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/TransformResult.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/UncommittedBundle.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/WatermarkCallbackExecutor.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/WindowEvaluatorFactory.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/artifact/LocalArtifactStagingLocation.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/artifact/LocalFileSystemArtifactRetrievalService.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/artifact/LocalFileSystemArtifactStagerService.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/artifact/UnsupportedArtifactRetrievalService.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/artifact/package-info.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/job/PreparingJob.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/job/ReferenceRunnerJobServer.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/job/ReferenceRunnerJobService.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/job/package-info.java
 delete mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/package-info.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/BundleFactoryOutputReceiverFactoryTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/CommittedResultTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/CopyOnAccessInMemoryStateInternalsTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/DirectMetricsTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/DirectTimerInternalsTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/DirectTransformExecutorTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/EvaluationContextTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/ExecutableGraphBuilder.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/FlattenEvaluatorFactoryTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/GroupByKeyOnlyEvaluatorFactoryTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/ImmutableListBundleFactoryTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/ImpulseEvaluatorFactoryTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/MockClock.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/PortableGraphTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/ReferenceRunnerTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/RemoteStageEvaluatorFactoryTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/StepTransformResultTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/TransformExecutorServicesTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/WatermarkCallbackExecutorTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/WindowEvaluatorFactoryTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/artifact/LocalArtifactStagingLocationTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/artifact/LocalFileSystemArtifactRetrievalServiceTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/artifact/LocalFileSystemArtifactStagerServiceTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/artifact/UnsupportedArtifactRetrievalServiceTest.java
 delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/job/ReferenceRunnerJobServiceTest.java
 create mode 100644 runners/flink/1.5/build.gradle
 copy runners/flink/{1.7 => 1.5}/job-server-container/build.gradle (100%)
 create mode 100644 runners/flink/1.5/job-server/build.gradle
 create mode 100644 runners/flink/1.5/src/main/java/org/apache/beam/runners/flink/translation/types/CoderTypeSerializer.java
 rename runners/flink/{ => 1.5}/src/main/java/org/apache/beam/runners/flink/translation/types/EncodedValueSerializer.java (100%)
 rename runners/flink/{ => 1.5}/src/test/java/org/apache/beam/runners/flink/streaming/FlinkBroadcastStateInternalsTest.java (100%)
 rename runners/flink/{ => 1.5}/src/test/java/org/apache/beam/runners/flink/streaming/FlinkStateInternalsTest.java (100%)
 rename runners/flink/{ => 1.5}/src/test/java/org/apache/beam/runners/flink/translation/types/CoderTypeSerializerTest.java (100%)
 create mode 100644 runners/flink/1.8/build.gradle
 copy runners/flink/{1.7 => 1.8}/job-server-container/build.gradle (100%)
 create mode 100644 runners/flink/1.8/job-server/build.gradle
 create mode 100644 runners/flink/1.8/src/main/java/org/apache/beam/runners/flink/translation/types/CoderTypeSerializer.java
 create mode 100644 runners/flink/1.8/src/main/java/org/apache/beam/runners/flink/translation/types/EncodedValueSerializer.java
 create mode 100644 runners/flink/1.8/src/test/java/org/apache/beam/runners/flink/streaming/FlinkBroadcastStateInternalsTest.java
 create mode 100644 runners/flink/1.8/src/test/java/org/apache/beam/runners/flink/streaming/FlinkStateInternalsTest.java
 create mode 100644 runners/flink/1.8/src/test/java/org/apache/beam/runners/flink/translation/types/CoderTypeSerializerTest.java
 delete mode 100644 runners/flink/build.gradle
 delete mode 100644 runners/flink/job-server-container/build.gradle
 delete mode 100644 runners/flink/job-server/build.gradle
 delete mode 100644 runners/flink/src/main/java/org/apache/beam/runners/flink/translation/types/CoderTypeSerializer.java
 delete mode 100644 runners/flink/src/main/java/org/apache/beam/runners/flink/translation/types/InspectableByteArrayOutputStream.java
 delete mode 100644 runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/UnboundedSocketSource.java
 delete mode 100644 runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/state/KeyGroupCheckpointedOperator.java
 delete mode 100644 runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/state/KeyGroupRestoringOperator.java
 create mode 100644 runners/flink/src/test/java/org/apache/beam/runners/flink/FlinkRunnerResultTest.java
 create mode 100644 runners/flink/src/test/java/org/apache/beam/runners/flink/FlinkStreamingPipelineTranslatorTest.java
 create mode 100644 runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/TimerReceiverFactory.java
 create mode 100644 runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/state/InMemoryBagUserStateFactory.java
 create mode 100644 runners/java-fn-execution/src/test/java/org/apache/beam/runners/fnexecution/jobsubmission/JobInvocationTest.java
 create mode 100644 runners/jet-experimental/build.gradle
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/DAGBuilder.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/FailedRunningPipelineResults.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/JetGraphVisitor.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/JetPipelineOptions.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/JetPipelineResult.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/JetRunner.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/JetRunnerRegistrar.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/JetTransformTranslator.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/JetTransformTranslators.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/JetTranslationContext.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/Utils.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/metrics/AbstractMetric.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/metrics/CounterImpl.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/metrics/DistributionImpl.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/metrics/GaugeImpl.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/metrics/JetMetricResults.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/metrics/JetMetricsContainer.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/metrics/package-info.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/package-info.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/processors/AbstractParDoP.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/processors/AssignWindowP.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/processors/BoundedSourceP.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/processors/FlattenP.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/processors/ImpulseP.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/processors/ParDoP.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/processors/StatefulParDoP.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/processors/UnboundedSourceP.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/processors/ViewP.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/processors/WindowGroupP.java
 create mode 100644 runners/jet-experimental/src/main/java/org/apache/beam/runners/jet/processors/package-info.java
 create mode 100644 runners/jet-experimental/src/test/java/org/apache/beam/runners/jet/JetTestRunnerRegistrar.java
 create mode 100644 runners/jet-experimental/src/test/java/org/apache/beam/runners/jet/TestJetRunner.java
 create mode 100644 runners/jet-experimental/src/test/java/org/apache/beam/runners/jet/TestStreamP.java
 delete mode 100644 runners/reference/job-server/build.gradle
 create mode 100644 runners/samza/src/main/java/org/apache/beam/runners/samza/SamzaPortablePipelineOptions.java
 delete mode 100644 runners/samza/src/main/java/org/apache/beam/runners/samza/util/PipelineDotRenderer.java
 delete mode 100644 runners/samza/src/main/java/org/apache/beam/runners/samza/util/PortablePipelineDotRenderer.java
 delete mode 100644 runners/spark/src/main/java/org/apache/beam/runners/spark/translation/SparkAbstractCombineFn.java
 create mode 100644 runners/spark/src/main/java/org/apache/beam/runners/spark/translation/SparkCombineFn.java
 delete mode 100644 runners/spark/src/main/java/org/apache/beam/runners/spark/translation/SparkGlobalCombineFn.java
 delete mode 100644 runners/spark/src/main/java/org/apache/beam/runners/spark/translation/SparkKeyedCombineFn.java
 create mode 100644 runners/spark/src/main/java/org/apache/beam/runners/spark/util/SparkCompat.java
 create mode 100644 runners/spark/src/test/java/org/apache/beam/runners/spark/translation/SparkCombineFnTest.java
 create mode 100644 sdks/go/pkg/beam/combine_test.go
 create mode 100644 sdks/go/pkg/beam/core/runtime/exec/datasource_test.go
 create mode 100644 sdks/go/pkg/beam/runners/spark/spark.go
 create mode 100644 sdks/go/pkg/beam/runners/vet/testpipeline/functions.go
 create mode 100644 sdks/go/pkg/beam/runners/vet/testpipeline/testpipeline.go
 create mode 100644 sdks/go/pkg/beam/runners/vet/testpipeline/testpipeline.shims.go
 create mode 100644 sdks/go/pkg/beam/runners/vet/vet.go
 create mode 100644 sdks/go/pkg/beam/runners/vet/vet_test.go
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/harness/JvmInitializer.java
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/harness/package-info.java
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/io/ShardingFunction.java
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/io/TextRowCountEstimator.java
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/AddFields.java
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/DropFields.java
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/RenameFields.java
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/testing/UsesKms.java
 delete mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/splittabledofn/Backlog.java
 delete mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/splittabledofn/Backlogs.java
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/splittabledofn/Sizes.java
 create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/coders/FloatCoderTest.java
 create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/io/TextRowCountEstimatorTest.java
 create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/transforms/AddFieldsTest.java
 create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/transforms/DropFieldsTest.java
 create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/transforms/RenameFieldsTest.java
 rename sdks/java/core/src/test/{avro => java}/org/apache/beam/sdk/state/StateContextsTest.java (100%)
 create mode 100644 sdks/java/core/src/test/resources/META-INF/services/org.apache.beam.sdk.util.common.ReflectHelpersTest$FakeService
 create mode 100644 sdks/java/extensions/euphoria/src/test/java/org/apache/beam/sdk/extensions/euphoria/core/client/util/SumsTest.java
 create mode 100644 sdks/java/extensions/sorter/src/main/java/org/apache/beam/sdk/extensions/sorter/HadoopExternalSorter.java
 create mode 100644 sdks/java/extensions/sorter/src/main/java/org/apache/beam/sdk/extensions/sorter/NativeExternalSorter.java
 create mode 100644 sdks/java/extensions/sorter/src/main/java/org/apache/beam/sdk/extensions/sorter/NativeFileSorter.java
 create mode 100644 sdks/java/extensions/sorter/src/test/java/org/apache/beam/sdk/extensions/sorter/ExternalSorterBenchmark.java
 create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamTableStatistics.java
 create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rule/BeamJoinAssociateRule.java
 create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rule/BeamJoinPushThroughJoinRule.java
 create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rule/JoinRelOptRuleCall.java
 create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/parquet/GenericRecordReadConverter.java
 create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/parquet/ParquetTable.java
 create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/parquet/ParquetTableProvider.java
 create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/parquet/package-info.java
 create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/seqgen/GenerateSequenceTable.java
 create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/seqgen/GenerateSequenceTableProvider.java
 create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/seqgen/package-info.java
 create mode 100644 sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamIOSourceRelTest.java
 create mode 100644 sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/rule/JoinReorderingTest.java
 create mode 100644 sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigquery/BigQueryRowCountIT.java
 create mode 100644 sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigquery/BigQueryTestTable.java
 create mode 100644 sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigquery/BigQueryTestTableProvider.java
 create mode 100644 sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/meta/provider/parquet/GenericRecordToRowTest.java
 create mode 100644 sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/meta/provider/parquet/ParquetTableReadTest.java
 create mode 100644 sdks/java/extensions/sql/src/test/resources/users.parquet
 create mode 100644 sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/JvmInitializers.java
 create mode 100644 sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/JvmInitializersTest.java
 create mode 100644 sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/dynamodb/AttributeValueCoder.java
 create mode 100644 sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/dynamodb/AttributeValueCoderProviderRegistrar.java
 create mode 100644 sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/dynamodb/AwsClientsProvider.java
 create mode 100644 sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/dynamodb/BasicDynamoDBProvider.java
 create mode 100644 sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/dynamodb/DynamoDBIO.java
 create mode 100644 sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/dynamodb/package-info.java
 create mode 100644 sdks/java/io/amazon-web-services/src/test/java/org/apache/beam/sdk/io/aws/dynamodb/AttributeValueCoderTest.java
 create mode 100644 sdks/java/io/amazon-web-services/src/test/java/org/apache/beam/sdk/io/aws/dynamodb/AwsClientsProviderMock.java
 create mode 100644 sdks/java/io/amazon-web-services/src/test/java/org/apache/beam/sdk/io/aws/dynamodb/DynamoDBIOTest.java
 create mode 100644 sdks/java/io/amazon-web-services/src/test/java/org/apache/beam/sdk/io/aws/dynamodb/DynamoDBIOTestHelper.java
 create mode 100644 sdks/java/io/amazon-web-services2/build.gradle
 create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/dynamodb/AttributeValueCoder.java
 create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/dynamodb/BasicDynamoDbClientProvider.java
 create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/dynamodb/DynamoDBIO.java
 create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/dynamodb/DynamoDbClientProvider.java
 create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/dynamodb/package-info.java
 create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/options/AwsModule.java
 create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/options/AwsOptions.java
 create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/options/AwsPipelineOptionsRegistrar.java
 create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/options/package-info.java
 create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/dynamodb/AttributeValueCoderTest.java
 create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/dynamodb/DynamoDBIOTest.java
 create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/dynamodb/DynamoDBIOTestHelper.java
 create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/dynamodb/DynamoDbClientProviderMock.java
 create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/options/AwsModuleTest.java
 delete mode 100644 sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticSearchIOTestUtils.java
 create mode 100644 sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTestUtils.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQuerySourceDef.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySchemaRetrievalException.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySourceDef.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryTableSourceDef.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableDestinationCoderV3.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableWriteResult.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableWriteResultCoder.java
 create mode 100644 sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryTimePartitioningClusteringIT.java
 create mode 100644 sdks/java/io/hcatalog/src/main/java/org/apache/beam/sdk/io/hcatalog/HCatalogUtils.java
 create mode 100644 sdks/java/io/hcatalog/src/main/java/org/apache/beam/sdk/io/hcatalog/PartitionPollerFn.java
 create mode 100644 sdks/java/io/hcatalog/src/main/java/org/apache/beam/sdk/io/hcatalog/PartitionReaderFn.java
 create mode 100644 sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/BeamSchemaInferenceException.java
 create mode 100644 sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcUtil.java
 create mode 100644 sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/LogicalTypes.java
 create mode 100644 sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/SchemaUtil.java
 create mode 100644 sdks/java/io/jdbc/src/test/java/org/apache/beam/sdk/io/jdbc/JdbcUtilTest.java
 create mode 100644 sdks/java/io/jdbc/src/test/java/org/apache/beam/sdk/io/jdbc/RowWithSchema.java
 create mode 100644 sdks/java/io/jdbc/src/test/java/org/apache/beam/sdk/io/jdbc/SchemaUtilTest.java
 create mode 100644 sdks/java/io/kafka/src/test/java/org/apache/beam/sdk/io/kafka/KafkaIOIT.java
 delete mode 100644 sdks/java/io/kinesis/src/main/java/org/apache/beam/sdk/io/kinesis/KinesisWatermark.java
 create mode 100644 sdks/java/io/kinesis/src/main/java/org/apache/beam/sdk/io/kinesis/WatermarkParameters.java
 create mode 100644 sdks/java/io/kinesis/src/main/java/org/apache/beam/sdk/io/kinesis/WatermarkPolicy.java
 create mode 100644 sdks/java/io/kinesis/src/main/java/org/apache/beam/sdk/io/kinesis/WatermarkPolicyFactory.java
 delete mode 100644 sdks/java/io/kinesis/src/test/java/org/apache/beam/sdk/io/kinesis/KinesisWatermarkTest.java
 create mode 100644 sdks/java/io/kinesis/src/test/java/org/apache/beam/sdk/io/kinesis/WatermarkPolicyTest.java
 delete mode 100644 sdks/java/javadoc/apiclient-docs/package-list
 delete mode 100644 sdks/java/javadoc/avro-docs/package-list
 delete mode 100644 sdks/java/javadoc/bq-docs/package-list
 delete mode 100644 sdks/java/javadoc/datastore-docs/package-list
 delete mode 100644 sdks/java/javadoc/guava-docs/package-list
 delete mode 100644 sdks/java/javadoc/hamcrest-docs/package-list
 delete mode 100644 sdks/java/javadoc/jackson-annotations-docs/package-list
 delete mode 100644 sdks/java/javadoc/jackson-databind-docs/package-list
 delete mode 100644 sdks/java/javadoc/joda-docs/package-list
 delete mode 100644 sdks/java/javadoc/junit-docs/package-list
 delete mode 100644 sdks/java/javadoc/oauth-docs/package-list
 create mode 100644 sdks/python/apache_beam/coders/avro_coder.py
 create mode 100644 sdks/python/apache_beam/coders/avro_coder_test.py
 copy sdks/python/apache_beam/{io/gcp/tests => examples/snippets/transforms}/__init__.py (100%)
 copy sdks/python/apache_beam/{io/gcp/tests => examples/snippets/transforms/element_wise}/__init__.py (100%)
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter_test.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/flat_map_test.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/keys.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/keys_test.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/kvswap.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/kvswap_test.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/map.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/map_test.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/pardo.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/pardo_test.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/partition.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/partition_test.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/regex.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/regex_test.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/to_string.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/to_string_test.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/values.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/values_test.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/with_timestamps.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/transforms/element_wise/with_timestamps_test.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/util.py
 create mode 100644 sdks/python/apache_beam/examples/snippets/util_test.py
 create mode 100644 sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py
 create mode 100644 sdks/python/apache_beam/io/gcp/bigquery_read_perf_test.py
 create mode 100644 sdks/python/apache_beam/io/gcp/bigquery_write_it_test.py
 create mode 100644 sdks/python/apache_beam/io/gcp/bigquery_write_perf_test.py
 create mode 100644 sdks/python/apache_beam/io/iobase_test.py
 create mode 100644 sdks/python/apache_beam/io/mongodbio.py
 create mode 100644 sdks/python/apache_beam/io/mongodbio_it_test.py
 create mode 100644 sdks/python/apache_beam/io/mongodbio_test.py
 create mode 100644 sdks/python/apache_beam/io/utils_test.py
 create mode 100644 sdks/python/apache_beam/runners/portability/flink_runner.py
 delete mode 100644 sdks/python/apache_beam/runners/portability/java_reference_runner_test.py
 create mode 100644 sdks/python/apache_beam/transforms/dofn_lifecycle_test.py
 create mode 100644 sdks/python/apache_beam/transforms/stats.py
 create mode 100644 sdks/python/apache_beam/transforms/stats_test.py
 delete mode 100644 sdks/python/test-suites/dataflow/build.gradle
 create mode 100644 sdks/python/test-suites/dataflow/py2/build.gradle
 create mode 100644 sdks/python/test-suites/dataflow/py37/build.gradle
 create mode 100644 sdks/python/test-suites/direct/py2/build.gradle
 create mode 100644 sdks/python/test-suites/direct/py37/build.gradle
 create mode 100644 sdks/python/test-suites/portable/py2/build.gradle
 create mode 100644 sdks/python/test-suites/portable/py35/build.gradle
 create mode 100644 sdks/python/test-suites/tox/py2/build.gradle
 create mode 100644 vendor/bytebuddy-1_9_3/build.gradle
 delete mode 100644 vendor/grpc-1_13_1/build.gradle
 create mode 100644 vendor/grpc-1_21_0/build.gradle
 delete mode 100644 vendor/guava-20_0/build.gradle
 create mode 100644 vendor/guava-26_0-jre/build.gradle
 create mode 100644 website/src/_posts/2019-03-18-beam-summit-site.md
 create mode 100644 website/src/_posts/2019-05-01-adding-data-sources-to-sql.md
 create mode 100644 website/src/_posts/2019-05-11-beam-summit-europe-2019.md
 create mode 100644 website/src/_posts/2019-05-22-beam-2.13.0.md
 create mode 100644 website/src/_posts/2019-05-30-beam-kata-release.md
 create mode 100644 website/src/_posts/2019-06-11-looping-timers.md
 delete mode 100644 website/src/documentation/dsls/sql/aggregate-functions.md
 create mode 100644 website/src/documentation/dsls/sql/calcite/aggregate-functions.md
 create mode 100644 website/src/documentation/dsls/sql/calcite/data-types.md
 create mode 100644 website/src/documentation/dsls/sql/calcite/lexical-structure.md
 create mode 100644 website/src/documentation/dsls/sql/calcite/overview.md
 create mode 100644 website/src/documentation/dsls/sql/calcite/query-syntax.md
 create mode 100644 website/src/documentation/dsls/sql/calcite/scalar-functions.md
 delete mode 100644 website/src/documentation/dsls/sql/create-external-table.md
 delete mode 100644 website/src/documentation/dsls/sql/data-types.md
 create mode 100644 website/src/documentation/dsls/sql/extensions/create-external-table.md
 create mode 100644 website/src/documentation/dsls/sql/extensions/joins.md
 create mode 100644 website/src/documentation/dsls/sql/extensions/set.md
 create mode 100644 website/src/documentation/dsls/sql/extensions/user-defined-functions.md
 create mode 100644 website/src/documentation/dsls/sql/extensions/windowing-and-triggering.md
 delete mode 100644 website/src/documentation/dsls/sql/joins.md
 delete mode 100644 website/src/documentation/dsls/sql/lexical-structure.md
 delete mode 100644 website/src/documentation/dsls/sql/scalar-functions.md
 delete mode 100644 website/src/documentation/dsls/sql/select.md
 delete mode 100644 website/src/documentation/dsls/sql/set.md
 delete mode 100644 website/src/documentation/dsls/sql/user-defined-functions.md
 delete mode 100644 website/src/documentation/dsls/sql/windowing-and-triggering.md
 create mode 100644 website/src/documentation/patterns/custom-io-patterns.md
 create mode 100644 website/src/documentation/patterns/file-processing-patterns.md
 create mode 100644 website/src/documentation/patterns/overview.md
 create mode 100644 website/src/documentation/patterns/pipeline-option-patterns.md
 create mode 100644 website/src/documentation/patterns/side-input-patterns.md
 create mode 100644 website/src/documentation/runners/jet.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/approximatequantiles.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/approximateunique.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/cogroupbykey.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/combine.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/combinewithcontext.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/count.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/distinct.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/groupbykey.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/groupintobatches.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/latest.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/max.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/mean.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/min.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/sample.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/sum.md
 create mode 100644 website/src/documentation/transforms/java/aggregation/top.md
 create mode 100644 website/src/documentation/transforms/java/element-wise/filter.md
 create mode 100644 website/src/documentation/transforms/java/element-wise/flatmapelements.md
 create mode 100644 website/src/documentation/transforms/java/element-wise/keys.md
 create mode 100644 website/src/documentation/transforms/java/element-wise/kvswap.md
 create mode 100644 website/src/documentation/transforms/java/element-wise/mapelements.md
 create mode 100644 website/src/documentation/transforms/java/element-wise/pardo.md
 create mode 100644 website/src/documentation/transforms/java/element-wise/partition.md
 create mode 100644 website/src/documentation/transforms/java/element-wise/regex.md
 create mode 100644 website/src/documentation/transforms/java/element-wise/reify.md
 create mode 100644 website/src/documentation/transforms/java/element-wise/tostring.md
 create mode 100644 website/src/documentation/transforms/java/element-wise/values.md
 create mode 100644 website/src/documentation/transforms/java/element-wise/withkeys.md
 create mode 100644 website/src/documentation/transforms/java/element-wise/withtimestamps.md
 create mode 100644 website/src/documentation/transforms/java/index.md
 create mode 100644 website/src/documentation/transforms/java/other/create.md
 create mode 100644 website/src/documentation/transforms/java/other/flatten.md
 create mode 100644 website/src/documentation/transforms/java/other/passert.md
 create mode 100644 website/src/documentation/transforms/java/other/view.md
 create mode 100644 website/src/documentation/transforms/java/other/window.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/approximatequantiles.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/approximateunique.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/cogroupbykey.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/combineglobally.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/combinewithcontext.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/count.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/distinct.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/groupbykey.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/groupintobatches.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/latest.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/max.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/mean.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/min.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/sample.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/sum.md
 create mode 100644 website/src/documentation/transforms/python/aggregation/top.md
 create mode 100644 website/src/documentation/transforms/python/element-wise/filter.md
 create mode 100644 website/src/documentation/transforms/python/element-wise/flatmap.md
 create mode 100644 website/src/documentation/transforms/python/element-wise/keys.md
 create mode 100644 website/src/documentation/transforms/python/element-wise/kvswap.md
 create mode 100644 website/src/documentation/transforms/python/element-wise/map.md
 create mode 100644 website/src/documentation/transforms/python/element-wise/pardo.md
 create mode 100644 website/src/documentation/transforms/python/element-wise/partition.md
 create mode 100644 website/src/documentation/transforms/python/element-wise/regex.md
 create mode 100644 website/src/documentation/transforms/python/element-wise/reify.md
 create mode 100644 website/src/documentation/transforms/python/element-wise/tostring.md
 create mode 100644 website/src/documentation/transforms/python/element-wise/values.md
 create mode 100644 website/src/documentation/transforms/python/element-wise/withkeys.md
 create mode 100644 website/src/documentation/transforms/python/element-wise/withtimestamps.md
 create mode 100644 website/src/documentation/transforms/python/index.md
 create mode 100644 website/src/documentation/transforms/python/other/create.md
 create mode 100644 website/src/documentation/transforms/python/other/flatten.md
 create mode 100644 website/src/documentation/transforms/python/other/passert.md
 create mode 100644 website/src/documentation/transforms/python/other/reshuffle.md
 create mode 100644 website/src/documentation/transforms/python/other/view.md
 create mode 100644 website/src/documentation/transforms/python/other/windowinto.md
 create mode 100644 website/src/images/blog/beam-kata/beam-kata-intellij-edu-1.png
 create mode 100644 website/src/images/blog/beam-kata/beam-kata-intellij-edu-2.png
 create mode 100644 website/src/images/blog/beam-kata/beam-kata-pycharm-edu-1.png
 create mode 100644 website/src/images/blog/beam-kata/beam-kata-pycharm-edu-2.png
 create mode 100644 website/src/images/logos/runners/jet.png


Mime
View raw message