beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From echauc...@apache.org
Subject [beam] branch spark-runner_structured-streaming updated (46ed555 -> 620a27a)
Date Thu, 24 Oct 2019 10:18:04 GMT
This is an automated email from the ASF dual-hosted git repository.

echauchot pushed a change to branch spark-runner_structured-streaming
in repository https://gitbox.apache.org/repos/asf/beam.git.


 discard 46ed555  Remove Encoders based on kryo now that we call Beam coders in the runner
 discard 25d0401  Fix: Remove generic hack of using object. Use actual Coder encodedType in
Encoders
 discard ebc53fd  Remove unneeded cast
 discard ad29daf  Use beam encoders also in the output of the source translation
 discard 507bbd8  Fix javadoc
 discard c980d4c  Fix typo
 discard fb3aa34  Add missing windowedValue Encoder call in Pardo
 discard ee2c0e6  Apply spotless
 discard 31c91a9  Apply new Encoders to Pardo. Replace Tuple2Coder with MultiOutputCoder to
deal with multiple output to use in Spark Encoder for DoFnRunner
 discard 868204f  Apply new Encoders to GroupByKey
 discard 30c662a  Create a Tuple2Coder to encode scale tuple2
 discard d093ffe  Apply spotless
 discard 6edcfa2  Apply new Encoders to AggregatorCombiner
 discard 5beb435  Apply new Encoders to Window assign translation
 discard ab7d24c  Ignore long time failing test: SparkMetricsSinkTest
 discard 3ac3c71  Improve performance of source: the mapper already calls windowedValueCoder.decode,
no need to call it also in the Spark encoder
 discard bcbb697  Apply new Encoders to Read source
 discard aa25e85  Apply new Encoders to CombinePerKey
 discard f0f2078  Catch Exception instead of IOException because some coders to not throw
Exceptions at all (e.g.VoidCoder)
 discard 3a333fb  Put Encoders expressions serializable
 discard cfdf4a4  Improve exceptions catching
 discard b879123  Apply spotless and checkstyle and add javadocs
 discard 0fe6f9b  Add an assert of equality in the encoders test
 discard d8b8b42  Fix generated code: uniform exceptions catching, fix parenthesis and variable
declarations
 discard ef69410  Fix equal and hashcode
 discard 4351304  Remove example code
 discard c4a4464  Remove lazy init of beam coder because there is no generic way on instanciating
a beam coder
 discard 91e923c  Cast coder instanciated by reflection
 discard 723c004  Add try catch around reflexion call in lazy init of beam coder
 discard 8bbf991  Fix beam coder lazy init using reflexion
 discard 959664f  Fix getting the output value in code generation
 discard 668227b  Fix ExpressionEncoder generated code: typos, try catch, fqcn
 discard cbd7c2b  Fix warning in coder construction by reflexion
 discard 2c94eef  Fix call to scala Fucntion1 in coder lazy init
 discard a758985  Lazy init coder because coder instance cannot be interpolated by catalyst
 discard b11e100  Fix code generation in Beam coder wrapper
 discard 2bf4cd9  Add a simple spark native test to test Beam coders wrapping into Spark Encoders
 discard e96af88  Fix visibility of serializer and deserializer
 discard 23735e4  Conform to spark ExpressionEncoders: pass classTags, implement scala Product,
pass children from within the ExpressionEncoder
 discard a5d49f5  Fix scala Product in Encoders to avoid StackEverflow
 discard 95fd272  type erasure: spark encoders require a Class<T>, pass Object and cast
to Class<T>
 discard 84f2cbd9 Fix EncoderHelpers.fromBeamCoder() visibility
 discard d613d6b  Wrap Beam Coders into Spark Encoders using ExpressionEncoder: deserialization
part
 discard 031754c  Wrap Beam Coders into Spark Encoders using ExpressionEncoder: serialization
part
 discard c350188  Use "sparkMaster" in local mode to obtain number of shuffle partitions +
spotless apply
 discard a524036  Improve Pardo translation performance: avoid calling a filter transform
when there is only one output tag
 discard 0cedc7a  Add a TODO on perf improvement of Pardo translation
     new 22d6466  Improve Pardo translation performance: avoid calling a filter transform
when there is only one output tag
     new 20d5bbd  Use "sparkMaster" in local mode to obtain number of shuffle partitions +
spotless apply
     new a5c7da3  Wrap Beam Coders into Spark Encoders using ExpressionEncoder: serialization
part
     new 5fa6331  Wrap Beam Coders into Spark Encoders using ExpressionEncoder: deserialization
part
     new c9e3534  type erasure: spark encoders require a Class<T>, pass Object and cast
to Class<T>
     new fff5092  Fix scala Product in Encoders to avoid StackEverflow
     new 2aaf07a  Conform to spark ExpressionEncoders: pass classTags, implement scala Product,
pass children from within the ExpressionEncoder, fix visibilities
     new e4478ff  Add a simple spark native test to test Beam coders wrapping into Spark Encoders
     new d5645ff  Fix code generation in Beam coder wrapper
     new e6b68a8  Lazy init coder because coder instance cannot be interpolated by catalyst
     new fdba22d  Fix warning in coder construction by reflexion
     new 8b07ec8  Fix ExpressionEncoder generated code: typos, try catch, fqcn
     new d7c9a4a  Fix getting the output value in code generation
     new 0cf2c87  Fix beam coder lazy init using reflexion: use .clas + try catch + cast
     new 50060a8  Remove lazy init of beam coder because there is no generic way on instanciating
a beam coder
     new ca01777  Remove example code
     new f48067b  Fix equal and hashcode
     new 34e8aa8  Fix generated code: uniform exceptions catching, fix parenthesis and variable
declarations
     new 78b2d22  Add an assert of equality in the encoders test
     new c6f2ac9  Apply spotless and checkstyle and add javadocs
     new 72c267c  Wrap exceptions in UserCoderExceptions
     new c8bfcf3  Put Encoders expressions serializable
     new c33fdda  Catch Exception instead of IOException because some coders to not throw
Exceptions at all (e.g.VoidCoder)
     new 7d456b4  Apply new Encoders to CombinePerKey
     new 3cc256e  Apply new Encoders to Read source
     new 68d3d67  Improve performance of source: the mapper already calls windowedValueCoder.decode,
no need to call it also in the Spark encoder
     new c48d032  Ignore long time failing test: SparkMetricsSinkTest
     new 7f1060a  Apply new Encoders to Window assign translation
     new 29f7e93  Apply new Encoders to AggregatorCombiner
     new 21accab  Create a Tuple2Coder to encode scala tuple2
     new 039f58a  Apply new Encoders to GroupByKey
     new c5e78a0  Apply new Encoders to Pardo. Replace Tuple2Coder with MultiOutputCoder to
deal with multiple output to use in Spark Encoder for DoFnRunner
     new 62a87b6  Apply spotless, fix typo and javadoc
     new 6a27839  Use beam encoders also in the output of the source translation
     new 27ef6de  Remove unneeded cast
     new 824b344  Fix: Remove generic hack of using object. Use actual Coder encodedType in
Encoders
     new 620a27a  Remove Encoders based on kryo now that we call Beam coders in the runner

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (46ed555)
            \
             N -- N -- N   refs/heads/spark-runner_structured-streaming (620a27a)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 37 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:


Mime
View raw message