beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-626) AvroCoder not deserializing correctly in Kryo
Date Tue, 01 Nov 2016 15:40:58 GMT

    [ https://issues.apache.org/jira/browse/BEAM-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625734#comment-15625734
] 

ASF GitHub Bot commented on BEAM-626:
-------------------------------------

GitHub user lukecwik opened a pull request:

    https://github.com/apache/incubator-beam/pull/1246

    [BEAM-626] Changes in AvroCoder serialization so it can serialize in Kryo

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
    
     - [x] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [x] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [ ] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
    
    ---


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/lukecwik/incubator-beam fix941

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/1246.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1246
    
----
commit 1f770d400ca9e5506199783551c0780a1c46f77e
Author: Aviem Zur <aviemzur@gmail.com>
Date:   2016-09-08T08:21:41Z

    Changes in AvroCoder serialization so it can serialize in Kryo

commit 3df987f76467c1e78ce3738a0f2c0df9cd94f82c
Author: Luke Cwik <lcwik@google.com>
Date:   2016-10-31T18:38:45Z

    !fixup Swap to use Serializable ThreadLocal with empty data block instead of memoization.

commit 3358d3d6cc036cb06fa37c6c2547909586bf048f
Author: Luke Cwik <lcwik@google.com>
Date:   2016-10-31T20:21:31Z

    fixup! Fix the memoization of the schema string to use readResolve/writeReplace with final
fields for Java. Kryo ignores readResolve/writeReplace and serializes the Schema object directly.

commit f92c39f14b49556f8e67c22c408520af0c204cce
Author: Luke Cwik <lcwik@google.com>
Date:   2016-10-31T20:24:49Z

    fixup! minor whitespace clean up.

commit a5f25e2e9edf9fe0c1fb59e594d5b450062ba77f
Author: Luke Cwik <lcwik@google.com>
Date:   2016-10-31T20:36:14Z

    fixup! Add FB suppression on limitation of FB with Serializable objects containing writeReplace.

----


> AvroCoder not deserializing correctly in Kryo
> ---------------------------------------------
>
>                 Key: BEAM-626
>                 URL: https://issues.apache.org/jira/browse/BEAM-626
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core
>            Reporter: Aviem Zur
>            Assignee: Aviem Zur
>            Priority: Minor
>
> Unlike with Java serialization, when deserializing AvroCoder using Kryo, the resulting
AvroCoder is missing all of its transient fields.
> The reason it works with Java serialization is because of the usage of writeReplace and
readResolve, which Kryo does not adhere to.
> In ProtoCoder for example there are also unserializable members, the way it is solved
there is lazy initializing these members via their getters, so they are initialized in the
deserialized object on first call to the member.
> It seems AvroCoder is the only class in Beam to use writeReplace convention.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message