flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "SmedbergM (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6866) ClosureCleaner.clean fails for scala's JavaConverters wrapper classes
Date Tue, 27 Jun 2017 16:42:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16065091#comment-16065091
] 

SmedbergM commented on FLINK-6866:
----------------------------------

A Map[K,V] is already Serializable, as is java.util.Map<K,V> -- it's just that before
2.12 scala.collection.convert.Wrappers.MapWrapper didn't inherit from the Serializable interface.
I'm not sure there's a non-hacky (i.e. explicitly checking whether the classname begins with
scala.collections.convert) way to get around this, because ObjectOutputStream checks whether
each field inherits from Serializable. Unfortunately, the scala library writers didn't give
the MapWrapper public access to its underlying wrapped Map.

I don't think the details of the particular code that we encountered this in is all that relevant;
in Scala, it's idiomatic to create immutable Maps and call `.asJava` on them when a library
(like Flink) requires a java.util.Map. Of course, one can avoid having the wrapper by directly
constructing the java.util.Map as in the MWE, but then one has all the mutability and concurrency
worries that good idiomatic Scala lets me not think about.

> ClosureCleaner.clean fails for scala's JavaConverters wrapper classes
> ---------------------------------------------------------------------
>
>                 Key: FLINK-6866
>                 URL: https://issues.apache.org/jira/browse/FLINK-6866
>             Project: Flink
>          Issue Type: Bug
>          Components: DataStream API, Scala API
>    Affects Versions: 1.2.0, 1.3.0
>         Environment: Scala 2.10.6, Scala 2.11.11
> Does not appear using Scala 2.12
>            Reporter: SmedbergM
>
> MWE: https://github.com/SmedbergM/ClosureCleanerBug
> MWE console output: https://gist.github.com/SmedbergM/ce969e6e8540da5b59c7dd921a496dc5



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message