spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean R. Owen (Jira)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-29292) Fix internal usages of mutable collection for Seq in 2.13
Date Fri, 01 May 2020 02:47:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-29292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17097132#comment-17097132
] 

Sean R. Owen commented on SPARK-29292:
--------------------------------------

Yep it is going to be a quite large internal change to even compile, and will change runtime
signature of most of the Scala API. That much will still be mostly source-compatible for programs,
and binary compatibility has always been not guaranteed across Scala versions. I think we
will largely have to 'take the hit' to perf where some of these changes introduce a copy,
as the change will otherwise be very hard to manage across 2.12 vs 2.13. Maybe we have to
rewrite some key internal code to avoid a copy where it matters. I take back the comment above;
ArrayBuffer.toSeq almost certainly can't return an immutable wrapper as it can't let underlying
changes be reflected in the 'immutable' .toSeq.

> Fix internal usages of mutable collection for Seq in 2.13
> ---------------------------------------------------------
>
>                 Key: SPARK-29292
>                 URL: https://issues.apache.org/jira/browse/SPARK-29292
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Spark Core
>    Affects Versions: 3.0.0
>            Reporter: Sean R. Owen
>            Assignee: Sean R. Owen
>            Priority: Minor
>
> Kind of related to https://issues.apache.org/jira/browse/SPARK-27681, but a simpler subset.

> In 2.13, a mutable collection can't be returned as a {{scala.collection.Seq}}. It's easy
enough to call .toSeq on these as that still works on 2.12.
> {code}
> [ERROR] [Error] /Users/seanowen/Documents/spark_2.13/core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala:467:
type mismatch;
>  found   : Seq[String] (in scala.collection) 
>  required: Seq[String] (in scala.collection.immutable) 
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message