spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean R. Owen (Jira)" <>
Subject [jira] [Commented] (SPARK-29292) Fix internal usages of mutable collection for Seq in 2.13
Date Fri, 01 May 2020 02:47:00 GMT


Sean R. Owen commented on SPARK-29292:

Yep it is going to be a quite large internal change to even compile, and will change runtime
signature of most of the Scala API. That much will still be mostly source-compatible for programs,
and binary compatibility has always been not guaranteed across Scala versions. I think we
will largely have to 'take the hit' to perf where some of these changes introduce a copy,
as the change will otherwise be very hard to manage across 2.12 vs 2.13. Maybe we have to
rewrite some key internal code to avoid a copy where it matters. I take back the comment above;
ArrayBuffer.toSeq almost certainly can't return an immutable wrapper as it can't let underlying
changes be reflected in the 'immutable' .toSeq.

> Fix internal usages of mutable collection for Seq in 2.13
> ---------------------------------------------------------
>                 Key: SPARK-29292
>                 URL:
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Spark Core
>    Affects Versions: 3.0.0
>            Reporter: Sean R. Owen
>            Assignee: Sean R. Owen
>            Priority: Minor
> Kind of related to, but a simpler subset.

> In 2.13, a mutable collection can't be returned as a {{scala.collection.Seq}}. It's easy
enough to call .toSeq on these as that still works on 2.12.
> {code}
> [ERROR] [Error] /Users/seanowen/Documents/spark_2.13/core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala:467:
type mismatch;
>  found   : Seq[String] (in scala.collection) 
>  required: Seq[String] (in scala.collection.immutable) 
> {code}

This message was sent by Atlassian Jira

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message