spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean R. Owen (Jira)" <>
Subject [jira] [Created] (SPARK-30132) Scala 2.13 compile errors from Hadoop LocalFileSystem subclasses
Date Thu, 05 Dec 2019 01:13:00 GMT
Sean R. Owen created SPARK-30132:

             Summary: Scala 2.13 compile errors from Hadoop LocalFileSystem subclasses
                 Key: SPARK-30132
             Project: Spark
          Issue Type: Sub-task
          Components: Spark Core
    Affects Versions: 3.0.0
            Reporter: Sean R. Owen

A few classes in our test code extend Hadoop's LocalFileSystem. Scala 2.13 returns a compile
error here - not for the Spark code, but because the Hadoop code (it says) illegally overrides
appendFile() with slightly different generic types in its return value. This code is valid
Java, evidently, and the code actually doesn't define any generic types, so, I even wonder
if it's a scalac bug.

So far I don't see a workaround for this.

This only affects the Hadoop 3.2 build, in that it comes up with respect to a method new in
Hadoop 3. (There is actually another instance of a similar problem that affects Hadoop 2,
but I can see a tiny hack workaround for it).

This message was sent by Atlassian Jira

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message