spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <>
Subject MIMA Compatiblity Checks
Date Sun, 08 Jun 2014 19:40:04 GMT
Hey All,

Some people may have noticed PR failures due to binary compatibility
checks. We've had these enabled in several of the sub-modules since
the 0.9.0 release but we've turned them on in Spark core post 1.0.0
which has much higher churn.

The checks are based on the "migration manager" tool from Typesafe.
One issue is that tool doesn't support package-private declarations of
classes or methods. Prashant Sharma has built instrumentation that
adds partial support for package-privacy (via a workaround) but since
there isn't really native support for this in MIMA we are still
finding cases in which we trigger false positives.

In the next week or two we'll make it a priority to handle more of
these false-positive cases. In the mean time users can add manual
excludes to:


to avoid triggering warnings for certain issues.

This is definitely annoying - sorry about that. Unfortunately we are
the first open source Scala project to ever do this, so we are dealing
with uncharted territory.

Longer term I'd actually like to see us just write our own sbt-based
tool to do this in a better way (we've had trouble trying to extend
MIMA itself, it e.g. has copy-pasted code in it from an old version of
the scala compiler). If someone in the community is a Scala fan and
wants to take that on, I'm happy to give more details.

- Patrick

View raw message