spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <>
Subject Re: MIMA Compatiblity Checks
Date Thu, 10 Jul 2014 07:33:09 GMT
You can take a look at


On Thu, Jul 10, 2014 at 12:21 AM, Liu, Raymond <>

> so how to run the check locally?
> On master tree, sbt mimaReportBinaryIssues Seems to lead to a lot of
> errors reported. Do we need to modify SparkBuilder.scala etc to run it
> locally? Could not figure out how Jekins run the check on its console
> outputs.
> Best Regards,
> Raymond Liu
> -----Original Message-----
> From: Patrick Wendell []
> Sent: Monday, June 09, 2014 3:40 AM
> To:
> Subject: MIMA Compatiblity Checks
> Hey All,
> Some people may have noticed PR failures due to binary compatibility
> checks. We've had these enabled in several of the sub-modules since the
> 0.9.0 release but we've turned them on in Spark core post 1.0.0 which has
> much higher churn.
> The checks are based on the "migration manager" tool from Typesafe.
> One issue is that tool doesn't support package-private declarations of
> classes or methods. Prashant Sharma has built instrumentation that adds
> partial support for package-privacy (via a workaround) but since there
> isn't really native support for this in MIMA we are still finding cases in
> which we trigger false positives.
> In the next week or two we'll make it a priority to handle more of these
> false-positive cases. In the mean time users can add manual excludes to:
> project/MimaExcludes.scala
> to avoid triggering warnings for certain issues.
> This is definitely annoying - sorry about that. Unfortunately we are the
> first open source Scala project to ever do this, so we are dealing with
> uncharted territory.
> Longer term I'd actually like to see us just write our own sbt-based tool
> to do this in a better way (we've had trouble trying to extend MIMA itself,
> it e.g. has copy-pasted code in it from an old version of the scala
> compiler). If someone in the community is a Scala fan and wants to take
> that on, I'm happy to give more details.
> - Patrick

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message