spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wenchen Fan <cloud0...@gmail.com>
Subject which classes/methods are considered as private in Spark?
Date Wed, 14 Nov 2018 02:25:39 GMT
Hi all,

Recently I updated the MiMa exclusion rules, and found MiMa tracks some
private classes/methods unexpectedly.

Note that, "private" here means that, we have no guarantee about
compatibility. We don't provide documents and users need to take the risk
when using them.

In the API document, it has some obvious private classes, e.g.
https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.serializer.DummySerializerInstance
, which is not expected either.

I looked around and can't find a clear definition of "private" in Spark.

AFAIK, we have several rules:
1. everything which is really private that end users can't access, e.g.
package private classes, private methods, etc.
2. classes under certain packages. I don't know if we have a list, the
catalyst package is considered as a private package.
3. everything which has a @Private annotation.

I'm sending this email to collect more feedback, and hope we can come up
with a clear definition about what is "private".

Thanks,
Wenchen

Mime
View raw message