spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <>
Subject [jira] [Commented] (SPARK-1439) Aggregate Scaladocs across projects
Date Wed, 16 Apr 2014 17:10:15 GMT


Sean Owen commented on SPARK-1439:

I had a run at this today. First I tried Maven-based formulas, but didn't quite do the trick.
I made some progress with unidoc although not all the way. Maybe an SBT expert can help me
figure how to finish it.


This works, but, generates *javadoc* for everything, including Scala source. The resulting
javadoc is not so helpful. It also complains a lot about not finding references since javadoc
doesn't quite understand links in the same way.

*Maven #2*

You can also invoke the scala-maven-plugin 'doc' goal as part of the site generation:


It lacks a goal like "aggregate" that the javadoc plugin has, which takes care of combining
everything into one set of docs. This only generates scaladoc in each module in exploded format.

*Unidoc / SBT*

It is almost as easy as:

- adding the plugin to plugins.sbt: {{addSbtPlugin("com.eed3si9n" % "sbt-unidoc" % "0.3.0")}}
- {{import sbtunidoc.Plugin.\_}} and {{UnidocKeys.\_}} in SparkBuild.scala
- adding "++ unidocSettings" to rootSettings in SparkBuild.scala

but it was also necessary to:

- {{SPARK_YARN=true}} and {{SPARK_HADOOP_VERSION=2.2.0}}, for example, to make YARN scaladoc
- Exclude {{yarn-alpha}} since scaladoc doesn't like the collision of class names:

  def rootSettings = sharedSettings ++ unidocSettings ++ Seq(
    unidocProjectFilter in (ScalaUnidoc, unidoc) := inAnyProject -- inProjects(yarnAlpha),
    publish := {}

I still get SBT errors since I think this is not quite correctly finessing the build. But
it seems almost there.

> Aggregate Scaladocs across projects
> -----------------------------------
>                 Key: SPARK-1439
>                 URL:
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Documentation
>            Reporter: Matei Zaharia
>             Fix For: 1.0.0
> Apparently there's a "Unidoc" plugin to put together ScalaDocs across modules:

This message was sent by Atlassian JIRA

View raw message