spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "holdenk (JIRA)" <>
Subject [jira] [Commented] (SPARK-10382) Make example code in user guide testable
Date Thu, 10 Sep 2015 18:12:45 GMT


holdenk commented on SPARK-10382:

My experience has mostly been that we just manually cut & paste over since the books I've
been involved with have been asciidoc & word respectively (also there it we wanted to
publish the examples publicly and not the rest of the book repo). It sounds like a good idea
though, I know Paco has been doing some stuff with integrating notebooks & docs/books
so it might be worth reaching out to him as well. That being said the current proposal sounds
like a good idea to me.

> Make example code in user guide testable
> ----------------------------------------
>                 Key: SPARK-10382
>                 URL:
>             Project: Spark
>          Issue Type: Brainstorming
>          Components: Documentation, ML, MLlib
>            Reporter: Xiangrui Meng
>            Assignee: Xiangrui Meng
>            Priority: Critical
> The example code in the user guide is embedded in the markdown and hence it is not easy
to test. It would be nice to automatically test them. This JIRA is to discuss options to automate
example code testing and see what we can do in Spark 1.6.
> One option I propose is to move actual example code to spark/examples and test compilation
in Jenkins builds. Then in the markdown, we can reference part of the code to show in the
user guide. This requires adding a Jekyll tag that is similar to,
e.g., called include_example.
> {code}
> {% include_example scala ml.KMeansExample guide %}
> {code}
> Jekyll will find `examples/src/main/scala/org/apache/spark/examples/ml/KMeansExample.scala`
and pick code blocks marked "guide" and put them under `{% highlight %}` in the markdown.
We can discuss the syntax for marker comments.
> Just one way to implement this. It would be nice to hear more ideas.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message