metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Miklavcic <michael.miklav...@gmail.com>
Subject Re: [PROPOSAL] up-to-date versioned documentation
Date Fri, 13 Jan 2017 00:19:35 GMT
Casey, Matt - These guys are using doxia
https://github.com/apache/falcon/tree/master/docs

Honestly, I kind of like Spark's approach -
https://github.com/apache/spark/tree/master/docs

Mike

On Thu, Jan 12, 2017 at 4:48 PM, Matt Foley <mattf@apache.org> wrote:

> I’m ambivalent; I think we’d end up tied to the doxia processing pipeline,
> which is “yet another arcane toolset” to learn.  Using .md as the input
> format decreases the dependency, but we’d still be dependent on it.
>
> I had anticipated that the web page would be a write-once thing that would
> be only a couple days for an experienced Web developer. But I was going to
> get an estimate from some co-workers before actually trying to get it
> implemented. And the script is a few hours of work with find and awk.
>
> On the other hand, doxia is certainly an expectable solution.  Is setting
> up that infrastructure less work than developing the web page?  Or is it
> actually just a matter of a few lines in pom.xml?
>
>
> On 1/12/17, 3:24 PM, "Casey Stella" <cestella@gmail.com> wrote:
>
>     Just a followup thought that's a bit more constructive, maybe we could
>     migrate the README.md's into a site directory and use doxia markdown
>     (example here <https://github.com/larrycai/doxia-markdown-demo>) to
>     generate the site as part of the build to resolve 1 through 3?
>
>     On Thu, Jan 12, 2017 at 6:19 PM, Casey Stella <cestella@gmail.com>
> wrote:
>
>     > So, I do think this would be better than what we currently do.  I
> like a
>     > few things in particular:
>     >
>     >    - I don't like the wiki one bit.
>     >    - We have a LOT of documentation in the README.md's and it's
> sometimes
>     >    poorly organized
>     >    - I like a documentation preprocessing pipeline to be present.
> For
>     >    instance, a major ask is all of the stellar functions in one
> place.  That's
>     >    solved by updating an index manually in the READMEs and keeping
> it in sync
>     >    with the annotation.  I'd like to make a stellar annotation ->
> markdown
>     >    generator as part of the build and this would be nice for such a
> task.
>     >
>     > My only concern is that the html generation/viewer seems like a fair
>     > amount of engineering.  Are you sure there isn't something easier
> that we
>     > could conform to?  I'm sure we aren't the only project in the world
> that
>     > has this particular issue.  Is there something like a maven site
> plugin or
>     > something?  Just a thought.  I'll come back with more :)
>     >
>     > Great ideas!  Keep them coming!
>     >
>     > Casey
>     >
>     > On Thu, Jan 12, 2017 at 6:05 PM, Matt Foley <mattf@apache.org>
> wrote:
>     >
>     >> We currently have three forms of documentation, with the following
>     >> advantages and disadvantages:
>     >>
>     >> || Docs || Pro || Con ||
>     >> | CWiki |
>     >>       Easy to edit, no special tools required, don't have to be a
>     >> developer to contribute, google and wiki search |
>     >> Not versioned, no review process, distant from the code, obsolete
> content
>     >> tends to accumulate |
>     >> | Site |
>     >>       Versioned and reviewed, only committers can edit, google
> search |
>     >>       Yet another arcane toolset must be learned, only web
> programmers
>     >> feel comfortable contributing, "asf-site" branch not related to code
>     >> versions, distant from the code, tends to go obsolete due to
>     >> non-maintenance |
>     >> | README.md |
>     >>       Versioned and reviewed, only committers can edit, tied to code
>     >> versions, highly local to the code being documented |
>     >>       Non-developers don't know about them, may be scared by
> github, poor
>     >> scoring in google search, no high-level presentation |
>     >>
>     >> Various discussion threads indicate the developer community likes
>     >> README-based docs, and it's easy to see why from the above.  I
> propose this
>     >> extension to the README-based documentation, to address their
> disadvantages:
>     >>
>     >> 1. Produce a script that gathers the README.md files from all code
>     >> subdirectories into a hierarchical list.  The script would have an
>     >> exclusion list for non-user-content, which at this point would
> consist of
>     >> [site/*, build_utils/*].  The hierarchy would be sorted
> depth-first.  The
>     >> resulting hierarchical list at this time (with six added README
> files to
>     >> complete the hierarchy) would be:
>     >>
>     >> ./README.md
>     >> ./metron-analytics/README.md  <== (need file here)
>     >> ./metron-analytics/metron-maas-service/README.md
>     >> ./metron-analytics/metron-profiler/README.md
>     >> ./metron-analytics/metron-profiler-client/README.md
>     >> ./metron-analytics/metron-statistics/README.md
>     >> ./metron-deployment/README.md
>     >> ./metron-deployment/amazon-ec2/README.md
>     >> ./metron-deployment/packaging/README.md  <== (need file here)
>     >> ./metron-deployment/packaging/ambari/README.md <== (need file here)
>     >> ./metron-deployment/packaging/docker/ansible-docker/README.md
>     >> ./metron-deployment/packaging/docker/rpm-docker/README.md
>     >> ./metron-deployment/packer-build/README.md
>     >> ./metron-deployment/roles/  <== (need file here)
>     >> ./metron-deployment/roles/kibana/README.md
>     >> ./metron-deployment/roles/monit/README.md
>     >> ./metron-deployment/roles/opentaxii/README.md
>     >> ./metron-deployment/roles/pcap_replay/README.md
>     >> ./metron-deployment/roles/sensor-test-mode/README.md
>     >> ./metron-deployment/vagrant/README.md  <== (need file here)
>     >> ./metron-deployment/vagrant/codelab-platform/README.md
>     >> ./metron-deployment/vagrant/fastcapa-test-platform/README.md
>     >> ./metron-deployment/vagrant/full-dev-platform/README.md
>     >> ./metron-deployment/vagrant/quick-dev-platform/README.md
>     >> ./metron-platform/README.md
>     >> ./metron-platform/metron-api/README.md
>     >> ./metron-platform/metron-common/README.md
>     >> ./metron-platform/metron-data-management/README.md
>     >> ./metron-platform/metron-enrichment/README.md
>     >> ./metron-platform/metron-indexing/README.md
>     >> ./metron-platform/metron-management/README.md
>     >> ./metron-platform/metron-parsers/README.md
>     >> ./metron-platform/metron-pcap-backend/README.md
>     >> ./metron-sensors/README.md  <== (need file here)
>     >> ./metron-sensors/fastcapa/README.md
>     >> ./metron-sensors/pycapa/README.md
>     >>
>     >> 2. Arrange to run this script as part of the build process, and
> commit
>     >> the resulting hierarchy list to someplace in the versioned and
> branched
>     >> ./site/ subdirectory.
>     >>
>     >> 3. Produce a "doc reader" web page that takes in this hierarchy of
> .md
>     >> pages, and presents a LHS doc tree of links, and a main display
> area for a
>     >> currently selected file.  If we want to get fancy, this page would
> also
>     >> provide: (a) telescoping (collapse/expand) of the doc tree; (b)
> floating
>     >> next/prev/up/home buttons in the display area.
>     >>
>     >> #4. Add to this web page a pull-down menu that selects among all the
>     >> release versions of Metron, and (if not running in the Apache site)
> a
>     >> SNAPSHOT version for the current filesystem version (for developer
>     >> preview).  Let it re-write the file paths per release version to
> the proper
>     >> release tag in github.  This web page will therefore be
>     >> version-independent.  Put it in the asf-site branch of the Apache
> site, as
>     >> the new "docs" sub-site from the home web page.  Update the list of
>     >> releases at each release, or if we want to get fancy, teach it to
> read the
>     >> release tags from github.
>     >>
>     >> 5. As part of the release process, the release manager (a) assures
> the
>     >> release is tagged in github with a consistent naming convention,
> and (b)
>     >> submits the new hierarchy of links to google search (there's an api
> for
>     >> that).
>     >>
>     >> 6. Deprecate the use of cwiki for anything but long-lived
>     >> demonstrations/tutorials that are unlikely to go obsolete.
>     >>
>     >>
>     >> Do folks feel this would be a good contribution to the visibility,
>     >> timeliness, and usability of our docs?
>     >> Is this an adequate solution for the current problems?
>     >>
>     >> Thanks,
>     >> --Matt
>     >>
>     >>
>     >>
>     >
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message