metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Casey Stella <ceste...@gmail.com>
Subject Re: [PROPOSAL] up-to-date versioned documentation
Date Thu, 12 Jan 2017 23:24:45 GMT
Just a followup thought that's a bit more constructive, maybe we could
migrate the README.md's into a site directory and use doxia markdown
(example here <https://github.com/larrycai/doxia-markdown-demo>) to
generate the site as part of the build to resolve 1 through 3?

On Thu, Jan 12, 2017 at 6:19 PM, Casey Stella <cestella@gmail.com> wrote:

> So, I do think this would be better than what we currently do.  I like a
> few things in particular:
>
>    - I don't like the wiki one bit.
>    - We have a LOT of documentation in the README.md's and it's sometimes
>    poorly organized
>    - I like a documentation preprocessing pipeline to be present.  For
>    instance, a major ask is all of the stellar functions in one place.  That's
>    solved by updating an index manually in the READMEs and keeping it in sync
>    with the annotation.  I'd like to make a stellar annotation -> markdown
>    generator as part of the build and this would be nice for such a task.
>
> My only concern is that the html generation/viewer seems like a fair
> amount of engineering.  Are you sure there isn't something easier that we
> could conform to?  I'm sure we aren't the only project in the world that
> has this particular issue.  Is there something like a maven site plugin or
> something?  Just a thought.  I'll come back with more :)
>
> Great ideas!  Keep them coming!
>
> Casey
>
> On Thu, Jan 12, 2017 at 6:05 PM, Matt Foley <mattf@apache.org> wrote:
>
>> We currently have three forms of documentation, with the following
>> advantages and disadvantages:
>>
>> || Docs || Pro || Con ||
>> | CWiki |
>>       Easy to edit, no special tools required, don't have to be a
>> developer to contribute, google and wiki search |
>> Not versioned, no review process, distant from the code, obsolete content
>> tends to accumulate |
>> | Site |
>>       Versioned and reviewed, only committers can edit, google search |
>>       Yet another arcane toolset must be learned, only web programmers
>> feel comfortable contributing, "asf-site" branch not related to code
>> versions, distant from the code, tends to go obsolete due to
>> non-maintenance |
>> | README.md |
>>       Versioned and reviewed, only committers can edit, tied to code
>> versions, highly local to the code being documented |
>>       Non-developers don't know about them, may be scared by github, poor
>> scoring in google search, no high-level presentation |
>>
>> Various discussion threads indicate the developer community likes
>> README-based docs, and it's easy to see why from the above.  I propose this
>> extension to the README-based documentation, to address their disadvantages:
>>
>> 1. Produce a script that gathers the README.md files from all code
>> subdirectories into a hierarchical list.  The script would have an
>> exclusion list for non-user-content, which at this point would consist of
>> [site/*, build_utils/*].  The hierarchy would be sorted depth-first.  The
>> resulting hierarchical list at this time (with six added README files to
>> complete the hierarchy) would be:
>>
>> ./README.md
>> ./metron-analytics/README.md  <== (need file here)
>> ./metron-analytics/metron-maas-service/README.md
>> ./metron-analytics/metron-profiler/README.md
>> ./metron-analytics/metron-profiler-client/README.md
>> ./metron-analytics/metron-statistics/README.md
>> ./metron-deployment/README.md
>> ./metron-deployment/amazon-ec2/README.md
>> ./metron-deployment/packaging/README.md  <== (need file here)
>> ./metron-deployment/packaging/ambari/README.md <== (need file here)
>> ./metron-deployment/packaging/docker/ansible-docker/README.md
>> ./metron-deployment/packaging/docker/rpm-docker/README.md
>> ./metron-deployment/packer-build/README.md
>> ./metron-deployment/roles/  <== (need file here)
>> ./metron-deployment/roles/kibana/README.md
>> ./metron-deployment/roles/monit/README.md
>> ./metron-deployment/roles/opentaxii/README.md
>> ./metron-deployment/roles/pcap_replay/README.md
>> ./metron-deployment/roles/sensor-test-mode/README.md
>> ./metron-deployment/vagrant/README.md  <== (need file here)
>> ./metron-deployment/vagrant/codelab-platform/README.md
>> ./metron-deployment/vagrant/fastcapa-test-platform/README.md
>> ./metron-deployment/vagrant/full-dev-platform/README.md
>> ./metron-deployment/vagrant/quick-dev-platform/README.md
>> ./metron-platform/README.md
>> ./metron-platform/metron-api/README.md
>> ./metron-platform/metron-common/README.md
>> ./metron-platform/metron-data-management/README.md
>> ./metron-platform/metron-enrichment/README.md
>> ./metron-platform/metron-indexing/README.md
>> ./metron-platform/metron-management/README.md
>> ./metron-platform/metron-parsers/README.md
>> ./metron-platform/metron-pcap-backend/README.md
>> ./metron-sensors/README.md  <== (need file here)
>> ./metron-sensors/fastcapa/README.md
>> ./metron-sensors/pycapa/README.md
>>
>> 2. Arrange to run this script as part of the build process, and commit
>> the resulting hierarchy list to someplace in the versioned and branched
>> ./site/ subdirectory.
>>
>> 3. Produce a "doc reader" web page that takes in this hierarchy of .md
>> pages, and presents a LHS doc tree of links, and a main display area for a
>> currently selected file.  If we want to get fancy, this page would also
>> provide: (a) telescoping (collapse/expand) of the doc tree; (b) floating
>> next/prev/up/home buttons in the display area.
>>
>> #4. Add to this web page a pull-down menu that selects among all the
>> release versions of Metron, and (if not running in the Apache site) a
>> SNAPSHOT version for the current filesystem version (for developer
>> preview).  Let it re-write the file paths per release version to the proper
>> release tag in github.  This web page will therefore be
>> version-independent.  Put it in the asf-site branch of the Apache site, as
>> the new "docs" sub-site from the home web page.  Update the list of
>> releases at each release, or if we want to get fancy, teach it to read the
>> release tags from github.
>>
>> 5. As part of the release process, the release manager (a) assures the
>> release is tagged in github with a consistent naming convention, and (b)
>> submits the new hierarchy of links to google search (there's an api for
>> that).
>>
>> 6. Deprecate the use of cwiki for anything but long-lived
>> demonstrations/tutorials that are unlikely to go obsolete.
>>
>>
>> Do folks feel this would be a good contribution to the visibility,
>> timeliness, and usability of our docs?
>> Is this an adequate solution for the current problems?
>>
>> Thanks,
>> --Matt
>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message