metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Miklavcic <michael.miklav...@gmail.com>
Subject Re: [DISCUSS] Metron documentation improvements
Date Fri, 21 Dec 2018 18:10:26 GMT
I spent a bit of time getting this to run. I currently get a host of 404's
and am going to need to dig in a bit and see just what they are. In the
meantime, I'm going to commit https://github.com/apache/metron/pull/1310
and have created a Jira to track adding site-book link validation to our
Travis checks - https://issues.apache.org/jira/browse/METRON-1952. For the
curious, I landed on the following modified commands to get this running:

nohup python -m SimpleHTTPServer 2>&1 &
wget -Dlocalhost --spider -e robots=off -w 1 -r -p http://localhost:8000
2>&1 > /dev/null | grep -C 2 -E "404"
if [ $? -eq 0 ]; then
  echo FAILED;
else
  echo PASSED;
fi

http.server is the impl for Python 3. Simple tweak using SimpleHTTPServer
for Python 2.

Cheers,
Mike


On Thu, Dec 20, 2018 at 1:24 PM Casey Stella <cestella@gmail.com> wrote:

> You will want to add a -Dlocalhost to the wget to ensure you're not
> checking domains linked from our docs and turn travis into google. ;)
>
>
> On Thu, Dec 20, 2018 at 3:19 PM Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
> > Well golly, I love this. I'll give that a whirl!
> >
> > On Thu, Dec 20, 2018 at 1:08 PM Casey Stella <cestella@gmail.com> wrote:
> >
> > > I definitely agree with option 3; that's a no-brainer IMO.  I thought
> for
> > > sure this was already happening, honestly.
> > >
> > > As for 2, we could even script the broken link check by:
> > >
> > >    - Serving up the site locally via python with `python -m
> http.server`
> > >    from the site-book output directory
> > >    - Looking at the output of wget --spider -e robots=off -w 1 -r -p
> > >    http://localhost:8000 for 404s
> > >
> > >
> > > As for 1, I'm fine with it if that's where we want to go.  I'm a +0
> > >
> > > On Thu, Dec 20, 2018 at 2:47 PM Michael Miklavcic <
> > > michael.miklavcic@gmail.com> wrote:
> > >
> > > > We recently had our site-book doc generation break due to our not
> > > including
> > > > it in the Travis build. The fix for a broken build seems simple
> enough
> > -
> > > > add it to our build process and assuming it doesn't cause build
> timeout
> > > > issues, we should be good to go.
> > > >
> > > > Beyond that, there are additional issues with the existing process.
> We
> > > have
> > > > a step in our PR review for validating that the docs are rendering
> > > > properly. I know I've gone back and corrected issues with broken
> images
> > > or
> > > > incorrectly rendering pages at least a few times now. On one hand, we
> > > might
> > > > say this is simply a matter of being better about validating
> > > documentation
> > > > during the review process. That may be true, but rather than fight
> > > upstream
> > > > like a salmon, I would prefer to simplify things, automate what we
> can,
> > > and
> > > > use technology to work with us. Based on this conversation on
> > > METRON-1950 -
> > > > (
> > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/e2acf91efc5f51ba0e26d76b00ca02415d3c6ee0adee74a037ab2beb@%3Cdev.metron.apache.org%3E
> > > > ),
> > > > I'd like to open up a general convo about improvements to our
> > > documentation
> > > > generation.
> > > >
> > > > *Current Issues:*
> > > >
> > > >    1. Duplicated effort - have to check pages render in Github and
> the
> > > >    Doxia-generated site-book
> > > >    2. Inconsistent model - what works for Github markdown may not
> work
> > > for
> > > >    Doxia, and vice versa
> > > >    3. Github is part of our workflow and easy to check, Doxia
> requires
> > an
> > > >    extra separate step - suffers unintentional bugs due to #2.
> > > >    4. Images have to be manually added to the site rendering code for
> > > >    copying to the "images" folder, and explicit src ref replacements
> > have
> > > > to
> > > >    be included for all affected pages/links as well.
> > > >    5. Page links and images are not validated - this currently
> requires
> > > >    manual review and intervention during PR review and whenever we
> > > create a
> > > >    new Metron release.
> > > >    6. Failed site-book build is not validated. Broken build does not
> > fail
> > > >    Travis
> > > >
> > > > *Options and Solutions:*
> > > >
> > > >    1. Otto has already brought up using Ascii doc as one option for
> > > solving
> > > >    a number of these issues.
> > > >    2. For issue #5, we can write a scraper that validates links or
> use
> > > >    tooling like Cypress for this.
> > > >    3. For issue #6, we can add site-book building to our Travis runs.
> > > It's
> > > >    pretty quick to generate and will catch the more egregious
> rendering
> > > > bugs.
> > > >    I plan to look at this presently.
> > > >
> > > > Mike
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message