cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From m..@apache.org
Subject [cassandra-website] 06/07: CASSANDRA-16066: Update README
Date Thu, 02 Sep 2021 11:43:45 GMT
This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-website.git

commit 07f0e129404014c8b1a5921d05aea0d1d1111760
Author: Anthony Grasso <anthony@thelastpickle.com>
AuthorDate: Sat Jun 26 13:05:29 2021 +1000

    CASSANDRA-16066: Update README
    
    * Added information and examples on how to use the docker wrapper script.
    * Added rationale information about why repository structure and tooling used.
    * Removed information about old build process and commands that have been
        replaced.
    * Updated and reused old information that is still applicable to new tooling.
    
    patch by Anthony Grasso; reviewed by Mick Semb Wever, Lorina Poland, Melissa Logan, Paul
Au for CASSANDRA-16066
---
 README.md | 318 ++++++++++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 236 insertions(+), 82 deletions(-)

diff --git a/README.md b/README.md
index 8103c1a..fda4bb4 100644
--- a/README.md
+++ b/README.md
@@ -1,13 +1,31 @@
 # Apache Cassandra website
 
-TODO: Add notes about the repository structure
+The website repository code is separated into two main parts. These parts are represented
by the directories at the root level of the project. Specifically the structure of the repository
is:
+
+```
+ROOT
+  - site-content
+  - site-ui
+```
+
+## Site UI
+
+The 'site-ui' directory contains only the UI styling files that determines the look and feel
of the site. A *ui-bundle.zip* file containing the styling information will be generated from
the contents of this directory. Generation of the *ui-bundle.zip* will be done using `gulp`
launched inside a Docker container.
+
+## Site Content
+
+The 'site-content' directory contains all the raw page information e.g. where to download,
developer guidelines, how to commit patches, etc. The live website HTML is generated from
the contents of this directory. Generation of the HTML content is done by `antora` launched
inside a Docker container. As part of the website HTML generation, the ui-bundle.zip file,
and the Cassandra documentation location are passed to `antora`. It uses the ui-bundle.zip
to style the website. The Cassandra d [...]
+
+## Further Reading
+
+For further details about why the directories are separated as described above and why we
use `antora` please see the [Details](#details) section at the bottom of this page.
 
 ## Development Cycle
 
 Making changes to the website content can be done using the following steps.
 
 1. Preview changes locally. See below for further details about how to do this.
-2. Commit changes in `site-content/source` to a fork and branch.
+2. Commit changes in `site-content/source/html` to a fork and branch.
 3. Create a pull request back to this repository.
 3. Get the pull request reviewed and merged to `trunk`.
 4. Preview the rendered site on https://cassandra.staged.apache.org/ (wait til [ci-cassandra.apache.org](https://ci-cassandra.apache.org/job/cassandra-website/)
has deployed it).
@@ -16,7 +34,7 @@ Making changes to the website content can be done using the following steps.
 
 # Developer Quickstart
 
-To test changes before committing, it is a requirement that you build the website locally.
Building the Apache Cassandra website takes a number of steps. To make things easier we have
provided a Docker container which can build the full website in a few simple commands and
have it ready to commit via git.
+To test changes before committing, it is a requirement that you build the website locally.
Building the Apache Cassandra website takes a number of steps. To make things easier we have
provided a suite of tools to build the full website in a few simple commands and have it ready
to commit via git. Please see the [Details](#details) section at the bottom of this page for
further details about the tooling.
 
 ## Building Prerequisites
 
@@ -31,150 +49,286 @@ $ git clone https://github.com/apache/cassandra-website.git
 $ cd ./cassandra-website
 ```
 
-To build the website run the following commands from within the `./cassandra-website` directory
(assuming you used the above clone command):
+A `run.sh` wrapper script has been provided to simplify generating the docs and building
the site. It provides a single commandline interface that generates the docker commands to
run the website and UI docker containers.
+
+The script has the following usage format
 
 ```bash
-$ export BUILD_USER=build
-$ docker-compose build website
-$ docker-compose run website
+$ ./run.sh <COMPONENT> <COMMAND> [OPTIONS]
 ```
 
-:warning: *Tip:* In order to prevent root-owned modified files in your repository, the container
user, `build`, is set up with a default UID=1000:GID=1000, which is usually the first user
configured on a linux machine. If your local user is different you should set up the container
user with your local host user's UID:GID, replace the above with:
+A complete list of components, commands and options can be found by running the following
command.
 
-:warning: *Tip:* Building cassandra-website may fail if the `CLOUDSDK_PYTHON` environment
variable is not set on your machine. For example, set the environment variable to export `CLOUDSDK_PYTHON=/usr/bin/python2`.
+```bash
+$ ./run.sh -h
+```
+
+To build the website only, run the following command from within the `./cassandra-website`
directory (assuming you used the above clone command).
 
 ```bash
-$ export BUILD_USER=build
-$ docker-compose build --build-arg UID_ARG=$(id -u) --build-arg GID_ARG=$(id -g) website
-$ docker-compose run website
+$ ./run.sh website build
 ```
 
-Once building has completed, the site content will be in the `./cassandra-website/site-content/build`
directory ready to be tested.
+This will build the website content using your local copy of the cassandra-website, and the
current checked-out branch. Use this command if you want to make a change to a top-level webpage
without building the docs for any versions of cassandra.
 
-### Customising the Website Build for Development
+Once building has completed, the HTML content will be in the `./site-content/build/html/`
directory ready to be reviewed and committed.
 
-#### You can use your own Cassandra fork
+:warning: *Tip:* In order to prevent root-owned modified files in your repository, the container
executes operations as a non-root user. By default, the user is `build` and has the user and
group permissions set to `UID=1000` and `GID=1000` respectfully. These permissions are usually
the first user configured on a linux machine.
 
-The container contains a local copy of the Apache Cassandra repository. The document generation
process commits the generated AsciiDoc to the local repository. This is done to allow Antora
to render to HTML the versioned documentation and website using the committed AsciiDoc in
each branch.
+If your local user has different user and group permissions you can set up the container
user with your local UID:GID. In addition, you can set the build user in the container your
local username. These changes can be made when building the container using the following
command:
 
-If you have a Cassandra fork you would like to use instead, you can override the repository
used when building the container:
+```bash
+$ ./run.sh website container -a BUILD_USER_ARG:$(whoami) -a UID_ARG:$(id -u) -a GID_ARG:$(id
-g)
+```
+
+If you need to customise the container user as noted above, you must do this before you build
the website or run any other website command.
+
+## Build the Website when Developing
+
+The website tooling is very flexible and allows for a wide range of development scenarios.
 
+### Build the website from a different branch
+
+You can tell the website builder to use a different branch to the one you are on. This can
be done using the following command.
+
+```bash
+$ ./run.sh website build -b cassandra-website:my_branch
 ```
-$ export BUILD_USER=build
-$ docker-compose build --build-arg CASSANDRA_REPOSITORY_URL_ARG=<git_url_to_your_fork>
website
-$ docker-compose run website
+
+This will build the website content using your local copy of the cassandra-website, and the
branch named `my_branch`.
+
+### Build the website using a local clone of the repository
+
+You can tell the website builder to use a different clone or fork of the repository.
+
+To build using another local copy of the cassandra-website run the following command.
+
+```bash
+$ ./run.sh website build -u cassandra-website:/local/path/to/another/clone/of/cassandra-website
 ```
 
-Note that you can specify multiple `--build-arg` options when calling `docker-compose build`.
So if you wanted to use a different user ID, group ID and repository you could run the following
command:
+This will build the website using the contents of the local repository located in */local/path/to/another/clone/of/cassandra-website*
 
+### Build the website using a remote clone of the repository
+
+To build using a remote copy of the cassandra-website run the following command.
+
+```bash
+$ ./run.sh website build -u cassandra-website:https://github.com/my_orgranisation/cassandra-website.git
 ```
-$ export BUILD_USER=build
-$ docker-compose build \
-  --build-arg CASSANDRA_REPOSITORY_URL_ARG=<git_url_to_your_fork> \
-  --build-arg UID_ARG=$(id -u) \
-  --build-arg GID_ARG=$(id -g) \
-  website
-$ docker-compose run website
+
+This will build the website using the contents of the remote repository located at *https://github.com/my_orgranisation/cassandra-website.git*
+
+:warning: *Tip:* The `HEAD` branch of the Cassandra Website repository is always used by
default unless an alternative branch is specified.
+
+In both cases above the `HEAD` branch is used and translates to different branches. In first
case where the repository is local, `HEAD` will translate to the currently checked out branch.
In the second case where the repository is remote and needs to be cloned, `HEAD` will translate
to default branch selected when the repository is cloned. You can specify a different branch
using the `-b` option as per the example in [Build a different branch](#build-a-different-branch).
+
+## Previewing the Website
+
+An offline preview mode exists if you want to view the website as you make changes to the
content. Preview mode can be launched using the following command.
+
+```bash
+$ ./run.sh website preview
 ```
 
-In addition to customising the container build, you can customise the environment variables
used inside the container. These can be used to influence Antora's rending of the site. For
example, Antora will generate documentation for all Cassandra releases starting at 3.11.5.
Hence, if you have a specific branch in your fork of Cassandra that you would like to render,
you can override the variable the specifies the branches to render.
+The site can be viewed on [http://localhost:5151](http://localhost:5151).
+
+The `preview` command operates the same as the `build` command. It will build the website
content using your local copy of the cassandra-website, and the current checked-out branch.
Additionally, it will then start a webserver to serve the HTML and a process that monitors
the content files in your local copy of the repository. If a change is made to a content file,
the website HTML will automatically be regenerated.
+
+Press `Ctrl+C` to stop the preview server and end the continuous build.
+
+:warning: *Tip:* You may need to refresh your browser when the auto rendering of the site
is complete.
+
+All options that are available in the `build` command can be used by the `preview` command.
Hence, the options used in the previous examples can be specified when using the `preview`
command.
 
-TODO: Explain that you need an env.dev file with variables in it and to run `docker-compose
run website-dev`
+# Developer Advanced Usage Guide
 
-Below is a select list of environment variables that can be overridden to control Antora's
rendering of the site and why you would want to override them.
+The cassandra-website tooling can also be used to perform a number of other operations:
 
-TODO: Add reason why you would override these
-`CASSANDRA_REPOSITORY_URL`
-`CASSANDRA_VERSIONS`
-`CASSANDRA_WEBSITE_REPOSITORY_URL`
-`CASSANDRA_WEBSITE_VERSIONS`
-`UI_BUNDLE_ZIP_URL`
+* Generate the Apache Cassandra documentation.
+* Update the website styling and behaviour.
 
+## Generating the Cassandra Documentation
 
+The website tooling provides the ability to generate the Cassandra AsciiDoc (.adoc) files.
The content of the AsciiDoc files are sourced from nodetool and the cassandra.yaml configuration
file. The AsciiDoc files need to exist in the Cassandra repository as either commits to the
branch or changes ready to be committed before `antora` can generate the HTML formatted equivalent
for publication to the Cassandra website.
 
-# WARNING: From here on is all wrong!!!!
+By default, the Docker container that generates the version documentation will clone the
Apache Cassandra project within the container when generating the documentation. The document
generation process commits the generated AsciiDoc to this repository. This is done so `antora`
can generate website HTML as well as the HTML for the various versions of the documentation
using the committed AsciiDoc in each branch.
 
-### Previewing the Website Build
+You can use your own local copy or fork of the Cassandra repository as the source for the
versioned documentation. Doing this will also allow you to view and access the generated AsciiDoc
files. To use your own copy or fork of the Cassandra repository, you can specify a local path
to the `run.sh` script. In this case, when rendering multiple versions of the documentation,
any changes to the generated AsciiDoc files will be committed to your local copy or fork.
If multiple source branches [...]
 
-The fastest way to preview the site is to run the following:
+### Generate documentation using a local clone of the repository
+
+To generate the latest version of the Cassandra docs using a local copy or fork of the Cassandra
repository, run the following command.
 
 ```bash
-$ docker-compose up preview
+$ ./run.sh website docs -u cassandra:/local/path/to/cassandra/repository -b cassandra:trunk
 ```
 
-Then view the site on http://localhost:8000
+The output of this command will be AsciiDoc (`.adoc`) files that `antora` can render into
HTML. Note that `antora` is never executed when only the Cassandra versioned documentation
is generated.
+
+If you are generating only the Cassandra version documentation, you should always specify
a local copy of a Cassandra repository. This is because the generated AsciiDoc files will
be placed in the Cassandra directory or committed to the branch used to generate them. You
will then be able to view the contents of the AsciiDoc files.
+
+If you are generating only the Cassandra version documentation, and you specify a remote
Cassandra repository location, the generated AsciiDoc files will be inaccessible. This is
because the Cassandra repository will be cloned inside the container before the AsciiDoc files
are generated. In this case a warning message will be displayed, and you will be asked whether
to proceed with the operation. Specifying a remote Cassandra repository location is useful
in cases where the HTML website  [...]
+
+### Generate documentation for multiple Cassandra versions
 
-If you want to preview the site as you are editing it run this command:
+To generate multiple versions of the Cassandra documentation using a local copy or fork of
the Cassandra repository, run the following command.
 
 ```bash
-$ docker-compose build cassandra-website
-$ docker-compose up cassandra-website-serve
+$ ./run.sh website docs -u cassandra:/local/path/to/cassandra/repository -b cassandra:trunk,cassandra-3.11,my_branch
 ```
 
-For information about the site layout see the **Layout** section of [README](src/README#layout)
in the _src_ directory.
+In the above command, multiple branches separated by a comma (`,`) can be specified in the
`-b` option.
 
+## Generating the website and versioned documentation HTML at the sametime
 
-How ci-cassandra.apache.org continuously deploys to cassandra.staged.apache.org
--------------------------------------------------------------------------------
+The website tooling can be used to run a complete end-to-end generation of the HTML website
and versioned documentation. That is, it can first generate the AsciiDoc files for each Cassandra
version, and then launch `antora` to generate the HTML website and versioned documentation
for publication.
 
-The `cassandra-website` job on ci-cassandra.apache.org performs the following steps to CD
master builds to cassandra.staged.apache.org
+### Generate the website and documentation using pre-generated Cassandra AsciiDoc in local
repositories
 
-```
-git checkout asf-staging
-git reset --hard origin/master
-docker-compose build --build-arg UID=$(id -u) --build-arg GID=$(id -g) cassandra-website
-docker-compose run cassandra-website
-#git status
-git add content/ src/doc/
-git commit -a -m "generate docs for $(git rev-parse --short HEAD)"
-#git show --stat HEAD
-git push -f origin asf-staging
+If you have already generated the Cassandra AsciiDoc (`.adoc`) files and committed them to
your repository, you can skip the Cassandra AsciiDoc generation process.
+
+To build the website using a local clone of the Cassandra repository that contains the generated
AsciiDoc files, and the Cassandra Website run the following command.
+
+```bash
+$ ./run.sh \
+  website \
+  build \
+    -i \
+    -u cassandra:/local/path/to/cassandra/repository \
+    -u cassandra-website:/local/path/to/cassandra-website/repository
 ```
 
+In the above command, the `-i` option is used to tell the tooling to include the Cassandra
repository when `antora` is generating the HTML. This ensures the versioned documentation
HTML is generated along with the website HTML. In this case, exiting AsciiDoc files in the
Cassandra repository are used to generate the versioned documentation HTML. That is, no additional
operations are run to pre-generate the Cassandra AsciiDoc files.
 
-Publishing to cassandra.apache.org :: Merging `asf-staging` to `asf-site`
------------------------------------
+### Generate the website and documentation using pre-generated Cassandra AsciiDoc in remote
repositories
 
-Updating the main website, after verifying the staged website, involves copying the `asf-staging`
branch to `asf-site`. A normal git merge is not used, because the `asf-staging` is forced
updated after each ci-cassandra.apache.org build. Instead make live the staged website by
copying the `asf-staging` to the `asf-site` branch.
+To build using a remote copy of the Cassandra repository that contains the generated AsciiDoc
files, and the Cassandra Website run the following command.
 
-    git fetch origin
-    git switch asf-site
-    git reset --hard origin/asf-staging
-    git push -f origin asf-site
+```bash
+$ ./run.sh \
+  website \
+  build \
+    -i \
+    -u cassandra:https://github.com/my_orgranisation/cassandra.git \
+    -u cassandra-website:https://github.com/my_orgranisation/cassandra-website.git
+```
 
+You can have a combination of local and remote repository paths. For example, you could have
a local copy of the Cassandra Website repository and want to use the remote Cassandra repository
to include the current documentation when rendering the website. To do this you can run the
following command.
 
-## Building the Site UI
+```bash
+$ ./run.sh \
+  website \
+  build \
+    -i \
+    -u cassandra:https://github.com/my_orgranisation/cassandra.git \
+    -u cassandra-website:/local/path/to/cassandra-website/repository
+```
+
+### Generate the website using local copy of the ui-bundle
 
-### Executing Tasks
+You can use your own *ui-bundle.zip* file containing the information on how to style the
website when building it. The *ui-bundle.zip* file can be generated using the `./run.sh` script.
See the [Building the Site UI](#building-the-site-ui) section for furher details on how to
build the *ui-bundle.zip*.
 
-To get a list of the tasks that can be executed run the following commands:
+To supply your own *ui-bundle.zip* file when building the website, run the following command.
 
+```bash
+$ ./run.sh website build -z /local/path/to/ui-bundle.zip
 ```
-$ export BUILD_USER=build
-$ docker-compose build website-ui:
-$ docker-compose run website-ui
+
+The path to the *ui-bundle.zip* file can also be a remote URL. You can supply the URL using
the following command.
+
+```bash
+$ ./run.sh website build -z https://github.com/apache/cassandra-website/archive/refs/tags/ui-bundle-1.0.zip
+```
+
+The styling contained in the *ui-bundle.zip* will be applied to docs if they are being rendered
as well.
+
+By default, the Docker container used to render the site will reference a GitHub release
version of the *ui-bundle.zip*.
+
+## Building the Site UI
+
+To get a list of the tasks that can be executed, run the following commands.
+
+```bash
+$ ./run.sh website-ui
 ```
 
 A task can be executed using the following commands:
 
 ```bash
-$ export BUILD_USER=build
-$ docker-compose build website-ui:
-$ docker-compose run website-ui <task>
+$ ./run.sh website-ui <task>
+```
+
+A full list of tasks can be found by running the following command
+
+```bash
+$ ./run.sh website-ui tasks
 ```
 
 ### Building Bundle
 
-TODO: Add info
+Antora needs a *ui-bundle.zip* when rendering the website and documentation content. It contains
CSS, Java Script, fonts, images and other assets which define the look and feel of the website.
+
+To generate the *ui-bundle.zip* from the assets in the *site-ui* directory, run the following
command.
+
+```bash
+$ ./run.sh website-ui bundle
+```
+
+This will build the UI Bundle using your local copy of the styling assets located in *./site-ui/*.
 
-### Releasing Bundle
+When packaged successfully, the *ui-bundle.zip* will be located in the *./site-ui/build/*
directory of the repository.
 
-TODO: Add info
+:warning: *Tip:* In order to prevent root-owned modified files in your repository, the container
executes operations as a non-root user. By default, the user is `build` and has the user and
group permissions set to `UID=1000` and `GID=1000` respectfully. These permissions are usually
the first user configured on a linux machine.
+
+If your local user has different user and group permissions you can set up the container
user with your local UID:GID. In addition, you can set the build user in the container your
local username. These changes can be made when building the container using the following
command:
+
+```bash
+$ ./run.sh website-ui container -a BUILD_USER_ARG:$(whoami) -a UID_ARG:$(id -u) -a GID_ARG:$(id
-g)
+```
+
+If you need to customise the container user as noted above, you must do this before you build
the UI Bundle or run any other website-ui command.
 
 ### Preview UI
 
+An offline preview mode exists if you want to view example website content as you make changes
to the UI. Preview mode can be launched using the following command.
+
 ```bash
-$ export BUILD_USER=build
-$ docker-compose build website-ui-preview
-$ docker-compose up website-ui-preview
-```
\ No newline at end of file
+$ ./run website-ui preview
+```
+
+The example content can be viewed on [http://localhost:5252](http://localhost:5252).
+
+While preview mode is running, any changes you make to the source files will be instantly
reflected in the browser. This works by monitoring the project for changes, running the build
task if a change is detected, and sending the updates to the browser.
+
+The files in the *preview-src/* folder provide the sample content that allow you to see the
UI in action. In this folder, you will primarily find pages written in AsciiDoc. These pages
provide a representative sample and kitchen sink of content from the real site.
+
+Press `Ctrl+C` to stop the preview server and end the continuous build.
+
+# Merging `asf-staging` to `asf-site`
+
+Updating the main website, after verifying the staged website, involves copying the `asf-staging`
branch to `asf-site`. A normal git merge is not used, because the `asf-staging` is forced
updated after each ci-cassandra.apache.org build. Instead make live the staged website by
copying the `asf-staging` to the `asf-site` branch.
+
+    git fetch origin
+    git switch asf-site
+    git reset --hard origin/asf-staging
+    git push -f origin asf-site
+
+# Details
+
+## Tooling components
+
+The tooling is made up of the following components
+
+* Run script: `./run.sh` - Provides a single commandline interface that generates the docker
commands to run the website and UI docker containers. Using the containers, it can build the
Cassandra website UI components, generate the Cassandra versioned documentation, and generate
the website HTML.
+* Website Docker container: `site-content/Dockerfile` and `site-content/docker-entrypoint.sh`
- Contains the libraries necessary to generate the Cassandra versioned documentation, and
generate the website HTML using Antora.
+* Antora Site YAML script: `bin/site_yaml_generator.py` - Used by the Website Docker container
to create the YAML file that defines the sources for the website content, optionally the cassandra
versioned documentation, and the UI styling bundle to apply.
+* Website UI Docker container: `site-ui/Dockerfile` and `site-ui/docker-entrypoint.sh` -
Contains the libraries necessary to generate the UI bundle ZIP file the is applied by Antora
to style the website and documentation.
+
+## Why is Antora being used
+
+Antora is being used for the website generation because it is designed to create websites
that have version documentation. For example, when a new version of Cassandra is released,
a new version of the documentation will be generated as well. Hence, if there is a change
in the behaviour of Cassandra or a tool in the project, it will be captured in the latest
version of the documentation. Users of the project will have the ability to select the version
of the documentation they are intere [...]
+
+## Why is the styling separated from the content
+
+Separating the layout/style, and the content means that stying changes can be made with little
to no impact on content and vice-versa. In addition, changes to the styling can happen in
parallel while website content is updated without conflict.
\ No newline at end of file

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message