Command verification¶

Every command in a tutorial or how-to guide is run in a disposable virtual machine, on each distribution the page lists, before the page is committed. The page's verified: date and distros: snapshot identifiers record exactly what was run.

Running it¶

You need a Linux machine with KVM, QEMU, xorriso and OpenSSH; Lab environments lists the requirements and explains how labs are built, controlled and troubleshot.

make verify PAGE=docs/how-to/storage/extend-lvm-volume.md

This boots a fresh lab per distribution (in parallel), runs the page's commands in order, and, if every distribution passes, updates the front matter. The first run downloads the official cloud images (about 4.5 GB).

Other useful commands:

scripts/uv run python scripts/verify_commands.py PAGE --plan        # show what would run where
scripts/uv run python scripts/verify_commands.py PAGE --distro arch # one distribution, no front-matter update
scripts/uv run python scripts/lab.py shell debian-13                # an interactive lab to experiment in
make lab-check                                                      # boot every lab image once
make lab-loadtest                                                   # measure how many labs this host runs at once

Lab images are checked before use¶

Before a lab image is first used, and again whenever its checks change, scripts/lab.py boots it and runs its acceptance checks: labs/common/accept.sh for every image (network, discoverable NICs and disks, sudo, a writable root filesystem), plus labs/<distribution>/accept.sh where an image needs more (the Gentoo image proves PID 1 is OpenRC's init, no systemd remains in the initramfs or cgroups, DNS works and the filesystem is clean). A baked image that fails isn't used. make lab-accept runs the checks on demand; scripts/uv run python scripts/lab.py list shows each image's state.

When you write a check, make it able to fail: assert that the tool exists and that its output isn't empty before interpreting it, capture output before matching it, and try the check once against a broken image or input.

The package cache¶

make lab-cache-up starts a caching proxy for package downloads (nginx in Docker, listening on the host's loopback only). While it runs, each lab's package sources point at it, so packages are downloaded once and shared by every lab; package managers still check every signature. The lab lists the files it changed in /etc/magellanic-lab-cache, and the result record notes whether the cache was used. Without Docker, or with MAGELLANIC_LAB_PKGCACHE=off, labs use their stock mirrors and nothing else changes. make lab-cache-status shows its size and hit counts; make lab-cache-purge empties it. The design is in ADR-018.

Results, image freezes and Gentoo¶

Every run records a result for each page and distribution, together with the exact lab image and the page's text it ran against, in ~/.cache/magellanic/labs/results/. make verify-status compares those records with the current images and lists each page that isn't a current pass, and why: never run, failed, run on an image that has since changed, or page edited since.

• Image freezes. During a content wave the lab images are pinned with make lab-freeze, so results stay comparable; images change only between waves (make lab-unfreeze).
• Gentoo is verified centrally. While writing, verify the other distributions with make verify PAGE=… SKIP=gentoo-openrc, and use scripts/uv run python scripts/lab.py shell gentoo-openrc to write the Gentoo tabs. Once the wave's images are frozen, make verify-central verifies every page without a current Gentoo pass.
• Front matter follows the records. make verify updates verified: and distros: only when every listed distribution has a current pass, whether from this run or an earlier one.

When a run fails¶

• Every block's script and complete output are written to a log under ~/.cache/magellanic/labs/logs/, one file per distribution per run; a failure prints the log's path. Reviewers read these logs instead of repeating the run.
• A lab that visibly fails to boot is stopped at once, with the reason and the end of its console log: a kernel panic, an emergency shell, an initramfs that can't find the root filesystem, firmware with no bootable disk, or a guest that hasn't written to its disk after a minute. Its files are kept for diagnosis.
• Otherwise the wait for a lab is three times that image's measured boot time (at least two minutes), not a fixed quarter of an hour. Measurements are kept in ~/.cache/magellanic/labs/boot-times.json.

What runs¶

• bash blocks run whole, as an unprivileged user. Show sudo where it's needed, exactly as a reader would type it.
• console blocks run the commands after $ (as the user) and # (as root) prompts. Output lines are shown to readers but not compared. Because prompts mark commands, an output line must not itself start with $ or # followed by a space; shorten or reword such output.
• Blocks inside a distribution tab run only on that family's labs. Other blocks run everywhere.
• Other code blocks (configuration files, output) don't run unless a file directive writes them.

Directives¶

Directives are HTML comments on their own line, so readers never see them.

Placeholder values are resolved to the lab's actual values before a block runs, so <INTERFACE> works inside single quotes, quoted heredocs, and file directive paths and contents.

Lab variables available to commands: $LAB_NIC1 (internet access over IPv4), $LAB_NIC2 (extra NICs are on an isolated segment with no other hosts, and unconfigured), $LAB_DISK1 (a blank 4 GiB disk), $LAB_DISTRO and $LAB_FAMILY.

For example, a how-to that extends a logical volume prepares one invisibly, then checks the result:

<!-- verify: lab disks=1 -->
<!-- verify-setup
pvcreate "$LAB_DISK1"; vgcreate vg_data "$LAB_DISK1"
lvcreate -n lv_data -L 1G vg_data; mkfs.ext4 -q /dev/vg_data/lv_data
mkdir -p /srv/data; mount /dev/vg_data/lv_data /srv/data
-->

The design is recorded in ADR-010.