Testing and Quality Gates

Status: Current Last modified: 2026-10-02 (commit 2d7e886b)

How local verification relates to CI. The local commands themselves live in Developer Verification Checks, which is their single owner; this page says which of them CI repeats and which it does not.

Local pre-merge contract

just gate runs everything CI runs; just push runs it and then pushes. See dev-checks for what it contains and why each step catches something the others cannot.

The list is deliberately not reproduced here: a set of commands assembled from memory forgets the easiest one to forget, cargo test --doc --workspace, which is exactly the one a green just test gives no signal about.

Never-regress gates

The CHAT core has five gates that must stay green for any change touching the grammar, parser, model, validation, serialization or alignment: parser equivalence, roundtrip idempotency (which carries reference-corpus coverage in the same test), the generated spec tests, the validation error corpus, and the gate registry. Each has a fast targeted command, listed with what it protects under Testing, Never-Regress Gates.

Those commands take --tests <filter>, not --test <name>: each crate has one integration binary, so a per-file target name errors out.

A red gate is a bug until proven otherwise, never a test expectation to quietly update. That rule has teeth in both directions: a diagnostic that LOOKS better after a change earns the same scrutiny as one that looks worse: a specific, plausible-looking error message can be a symptom of corruption rather than an improvement.

What CI actually runs

.github/workflows/ci.yml is the authoritative shared signal, and it runs these jobs:

Separate workflows cover release-time lint (release-lint.yml: clippy over both workspaces plus the feature-off build, on a tag or on demand), cross-platform builds (cross-platform.yml), the weekly scheduled clippy (clippy-rolling.yml), crates.io readiness, and the release and desktop pipelines.

What CI does NOT cover

Worth knowing, because these are the gaps where a local run is the only signal:

• The vendored re2c lexer. No workflow installs re2c, so nothing verifies that the committed lexer matches lexer.re. just verify-vendored-lexer is the only check, and it must be run by hand.
• The observation snapshot (spec/observations/example-diagnostics.json) records, for every spec example, the codes each stage emitted and whether the parsed model serializes back byte-exact. It IS in CI, through its currency test, but the gap is human: a regenerated snapshot with a changed entry passes the test, so every diff in it must be adjudicated in the commit as intended or unintended rather than committed because just regen produced it.
• A consumer’s behaviour after regenerating a generated module. A differential over generated TEXT is blind to a change in behaviour precisely when the text is expected to change; only running the consumer’s own suite sees it.

Legacy labels

References to numbered gates such as G0-G14 come from the predecessor workspace and name nothing here. There is no Makefile in this repository.

This page last changed: 2026-10-02 (commit 2d7e886b). The whole book last changed: 2026-10-07 (commit 5e895791).