Check & review: check, changes, diff, review, ack, accept¶
vouch check¶
The gate: cited values exist, are fresh, and are acknowledged. Read-only, never runs your code, targets under a second.
| Flag | Meaning |
|---|---|
--strict |
treat warnings as errors (CI, agents) |
--quiet |
print only problems; pending expect() values, the planned state of a draft, are one line pointing at vouch todo (the pre-commit hook uses this). Changes a build has already reported are one line in either mode |
--verbose |
also print informational notes |
--json |
the vouch/1 JSON envelope |
--no-env |
skip the package-version drift check |
--paper FILE |
check only this paper, when vouch.toml lists several; FILE is its main, relative to the project root or the working directory. A file no [[paper]] lists is an error (exit 2). |
Exit codes: 0 pass, 1 fail, 2 could not check.
Full check list: Checks and lints.
vouch lint¶
The edit hook's check, on any .tex file: unknown keys, malformed macros, and
numbers typed by hand, each with its fix. The hook runs only after an agent's own
edits, so a file changed by a script, or by hand, waits for the next vouch check;
vouch lint checks it at once, in the hook's time (it reads the index
vouch build writes).
With no file, it checks every file of every paper. It exits 1 if any file has a problem, and also lists the values the files cite that are owed.
vouch changes¶
Cited values that changed since last acknowledged, with the citing sentences.
--md FILE writes a shareable Markdown review report, e.g. for co-authors.
vouch diff¶
What moved between two states of the store: every recorded value, parameter,
claim, table cell and figure, not only the cited ones. The default is the last
commit (HEAD) against the values on disk now. Run it before committing a
re-run's records, or to write up a PR.
| Form / flag | Meaning |
|---|---|
vouch diff |
HEAD vs the working tree |
vouch diff REV |
REV (a branch, tag, commit, HEAD~3, ...) vs the working tree |
vouch diff REV REV2 |
two revisions |
--key PATTERN |
only keys containing this substring or matching this glob (repeatable) |
--cited |
only keys the paper cites |
--all |
also each mean ± std's subfields (.mean, .std, ...) and tuple elements |
--json |
the vouch/1 envelope: from, to, counts, unchanged, items, notes |
--md FILE |
a Markdown report, one table per group, e.g. for a PR description |
$ vouch diff
vouch diff: HEAD → working tree · 5 changed · 1 added · 0 removed · 12 unchanged
run train
● toy.centroid.acc 80.2 ± 3.5% → 79.3 ± 3.1% ↓ Δ -0.867 pts, -1.1% relative worse ! sample size changed (n=5 -> 3)
● toy.majority.acc 49.9 ± 4.5% → 50.0 ± 3.5% ↑ Δ +0.1 pts, +0.2% relative better ! sample size changed (n=5 -> 3)
● toy.centroid.n_test 200 → 50 ↓ Δ -150, -75.0% ! large move (75%)
● train.param.seeds 5 → 3 ↓ Δ -2, -40.0% ! large move (40%)
toy.majority.n_test 200 → 50 ↓ Δ -150, -75.0% ! large move (75%)
+ toy.centroid.f1 0.81
● cited in the paper; `vouch changes` lists the sentences to re-read
How to read a line:
+added,-removed,~a figure file that changed.●means the key is cited.- Δ is the absolute and relative change. For percentages it is in percentage points, as the paper prints them. For a mean ± std it is the change in the mean.
- ↑ / ↓ is the direction. better / worse appears when the key has
better = "higher"or"lower"(fromrecord(..., better=)or[metrics]). A claim that stops holding isworse. - ! lists the same problem heuristics
vouch builduses: sign flip, large move (changes.rel_threshold), order of magnitude, sample size changed, moved to another run, type changed.
Groups come in this order: each run, derived values, claims, tables, removed
keys, figures. Within a group, cited keys come first and worse before the rest.
Revisions are read with git (git ls-tree, git cat-file), and nothing is
checked out. Formats, units and better come from the current vouch.toml
on both sides, so both sides are rendered and judged the same way. The
working-tree side uses the last vouch build's derived values. If they are out of
date, a note says so. Exit codes: 0 (with or without differences), 2 when
it could not compare (not a git repository, unknown revision).
More on reading the output and choosing revisions: Comparing runs and live rebuilds.
vouch diff or vouch changes?
vouch changes covers cited values against what someone last
acknowledged, and waits until the sentences are re-read. vouch diff
covers every recorded value against a git revision and answers "what
did this re-run do?". It never acknowledges anything.
vouch review¶
Step through pending changes interactively.
$ vouch review
SUSPICIOUS cifar.vit.acc 91.2\% → 72.4\% … (sentences shown)
[a]ck [A]ck all remaining [s]kip [o]pen [q]uit >
o opens $EDITOR / code -g at the first citing line. A (or all)
acknowledges the change shown and every one after it. It asks first, with their
count and how many are suspicious, and a no returns to the change shown. After
the last change it asks why, and records the answer with each acknowledgment. An end of input (a stdin
redirected from /dev/null, which Windows reports as a terminal) quits, and
acknowledges nothing. In a script, use vouch changes and vouch ack KEY.
vouch ack¶
Acknowledge changed values after re-reading their sentences.
| Flag | Meaning |
|---|---|
KEY... |
keys, globs ('cifar.*'), or a table name — every changed cell of it |
--all |
every pending change |
--run RUN |
every pending change from this run |
--why TEXT |
recorded with the acknowledgment (default "acknowledged") |
One of KEY..., --all or --run is required. vouch ack and vouch accept
rebuild the generated files themselves, so highlights and tooltips update
without a separate vouch build. Acknowledging is a human action — see
Change notification.
vouch accept¶
Record that a stale run's result still stands.
--why is required and is recorded with who accepted it and when, in
.vouch/accepted.toml. See Freshness.