1# Chapter 15: tests/fixtures, money spent once
2
3A passing test suite proves the extension does what the tests say. It does
4not prove Jev answers the way the design assumes. Some claims can only be
5settled against the real model: that two layouts give the same answers, how
6far repeats of one question spread, whether a label-tree search lands on the
7right leaf, whether `ORDER BY jev_prob … LIMIT` ranks well. Those are the
8*release gates*, and each needs live answers, which cost real money.
9
10So each gate's live run was recorded once, on 2026-09-26, against
11`jev-1.13.0`, by `tools/record-gates --send` (chapter 24), with the spend
12capped by `--max-cost`. The four files here are those recordings: every
13request exactly as sent, and every response exactly as it came back,
14headers included. From then on the `gate_*` tests replay them through the
15extension, and spend nothing.
16
17```mermaid
18flowchart LR
19  R["tools/record-gates --send<br/>(once, on the live key)"] -->|"writes"| F["tests/fixtures/*.json"]
20  F -->|"read by"| M["jev-mock in replay mode"]
21  E["the extension, as it is today"] -->|"its requests"| M
22  M -->|"same bytes: the recorded answer<br/>other bytes: fail, naming the request"| E
23```
24
25A replay matches by the **exact request bytes**. If a change to the
26extension alters a single byte of what it sends (a key order, a space, a
27question's wording, a lookahead the search now asks), the mock has no
28recorded answer for it, and the test fails naming the request. That makes
29these files a tripwire as much as a measurement.
30
31> **Aside: what the money bought.** Read
32> [layout_parity.json](layout_parity.json) and you will meet four people,
33> each asked "Could this person work from home?" on their own: Ana the
34> backend engineer 0.88, Rui the nurse 0.27, Inês the accountant 0.81, João
35> the bus driver 0.05. Then the same four in one request, each row inside
36> its own question: 0.84, 0.21, 0.78, 0.06. Close, but not the same, and
37> four rows prove nothing either way, which is why that layout stays
38> switched off until a proper measurement says it is safe. And in [ranking.json](ranking.json), a
39> separate recording, Ana is 0.87 and Rui 0.28: the same bytes sent, a
40> hundredth apart. That is why the contract calls a cached answer "one
41> recorded draw, not a fixed point".
42
43> **Try it.** `nix develop -c buck2 test //tests:gate_ranking` replays the
44> ranking gate through a throwaway Postgres, with no network at all.
45
46## For the people who maintain it
47
48| File | What |
49| --- | --- |
50| [layout_parity.json](layout_parity.json) | Row as state (one request per row) and row in instructions (every row a question in one request), for the same rows and question. The extension sends only the first; the second is checked for an answer per row. |
51| [unsure_band.json](unsure_band.json) | One question about one row; its repeats are one distinct request, so it holds one answer. |
52| [label_tree.json](label_tree.json) | The whole label-tree search for one row: both rounds, lookahead included, and the "does any label fit" question. |
53| [ranking.json](ranking.json) | `ORDER BY jev_prob … LIMIT` over two rows, one request each. |
54
55Each file is `{"model", "exchanges": [{"request", "responses": [{"status",
56"headers", "body"}]}]}`, the format `tests/replay.rs` pins. The rows, the
57question and the SQL that produce these requests are in
58[../support/gates.rs](../support/gates.rs).
59
60← Previous: [Chapter 14, support/](../support/) · Up: [tests](../) · Next: [Chapter 16, nix/](../../nix/) →