1# Chapter 12: native, the window
2
3Everything so far comes together here: the SNES in a window, played by the
4zbanks/alttp C bot, with Jev at its goal choice if you ask, and panels around
5the picture saying what is going on. The picture is never drawn over, so
6nothing covers the game's own HUD.
7
8```bash
9nix develop -c buck2 run //apps/native:native -- \
10  "roms/Legend of Zelda, The - A Link to the Past (USA).sfc"
11```
12
13With `--jev` (or `ZB_JEV=1`) Jev makes the bot's goal choice (chapter 5),
14which needs the key in the window's environment:
15
16```bash
17op-env-run -- nix develop -c buck2 run //apps/native:native -- --jev \
18  "roms/Legend of Zelda, The - A Link to the Past (USA).sfc"
19```
20
21Without `--jev` nothing in the window asks Jev and the bot plays exactly as
22upstream's. With it and no key, it plays the same and the Jev panel says why
23Jev is off. Each question blocks the frame loop for its round trip (about
24150 ms measured), so the game pauses for a blink while Jev thinks.
25
26> **Aside: you can take the controller.** Every frame the window hands the
27> bot the human's pad (your keyboard, plus anything an MCP `press` holds)
28> and holds whatever pad the bot returns, exactly the hook upstream's
29> Snes9x used. The bot zeroes the pad and plays, except that **holding X
30> (the `S` key) makes it step aside and pass your pad through**: upstream's
31> own manual override (`alttp.c:57`). The Bot panel says when it has stepped
32> aside.
33
34| Key | Pad |
35| --- | --- |
36| Arrows | D-pad |
37| `X` / `Z` | A / B |
38| `S` / `A` | X / Y |
39| `Q` / `W` | L / R |
40| Enter / Backspace | Start / Select |
41| `-` / `=` | Playback speed down / up |
42
43## What happens before the window opens
44
451. The ROM is patched the way the Randomizer the bot was written for differs
46   from vanilla where the bot can tell (`zbanks::rando::patch_rom`).
472. The bot starts: `ap_init` caches its RAM pointers and loads the states
48   `home` and `hpegs` from `<rom>.states/`. Make `home` once with chapter
49   11's `--make-home home --states "roms/<rom>.states"`. Without it the bot
50   starts at power-on, where it presses nothing, and the log says so.
513. Unless `--fresh`, the machine then goes back to `resume`, the state the
52   last window kept, as a Snes9x user loading their own state after the bot
53   was up.
54
55The bot's printf trace goes to `run/zbanks-window/bot.log` (this process's
56stdout is a pipe that a thread copies into that file, rotated at 50 MB with
57three old files kept). Its debug files (`goals.txt`, `map`, `map.pbm`,
58`full_map.dot`, ...) go beside it. The window's own messages are stderr.
59
60## The panels
61
62- **Bot**: the bot's own line (`ap_info_string`, what upstream's Snes9x drew
63  over the picture), the frame, where it puts Link, its task list (current
64  first, green) and its goals with the score the planner last gave each
65  (dim: unsatisfiable or over the limit; green: complete), `assert_bp` traps
66  it has stepped over and reads of unmapped memory (red). A red "STALLED"
67  line when the bot has stopped playing, and under it what the bounded
68  recovery is doing about it: amber while it is still trying, red once it
69  has given up for good.
70- **MCP**: the server's address, a light (amber: listening; green: a client
71  has been heard from; red: the port would not bind, with the reason), and
72  a log of every tool call with its arguments and outcome.
73- **State**: the game in words, from chapter 3.
74- **Jev**: whether it is on; the shared spend guards in one line (questions
75  in the last minute against the bucket of 30, every process's; dollars in
76  the last hour; spent so far, with no cap); in amber, whatever is stopping
77  questions right now ("throttled: bucket empty, next question in 12s"
78  clears by itself, "out of credit (API)" does not); the totals; a usage
79  graph by game frame across the current recording, with a playhead that
80  follows the scrubber; and a newest-first history of every choice this
81  window made, plus the last window's (loaded from
82  `run/zbanks-window/jev.jsonl`). Click a row to see every option in the
83  words Jev was given, its probability, the pick (`>`) and upstream's own
84  (`u`), and for a choice that asked, the exact request, tokens, latency and
85  cost.
86- **Pad**: the pad the console held on the last frame (the bot's), lit; a
87  button an MCP `press` holds carries its confidence as a percentage.
88- **Scrubber**: directly under the picture: play/pause, skip to end, a speed
89  control (0.25x to 8x), "run the bot from here", and a timeline bar to click
90  or drag. Marks show where other branches forked off.
91
92Bot and MCP share the left column; State and Jev share the right. The layout
93is one function of the window's size (chapter 7).
94
95## The replay timeline
96
97Every emulated frame of live play is recorded (chapter 6) to
98`run/zbanks-window/replay/<recording-id>/`, and the scrubber turns that into
99VCR controls:
100
101- **Click or drag the bar** to jump to any recorded frame and pause there:
102  the nearest keyframe, then the log replayed forward, headless. The bot is
103  never asked anything while scrubbing.
104- **Play** replays the log from a paused frame; running off the end hands
105  back to the live bot at exactly the frame it had reached, byte for byte,
106  so nothing is reconstructed and nothing stutters.
107- **Speed** scales both replay and live play; sound mutes at anything but
108  1x, and the choice survives a restart.
109- **Run the bot from here** restarts the window with `--branch-from`, which
110  replays the whole history up to that frame through a *fresh* bot (a second
111  one cannot exist in this process) before going live as a new branch. The
112  scrubber shows "rebuilding bot: N/M frames" meanwhile. The old future is
113  kept.
114
115> **Try it.** Let the bot play for a minute, drag the scrubber back thirty
116> seconds, and press play. Then try "run the bot from here" and watch the
117> new branch's mark appear on the bar.
118
119## Restarting without losing the game
120
121A snapshot is the whole machine (1.3 MB), kept as `<rom>.states/<name>.state`
122beside the ROM (gitignored). The window keeps one for itself, `resume`: every
12330 seconds, on exit, and before a `restart`. It resumes from it unless given
124`--fresh`, and a `restart` drops `--fresh` from the arguments it re-runs
125with, so it carries on the game.
126
127The bot's state is *not* in a snapshot: it is the C bot's globals, and a
128restart starts a new bot. What it learned of the map is carried the way
129upstream carried it, as a map file: whenever the window keeps `resume` it
130also exports the bot's map to `run/zbanks-window/map_export.txt` (written to
131a `.part` name and renamed, so a killed window leaves the last whole one),
132and a window starting copies that to `map.19.txt`, which the new bot's first
133tick imports. Each goal's outcome is carried too (chapter 4's
134`goal_outcomes.jsonl`); the goal list itself is rebuilt from the map.
135
136A snapshot names the core version and the cartridge it was taken from, and
137is refused by any other; the window then plays from `home` and says why. The
138cartridge here is the ROM as patched for the bot, so snapshots of the
139unpatched image are refused.
140
141The cartridge save is read from and written to a `.sav` beside the ROM
142(gitignored), every few seconds and on exit. `--sound` turns audio on; it is
143off by default because under WSLg it cuts in and out (heard 2026-09-20).
144
145## MCP: the window answers questions
146
147The window serves MCP (chapter 8) over streamable HTTP at
148`http://127.0.0.1:7637/mcp`. A client does not connect to that. It starts
149`native mcp`, the same binary with no window, which serves MCP on stdio and
150passes everything on to whichever window is running; `../../.mcp.json`
151starts it through `../../tools/mcp/launch.sh` (chapter 25).
152
153| Always | What |
154| --- | --- |
155| `state` | The game state, as JSON. |
156| `bot` | What the bot says about itself, as JSON: info line, tasks, the first goals with their scores, goal count, where it puts Link, traps, stray reads, and the savestate calls `ap_init` made. |
157| `jev_history` | Jev's recent goal-choice history, newest first, as JSON: what the Jev panel's history shows, with the exact request sent when it asked. |
158| `frame` | The game picture as a PNG, at the console's resolution. |
159| `window` | The whole window as drawn, panels included. |
160| `read_wram` | Bytes of work RAM by SNES address, as hex. |
161| `states` | The names of the snapshots kept beside the ROM. |
162| `budget` | Jev's spend guards from the shared ledger: spent so far, with no cap; the question bucket (and seconds until it refills); the hour breaker (and seconds until it reopens); holds in flight; whether the vendor says the account is out of credit. Nothing in it needs clearing by hand. |
163| `dev_mode` | `on: true` adds the tools below; `false` takes them away. |
164| `patch_info` | This process's pid and `subsecond::aslr_reference()`, for `tools/hotpatch/patch.sh`. |
165
166| Dev mode only | What |
167| --- | --- |
168| `press` | Hold buttons for a number of emulated frames, then return the state. The buttons go to the bot as the human's pad, so they reach the game only while X is among them (the bot steps aside) or the bot is in a mode it does not play. Optional `confidence`, 0 to 1. |
169| `save_state` | Keep the machine exactly as it stands, under a name. |
170| `load_state` | Become the machine kept under a name. The bot carries on. |
171| `power_cycle` | Turn the SNES off and on again; the battery save is kept. |
172| `hot_patch` | Apply a `tools/hotpatch`-built patch (a path to its `patch-N.json`): no restart, the game keeps running. Rust host code only. |
173| `restart` | Re-run the binary this window was started from, to pick up a new build. |
174
175## Changing the code while it runs
176
177```bash
178nix develop
179tools/hotpatch/patch.sh //apps/native:native
180```
181
182rebuilds the window's own code (`//apps/native:app`: `src/app.rs`,
183`src/activity.rs`) and `packages/panels`, builds a patch from the fresh
184objects (chapter 27), and delivers it over MCP: dev mode on, `hot_patch`,
185dev mode off. The picture and the panels are redrawn on the very next frame,
186same process, same game. The executable itself is never rebuilt for this:
187`src/main.rs` is a three-line dispatcher with no patch points, so a panels
188edit never pays for relinking the whole eframe/wgpu binary.
189
190Most edits to `logic`/`ui`'s bodies and to `packages/panels` patch. Changing
191the *shape* of `App`, or of a struct crossing the patch boundary, does not,
192and needs a `restart`. The C bot is not hot-patchable at all: a change there
193is a `buck2 build` and a `restart`. The window closes and reopens about a
194second later on the same frame of the same game, and the client is still
195connected, because what it is connected to is `native mcp`.
196
197## For the people who maintain it
198
199Left running in the background, start it so that its end is written down
200whatever the cause (a window once vanished with no trace, 2026-09-21):
201
202```bash
203nohup nix develop -c sh -c 'buck2 run //apps/native:native -- \
204  "roms/Legend of Zelda, The - A Link to the Past (USA).sfc"; \
205  echo "native exited with status $? at $(date -Is)"' > run/native.log 2>&1 &
206```
207
208### In this folder
209
210| Path | What |
211| --- | --- |
212| [src/main.rs](src/main.rs) | The executable: `native mcp` goes to `mcp::proxy::run`, anything else to `app::window`. |
213| [src/app.rs](src/app.rs) | The window, as its own `rust_library` (`//apps/native:app`): arguments, the bot and its stall/recovery, Jev, sound, saves, `resume`, the map carry-over, the replay recorder and scrubber, `App::answer` for every MCP `Request`, the patch points `logic_impl`/`ui_impl`, and the bot.log rotation. |
214| [src/activity.rs](src/activity.rs) | The MCP panel: where the server listens, whether a client has been heard from, and every tool call that came in. |
215| [BUCK](BUCK) | `:app` (with the hot-patch flags), `:test` (the bot.log rotation tests), and `:native`, the executable, which keeps `-Csave-temps`/`-Clink-dead-code` for the "main" sentinel object a patch needs. |
216
217Measured 2026-09-20: the window opens as a native Wayland surface, wgpu runs
218on Vulkan (Dozen, under WSL), and the process sits at about half of one core.
219The hot-patch build flags cost nothing measurable (50.6% CPU before and after
220loading a build with them); the emulator core is in `packages/console`, which
221does not carry them.
222
223← Previous: [Chapter 11, zbanks/](../zbanks/) · Up: [apps](../) · Next: [Chapter 13, jevprobe/](../jevprobe/) →