jevsnes.git / research / subsecond-patch-build.md
subsecond-patch-build.mdpreviewsubsecond-patch-build.mdsource812 lines · 47.1 KB · raw

Subsecond hot-patching: what dx does, and how to reimplement it under buck2

Source: ~/src/github.com/DioxusLabs/dioxus at tag v0.7.10 (commit 57d6794ad60b949e5bd8aa282f6f8c3dc97a365e), a blobless clone (git show fetches blobs on demand). All cited files are dual MIT OR Apache-2.0 (packages/subsecond/subsecond/Cargo.toml:7, packages/subsecond/subsecond-types/Cargo.toml:6, packages/cli/Cargo.toml:8, packages/devtools/Cargo.toml:6, packages/devtools-types/Cargo.toml:6; repo root carries LICENSE-MIT + LICENSE-APACHE), so copying code/logic from them and keeping the licence notice is fine.

Scope note: subsecond-cli/a standalone harness binary does not exist in this checkout — I grepped git ls-tree -r --name-only HEAD | grep subsecond and the only subsecond dirs are subsecond/, subsecond-types/, and subsecond-tests/{cross-tls-crate,cross-tls-crate-dylib,cross-tls-test} (cross-TLS regression fixtures, not a build-tool example). All "how does the patch get built" logic lives in packages/cli/src/build/{request.rs,link.rs, patch.rs} and packages/cli/src/{rustcwrapper.rs,cli/link.rs}, not in a separate crate.

The mechanism dx uses to see rustc/linker invocations at all

dx never parses cargo's own progress output for build flags — it makes itself both the rustc wrapper and the linker, for every workspace-member crate, on every build (fat or thin):

  • request.rs:1646: cmd.env("RUSTC_WORKSPACE_WRAPPER", Workspace::path_to_dx()?); — this wraps only workspace-member crates (not external deps), letting cargo invoke dx rustc <real-args> for each one.
  • request.rs:1637-1641: DX_RUSTC_WRAPPER_ENV_VAR ("DX_RUSTC", defined rustcwrapper.rs:36) points at a per-build-mode scope directory where captured args get written.
  • rustcwrapper.rs:46-48: is_wrapping_rustc() just checks that env var is set.
  • rustcwrapper.rs:53-94 (run_rustc): captures args() + vars() into a RustcArgs{args, envs}, always writes it to {crate_name}.{lib|bin}.json (write_rustc_args, rustcwrapper.rs:96-136) even for link steps ("the tip crate's bin target is typically only observed during the final link invocation" — comment at link.rs:68), then either delegates to the linker-interception path (has_linking_args(), rustcwrapper.rs:138-170, which checks for a .o arg or -flavor, including inside @responsefile command files) or actually execs rustc with the captured args/envs (rustcwrapper.rs:79-93).
  • Simultaneously dx sets itself as the linker too (cargo_build_arguments, request.rs:1748-1753: -Clinker={path to dx}, gated on BuildMode::Thin | BuildMode::Fat), and LinkAction::write_env_vars (cli/link.rs:85-109) exports DX_LINK=1, DX_LINK_ARGS_FILE, DX_LINK_ERR_FILE, DX_LINK_TRIPLE, optionally DX_LINK_CUSTOM_LINKER.
  • LinkAction::run_link_inner (cli/link.rs:131-242) is "NoLink": it writes the linker's argv to DX_LINK_ARGS_FILE and then, if no real linker was given, does not link at all — it writes a dummy empty object file (object::write::Object::new(...).write()) at the expected -o//OUT: path "to satisfy rustc's use of llvm-objcopy" (comment cli/link.rs:191-196). The doc comment on LinkAction (cli/link.rs:7-28) is explicit about why: rustc has no supported way to stop short of a full link, so this is a hack to get the arguments rustc would have used without paying for (or trusting) a real link.

This two-pronged interception (RUSTC_WORKSPACE_WRAPPER + -Clinker=dx) is how dx gets a reliable, per-crate, per-platform record of every rustc invocation's exact args/envs and the exact final link line, without depending on cargo's own diagnostic JSON (comment link.rs:1-16 explains the alternative — parsing cargo's printed link args — was rejected as unreliable across platforms, esp. Windows response files).

1. The "fat" (initial) build

Fat is a normal cargo rustc invocation (request.rs:1613-1649, cargo_build_command), with these additions layered on top of the user's own profile/rustflags:

  • -Csave-temps=true and -Clink-dead-code (request.rs:1799-1801), added whenever build_mode is Thin or Fat (request.rs:1795). Comment: "link-dead-code: prevents rust from passing -dead_strip to the linker since that's the default" / "save-temps=true: keeps the incremental object files around, which we need for manually linking" (request.rs:1796-1798). These are captured by the rustc wrapper and so get replayed automatically on every future thin build too (comment request.rs:1792-1794).
  • -Clinker={dx} (request.rs:1748-1753) — always for Thin/Fat, so the "NoLink" capture above fires.
  • Platform rpath flags (-Wl,-rpath,$ORIGIN etc., request.rs:1775-1778 for Linux) and, for wasm only, PIC/export flags (request.rs:1852-1866) — not relevant to native Linux.
  • No -Zunstable-options, no nightly requirement, and no explicit -Crelocation-model=pic on native (non-wasm) targets. Proof of absence: grep -n "relocation-model" packages/cli/src/build/*.rs returns exactly two hits, both gated on wasm/wasi (request.rs:1594-1596, link.rs:573-575, link.rs:358-363 doc comment "I think we can make relocation-model=pic work for non-wasm platforms" — i.e. not done today for native). Nightly is used only for an unrelated unit-count estimate (cargo +nightly unit-graph, request.rs:2349-2360), not for hotpatch compilation; grep -rn nightly over packages/cli/src and packages/subsecond turns up nothing else load-bearing.
  • No --export-dynamic/-rdynamic blanket flag on Linux at the cargo_args level; instead the fat link step (next section) surgically exports just main via -Wl,--export-dynamic-symbol,main (link.rs:995).
  • Debug-assertions/opt-level are not forced by dx — they come from whatever cargo profile the user picked. But subsecond::call/try_call are cfg!(debug_assertions)-gated no-ops in release (subsecond/src/lib.rs:250-254, :411-414), so hot-patching is a no-op unless the profile has debug-assertions = true (dev profile, by default).

What's captured, and by what: the rustc-wrapper writes one JSON per workspace crate ({crate}.lib.json / {crate}.bin.json) containing {args: Vec<String>, envs: Vec<(String,String)>} (rustcwrapper.rs:26-30); the linker-wrapper writes the raw final link argv, one arg per line, to link_args_file (cli/link.rs:145). Both are read back after the cargo build child process exits: workspace_rustc_args = self.load_rustc_argset() (request.rs:1152) assembles a WorkspaceRustcArgs{link_args, rustc_args: HashMap<String,RustcArgs>} (rustcwrapper.rs:11-24) from the on-disk JSONs plus the plain-text link-args file.

For Fat specifically, run_fat_link (link.rs:788-1142) is then invoked (request.rs:1157-1160) to do the real link, because the wrapper only wrote a dummy object file:

  • Takes the tip crate's captured .bin.json args + the captured link_args.
  • Filters link_args down to the .rlib entries (link.rs:809-815).
  • Builds (and caches, keyed on rlib name+size+mtime+dx's own git commit hash, link.rs:817-849) a "fat archive": an ar archive (libdeps-{hash}.a) containing every .rcgu.o/.obj member pulled out of every workspace .rlib (external/toolchain rlibs are kept as normal .rlib refs instead, link.rs:865-922 — "Skip compiler rlibs since they're missing bitcode").
  • Splices that archive into the original link command with -Wl,--whole-archive … -Wl,--no-whole-archive on the Gnu flavor (link.rs:957-965) — i.e. it forces every object, not just what the linker would keep, into the final binary, so those symbols exist to be jumped back to later.
  • Appends -Wl,--export-dynamic-symbol,main (Gnu/Linux, link.rs:993-996; _main+-Wl,-exported_symbol,_main on Darwin, /EXPORT:main on MSVC) — this is the only blanket "export a symbol" flag on Linux, and it's main specifically, used purely as the ASLR reference point (see §3), not a general --export-dynamic.
  • Runs the real linker (select_linker, link.rs:1240-1277: cc on Gnu/Linux) directly (link.rs:1084-1089), i.e. dx invokes cc itself here, bypassing rustc entirely for this final link.

2. The "thin" (patch) build

Trigger: BuildMode::Thin{ modified_crates, workspace_rustc_args, aslr_reference, cache, .. } (compile_workspace_hotpatch, link.rs:132-335).

Which crates get recompiled: every crate in the cumulative (since the last fat build) modified_crates set gets replayed — not just the tip. AppBuilder::patch_rebuild (builder.rs:364-456) BFS-walks workspace_dependents_of from the changed files' crates to add the cascade (comment builder.rs:353-358: "editing a leaf crate forces parent crates' generic instantiations to change too"), then keeps that set forever until the next full/fat rebuild (builder.rs:466-467: "A full rebuild resets all accumulated hotpatch state"). This contradicts the subsecond crate's own doc comment ("Subsecond currently only patches the 'tip' crate… We plan to add full workspace support in the future," subsecond/src/lib.rs:67-75) — that doc appears stale relative to the CLI's actual v0.7.10 implementation. UNVERIFIED: whether this workspace-replay path is fully reliable for arbitrary dependency-crate changes (the subsecond doc's caveats about non-deterministic build graphs and generic-forwarding cascades may still apply in edge cases) — I found no test exercising a multi-crate thin build in this checkout.

How non-tip crates are recompiled: workspace_hotpatch_replay_order (link.rs:612-662, Kahn's-algorithm topological sort over the modified subgraph only) determines order; for each, workspace_hotpatch_replay_args (link.rs:586-603) looks up its original captured .lib.json args, and compile_dep_crate (link.rs:518-584) runs rustc directly (not cargo) with those exact args verbatim (env-cleared, replaying the captured envs, stripping -Clinker=… so it doesn't recurse into dx's link interception, and adding -Crelocation-model=pic only for wasm/wasi). This overwrites the crate's .rlib in place at its original path (link.rs:46-47 doc comment).

The tip crate is then rebuilt via the normal cargo_build path (link.rs:161, i.e. cargo rustc again) but under BuildMode::Thin, which routes cargo_build_command into the other branch (request.rs:1571-1601): runs rustc directly (not cargo) with the tip's captured fat-build args (rustc_args.args[1..]), env-cleared and wrapper vars removed, plus -Clinker={dx} again so this recompile also gets intercepted for object capture. No new flags are added for the tip crate on native platforms in this thin path — it reuses exactly what was captured during the fat build (the -Csave-temps/-Clink-dead-code already baked in).

What gets linked into the patch, and into what:

  • temp_objects: every .rcgu.o from the tip's fresh link args (link.rs:190-197) — these are per-codegen-unit object files straight from rustc's -Csave-temps output, i.e. only the tip crate's own new/changed translation units, not whole-crate objects.
  • workspace_rlibs: the (now freshly overwritten) .rlib files for every other replayed crate (link.rs:199-200, workspace_hotpatch_link_rlibs at link.rs:672-723), resolved by reconstructing the exact rlib filename from each crate's captured --out-dir + -C extra-filename (find_rlib_for_crate, link.rs:1285-1349).
  • On non-wasm (our case): a generated stub object — create_undefined_symbol_stub (patch.rs:852-1259) — is appended (link.rs:219-231). This is the "resolve missing symbols against the running binary" step (§ next paragraph).
  • Everything is linked with thin_link_args (link.rs:341-511), which for LinkerFlavor::Gnu (Linux) starts from -shared (link.rs:429-436: -shared -Wl,--eh-frame-hdr -Wl,-z,noexecstack -Wl,-z,relro,-z,now -nodefaultlibs -Wl,-Bdynamic), i.e. the patch output is a shared object (.so), not an executable — confirmed by patch_exe (link.rs:737-756): extension is "so" for LinkerFlavor::Gnu, filename lib{name}-patch-{timestamp_ms}.so next to the main exe. Original args are filtered down to just -L, -l*, -m*, -fuse-ld*/-Wl,-fuse-ld*, -B* (needed because "Rust 1.86+ injects -B/gcc-ld + -fuse-ld=lld", comment link.rs:442-444), and -ld-path (link.rs:447-463) — i.e. no rlibs/objects from the original fat link are reused as linker inputs here, only search-path/toolchain-selection flags. The actual linker binary is select_linker() → self.workspace.cc() for Gnu (link.rs:1255), i.e. the same cc used for the fat link, invoked directly (not through rustc), env-cleared but with PATH restored on Linux specifically (link.rs:277-280, compile_workspace_hotpatch's own linker call at link.rs:287-292).

How symbols that live in the ORIGINAL binary are resolved (Linux/native, not the WASM ifunc path): create_undefined_symbol_stub (patch.rs:852-1259) — not --just-symbols, not an asm/ELF diff, a hand-generated object file of absolute-address jump stubs:

  1. Collect every undefined symbol across the objects about to be linked (collect_stub_symbols_from_path/_bytes, patch.rs:1262-1308, via the object crate reading .o/.rlib/.a member symbol tables), minus anything also defined among them (patch.rs:867-870).
  2. Compute aslr_offset = aslr_reference - aslr_ref_address where aslr_ref_address is the original fat binary's on-disk main symbol address (from the cached symbol table, patch.rs:926-939) and aslr_reference is the live process's actual main address, reported by the running app over the devtools websocket (§4) and threaded in from BuildMode::Thin{ aslr_reference, .. }.
  3. For each undefined symbol found in the original binary's cached symbol table (HotpatchModuleCache.symbol_table, populated once per fat build via the object crate, patch.rs:263-291, and reused across thin builds for speed — "dropped the patch time from 3s to 1.1s", patch.rs:66): abs_addr = sym.address + aslr_offset (patch.rs:966), then hand-emit machine code that jumps to that absolute address and register it as a new defined symbol with the undefined symbol's exact name in a freshly built object::write::Object:
    • Linux/x86_64 text symbols: SymbolKind::Text branch (patch.rs:1083-1091): FF 25 00 00 00 00 (jmp [rip+0]) followed by the raw 8-byte absolute address appended right after — i.e. a RIP-relative indirect jump whose target slot is the very next 8 bytes.
    • Data symbols: written as SymbolSection::Absolute at abs_addr directly (patch.rs:1245-1254) — no jump stub needed, the symbol just is that address.
    • TLS symbols: handled specially by copying real init bytes out of the cached .tdata/__thread_data section into a new TLS symbol (patch.rs:1163-1226) rather than writing a bogus absolute address.
  4. This generated object (stub.o) is linked in alongside the tip's .rcgu.os and the workspace .rlibs.

So on Linux/x86_64 the whole "resolve against the live process" trick is: one small hand-built ELF object of jmp [rip+0]; <8-byte address> stubs, computed from (a) the fat build's own captured symbol table and (b) the process-reported ASLR reference — no --just-symbols, no dlopen-time symbol patching, no linker feature at all beyond ordinary symbol resolution against this synthetic object.

3. The jump table

subsecond_types::JumpTable (subsecond-types/src/lib.rs:8-47):

pub struct JumpTable {
    pub lib: PathBuf,               // the patch .so/.dylib/.dll/.wasm path
    pub map: AddressMap,            // old_addr -> new_addr, u64 keys, identity-hashed (no re-hash of an address)
    pub aslr_reference: u64,        // "main" symbol's address in the OLD (running) binary
    pub new_base_address: u64,      // "main" symbol's address as recorded IN the patch's own symbol table
    pub ifunc_count: u64,           // wasm-only: size to grow the indirect function table by
}

AddressMap = HashMap<u64, u64, BuildHasherDefault<AddressHasher>> where AddressHasher (subsecond-types/src/lib.rs:53-92) is a no-op/identity hasher (write_u64 just stores the value) — addresses are unique by construction so hashing them is wasted work.

How "changed" is decided — it isn't, at the jump-table level. There is no symbol-diffing step anywhere I could find (grepped patch.rs/link.rs for "diff"/"changed" — only comments describing intent, e.g. link.rs:177 "ThinLink… Diffing of object files for function invalidation" is aspirational doc-prose, not implemented code found in this checkout). What actually happens: create_native_jump_table (patch.rs:403-443) parses the whole old binary's symbol table (cached) and the whole freshly-linked patch .so's symbol table (obj2.symbol_map()), and inserts an entry for every name present in both (patch.rs:420-424):

for (new_name, new_addr) in new_name_to_addr.iter() {
    if let Some(old_addr) = old_name_to_addr.get(*new_name) {
        map.insert(old_addr.address, *new_addr);
    }
}

"Changed" is decided upstream, structurally: only the tip crate's .rcgu.os (i.e. whatever codegen units rustc actually re-emitted) plus replayed dependency crates make it into the patch .so at all; anything not recompiled simply isn't in new_name_to_addr and gets no entry (its old address is used unchanged, since callers only redirect through entries that exist in jump_table.map, subsecond/src/lib.rs:396-403,919/936). So the "diff" is really "what did rustc choose to recompile", not a post-hoc comparison — every symbol common to old and new gets remapped whether or not its bytes actually changed, which is why subsecond/src/lib.rs:118-120 notes ptr_address "will always return the most up-to-date version… equivalent to a version of the function where every function is considered 'new'."

sentinel = main_sentinel(triple) (referenced patch.rs:426-434, "main" on Linux) is used as the reference: new_base_address = patch's own main address, aslr_reference = the original binary's cached main address (not yet corrected for the live process — that correction happens at apply time, see below).

What apply_patch does on Linux/unix (subsecond/src/lib.rs:498-547, pub unsafe fn apply_patch(mut table: JumpTable) -> Result<(), PatchError>):

  1. libloading::Library::new(&table.lib) — an ordinary dlopen (subsecond/src/lib.rs:507-512), leaked forever (never dlclose'd — comment explains dropping could run destructors mid-crash-prone-state).
  2. old_offset = aslr_reference() - table.aslr_reference — aslr_reference() (subsecond/src/lib.rs:716-750) is dlsym(RTLD_DEFAULT, "main") in the currently running process, so this is the ASLR slide of the original binary between compile time and now.
  3. new_offset = dlopen'd_lib.get::<*const ()>("main") - table.new_base_address — same idea for the just-loaded patch .so.
  4. Every entry in table.map gets rebased: (old_key + old_offset, new_val + new_offset) (subsecond/src/lib.rs:535-544).
  5. commit_patch(table) — atomically swaps a leaked, Box-owned JumpTable into a global AtomicPtr (Relaxed ordering, subsecond/src/lib.rs:279, 308-321) and runs any registered hot-reload handlers.

Callers then look up known_fn_ptr (the function's compile-time address, taken via transmute of the function item, subsecond/src/lib.rs:396/430) in jump_table.map and, if present, call through the mapped address instead (HotFn::try_call, subsecond/src/lib.rs:411-441; the macro-generated call_as_ptr, subsecond/src/lib.rs:902-941, handles Android pointer-tagging specially but is a no-op branch on Linux).

4. The wire protocol

Types: packages/devtools-types/src/lib.rs.

pub enum DevserverMsg {
    HotReload(HotReloadMsg), HotPatchStart, FullReloadStart,
    FullReloadFailed, FullReloadCommand, Shutdown,
}                                                    // lines 9-28
pub struct HotReloadMsg {
    pub templates: Vec<HotReloadTemplateWithLocation>,
    pub assets: Vec<PathBuf>,
    pub ms_elapsed: u64,
    pub jump_table: Option<JumpTable>,
    pub for_build_id: Option<u64>,
    pub for_pid: Option<u32>,
}                                                    // lines 42-50

Both derive plain Serialize, Deserialize with no #[serde(tag=...)] override, so DevserverMsg serializes as serde's default externally-tagged JSON, e.g. {"HotReload":{"templates":[],"assets":[],"ms_elapsed":0, "jump_table":{...JumpTable fields...},"for_build_id":1,"for_pid":12345}}. Transport is a plain tungstenite WebSocket carrying Message::Text(json) (devtools/src/lib.rs:98-109), explicitly not authenticated: "This doesn't use any form of security or protocol, so it's not safe to expose to the internet" (devtools/src/lib.rs:59).

Connection URL (connect_at, devtools/src/lib.rs:89-101):

{endpoint}?aslr_reference={subsecond::aslr_reference()}&build_id={dioxus_cli_config::build_id()}&pid={std::process::id()}

— the client (the running app) reports its own live main address as a query param when it connects; this is exactly the aslr_reference value patch_rebuild threads into BuildMode::Thin (builder.rs:380-395 — dx waits for a client to be connected before it will do a thin build on non-wasm targets, since it needs this value: "Ignoring hotpatch since there is no ASLR reference. Is the client connected?", builder.rs:390-391).

Receiving/applying (connect_subsecond, devtools/src/lib.rs:76-86):

pub fn connect_subsecond() {
    connect(|msg| {
        if let DevserverMsg::HotReload(hot_reload_msg) = msg {
            if let Some(jumptable) = hot_reload_msg.jump_table {
                if hot_reload_msg.for_pid == Some(std::process::id()) {
                    unsafe { subsecond::apply_patch(jumptable).unwrap() };
                }
            }
        }
    });
}

for_pid gates whether this process applies it (a dev server can be talking to several connected clients/processes at once); for_build_id is checked similarly in the dioxus-VirtualDom-aware try_apply_changes (devtools/src/lib.rs:19-55, checks msg.for_build_id == Some(dioxus_cli_config::build_id())) but not in the bare connect_subsecond path — for a non-Dioxus app (jev is eframe/egui, not Dioxus), connect_subsecond's simpler pid-only check is the relevant precedent.

Confirmed: subsecond::apply_patch is pub unsafe fn apply_patch(table: JumpTable) -> Result<(), PatchError> (subsecond/src/lib.rs:498) — it can be called directly with a hand-constructed JumpTable, with no dependency on the websocket protocol, dx, or Dioxus at all. A buck2-driven tool can build its own JumpTable (e.g. by shelling out to the same create_native_jump_table logic, or a from-scratch equivalent using the object crate) and either (a) open its own tiny websocket/HTTP server that jev's MCP server dials into and speaks a {jump_table: JumpTable}-shaped JSON, mirroring DevserverMsg, or (b) skip the network entirely and have the MCP hot_patch tool call subsecond::apply_patch in-process, since jev's own binary already links subsecond and already has an MCP server loop (apps/native/src/mcp.rs) that can receive a file path + deserialize a JumpTable from disk/stdin and call apply_patch directly on the UI/tool thread. This avoids reimplementing the websocket handshake and the aslr_reference query-param dance — the tool call itself can read subsecond::aslr_reference() in-process before invoking the build.

5. Hard requirements/limits for the buck2 design

  • No nightly rustc requirement. (See §1 — the only nightly use is an unrelated build-progress estimate.) Stable rustc + stable -C/-Z-free flags (-Csave-temps=true, -Clink-dead-code) suffice for native/Linux.
  • debug_assertions must be on in the profile being patched — call/ try_call collapse to a plain call in release (subsecond/src/lib.rs:250-254, :411-414); patching a release-profile binary compiles but every subsecond::call site is a dead no-op, so apply_patch succeeds but nothing ever redirects. jev's buck2 system_rust_toolchain sets rustc_flags = ["-Copt-level=3", "-Cdebuginfo=1"] (toolchains/BUCK:16) with no explicit -Cdebug-assertions. VERIFIED 2026-09-20, measured: built //apps/hotdemo:hotdemo with println!("{}", cfg!(debug_assertions)) at the top of main and no -Cdebug-assertions flag on the target (toolchain's -Copt-level=3 only) — ran it and it printed hotdemo debug_assertions=false. So buck2's system_rust_toolchain at opt-level=3 does not imply debug-assertions on; it must be set explicitly per hot-patch target. apps/hotdemo/BUCK does this (-Cdebug-assertions=yes in rustc_flags).
  • Thread-locals in the tip crate reset on every patch — "not currently bind[ing] thread-locals in the patches to their original addresses" (subsecond/src/lib.rs:88-92). Not a buck2-specific concern, but a correctness limit to document for whoever writes jev's hot-patched code.
  • No struct-layout/ABI safety net. Layout or size changes to structs referenced across the patch boundary crash outright — frameworks are expected to throw away and rebuild affected state (subsecond/src/lib.rs:94-109).
  • "Tip crate only" in the subsecond crate's doc comment is stale relative to the CLI's actual v0.7.10 code: compile_workspace_hotpatch + workspace_hotpatch_replay_order/_link_rlibs (link.rs:132-335, 586-723) DO recompile and relink modified library crates (topologically replayed via direct rustc invocations reusing captured args), not just the tip. So //packages/panels (or //packages/console, //packages/alttp) changing can be folded into a patch, provided the buck2 tool replays that crate's rustc invocation with --out-dir/extra-filename unchanged (so the resulting .rlib lands exactly where the tip's link line already expects it) and the tip crate itself gets relinked against the new .rlib. UNVERIFIED beyond what's in this checkout: no test in this repo exercises a multi-crate thin build, so treat this as "the code path exists and is structured to support it" rather than "empirically proven robust."
  • The stub-symbol resolver (create_undefined_symbol_stub) is unimplemented for WASM ("this function is not defined to run on WASM binaries", patch.rs:849-850) — irrelevant to jev's native build but worth noting if a wasm target is ever added; that path instead rewrites the .wasm module's ifunc/GOT imports directly (create_wasm_jump_table, patch.rs:458-761).
  • A live client connection is required before any thin build can even start, because aslr_reference (the process's real main address) has to be known (builder.rs:380-395). For jev's own tool, the MCP server already running inside the app window is the natural source of this value — it can report subsecond::aslr_reference() directly to the hot_patch tool caller instead of over a websocket query param.

6. Buck2 prelude: what's available, and a concrete design

Prelude location (this repo): ~/nixos-config-independent, resolved via nix develop -c buck2 audit cell prelude → /home/nixos/jev/prelude (which is itself a redirect to the bundled cell at /home/nixos/jev/buck-out/v2/external_cells/bundled/prelude, where the actual .bzl files live).

Relevant prelude machinery (paths relative to that prelude root):

  • rust/build_params.bzl:22-30 — CrateType = enum("bin","rlib","dylib", "proc-macro","cdylib","staticlib"). cdylib already exists as a first-class crate type — the natural output type for a hand-built rust_binary-like "patch" rule, since it's exactly a .so with C-style exports, matching what dx's thin link produces by hand. crate_type_linked (build_params.bzl:35-36) includes "cdylib".
  • rust/build.bzl:950 (_rustc_flags) and the emit-keyed dispatch starting build.bzl:1012 — this is where per-target rustc_flags (a plain attrs.list(attrs.arg()) on rust_library/rust_binary, standard prelude attr) get assembled with toolchain-level rustc_flags/extra_rustc_flags (build.bzl:1226-1232). A buck2 rule can add -Csave-temps=true -Clink-dead-code (and, if confirmed necessary per §5, -Cdebug-assertions=yes) the same way jev's toolchains/BUCK:16 already adds -Copt-level=3 -Cdebuginfo=1 — no new mechanism needed, just more flags on the existing rustc_flags toolchain attr or a target-level override.
  • rust/build.bzl:933-947 (_check_restricted_rustc_flags) — the prelude can restrict certain rustc flags per toolchain (uses_restricted_rustc_flags escape hatch); worth checking jev's toolchain doesn't block -Csave-temps/ -Clink-dead-code (it doesn't set restricted_rustc_flags today — toolchains/BUCK has no such field).
  • rust/rust_library.bzl:557-558,627-628,636-637 — exported_linker_flags/ exported_post_linker_flags (plus the plain non-exported linker_flags attr inherited from the common Rust rule attrs) is the existing plumbing for arg like -Wl,--export-dynamic-symbol,main.
  • rust/rust_binary.bzl:148 — link_style/LinkStrategy attr, confirming buck2's rust rules already have a first-class notion of static vs shared-dependency linking, relevant to deciding whether the "fat" analogue (a binary linking every workspace .rcgu.o unconditionally, undead-code stripped) is better modeled as a link_style = "static" rust_binary with the save-temps/link-dead-code flags, vs. a wholly custom genrule/action.
  • Object files: --emit/temp objects come out of the ordinary rustc action buck2 already runs per crate (build.bzl Emit("link") etc, build.bzl:1012 on); with -Csave-temps=true those .rcgu.o files land in rustc's own temp dir the same way they do under cargo — buck2's rust rules don't need new emit plumbing, just the flag, though the tool will need to locate rustc's actual (buck2-managed, hashed) --out-dir/temp dir rather than cargo's, since buck2 controls that path differently than cargo does. VERIFIED 2026-09-20, measured — see §7.1: they land in a normal declared buck2 output directory, not ephemeral scratch, at a path mechanically derivable from the target's own main-output path.

Proposed design, given all of the above:

  1. A hotpatch Rust binary (new crate, e.g. //tools/hotpatch) that reimplements, against the object crate directly (no dioxus/cli dependency needed — subsecond-types is a tiny, license-compatible, directly-vendorable crate for the JumpTable/AddressMap types):
    • create_native_jump_table-equivalent (patch.rs:403-443): parse old binary + new .so symbol tables via object, build the old→new address map keyed on shared names, using main as the ASLR sentinel.
    • create_undefined_symbol_stub-equivalent (patch.rs:852-1259, Linux x86_64 branch only needed: patch.rs:1083-1091 for the jmp [rip+0] text stub, patch.rs:1245-1254 for data, TLS can likely be deferred given jev is a single-binary eframe app unlikely to lean on TLS initializers).
    • A hot_patch mode driven by a small CLI: given the base exe path (buck2 output), the new .rcgu.o/.rlib set, and the live process's aslr_reference (queried from jev's own MCP server), emit the stub object, link (cc -shared -Wl,--eh-frame-hdr … -nodefaultlibs per link.rs:429-436, plus -Wl,--export-dynamic-symbol,main carried over from the base build) and print/serialize the resulting JumpTable as JSON.
  2. Buck2 side: two new targets alongside apps/native:native —
    • native itself gains -Csave-temps=true -Clink-dead-code -Wl,--export-dynamic-symbol,main (via rustc_flags/linker_flags) only under a hotpatch-flavored build (e.g. a buck2 select() on a config_setting, so ordinary release/dev builds are unaffected) — the buck2 analogue of dx's "fat" build. No archive-splicing step is necessary the way run_fat_link does it (link.rs:788-1142): object files inside a bin crate that are reachable from main are always linked (the compiler wouldn't have generated them otherwise), and -Clink-dead-code alone is what stops the linker from then dropping dependency-rlib symbols that are compiled in (rlibs retain every public item, cargo/buck2-reachability rules) but not called from anywhere in the current build. VERIFIED 2026-09-20, measured — see §7.2: without -Clink-dead-code, roughly two-thirds of all Rust-mangled symbols in hotdemo's own binary were gone from nm, including half of the subsecond/subsecond-types-related symbols. dx's fat-archive dance is unnecessary for this design, but the flag is not optional.
    • A per-crate (or per-changed-crate) thin recompile path: since buck2 already tracks per-crate build graphs and caches, there is no need to replay captured rustc args the way compile_dep_crate does (link.rs:518-584) — buck2 already knows how to rebuild exactly the crates whose inputs changed, with the same flags, because that's buck2's whole job. The buck2-native replacement for workspace_hotpatch_replay_order is simply: ask buck2 to build the (now on-disk-cached, per-cdylib- or per-rlib-target) outputs for the changed target and its buck2-graph dependents, then hand those .rlib/.rcgu.o paths to the hotpatch tool for the final ad-hoc link — i.e. buck2's own incrementality subsumes dx's rustc-arg-replay machinery entirely; only the final, ad-hoc, non-buck2-native stub-object-and-link step needs the custom tool, mirroring compile_workspace_hotpatch's tail (link.rs:186-334) but sourcing its object/rlib inputs from buck2 outputs instead of a hand-rolled JSON cache.
  3. Delivery to the running window: extend apps/native/src/mcp.rs's dev-mode tool set (already gated by DevMode, per its module doc, mcp.rs:8-12) with a hot_patch tool that (a) on request, reports subsecond::aslr_reference() back to the caller (buck2/the hotpatch tool invocation) so it can compute aslr_offset, and (b) accepts a path to a just-built JumpTable (JSON) or patch .so + JSON sidecar, and calls unsafe { subsecond::apply_patch(table) } on it — in-process, no websocket, no dioxus_cli_config/dioxus-devtools dependency at all. This is strictly simpler than reimplementing DevserverMsg/tungstenite (§4), since jev already has a live, bidirectional, already-authenticated (loopback-only) channel into the running window via MCP.

7. The two UNVERIFIED points from §6, resolved by measurement (2026-09-20)

Both measured against //apps/hotdemo:hotdemo (apps/hotdemo/BUCK, rustc_flags = ["-Csave-temps=true", "-Clink-dead-code", "-Cdebug-assertions=yes"]), in this repo's own buck2/nix devshell, not the dioxus checkout.

7.1 Where -Csave-temps=true leaves the .rcgu.o objects

They land in a normal declared buck2 output directory, not scratch — no ar x fallback needed.

buck2 build //apps/hotdemo:hotdemo --show-full-output reports the main binary at buck-out/v2/art/root/<cfg-hash>/apps/hotdemo/__hotdemo__/hotdemo. buck2 log what-ran (after touching a source line to force a rebuild) shows the actual rustc invocation's args file (…/apps/hotdemo/__hotdemo__/<hash>/XIPL/hotdemo-link-diag.args), which contains:

-Csave-temps=true
-Clink-dead-code
-Cdebug-assertions=yes
--emit=link=buck-out/v2/art/root/<cfg-hash>/apps/hotdemo/__hotdemo__/hotdemo
--out-dir=buck-out/v2/art/root/<cfg-hash>/apps/hotdemo/__hotdemo__/XIPL/extras/hotdemo

The prelude explains why that --out-dir exists at all, independent of -Csave-temps: rust/build.bzl:1451-1453 always declares an "extras" directory as a real buck2 output (extra_dir = subdir + "/extras/" + base; ctx.actions.declare_output(extra_dir, dir = True, …)) for per-codegen-unit side artifacts (originally split-DWARF .dwo files, build.bzl:1173-1180's comment). -Csave-temps=true doesn't need new buck2 plumbing — it just makes rustc also drop its .rcgu.o/.rcgu.bc files into whatever --out-dir it's already given, and that directory happens to be a persistent, declared output rather than the action's TMPDIR (a separate, genuinely-ephemeral path under buck-out/v2/tmp/…/rustc/_buck_… that the args file also references, for the compiler's own scratch use). Confirmed by listing the directory after a clean build — it survives, with one .rcgu.o per codegen unit (hotdemo.hotdemo.<hash>-cgu.<N>.rcgu.o) plus one for the crate's monomorphization glue (hotdemo.<hash>.rcgu.o).

So the relationship a caller needs is entirely mechanical, from --show-full-output's own answer:

extras_dir = dirname(main_output) + "/XIPL/extras/<crate-file-stem>"

tools/hotpatch/patch.sh computes it this way rather than hard-coding a path.

No — -Clink-dead-code is doing real, necessary work, not a defensive no-op.

Built hotdemo twice, flipping only -Clink-dead-code (-Csave-temps=true/-Cdebug-assertions=yes held fixed), and compared nm on the resulting binary:

total symbols (nm | wc -l)subsecond/subsecond-types-mangled symbols
with -Clink-dead-code384767
without -Clink-dead-code139436

Without the flag, the plain static link (buck2's default, no --gc-sections override of its own) drops roughly two-thirds of every Rust-mangled symbol in the binary, including about half of the subsecond/subsecond-types ones — dependency rlibs (subsecond, subsecond-types, hashbrown, serde, …) retain every public item at compile time (a rlib's reachability set is "visible from the crate root", not "called by this program"), and the linker's default dead-code elimination then strips whichever of those never end up referenced by hotdemo's own call graph. A function a future patch needs to jump back into — one hotdemo doesn't call today but a later edit will — is exactly the kind of thing this strips if not held open. apps/hotdemo's own tick/main/apply_pending_patch were unaffected either way (directly reachable from main, so always kept), which is why this had to be measured against the dependency rlibs' symbol counts, not hotdemo's own three functions, to be a real test. This matches dx's own rationale (request.rs:1796-1798's comment, quoted in §1) exactly, and confirms the flag stays on every hot-patch-flavored buck2 target — including, later, apps/native and any of its library deps whose changes should be patchable.

7.3 Bonus: the -Cdebug-assertions default at -Copt-level=3, resolved

§5 flagged this as unconfirmed too. Built hotdemo with the toolchain's plain -Copt-level=3 -Cdebuginfo=1 and no explicit -Cdebug-assertions flag, printing cfg!(debug_assertions) at startup: it printed hotdemo debug_assertions=false. So buck2's system_rust_toolchain does not imply debug-assertions from opt-level=3 — every hot-patch target needs -Cdebug-assertions=yes explicitly, which apps/hotdemo/BUCK now carries.

8. Two more traps found getting the end-to-end proof to actually redirect (2026-09-20)

tools/hotpatch/apps/hotdemo built and linked cleanly, apply_patch returned Ok(()), and "patch applied" printed - and the log kept printing the OLD string anyway. Both root causes below are nm/disassembly-verified, not guessed, and both are silent: nothing errors, the patch mechanism just does nothing.

8.1 subsecond needs its OWN -Cdebug-assertions=yes, not just the tip crate's

subsecond::call/try_call are cfg!(debug_assertions)-gated no-ops (§5). apps/hotdemo/BUCK had -Cdebug-assertions=yes from the start. The redirect still never fired.

cfg!() is resolved against the compilation that contains the macro invocation, not the crate that later monomorphizes a generic function. Under cargo, a single profile's debug-assertions setting applies uniformly to the whole dependency graph — subsecond gets built with whatever flag the TIP crate's profile carries, because cargo compiles every crate in that one profile. Under buck2/reindeer, each third-party crate is its own independent cargo.rust_library target, built with whatever rustc_flags ITS OWN target carries — which, absent a fixup, is just the toolchain default (-Copt-level=3 -Cdebuginfo=1, no debug-assertions, confirmed false in §7.3). apps/hotdemo's own -Cdebug-assertions=yes therefore had zero effect on subsecond's internal gate — verified by nm on the built binary: no call to subsecond's own (real, addressable — _RNvCs41PE3uYsSey_9subsecond14get_jump_table) get_jump_table function appears anywhere inside main's compiled code. The fast if !cfg!(debug_assertions) { return f() } path was compiled in as unconditionally true, because subsecond's OWN build never turned debug-assertions on.

Fix: third-party/rust/fixups/subsecond/fixups.toml —

rustc_flags = ["-Cdebug-assertions=yes"]

reindeer's FixupConfig supports rustc_flags per crate directly; no custom buck2 plumbing needed. Any first-party crate that hot-patches under buck2/reindeer needs this same fixup on subsecond — it is not specific to apps/hotdemo and will apply equally to apps/native.

8.2 subsecond::call needs a bare fn item, not a closure — for two independent reasons

The first working version of tick was fn tick(n: u64), called as subsecond::call(|| tick(n)). Even after fixing §8.1, the redirect still never fired. Two separate, unrelated causes, found by disassembling main (objdump -d, nm) and reading HotFn::try_call/call_as_ptr (subsecond/src/lib.rs:395-424, :877-943) line by line:

  1. The redirect key depends on size_of::<F>(). try_call branches: if the closure/fn type F is exactly pointer-width (size_of::<F>() == size_of::<fn() -> ()>(), 8 bytes on x86_64), it calls call_as_ptr, which does std::mem::transmute_copy::<Self, Self::Real>(&self) — reinterpreting F's own in-memory bytes AS a function pointer. This is correct only when F genuinely IS a bare fn pointer value (then Self == Self::Real and the transmute is an identity). Our closure captured n: u64 — also 8 bytes — so it took the SAME branch, and the transmute reinterpreted n's VALUE as a function address: garbage, not a lookup key subsecond's map would ever contain. Any closure whose captured environment happens to be pointer-sized (one u64, one reference, …) hits this, silently. apps/hotdemo's fix: no closure at all — tick takes no parameters, reads a static AtomicU64 for its counter, and main calls subsecond::call(tick) directly, so F is a genuine zero-sized function-item type and the other branch (<F as HotFunction<A,M>>::call_it as *const (), subsecond/src/lib.rs:428-435) runs instead.
  2. -Copt-level=3's default cross-CGU ThinLTO inlines small, single-call-site functions across the whole chain. Fixing (1) alone was not enough: nm on the resulting binary still showed no tick symbol and no call_it/HotFunction symbol anywhere — the entire subsecond::call → HotFn::try_call → HotFunction::call_it → tick chain had been inlined into one block inside main, leaving nothing to redirect a call to. subsecond's own source carries no #[inline(never)] anywhere (confirmed by grep) — it is written assuming a dev-profile-style build (opt-level = 0), where LLVM performs little to no inlining by default and this is a non-issue. This project forces -Copt-level=3 for every target unconditionally (toolchains/BUCK, a real requirement — the emulator core is unusable below realtime at -O0), which is a mode subsecond was never designed against. Fix (for the tip crate's OWN patchable functions): #[inline(never)] on tick. This does not, by itself, force call_it to remain a distinct out-of-line symbol too — but with tick pinned, call_it's inlined copy contains a genuine call instruction to a real, address-stable tick, which turned out to be sufficient in this build (confirmed by the working v1→v2→v3 proof, §8.3) even though nm still shows no separate call_it symbol. Treat this as "empirically sufficient for a trivial single-statement function," not as a proof that inlining of the wrapper machinery itself can never matter — a much larger patchable function, or a different LLVM version's inlining heuristics, could still behave differently.

Consequence for apps/native: every function meant to be hot-patchable needs #[inline(never)], and the call site needs to pass it as a bare fn item (or a zero-sized, non-capturing closure) — not a closure over any pointer-sized local state. A method taking &mut self is already safe from the size-8 coincidence in the common case (&mut self alone is exactly one pointer is 8 bytes, so this needs re-checking per call site, not assumed safe by shape).

8.3 The actual end-to-end proof, once both were fixed

tools/hotpatch/patch.sh //apps/hotdemo:hotdemo, twice in a row against a single running process (pid unchanged throughout, run/hotdemo.log):

hotdemo pid=42801 aslr_reference=0x5555555adf20
hotdemo v1 tick=0
...
hotdemo v1 tick=14
hotdemo: patch applied
hotdemo v2 tick=0
...
hotdemo v2 tick=45
hotdemo: patch applied
hotdemo v3 tick=0
...

TICKS (a static) resets to 0 on each patch rather than continuing its count — expected, not a bug: it lives in the tip crate, which gets fully recompiled every patch, so the freshly-dlopen'd .so gets its own fresh .bss copy rather than being rebased onto the original process's static (subsecond's "globals are tracked" claim, subsecond/src/lib.rs's module doc, applies to statics that live in an UNCHANGED dependency crate reused via its cached .rlib — not to one recompiled as part of the tip). Same family of limitation as the already-documented thread-local reset (§5).

Files most load-bearing for this design, for a follow-up implementer to open directly: packages/cli/src/build/link.rs (thin/fat link logic), packages/cli/src/build/patch.rs (jump table + stub generation), packages/cli/src/rustcwrapper.rs + packages/cli/src/cli/link.rs (why/how dx wraps rustc+linker — buck2 needs none of this, since it already owns its own build graph), packages/subsecond/subsecond/src/lib.rs (apply_patch, aslr_reference, the HotFn/call runtime), packages/subsecond/subsecond-types/src/lib.rs (JumpTable), packages/devtools/src/lib.rs + packages/devtools-types/src/lib.rs (wire protocol, useful only as a fallback if in-process MCP delivery turns out to be insufficient), and, in this repo, /home/nixos/jev/toolchains/BUCK

  • /home/nixos/jev/apps/native/BUCK + /home/nixos/jev/apps/native/src/mcp.rs as the integration points.