Up version 0.4.0

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

.github/pull_request_template.md

@@ -0,0 +1,36 @@
+<!--
+Target branch: dev. Rules: CONTRIBUTING.md, "Pull requests". Write each section in your own words about this diff;
+delete the rows that do not apply. Never attach game files, keys, extracted data or logs that contain them.
+-->
+
+## Problem
+
+<!-- What the player sees or what is wrong in the code. Issue link if there is one. -->
+
+## Change
+
+<!-- What the diff does and why this way. Name the switch (totk.ini setting or TOTK_* variable) that turns it off. -->
+
+## Where it reproduces
+
+<!-- OS, CPU, GPU and driver, game update (1.2.1 / 1.4.3), settings. Hardware it cannot appear on, if known. -->
+
+## Proof
+
+<!-- Before and after on the same repro: screenshots, log lines, a test that failed and now passes. -->
+
+## Checks run
+
+<!-- The rows of CONTRIBUTING.md "Checks by what you changed" that match the diff, each with its result. -->
+
+- [ ] `python tools/check_content.py --staged`
+- [ ] `totk.exe --selftest`
+
+## Measurements
+
+<!-- Frame path, timing or scheduling only. Per run, switch off and on, at least two each: average fps, 1% low or
+worst frame, `perf: slow frame` count. Scene, save, seconds, CPU, GPU, driver, resolution. -->
+
+## Not run
+
+<!-- Paths you could not run (Linux, HDR, a vendor you do not have) and how they are gated. -->

AGENTS.md

@@ -29,6 +29,15 @@ This repository is public. Game data lives outside it and every derived file sta
 - Graphics bugs: [docs/maxwell.md](docs/maxwell.md) lists the Maxwell behaviour the Vulkan path must match and the TotK bugs other ports fixed.
 - Comments say what the code does and why, in words that stand on their own. Cite our code by file plus function (a line number may follow as a hint) and upstream code by project, file and function: `refs/` moves, so its line numbers rot. Commit hashes belong to the attribution of copied code.
 
+## Running and debugging
+
+- Running the game, screenshots, frame dumps, timings: [docs/agents/running.md](docs/agents/running.md).
+- Investigating a bug, glitch or crash: search [docs/agents/symptoms.md](docs/agents/symptoms.md) by the player's words first, then follow [docs/agents/debugging.md](docs/agents/debugging.md). Read the code before running the game.
+
+## Pull requests
+
+Preparing a pull request: target `dev`; the checks by changed area, the proof and the performance runs it carries are in [CONTRIBUTING.md](CONTRIBUTING.md), "Pull requests". Reviewing one: [docs/pr-review-kit/](docs/pr-review-kit/).
+
 ## Writing
 
 Code, comments, docs and commit messages are in English. Commit messages are plain sentences without type prefixes.

CONTRIBUTING.md

@@ -29,14 +29,93 @@ Use the issue form. Include the port version and game update from the title line
 
 ## Pull requests
 
-- One topic per pull request.
-- Describe the problem, then what changed.
-- Say how you tested it: which version, which scene, how long.
-- A change to the lifter (`recomp/`) or the CPU headers passes the differential tests in [docs/building.md](docs/building.md), "Checks".
-- Commit messages are plain sentences, without `feat:` or other prefixes.
-- AI-assisted code is welcome. You are responsible for having read and run it.
+### Branch
+
+Open pull requests against `dev`. Branch from the current `dev` and rebase on it before you open the pull request. `main` receives `dev` at a release; a pull request against `main` is retargeted to `dev` before review.
+
+### Before you start
+
+- Search the open pull requests and issues on the topic. When another pull request already covers it, improve that one or say in yours how they differ.
+- Fixes are reviewed before features, in order of how many players the bug hits. A feature with a design in [docs/milestones/](docs/milestones/) (mods, cheats, the menu) follows that design. For a feature without one, open an issue first.
+- Find out where the bug reproduces: OS, GPU vendor and generation, driver, game update, settings. Put that in the description.
+
+### What the change must keep
+
+- **The default for everyone.** A fix for one setup (a GPU vendor, a memory size, a display) is gated on that setup. Players without the problem see no change.
+- **A switch.** New behaviour on the frame path or in rendering sits behind a `totk.ini` setting or a `TOTK_*` variable, so a player can turn it off and a reviewer can compare. A new setting is described in [totk.ini.example](totk.ini.example) and [docs/settings.md](docs/settings.md).
+- **The fix, not a disabled feature.** A crash that goes away with a feature off points at the bug in that feature. Turning the feature off for a vendor is not a fix.
+- **The frame time on weak hardware.** Game threads and the render thread do no file I/O, no GPU allocations, no per-draw state copies and no global atomic counters. Memory kept per frame or per object is freed. The full list with measured costs: [docs/pr-review-kit/02-performance.md](docs/pr-review-kit/02-performance.md), "What the hot path must not do".
+- **Other GPUs.** New Vulkan code queries each feature, extension, format and limit it uses and keeps a working path without it. Nothing is hardcoded from what one GPU reports.
+- **Text in the catalogs.** Text a player sees gets an id in `runtime/ui_text.h` and an entry in `runtime/ui_text.c` for every menu language (English, Russian, Brazilian Portuguese, Simplified Chinese); installer text goes into `installer/text/*.json`.
+- **The generator owns generated C.** Change `recomp/`, never `generated/`.
+
+### Checks by what you changed
+
+Run every row that matches your diff and list the results in the description. Commands and details: [docs/building.md](docs/building.md), "Checks".
+
+| You changed | Run |
+|---|---|
+| anything | `python tools/check_content.py --staged`; `build\<ver>\totk.exe --selftest` |
+| `recomp/`, `runtime/cpu.h`, `runtime/simd.h` | the differential tests against Unicorn: `tests\diff\lift.py`, `simd.py`, `atomic.py`, `func.py`, `tests\flags_check.py`, `tests\excl_mt.py`; the same with a `--cpu v2` build |
+| `recomp/jumptables.py` | `tests\jumptables_check.py`: 0 disagreements on 1.2.1 and 1.4.3 |
+| `runtime/` | `--probe-kernel`, `--probe-vkexec`, `--probe-nvn-tex`, `--probe-shader`, plus the probe for the file you touched (list at the top of `runtime/main.c`) |
+| settings | `totk.exe --probe-cfg`; `python tests\settings_doc_check.py` |
+| button prompts | `.venv-py314\Scripts\python tests\romfs_prompts_test.py` |
+| mods, ExeFS patches | `.venv\Scripts\python tests\exefs_patches_test.py`; `totk.exe --probe-mods` |
+| `installer/` | `python tests\installer_test.py`; `python tests\installer_gui_test.py` |
+| shader translator | `tools\xlate_smoke.py`; `tools\shader_census.py --version <ver> --translate` with no new failure groups |
+| rendering | a run in a scene where the change acts, with `TOTK_VK_VALIDATION=1`: no new validation messages; screenshots before and after |
+| frame path, timing, scheduling | the measurements below |
+
+Checks that need game files run on your own copy of the game. Results go into the pull request as text: pass or fail, counts, log lines. Never attach logs or dumps that contain game data.
+
+### Proof
+
+- **A bug fix** shows the bug before the change and gone after it, on the same repro: a screenshot pair, a log line, a probe or test that fails before and passes after. A scripted repro (`TOTK_INPUT_RECORD`, [docs/controls.md](docs/controls.md), "Scripted input") is the strongest. If the bug cannot be reproduced on your hardware, say so and point at the code that causes it.
+- **A crash fix** names the crash: the log's crash report, `vk: pass marks` lines for a lost device.
+- **Parts you could not run** (Linux-only code, HDR, vendor-only paths) are listed as unrun and gated away from everyone else.
+
+### Performance measurements
+
+A change on the frame path, in timing or in scheduling carries these numbers, or it waits until someone measures it.
+
+1. **Ceiling first.** Before the work, estimate the share of the frame the changed code holds on the critical path (`TOTK_CRIT` + `tools/crit_path.py`). A ceiling of a few percent is not worth a hot-path change.
+2. **Build 1.4.3** with prepared shaders in that build folder (`tools/prepare_shaders.py`, [docs/building.md](docs/building.md) step 6). Caches copied from another build folder do not count.
+3. **A heavy scene** where the change acts: a fight with a large enemy and a crowd, or a town, with `TOTK_FPS_LIMIT=360`.
+4. **Whole runs, switch off and on, at least two each**, the same save and the same input script, timed by game seconds:
+
+   ```powershell
+   .venv\Scripts\python tools\run_game.py build\v143 --script <input.txt> --seconds 60 --quiet --env TOTK_TRACE=perf,sched --env TOTK_FPS_LIMIT=360 --env <SWITCH>=0
+   ```
+
+   `--quiet` waits until no other game or build runs. Close everything heavy on the machine. Before reading the numbers, check that each run reached the world and showed the same scene.
+5. **Report per run**: average fps, 1% low or worst frame, the number of `perf: slow frame` lines, VRAM; `tools/perf_summary.py` condenses the log. Add CPU, GPU, driver, RAM, resolution and the settings you changed.
+
+A switch toggled inside one run on a moving scene gives false results: compare whole runs. Zero new slow frames is a hard bar; an fps gain does not buy back new hitches. Background and the reasons behind each rule: [docs/pr-review-kit/02-performance.md](docs/pr-review-kit/02-performance.md).
+
+### Description and commits
+
+Fill in the pull request template: problem, change, where it reproduces, checks, measurements, what stayed unrun. One topic per pull request.
+
+Commit messages are plain English sentences, without `feat:`, `fix:` or other prefixes. A commit that works on an issue ends its subject with the issue numbers: `Render the handheld scene at the asked size on 1.4.3 (#998, #536)`.
+
+### Review and merge
+
+The maintainers review every pull request against [docs/pr-review-kit/](docs/pr-review-kit/): read it to see what gets a pull request rejected, which bug classes are already fixed and which invariants a change must keep. A good pull request that misses the project's conventions is fixed up while merging rather than sent back. Part of a pull request can be taken; the reason for what was left out or rejected is posted on the pull request.
+
+Accepted work merges into `dev` and reaches players through test builds. `main` moves only at a release.
+
+## Coding agents
+
+AI-assisted code is welcome. You are responsible for having read and run it, and the pull request says which checks ran on your machine.
+
+Point your agent at [AGENTS.md](AGENTS.md) (Claude Code reaches it through `CLAUDE.md`), this file and [docs/pr-review-kit/](docs/pr-review-kit/) before it writes code. [docs/agents/](docs/agents/) holds what it needs to run the game unattended, measure, and debug: the method, the bug classes of a recompiled Switch game, and a symptom catalog. The rules in them bind the agent as they bind you:
+
+- Key values stay out of prompts, logs, commands and files. The agent passes the keys file path to the tools.
+- Results in the pull request come from runs that happened. A check the agent did not run is listed as not run.
+- The agent writes the description in its own words about this diff. A review kit checklist pasted in with every box ticked tells the reviewer nothing.
 
 ## Code and docs
 
 - Code, comments, commit messages and documentation are in English.
-- Generated code is never edited by hand: change the generator.
+- Comments say what the code does and why. Cite our code by file and function, upstream code by project, file and function.

README.md

@@ -26,6 +26,7 @@ Report problems as issues, following [CONTRIBUTING.md](CONTRIBUTING.md).
 - 16:10 (Steam Deck, handhelds), 3:2 (Surface, Framework), 21:9 and 32:9 (ultrawide) game resolutions, with the camera and the interface in that shape
 - Anisotropic texture filtering up to 16x, for sharp ground into the distance
 - In-game settings menu (F1) and performance overlay (F10)
+- Play time across runs on the Game tab of the settings menu, kept in `playtime.txt` next to the saves
 - Discord Rich Presence ("Playing TotK Recomp" with the play time), off by default, own application id supported
 - Setup and in-game menu in English, Russian, Brazilian Portuguese and Simplified Chinese
 - Keyboard, mouse and gamepad with remapping, rumble and gyro
@@ -34,7 +35,8 @@ Report problems as issues, following [CONTRIBUTING.md](CONTRIBUTING.md).
 - Shaders prepared at install instead of during play
 - Mods made for emulators: unpack a mod into the `mods` folder next to `totk.exe`; switch and order them in the menu. RomFS
   mods, TKMM's merged output and ExeFS code patches (`.ips`, `.pchtxt`) work; the game rebuilds itself for code patches
-  ([docs/mods.md](docs/mods.md))
+  ([docs/mods.md](docs/mods.md)). HD texture packs work: the port grows the texture heap a pack asks for by itself. Tested
+  with the HDPacks.com 4K Texture Pack v1.0 on 1.4.3 and 1.2.1; other packs come with no guarantee
 - Atmosphère cheats: put a cheat file into a mod and switch its cheats in the menu ([docs/cheats.md](docs/cheats.md))
 
 ## Planned

THIRD_PARTY_NOTICES.md

@@ -27,7 +27,7 @@ Code copied or adapted from another project is listed here with its source, the
 | deko3d block-linear image layout, as ported in MacNativeGames' `nvn_tex.c` (Apple parts dropped, layout math unchanged; licence text in `games/totk/third_party/d2r_gfx/LICENSE.deko3d`, reproduced in the file header) | https://github.com/devkitPro/deko3d (`source/dk_image.cpp`), via MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/third_party/d2r_gfx/gfx/nvn/nvn_tex.c`, `nvn_tex.h`), Copyright (C) 2018-2020 fincs | zlib | `runtime/nvn_tex.c`, `runtime/nvn_tex.h` |
 | MacNativeGames texture auto-layout goldens (40 original-SDK size and block-height cases) | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/tests/texture_auto_layout/probe.c`), Copyright (c) 2026 jhaizhou | MIT | `runtime/nvn_tex_probe.c`; `runtime/nvn_probe.c` (four of the cases, through the procs) |
 | MacNativeGames block-linear deswizzler (address function; copy loop split out as `tex_bl_copy`) | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/third_party/d2r_gfx/gfx/tex/nvn_deswizzle.c`), Copyright (c) 2026 jhaizhou | MIT | `runtime/tex_deswizzle.c`, `runtime/tex.h` |
-| SDL 3.4.18, dynamic link, `SDL3.dll` copied next to `totk.exe`. Developer builds use the MSYS2 build (`mingw-w64-ucrt-x86_64-sdl3`, which imports `libiconv-2.dll`, LGPL-2.1, unmodified, copied beside it); the installer release ships the upstream release `SDL3-3.4.18-win32-x64.zip`, which imports no libiconv | https://libsdl.org, Copyright (c) 1997-2026 Sam Lantinga | zlib | `runtime/input_win.c` (gamepad input, rumble, gyro and accelerometer) |
+| SDL 3.4.20, dynamic link, `SDL3.dll` copied next to `totk.exe`. Developer builds use the MSYS2 build (`mingw-w64-ucrt-x86_64-sdl3`, which imports `libiconv-2.dll`, LGPL-2.1, unmodified, copied beside it); the installer release ships the upstream release `SDL3-3.4.20-win32-x64.zip`, which imports no libiconv | https://libsdl.org, Copyright (c) 1997-2026 Sam Lantinga | zlib | `runtime/input_win.c` (gamepad input, rumble, gyro and accelerometer) |
 | libopus (MSYS2 `mingw-w64-ucrt-x86_64-opus`), dynamic link, `libopus-0.dll` copied next to `totk.exe` | https://opus-codec.org, Copyright (c) 2001-2023 Xiph.Org, Skype Limited, Octasic, Jean-Marc Valin, Timothy B. Terriberry, CSIRO, Gregory Maxwell, Mark Borgerding, Erik de Castro Lopo, Mozilla, Amazon | BSD-3-Clause | `runtime/ov_media.c` (nn::codec hardware Opus decoder) |
 | Dear ImGui 1.91.9 (core and Vulkan back end, unmodified), as vendored by LAS Recomp | https://github.com/ocornut/imgui, via https://github.com/peiikai/LASRecomp @ `da35b83a9611d046a9a681aae668d7085954d37f` (`third_party/imgui/`), Copyright (c) 2014-2025 Omar Cornut | MIT | `runtime/third_party/imgui/` (compiled into `totk.exe` for the in-game menu, `runtime/ui/`) |
 | SavageWindRecomp settings menu: two-pane option rows (row look, left / right stepping, hold speed-up, description pane), the menu style and tab bar | https://github.com/Revivedx/SavageWindRecomp @ `575ad88e02f9a89a9a885d90039d48af33c982f4` (`runtime/ui/option_rows.mm`, `runtime/ui/menu.mm`), Copyright (c) 2026 Revivedx and the Savage Wind Recomp contributors, Copyright (c) 2026 Peiikai | MIT | `runtime/ui/option_rows.cpp`, `runtime/ui/option_rows.h`, `runtime/ui/menu.cpp` |
@@ -73,7 +73,7 @@ These tools run on the developer's or player's machine during the build. They ar
 | CPython 3.14.9 embeddable package, with `tkinter` and Tcl/Tk 9.0.4 taken from the python.org 3.14.9 installer's component MSI (unpacked, never run); includes `vcruntime140*.dll` (Microsoft, redistributable with Python), OpenSSL, libffi, SQLite, zlib, libtommath | https://www.python.org/downloads/release/python-3149/ | PSF-2.0 (Tcl/Tk: Tcl/Tk licence, BSD-style) | `python/` |
 | numpy 2.5.4 (with its bundled OpenBLAS), lz4 4.4.5, cryptography 50.0.2 (with its bundled OpenSSL), fontTools 4.68.0, cffi 2.1.1, pycparser 3.11 | PyPI binary wheels, pins in `tools/requirements-installer.txt` | BSD-3-Clause; lz4 BSD-3-Clause; cryptography Apache-2.0 OR BSD-3-Clause; fontTools MIT; cffi MIT; pycparser BSD-3-Clause | `python/Lib/site-packages/` |
 | llvm-mingw 20261020 (LLVM/Clang/LLD 23, mingw-w64, libc++, libunwind, compiler-rt), x86_64 subset | https://github.com/mstorsjo/llvm-mingw/releases/tag/20261020 | Apache-2.0 with LLVM exception; mingw-w64 ZPL-2.1/BSD/public domain; libc++ and libunwind Apache-2.0 with LLVM exception and MIT/UIUC; winpthreads MIT | `llvm/` |
-| SDL 3.4.18 upstream Windows release (`SDL3.dll`) and mingw development package (headers, `libSDL3.dll.a`) | https://github.com/libsdl-org/SDL/releases/tag/release-3.4.18 | zlib | `bin/SDL3.dll`, `runtime/deps/` |
+| SDL 3.4.20 upstream Windows release (`SDL3.dll`) and mingw development package (headers, `libSDL3.dll.a`) | https://github.com/libsdl-org/SDL/releases/tag/release-3.4.20 | zlib | `bin/SDL3.dll`, `runtime/deps/` |
 | libopus 1.6.1 (MSYS2 `mingw-w64-ucrt-x86_64-opus` 1.6.1-2; `libopus-0.dll` imports only KERNEL32 and the UCRT api-ms-win-crt DLLs), headers and import library | https://repo.msys2.org/mingw/ucrt64/ | BSD-3-Clause | `bin/libopus-0.dll`, `runtime/deps/` |
 | libvpx 1.17.0 (MSYS2 `mingw-w64-ucrt-x86_64-libvpx` 1.17.0-1), headers and static library compiled into `totk.exe`; `libvpx-LICENSE.txt` and `libvpx-PATENTS.txt` come from the v1.17.0 source tag | https://github.com/webmproject/libvpx | BSD-3-Clause with the WebM patent grant | `runtime/deps/`, `licenses/` |
 | Zstandard 1.5.7 (MSYS2 `mingw-w64-ucrt-x86_64-zstd` 1.5.7-2), headers and static library compiled into `totk.exe`; `zstd-LICENSE.txt` from the package | https://github.com/facebook/zstd | BSD-3-Clause | `runtime/deps/`, `licenses/` |

VERSION

@@ -1 +1 @@
-0.3.11
+0.4.0

docs/agents/debugging.md

@@ -0,0 +1,117 @@
+# Debugging the port: method for agents
+
+A game run costs minutes and a full rebuild costs several more, while reading code costs seconds. Order the work so the
+cheap steps decide as much as they can before the game starts. Each step ends on its stated condition.
+
+## Steps
+
+1. **Evidence.** Hold one artifact that shows the symptom: the player's screenshot, log or input recording, a
+   flight-recorder dump, a frame dump ([running.md](running.md), "Observing"). Done when you can point at the wrong
+   pixel, line or frame.
+2. **Where it reproduces.** OS, GPU vendor and generation, driver, game update, settings, mods, the save. Done when you
+   know whether your hardware can show the bug at all. NVIDIA bugs do not reproduce on AMD; RDNA1/2 bugs do not
+   reproduce on RDNA3/4 (another driver branch); a Ryzen 7000 iGPU is RDNA2 and runs that branch. When your hardware
+   cannot show it, say so and work from the code and the reporter's logs; runs that cannot decide anything are skipped.
+3. **Known already?** Search [symptoms.md](symptoms.md), [pr-review-kit/04-knowledge.md](../pr-review-kit/04-knowledge.md)
+   and [maxwell.md](../maxwell.md) sections 6-7 with the words the player used (black, magenta, flicker, slow motion,
+   missing, stale, streaks, stuck) and the object (sky, grass, shadow, fog, HUD, weapon). Then the upstream notes:
+   `refs/nxnative/docs`, `refs/MacNativeGames/docs` and the SavageWindRecomp issue tracker (`tools/fetch_refs.sh`
+   fetches `refs/`). Done when every hit is either ruled out or is your first hypothesis. A hit on a class we already
+   fixed: confirm that fix still holds before deriving anything new.
+4. **Read the code.** The game's function in Ghidra ([building.md](../building.md), "Reverse engineering") or in
+   `generated/<ver>/`, and our runtime path in `runtime/`. Done when the code either rules the hypothesis out or names
+   the exact condition a run must show. A run confirms what the code could not rule out.
+5. **The cheapest falsifying check.** An existing switch (a `totk.ini` setting or a `TOTK_*` variable), a dump, a trace
+   line, a probe. When it refutes the hypothesis, stop: write down what was checked and why it fell, and start again
+   at step 3 with the next hypothesis. No build, no benchmark for a refuted idea.
+6. **Repro red, then green.** The same repro shows the bug before the fix and its absence after. A fix without a
+   reproduced failure is a guess.
+7. **Fix the cause.** A crash that goes away with a feature off points at the bug in that feature; fix the feature.
+   A fix that one setup needs is gated on that setup, and everyone else keeps today's behaviour.
+
+## Traps
+
+- **One-off failures.** A failure that happens once and not again: rerun before you debug it as a code bug. A crash or
+  a forced shutdown leaves zero-filled cache files, and memory errors on the host flip bits in files and objects.
+- **Every build fails the same way at once.** Suspect the inputs and caches (game files, the OS file cache, shader and
+  pipeline caches), not the code. Bisecting code is pointless when the old builds fail too.
+- **Compile errors in generated C** on a line the generator could not have written are a flipped bit. Regenerate and
+  compare hashes; `recomp/` is fine.
+- **A crash that looks like a race.** Try the save first: some saves crash the same way on a real Switch.
+- **Theories about locks and threads.** Read the game's own protocol in Ghidra before running anything. Ten runs on
+  lock theories once went where one read of the game's double-buffer code would have narrowed it.
+- **Numbers from a fast machine.** A change neutral on a fast desktop can matter on a weak laptop or the Steam Deck,
+  in either direction. Say which hardware your numbers come from.
+
+## Bug classes of a recompiled Switch game
+
+The port runs the game's own code on another OS and another GPU. Most bugs come from places where the host differs
+from the console.
+
+### Variable frame step and epsilon-gated timers
+
+The console steps the game by whole 30 Hz units, so a countdown started at a whole value reaches exactly 0. With the
+real-time step (`realtime = 1`, the default) the step is a measured ratio, and the last step can leave a remainder of
+about 1e-8. Many of the game's updaters step a timer only while it is above `FLT_EPSILON`; a remainder in
+(0, `FLT_EPSILON`] is never stepped again. A consumer that tests the same field with `> 0` then sees it active
+forever, until a load reinitialises the object. Issue #18 (weapons locked after a Fuse) was this.
+
+Symptom words: stuck until a reload, a teleport or a shrine; locked after an action; works sometimes; only above
+30 fps. To check a report: find the field the stuck behaviour reads, then compare the updater's threshold with the
+consumer's test. A change to the frame step, the pacer or the fps limit is checked for this class.
+
+### Thread scheduling: Horizon is not a desktop OS
+
+- Horizon runs a woken better-priority thread at once on its core, before the thread that woke it goes on. A host
+  runs both in parallel, so state can be read one step early or freed while another thread still uses it.
+- Threads created with ideal core -2 start pinned to one core on Horizon. The game keys per-core data by
+  `GetCurrentProcessorNumber` without locks and expects that pinning.
+- The game has hundreds of spin loops without a system call. On the console a spinning thread holds one core of three;
+  on the host it burns a host core and can keep an equal-priority waiter out.
+- ARMv8 `stlr` followed by `ldar` is ordered (RCsc). C11 release/acquire lets x86 reorder that store and load. A
+  two-thread flag handshake that loses a wake points here; lifter changes to barriers and atomics pass
+  `tests\diff\atomic.py` and `tests\excl_mt.py`.
+- A scheduling change is judged by how long a ready, better thread waits for a core it may use, plus the slow-frame
+  count. A check that made one bug rarer once added 50-75 ms hitches.
+
+### Maxwell behaviour versus Vulkan defaults
+
+The port must do what the Switch GPU does: last-vertex provoking, (0,0,0,1) for unwritten vertex outputs, colour
+targets outside the shader's output map keep their texels, the G-buffer pass samples its own targets. The full list
+with sources: [maxwell.md](../maxwell.md), section 7.
+
+### Memory shared by the CPU and the GPU
+
+- The game fills textures and buffers with the CPU and samples them on the GPU without the flush a desktop API needs.
+  Missing uploads show as magenta, brown or stale textures.
+- The game reads GPU-rendered textures with the CPU (grass maps, cut and burn masks). Missing write-backs show as
+  effects that never happen: cut grass stays up, fire does not spread.
+- Memory only the GPU writes looks idle to write-watch tracking: a store from a shader leaves no fault and no dirty bit.
+- Anything kept per texture, per frame or per object without a free grows host memory until a commit failure after
+  hours. Watch private memory over a few idle minutes.
+
+### Device lost and GPU hangs
+
+- The `vk: pass marks` lines name the pass in flight. Validation names the misuse when there is one.
+- Submits and presents share one queue under one lock. A present from its own thread on a second queue gave device
+  lost on NVIDIA before Turing in another port.
+- Caches from another driver or GPU (pipeline caches, translated shaders) gave black windows after driver updates;
+  every cache is validated on load and dropped when vendor, device or `pipelineCacheUUID` change.
+- A first-time shader stall over 2 s on a slow GPU trips the Windows GPU timeout (display event 4101). Large one-shot
+  submissions (a boot-time decode of every texture) do the same.
+- Point lists from vertex shaders that write no point size are undefined without `VK_KHR_maintenance5`.
+
+### Windows and Wine host costs
+
+What the hot path must not do, with measured costs: [pr-review-kit/02-performance.md](../pr-review-kit/02-performance.md),
+"What the hot path must not do". For debugging, also:
+
+- `SuspendThread` returns at once, but `GetThreadContext` waits until the thread leaves the kernel. A sampler or a
+  debugger stalls for as long as the stopped thread's file open or driver call runs.
+- A thread stopped between its `sub rsp` and its stack probe has its stack pointer inside the guard page. Check
+  `PAGE_GUARD` with `VirtualQuery` before reading another thread's stack.
+- Paths go through the W APIs with UTF-8 to UTF-16 conversion. An A API call breaks on a path with non-Latin letters,
+  such as a Windows user name in Cyrillic or Chinese.
+- Wine emulates `GetWriteWatch` with `mprotect` and one fault per first write. Anything relying on cheap OS memory
+  tricks (write-watch, protection changes, fault handlers) costs more under Proton.
+- `win32u.dll` frames in a host stack are kernel graphics calls: memory allocation or submission.

docs/agents/running.md

@@ -0,0 +1,90 @@
+# Running and observing the game as an agent
+
+How an agent runs a build into gameplay without a person at the controls, captures what the game shows, and reads the
+results without flooding its context. Setting names and file formats live in [settings.md](../settings.md) and
+[controls.md](../controls.md); this page says which to use when. How to judge a performance change:
+[pr-review-kit/02-performance.md](../pr-review-kit/02-performance.md).
+
+## Before the first run
+
+- **A build dir per change.** Build into its own folder (`tools/build_game.py --out build/<ver>-<name>`) so a
+  comparison has two exes side by side. After every build check the exe's modification time: a failed link leaves the
+  old `totk.exe` in place and you test old code.
+- **Prepared shaders in that folder.** `tools/prepare_shaders.py build/<dir> --version <ver>` ([building.md](../building.md)
+  step 6). Without it the game compiles shaders during play, and every timing and every "a draw is missing" shot of the
+  first minutes says nothing about your change. Shader and pipeline caches are not copied between build folders.
+- **The same `totk.ini` on both sides** of any comparison. A folder without one runs at the defaults; check the log's
+  `resolution:` line of each run before comparing them.
+- **Saves.** `tools/run_game.py` copies the profile `build/profile`, swaps in a save from `build/save-backup/<name>/`
+  and deletes the copy afterwards, so the real profile is never written. To make a backup, copy the user folder from
+  `build/profile/save/` into `build/save-backup/<name>/`. Pick a save that starts where the change acts.
+
+## Running
+
+```powershell
+.venv\Scripts\python tools\run_game.py build\<dir> --save <name> --seconds 30 [--script <input.txt>] [--shot out.png] [--env KEY=VALUE] [--quiet] [--tag <name>]
+```
+
+- `--seconds` is game time since the world loaded (the log line `frame step: world clock starts` marks the start), at
+  any frame rate. Measure and script by game time; a frame count at another fps is another scene.
+- `--script` replays input ([controls.md](../controls.md), "Scripted input"). A new script starts with the presses of
+  `tools/input/load_save.txt`, which load the first save from the title. `TOTK_INPUT_RECORD` turns a person's play into
+  a script; ask the user to record the repro rather than guessing the button sequence.
+- `--shot` saves the last frame and builds pipelines synchronously: use it for pictures, never for timing.
+- `--quiet` waits until no other `totk.exe` or `ninja.exe` runs. Every timing you report comes from a `--quiet` run.
+- The log goes to `<dir>\logs\<tag>.log`; the tool prints `<log>: rc=<code>`.
+
+Run one game at a time: two instances share the GPU's memory and the CPU, and both runs' numbers are void.
+
+Running `totk.exe` yourself: set `TOTK_ASSUME_FOCUS=1` (an unfocused window otherwise pauses the game),
+`TOTK_INPUT=none` (the user's keyboard and pad stay out) and `TOTK_BACKGROUND_WINDOW=1`. Stop only the process you
+started, by PID: `taskkill /IM totk.exe` also closes the user's game.
+
+**Check that the run reached the world** before trusting it: the world clock line is in the log and the shot shows the
+scene you expected. Title menus on a slow boot eat presses, and runs of one script drift apart after 10-15 game seconds
+(enemy AI, physics, weather). Two runs are comparable only when shots at the same game second show the same scene.
+
+## Observing
+
+| You need | Use |
+|---|---|
+| pictures over a time span | `TOTK_SHOTS=w<lo>-<hi>:<step>` (world seconds) and `TOTK_SHOTS_DIR`; 480x270 by default |
+| the frames around a hitch or a one-frame glitch | `TOTK_FLIGHT=1`: keeps the last 600 frames with timings and thumbnails, dumps on F8, on a frame over `TOTK_FLIGHT_SPIKE_MS` or at `TOTK_FLIGHT_AT=<frame>`; `python tools/flight_view.py <dump dir>` lists the worst frames and writes a contact sheet |
+| every render pass of one frame | `TOTK_DUMP=<frame>` and `TOTK_DUMP_DIR`: a PNG per pass target plus `passes.txt` (draws, bound textures with swizzles, storage buffers, copies, clears) and `images.txt`; `TOTK_DUMP_BUF` adds a guest buffer |
+| Vulkan misuse | `TOTK_VK_VALIDATION=1` (needs the Vulkan SDK), run into the world; compare the message kinds with a run without your change. The layer caps repeats, so counts are not totals |
+| which pass the GPU was in at a device lost | the `vk: pass marks` lines; on by default on AMD, `TOTK_PASS_CRUMBS=1` elsewhere |
+| a crash | the crash report at the end of the log: exception, guest and host addresses, registers |
+| a run on a large card that acts like a small one | `TOTK_VRAM_LIMIT_MIB=<MiB>` |
+
+Frame dumps: a draw's argument lines come before its pass header. Pass numbers shift between runs, so find a pass by
+its draw count or its targets, not by its number. RenderDoc cannot capture this renderer ([controls.md](../controls.md),
+"RenderDoc capture").
+
+## Measuring
+
+| What | How |
+|---|---|
+| fps, worst frame, render-thread waits, GPU busy time, VRAM, `bound: gpu\|cpu\|pace` | `--env TOTK_TRACE=perf,sched`, then `python tools\perf_summary.py <log> <lo> <hi>` |
+| slow frames | count the `perf: slow frame` lines; each names what the render thread was blocked on (pipeline, translation, texture, fence, pacing) |
+| the frame's critical path, by thread and state | `TOTK_CRIT=w<lo>-<hi>` and `TOTK_CRIT_OUT=<absolute path>` with `sched` tracing, then `python tools\crit_path.py <file>` |
+| where CPU time goes | `TOTK_PROFILE=<path>` and `TOTK_PROFILE_FRAMES=w<lo>-<hi>`, then `python tools\prof_report.py <profile> <totk.map>`; `TOTK_PROFILE_RENDER=1` samples the render thread with call chains |
+| a switch on a static scene | a setting set to `ab` with `TOTK_AB=<seconds>` alternates the two sides in blocks of game time; `python tools\ab_summary.py <log> --skip <seconds>`. On a moving scene compare whole runs instead |
+
+A single run proves little: runs of one script on one build spread by a third in their 1% lows. Compare medians of at
+least two whole runs per side, on a quiet machine.
+
+## Probes
+
+Each probe is a mode of `totk.exe`, run from the build folder; the list with what each covers is at the top of
+`runtime/main.c`. `--selftest` needs no game files. `--probe-kernel`, `--probe-vkexec`, `--probe-nvn-tex` and
+`--probe-shader` cover most runtime changes and run in seconds: run them before any game run.
+
+## Reading output without filling the context
+
+- Game logs run to megabytes. `grep` the lines you need, `tail` the end, pipe `perf_summary.py` through `grep SUMMARY`.
+- Generated C runs to hundreds of thousands of lines per file. `grep -n` the function, then print its line range.
+- Look at screenshots downscaled: paste several into one 1280x720 grid and read the grid once. Write down what it shows
+  instead of opening it again.
+- Send long builds and runs to the background with their output in a file; read the last lines when they end.
+- Keep a notes file in the build folder from the first finding: what is ruled out, the leading hypothesis, the next
+  step, temporary debug edits with file and line. A fresh session continues from it.

docs/agents/symptoms.md

@@ -0,0 +1,71 @@
+# Symptoms and their usual causes
+
+What a player reports, the causes this port and other Switch ports (nxnative, MacNativeGames, SavageWindRecomp,
+LAS Recomp) found behind it, and the first check. Search by the player's words and by the object. Which of these are
+already fixed here, and where: [pr-review-kit/04-knowledge.md](../pr-review-kit/04-knowledge.md), "Fixed bug classes".
+GPU semantics behind the graphics rows: [maxwell.md](../maxwell.md). The method around this table:
+[debugging.md](debugging.md).
+
+## Graphics
+
+| Symptom | Usual causes | First check |
+|---|---|---|
+| magenta, brown or stale textures; caves or sky wrong | the CPU filled the texture and the GPU never got the upload (no flush on the console) | frame dump: the texture's upload count and mirror state in `passes.txt` |
+| effects that never happen: cut grass stays up, fire does not spread | the CPU reads a GPU-rendered texture that was never written back | which texture the game reads after its fence; write-back of that target |
+| black scene, black dialog background or loading map | deferred binds read at record time; a 3D target rejected; a texture compare ignoring the channel mapping | frame dump: the first pass whose target goes black |
+| lime silhouettes, rectangles in the sky | a target outside the shader's output map overwritten instead of kept | dump: the target's previous contents versus after the pass |
+| dark or magenta 16x16 blocks, missing characters | a pass samples its own render targets (G-buffer feedback) | feedback-loop layout and barriers on that pass |
+| tile rectangles, wrong data in tiles | storage buffers bound in the wrong slot order | `passes.txt`: storage buffers with their c0 slots |
+| streaks on cliffs, characters stretched into streaks | primitive restart ignored; integer vertex inputs against a float format | validation run; the draw's topology and vertex formats in the dump |
+| garbage geometry on title or load screens | 8-bit indices widened on the CPU before a copy in the same submission wrote them | whether a copy to the index buffer precedes the draw in one submission |
+| black tiles, masks not cleared | clears with a partial channel or stencil mask skipped or done in full | `passes.txt` clears and their masks |
+| garbage in feedback effects at first use | new GPU-only images start undefined; the console starts them at zero | clear on creation and compare |
+| empty clouds or fog for the whole session | a one-shot draw skipped while its pipeline built, never drawn again | `perf: slow frame` lines naming pipeline waits; the draw's first frame |
+| fog that never parts, flicker in tile rows | driver constants (c0: dispatch size, point size, base instance) not filled | the c0 contents the translator expects for that stage |
+| no sun shadows, wrong lighting | unwritten vertex outputs read as zero instead of (0,0,0,1) | the shader's output map |
+| one flat ambient colour | a 1-layer 2D array sampled as 2D | the view type the shader expects |
+| upside-down text | lower-left window origin | origin mode in the log |
+| flicker or a texture swapped with a similar one | the newest texture picked at shared storage instead of the one with the matching view | texture lookup by id in the dump |
+
+## Timing and frame pacing
+
+| Symptom | Usual causes | First check |
+|---|---|---|
+| stuck until a reload or teleport, locked after an action | a timer remainder below `FLT_EPSILON` with the real-time step ([debugging.md](debugging.md)) | `realtime = 0` in `totk.ini`; if the bug is gone, find the timer field |
+| slow motion in GPU-heavy scenes, dynamic resolution never drops | the engine's GPU time comes from report timestamps; wrong timestamps make the frame look free | what the engine reads as GPU time and what the report writes |
+| frame counter wanders 50-60 | two 60 Hz clocks (vsync event and present acquire) drifting apart | whether both follow one display tick |
+| a 30 Hz thread runs at 27 Hz, slow cutscenes | host sleep overshoot: Windows waitable timers wake 0.3-0.5 ms late | `TOTK_TRACE=sched` sleep asked versus took |
+| 20 fps after a scheduler change | a yield back-off that starts too early | yield handling against the console's |
+| climbing or movement wrong only above some fps | game code that assumes a fixed step count per frame | compare at 30, 60 and the failing rate |
+
+## Threads and crashes
+
+| Symptom | Usual causes | First check |
+|---|---|---|
+| state read one step early, one frame of inconsistent data | a woken better-priority thread does not run before the waker goes on | sched trace around the handoff |
+| per-core data corrupted, rare crash in a container walk | threads the console pins to one core run on any host core | the thread's core mask from its creator; a run on one core |
+| rare hang or lost wake | address-arbiter signal values or wake order differ from Horizon | the arbiter path against mesosphere |
+| a flag handshake loses a wake | `stlr`/`ldar` ordering lost in x86 release/acquire | the guest code's barrier pattern |
+| a host core at 100% while the game waits | game spin loops without system calls | profile: which guest function spins |
+| crash on world load | sizes the game reads from the driver (command memory used) differ from the console's | the sizes the game asks for against the console's |
+| crash with one save only | the save itself; some crash on a real Switch | the same save on another build and, if possible, on hardware |
+
+## Device lost, hangs, black window
+
+| Symptom | Usual causes | First check |
+|---|---|---|
+| device lost a few frames in, GPU writes to address 0 (older NVIDIA) | present from another thread or queue | every submit and present on one queue under one lock |
+| device lost or garbage on one AMD generation | views the driver branch mishandles (BC block views of 3D textures) | `vk: pass marks`, validation; the formats in the pass |
+| device lost on one scene, every run, one vendor | a shader or pass that driver cannot run | pass marks name the pass; skip shaders by hash to bisect |
+| GPU timeout after 1-2 min, single frames over 2 s | first-time shader stalls or a huge one-shot submission on a slow GPU | prepared shaders; submission sizes at boot |
+| GPU hang at boot with pipeline libraries | point lists without a point size | the shader writes point size or the device clamps it |
+| black window after a driver update | caches from another driver | the log line `vk_pipeline: ... written by another device or driver`; delete the caches next to `totk.exe` |
+
+## Memory
+
+| Symptom | Usual causes | First check |
+|---|---|---|
+| host memory grows in play until a commit failure after hours | state kept per texture init or per frame and never freed | private memory over a few idle minutes |
+| multi-GB peaks while travelling | upload staging grows inside one submission | staging bytes per submission |
+| out of memory after minutes | dead textures re-entering tables | live images and allocator bytes over time |
+| stutter as VRAM fills | new memory blocks allocated on the render thread (8-23 ms each) | profile the render thread over the hitch: `vkAllocateMemory` under the allocator's lock |

docs/architecture.md

@@ -85,11 +85,17 @@ Every guest thread is a host thread. TotK assumes Horizon's scheduler: three app
 - Vulkan 1.3 back end in place of MacNativeGames' Metal one. At `QueueSubmitCommands` the recorded NVN commands are replayed into Vulkan command buffers (`vk_exec*.c`).
 - Memory pools. Apple's unified memory let MacNativeGames use the game's pools zero-copy. The back end imports the pools as host memory (`VK_EXT_external_memory_host`). On a discrete GPU, pools bound as buffers are mirrored in device-local memory and the pages the CPU wrote (write watch, above) are uploaded before a replay; idle pool regions are made read-only so the write-watch queries skip them (`vk_memory.c`). GPU-rendered textures are written back when the CPU reads them: on a pool invalidate, on a CPU copy, and for small targets once the CPU waits for a fence (`vk_texture.c`).
 - Replay thread. `TOTK_REPLAY_ASYNC=1` moves the replay off the submitting thread onto a queue at most one frame behind the game; reads of GPU results wait for the newest writer instead of draining the queue (`vk_backend.c`). It is off by default. Where the game waits for the GPU it overlaps the replay with the game threads (a GTX 1650 in the Depths, issue #88), a gain measured only there; while it was the default, a GTX 1660 Ti at the 60 fps cap with the GPU at 26 % reported uneven motion (#659). It adds up to a frame of latency; `replay_async = 1` turns it on.
+- Chunked replay. A large submission is recorded in chunks by several threads (`TOTK_REPLAY_WORKERS`, 0 by default: the sequential replay; 4 is a good value from 8 logical processors) and the chunks go to the queue in order (`vk_exec.c`). A sequential scan keeps a copy of the queue state where each chunk starts and lists what a chunk would otherwise learn from the chunks before it: inline uniform updates, conditions, contact shadow marches, uniform drops. It also makes render targets live in replay order and notes the op, since a chunk can sample a storage before the chunk that renders it got there or after the scan went past a later target there: a chunk's texture slots resolve as of its own op, and the chunks after a texture with no entry yet wait for the chunk that makes it. DLSS's decisions (the G-buffer, the light target, the end of the scene, the scaling pass) are the scan's too, and its copy of the lit scene and its evaluation are records of the chunk that holds the op; a chunk's draws are jittered while they come before the scan's end of the scene. The evaluation waits until the chunks before it are recorded, as it reads what their draws noted (the camera block, the jitter, the per-object motion list, which chunks add to under one lock), and the chunks after it wait for it, as they sample its output. `TOTK_REPLAY_STATE_CHECK=1` compares the state after the last chunk with the scan's.
+- Frame ahead. `TOTK_SYNC_AHEAD=1` (`sync_ahead`, off by default) makes the one `nvnSyncWait` per frame that waits for the frame presented last wait for the frame before it, so the game builds the next frame while the GPU still runs the previous one (`nvn_api_core.c`, `nvnSyncWait`). The game then writes memory that in-flight work still reads. This is safe only with shadow storage (`vk_memory.c`, the CPU-visible pools are host-visible copies of guest memory, on by default on Linux and Wine, `TOTK_SHADOW=1` on native Windows; without it the switch is ignored): a CPU write reaches the GPU only through the shadow copy made before each submit (`vk_mem_shadow_sync`), and a copy that would overwrite bytes an unfinished submission reads is queued as a GPU copy in the prologue command buffer of the part submitted next (`vk_mem_shadow_prologue`, `TOTK_SHADOW_DEFER=1`, the default with the feature) or, with `=0`, waited for on the replay thread (`shadow_reader_wait`). Only that one wait lags (identified by its call site, known for 1.2.1 and 1.4.3); results the game reads back from the GPU after it can be a frame old.
 - Recorder thread. The Vulkan calls the replay makes into its command buffer are made by a thread of their own (`vk_rec.c`, after Savage Wind Recomp and LAS Recomp): each `vkCmd*` copies its arguments into a packet of a ring and the recorder makes the calls in order; calls into other command buffers are made at once; the recorder also ends the command buffer, and the submit waits until it has. In the town on an RX 9070 the replay takes 17% less time on the submitting thread. It is off by default on AMD GPUs, where the driver loses the device or hangs the GPU at startup with it (`TOTK_VK_RECORDER=1` forces it on, `=0` makes every call on the submitting thread).
 - Uniform snapshot. The game rewrites per-object uniform blocks on other threads soon after it submits the frame
   that reads them, while the replay still records it. At the start of a submission's replay every uniform range it
   binds from CPU-written pools is copied, and the draws bind the copy (`vk_usnap.c`, after MacNativeGames'
   `TOTK_UBO_SNAPSHOT`); without it, late draws read another object's matrix.
+- Uniform updates. `nvnCommandBufferUpdateUniformBuffer` between the draws of a pass is replayed as an overlay: the
+  bytes go to scratch memory, later bindings of the range bind them, and the copies into the game's buffers are
+  recorded behind one barrier when the pass ends (`vk_exec_draw.c` `op_update_ub`, `TOTK_UB_OVERLAY`). A copy at the
+  update itself would end the pass with a full barrier, which the terrain paid before most of its draws.
 - Driver data. The NVN driver keeps a per-stage constant bank (c0) that shaders read for things like point size, dispatch size and base instance. The back end fills it the way the driver does.
 - Shaders: the Ryujinx translator 1.3.2 with MacNativeGames' patches and ours (`tools/xlate/patches/`), built with .NET 10 NativeAOT as `totk_xlate.exe` and run as a separate process. Its SPIR-V goes to Vulkan directly. `tools/prepare_shaders.py` and `totk.exe --prepare-shaders` translate and compile the variants of MacNativeGames' shared list ahead of play (29,587 for 1.4.3); the rest are translated on first use and cached on disk.
 - Pipelines: `VK_EXT_graphics_pipeline_library` where the driver has it. Shader stages are compiled ahead into libraries; a draw that meets a new state links them at once (fast link), and an optimised pipeline replaces the linked one when a background thread finishes it. Pipeline keys seen in play are saved and built at boot. A draw whose pipeline is not ready is skipped when one of its pass's targets was rendered in the previous frame, and waits otherwise, so one-shot passes are never lost; a full-screen draw is skipped as well rather than waiting (`TOTK_PIPE_SKIP`); the stage libraries a waiting draw needs are built on idle workers in parallel (`TOTK_PIPE_WAIT`). Without the extension, a missing pipeline is built on a worker and the draw waits or is skipped by the same rule.

docs/building.md

@@ -118,6 +118,13 @@ of them.
    button prompts"), which `build_game.py` copies next to `totk.exe` like `romfs-patches`. `--prompts pc` writes
    `romfs-patches-pc`, the keyboard and mouse prompts ([controls.md](controls.md), "PC button prompts").
 
+   A build from source has no layouts for other shapes than 16:9. At an `output_res` such as 5120x1440 the game then
+   draws the largest 16:9 size inside it (2560x1440) and the present adds bars, instead of a wide camera over a
+   stretched HUD (`runtime/resolution.c`, `shape_fallback`). For the full shape make `romfs-patches-<w>x<h>` once per
+   size: `make_romfs_patches.py --version <ver> --aspect <w>x<h> --out build\<ver>\romfs-patches-<w>x<h>` (the sizes
+   the installer makes are `ASPECT_SIZES` in `installer/steps.py`); `build_game.py` copies every `romfs-patches*`
+   folder next to `totk.exe`.
+
 6. Prepare the shaders (once per build folder and translator). It needs `refs\MacNativeGames` (`tools\fetch_refs.sh`)
    for the shared shader list:
 
@@ -153,30 +160,9 @@ folder. The folder holds code and data derived from your game: for your own mach
 
 ### Profile-guided build
 
-`build_game.py --pgo <file>.profdata` compiles the generated code at -O2 with an execution profile, and the runtime
-with it too: hot paths laid out straight, hot calls inlined, code that never ran compiled for size. On 1.4.3 the fight
-bench ran 12-15% faster per frame and a fight outside the training about 6%. The installer builds with
-`data/pgo-<ver>.profdata` of the release when the update has one, else at -O1.
-
-A profile only fits the generated C it was trained on: functions are matched by name and control-flow hash, so a
-function whose C changes loses its counts without a warning. Retrain after a lifter or generator change, from a clean
-`recomp/`. The instrumented build needs llvm-mingw, as MSYS2's clang has no profile runtime:
-
-```powershell
-$llvm = "toolchain\llvm-mingw-<date>\llvm-mingw-<date>-ucrt-x86_64"
-.venv\Scripts\python tools\build_game.py --version v143 --out build\v143-pgogen --toolchain $llvm --pgo gen
-```
-
-Play `build\v143-pgogen\totk.exe` through varied scenes (fights, towns, riding, building, the Depths, menus) for 20-30
-minutes; it is slower than a normal build. It writes `build\v143-pgogen\pgo\*.profraw` every minute and at exit.
-Scripted runs from saves (`tools/run_game.py`) add to the same file. Then:
-
-```powershell
-.venv\Scripts\python tools\pgo.py merge build\v143-pgogen --version v143
-```
-
-writes `build\pgo\pgo-v143.profdata` and its record (`.json`: the `recomp/` tree it fits). `make_release.py` ships every
-profile of `build\pgo` in `data\` and warns when a record names another `recomp/` tree than the one released.
+`build_game.py --pgo <file>.profdata` compiles the generated code and the runtime at -O2 with an execution
+profile, 6-12% shorter frames. Which build uses which profile, when to retrain, where and how to train and how to
+measure a new profile: [pgo.md](pgo.md).
 
 ## Linux (in progress)
 
@@ -231,7 +217,12 @@ Before a commit:
 A change to the button prompts (`tools/make_romfs_patches.py --prompts`, `tools/button_icons.py`) passes
 `.venv-py314\Scripts\python tests\romfs_prompts_test.py`, which runs the tool on a RomFS it makes itself (no game
 files); a change to the settings passes `totk.exe --probe-cfg` and `python tests\settings_doc_check.py`, which fails
-when a setting is described neither in `totk.ini.example` nor in [settings.md](settings.md).
+when a setting is described neither in `totk.ini.example` nor in [settings.md](settings.md). A change to the mods' ExeFS patches
+(`tools/exefs_patches.py`, `runtime/mods.c`, the `folded.txt` of `recomp/gen.py`) passes
+`.venv\Scripts\python tests\exefs_patches_test.py`, which makes its own modules and runs `gen.py` on one with a jump
+table (no game files), and `totk.exe --probe-mods`. A change to the cheat VM or the cheat opcode tables
+(`runtime/cheats.c`, `tools/cheats.py`) passes `.venv\Scripts\python tests\cheats_vm_check.py`, which runs both on
+cheats of its own with clang from PATH (`--cc` names another).
 
 The lifter (`recomp/`) and the CPU headers (`runtime/cpu.h`, `runtime/simd.h`) are checked against Unicorn. Every lifter
 change passes these, all run with `.venv`:

docs/cheats.md

@@ -58,7 +58,7 @@ The game's code is compiled ahead of time, so a cheat cannot write new instructi
 |---|---|---|
 | code | Opcode 0 writes into main's code | Built into the game at setup as a patch set (`recomp/patches.py`); switches at once. |
 | key | Code writes inside a button condition (`8` mask, writes, `21` else, the original words, `20`) | A patch set that is on while the cheat is on and the buttons are held. |
-| data | Writes data only: heap, main's data, pointer chains, button conditions | Runs in a cheat VM after dmnt's, opcodes 0 to A, every presented frame while on. |
+| data | Writes data only: heap, main's data, pointer chains, button conditions | Runs in a cheat VM after dmnt's, opcodes 0 to A and C0 to C3, every presented frame while on. |
 | master | `{name}` with code writes | A patch set that is always on. |
 | restore | Writes back only the game's original instructions | Not needed: switching a cheat off restores them. Greyed out. |
 
@@ -66,7 +66,7 @@ A cheat whose code writes call or jump into a code cave the cheat writes itself
 
 Not supported, and shown greyed out with the reason:
 
-- the 0xC and 0xF opcode extensions;
+- opcode C4 (extended button condition) and the 0xF extensions (FF0 pause, FF1 resume, FFF debug log);
 - opcode 5 with a register offset;
 - code writes mixed with other opcodes, except the button pattern above;
 - code writes that change the CPU flags differently from the game's instruction, or code caves with branches or pc-relative instructions.

docs/controls.md

@@ -14,7 +14,7 @@ The log names the layout in effect (`input: gamepad layout ...`); the menu shows
 |---|---|---|---|
 | Left stick | left stick | W A S D | move |
 | Right stick | right stick | mouse; arrow keys as a second source | camera |
-| Y | west | left mouse button | attack |
+| Y | west | left mouse button, T | attack |
 | ZL | left trigger (over 50%) | right mouse button, Z | shield, lock on |
 | ZR | right trigger (over 50%) | F, X | bow |
 | X | north | Space | jump |
@@ -29,7 +29,7 @@ The log names the layout in effect (`input: gamepad layout ...`); the menu shows
 | Minus | Back | M | map |
 | Right stick shaken left and right | shake the right stick | G | detach a fused part from what Ultrahand holds |
 
-Esc is unbound in the game; it opens the settings menu and releases the mouse cursor (see Mouse camera). Z and X duplicate ZL and ZR so menu pages turn with two neighbouring keys; the menu tabs (L, R) are Q and R. The mouse wheel zooms the Purah Pad's map and is unbound elsewhere. The bindings are the tables of `input_map` in `runtime/input_map.c`: the keys (platform-neutral key ids) and one gamepad table per layout (gamepad inputs by position); `runtime/input_win.c` translates Win32 virtual keys and SDL buttons.
+Esc is unbound in the game; it opens the settings menu and releases the mouse cursor (see Mouse camera). Z and X duplicate ZL and ZR so menu pages turn with two neighbouring keys; the menu tabs (L, R) are Q and R. T duplicates Y for the game's menus, where the left mouse button is A while the pointer is shown (sort the inventory with T). The mouse wheel zooms the Purah Pad's map and is unbound elsewhere. The bindings are the tables of `input_map` in `runtime/input_map.c`: the keys (platform-neutral key ids) and one gamepad table per layout (gamepad inputs by position); `runtime/input_win.c` translates Win32 virtual keys and SDL buttons.
 
 ## Xbox button prompts
 
@@ -46,7 +46,7 @@ Esc is unbound in the game; it opens the settings menu and releases the mouse cu
 | Plus, Minus | Menu, View | Menu, View |
 | D-pad | cross | cross |
 
-The action prompt (`DoCommand_00`) shows its button in a diamond whose shape gives the face buttons' positions as on a Switch, with the other three letters around it. With `nintendo` its letters are swapped with the rest and every one names the button in that place. With `xbox` and `auto` the letters are right but the diamond keeps the Switch positions: A shows on its right, while the button labelled A is at the bottom. The sticks keep the Switch icons, and so do prompts drawn as fixed images (some tutorial animations).
+The action prompt (`DoCommand_00`) shows its button in a diamond whose shape gives the face buttons' positions as on a Switch, with the other three letters around it. With `nintendo` its letters are swapped with the rest and every one names the button in that place. With `xbox` and `auto` the letters stay and the diamond is laid out as an Xbox pad: A at the bottom, B on the right, X on the left and Y at the top, the button the prompt names lit in its place, its bar reaching left from X and right from the others (`tools/make_romfs_patches.py`, `bflyt_xbox_diamond`: the panes of `blyt/DoCommand_00.bflyt` take their partner's place, A's of B and X's of Y). The sticks keep the Switch icons, and so do prompts drawn as fixed images (some tutorial animations).
 
 The prompts are part of the game's fonts, textures and messages, so they follow the layout and not the bindings. A button remapped with a `bind_` line or on the Controls tab keeps the prompt the game draws for it, which then names the wrong button. Use the Xbox prompts with the default bindings.
 
@@ -55,7 +55,9 @@ The game reads its UI archive and fonts once at boot, so the prompts change at t
 - every prompt but the button guide in play is text in the button font, `scft/External.bfotf` in each `Font/Font*.Nin_NX_NVN.bfarc.zs`: glyphs U+E040 to U+E043 (A B X Y), U+E044 to U+E047 (L R ZL ZR), U+E04B and U+E04C (Plus, Minus), U+E049 and U+E060 to U+E065 (the d-pad). fontTools writes the font again, so the archives grow and the folder also carries the resource size table (`System/Resource/ResourceSizeTable.Product.<n>...rsizetable.zs`) with their entries raised by as much; without that the game stops at start.
 - the button guide in play draws L and ZL from the images `Nt_KeyTexL_00` and `Nt_KeyTexZL_00` of the Common archive's `timg/__Combined.bntx`, painted with the same icons. The action prompt draws its button from `Nt_KeyTexA/B/X/Y_00`, whose image data is swapped for `nintendo`, and the diamond's letters are text the game takes from `LayoutMsg/DoCommand_00.msbt` of each language's `Mals` archive (and the layout's own text panes behind it), swapped the same way.
 
-The Switch prompts stay, with a note under *Button prompts* and a `RomFS overlay: ...` line in the log, when the folder is missing (an install made by an older setup, or one where the setup could not prepare it and showed a warning; running the setup again on the install folder writes it), when it fails its check, with a game resolution of another shape than 16:9 (its own UI layout wins; 16:10 and 21:9), and when the `romfs_mods` folder replaces one of its files: `romfs-patches` then loads in its place, where two overlays replacing one file would stop the game. In a source build, `.venv-py314\Scripts\python tools\make_romfs_patches.py --version <ver> --prompts xbox` (and `--prompts xbox-label`) writes `build\<ver>\romfs-patches-xbox` (`-label`); it needs fontTools from `tools/requirements-py314.txt`. `--xbox-prompts` is the older name of `--prompts xbox`.
+The Switch prompts stay, with a note under *Button prompts* and a `RomFS overlay: ...` line in the log, when the folder is missing (an install made by an older setup, or one where the setup could not prepare it and showed a warning; running the setup again on the install folder writes it), when it fails its check, with a game resolution of another shape than 16:9 that has no folder of its own (below), and when the `romfs_mods` folder replaces one of its files: `romfs-patches` then loads in its place, where two overlays replacing one file would stop the game. In a source build, `.venv-py314\Scripts\python tools\make_romfs_patches.py --version <ver> --prompts xbox` (and `--prompts xbox-label`) writes `build\<ver>\romfs-patches-xbox` (`-label`); it needs fontTools from `tools/requirements-py314.txt`. `--xbox-prompts` is the older name of `--prompts xbox`.
+
+At a game resolution of another shape than 16:9 the game loads `romfs-patches-xbox-<w>x<h>` (or `romfs-patches-xbox-label-<w>x<h>`) in place of `romfs-patches-<w>x<h>`: the same prompts with that shape's UI layout, `make_romfs_patches.py --prompts xbox --aspect <w>x<h> --reuse <install>\romfs-patches-xbox --out <install>\romfs-patches-xbox-<w>x<h>`. The setup writes both for each of the fifteen sizes it lays the UI out for, as for the PC prompts below, so on a display of one of those shapes the game resolution follows the display with these prompts too. At another size, without that folder, the Switch prompts stay (*off at this game resolution*).
 
 ## PC button prompts
 
@@ -70,10 +72,10 @@ The Switch prompts stay, with a note under *Button prompts* and a `RomFS overlay
 | Left stick, its press | W A S D (or the keys of the directions meant), Ctrl |
 | Right stick, its press | the mouse, with an arrow for each direction meant, C |
 
-- The glyphs keep the width of the game's own, so the right-aligned guides do not run into their text. A key is a rounded square with its label cut out in the game's own text fonts; a mouse button is a mouse with that button filled and the others hollow.
+- The glyphs keep the width of the game's own, so the right-aligned guides do not run into their text. A key is a rounded square with its label cut out in the game's own text fonts (Shift: an arrow over its name); a mouse button is a mouse with that button filled and the others hollow.
 - The button guide in play draws every button from images of the Common archive (`Nt_KeyTexA/B/X/Y/L/ZL_00`, `Nt_KeyTexLSt_00` and `_01`, the stick tilted and pressed): all of them are painted with the same icons. The letters of the action prompt's diamond are one character of the Normal font each: E, an arrow for Shift, a bar for Space, L and R for the mouse buttons.
 - Like the Xbox prompts they follow the defaults and not the bindings. `--binds <totk.ini>` draws the first key of each `bind_` line of that file instead, for a folder made by hand.
-- At a game resolution of another shape than 16:9 the game loads `romfs-patches-pc-<w>x<h>` in place of `romfs-patches-<w>x<h>` when it is there: the same prompts with that shape's UI layout, `make_romfs_patches.py --prompts pc --aspect <w>x<h> --out <install>\romfs-patches-pc-<w>x<h>`. The setup does not write those, so without one the Switch prompts stay (*off at a game resolution not 16:9*).
+- At a game resolution of another shape than 16:9 the game loads `romfs-patches-pc-<w>x<h>` in place of `romfs-patches-<w>x<h>`: the same prompts with that shape's UI layout, `make_romfs_patches.py --prompts pc --aspect <w>x<h> --reuse <install>\romfs-patches-pc --out <install>\romfs-patches-pc-<w>x<h>`. The setup writes one for each of the fifteen sizes it lays the UI out for, so on a display of one of those shapes the game resolution follows the display with these prompts too. At another size, without that folder, the Switch prompts stay (*off at this game resolution*).
 
 ## Window keys
 
@@ -96,7 +98,7 @@ The tab writes `totk.ini` lines, one per Switch button or stick direction: `bind
 - `key:default` and `pad:default` keep that part's defaults (`pad:default` is each layout's own button).
 - `none` leaves the button unbound.
 
-A line replaces every default of that button, pad and keyboard, so list the keys you still want or use `key:default`. Pad names are physical positions, so a line means the same in both layouts and replaces the layout's own button for that Switch button. Stick directions take keys only: the controller's sticks and the mouse camera always work. Buttons without a line keep the table above. Names are case-insensitive. An invalid entry logs one `input: bind ...` line and is skipped; a line without a valid entry changes nothing. The `TOTK_BIND_<button>` environment variables work like any other setting. A changed line applies at once (`runtime/input_win.c` builds the tables again at its next poll).
+A line replaces every default of that button, pad and keyboard, so list the keys you still want or use `key:default`. Pad names are physical positions, so a line means the same in both layouts and replaces the layout's own button for that Switch button. Stick directions take keys only: the controller's sticks and the mouse camera always work. Buttons without a line keep the table above, less any key a line binds: with `bind_X = key:t`, T presses X only, not also Y. Names are case-insensitive. An invalid entry logs one `input: bind ...` line and is skipped; a line without a valid entry changes nothing. The `TOTK_BIND_<button>` environment variables work like any other setting. A changed line applies at once (`runtime/input_win.c` builds the tables again at its next poll).
 
 A pad that reports its face buttons swapped (A/B and X/Y, as some GuliKit models do) is fixed with:
 
@@ -197,7 +199,7 @@ The stick direction bits (`StickLLeft` to `StickRDown`) are derived from the sti
 
 ## Mouse in the game's menus
 
-On game 1.4.3, while one of the game's menus has its focus box (the frame the d-pad moves: the title, the save list, the inventory and its System tab, Options, shops, dialogue choices), the cursor is released from the mouse camera and shown in the middle of the window. A focus gap of up to 300 ms (a tab switch, a popup opening: the game drops the focus box for a frame or two) keeps the pointer where it is instead of capturing and centring it again. The item under the pointer takes the focus as if the d-pad had moved there, with the game's highlight, sound and description; a left click is A on it, a right click is B. The keys and the pad keep working, and the focus moves only when the pointer moves. This happens only while the keyboard and mouse are the device used last: after a gamepad button press or a stick pushed past half way the menus are as on the console, with no pointer, until the mouse moves or a key is pressed (`input_kbm_last` in `runtime/input.c`). The ability wheel, the d-pad strips, Autobuild and the Purah Pad's camera stay gameplay: the mouse keeps the camera there. The map has no focus box; it is found in `eui::ScreenMgr`'s list of open screens (every 100 ms: the Purah Pad's frame `AppMenuOverlay_00` with `AppMap_00` and the menu background `AppMenuBG_00`), and there the mouse moves the map's cursor as the left stick does, with the cursor still captured, and the scroll wheel zooms: each notch is a full-tilt push of the right stick (the map's zoom) for 100 ms with 50 ms of neutral after it, up (in) for a turn away from you; a fast scroll's notches wait their turn (at most four), a turn the other way drops them. The Scope opens the same map screen without the menu background and keeps the mouse camera. When the menu closes the cursor is captured again.
+On game 1.4.3, while one of the game's menus has its focus box (the frame the d-pad moves: the title, the save list, the inventory and its System tab, Options, shops, dialogue choices), the cursor is released from the mouse camera and shown in the middle of the window. A focus gap of up to 300 ms (a tab switch, a popup opening: the game drops the focus box for a frame or two) keeps the pointer where it is instead of capturing and centring it again. The item under the pointer takes the focus as if the d-pad had moved there, with the game's highlight, sound and description; a left click is A on it, a right click is B. The middle and side mouse buttons keep their bindings (bound to L and R they turn the tabs), and T presses Y (sort). The keys and the pad keep working, and the focus moves only when the pointer moves. This happens only while the keyboard and mouse are the device used last: after a gamepad button press or a stick pushed past half way the menus are as on the console, with no pointer, until the mouse moves or a key is pressed (`input_kbm_last` in `runtime/input.c`). The ability wheel, the d-pad strips, Autobuild and the Purah Pad's camera stay gameplay: the mouse keeps the camera there. The map has no focus box; it is found in `eui::ScreenMgr`'s list of open screens (every 100 ms: the Purah Pad's frame `AppMenuOverlay_00` with `AppMap_00` and the menu background `AppMenuBG_00`), and there the mouse moves the map's cursor as the left stick does, with the cursor still captured, and the scroll wheel zooms: each notch is a full-tilt push of the right stick (the map's zoom) for 100 ms with 50 ms of neutral after it, up (in) for a turn away from you; a fast scroll's notches wait their turn (at most four), a turn the other way drops them. The Scope opens the same map screen without the menu background and keeps the mouse camera. When the menu closes the cursor is captured again.
 
 `runtime/menu_cursor.c` reads eui's screens at each pad read and moves the focus with the writes `eui::BoxCursorMgr::moveBoxCursor` makes, so no game code is called. `menu_cursor = 0` (`TOTK_MENU_CURSOR=0`, live; the *Mouse in the game's menus* row of the Controls tab) turns it off; `TOTK_MENU_CURSOR_LOG=1` logs every screen it reads (by default the first six), with the layout size, the node count and the hit panes it uses.
 

docs/installer.md

@@ -60,7 +60,9 @@ loads the saves 1.4.3 wrote.
 4. **Lift** the ExeFS to C (`recomp/gen.py`, `recomp/patches-v121.txt` or `patches-v143.txt`) into a temp folder next
    to the installer. The code cheats of the mods' cheat files and the mods' ExeFS patch files become patch sets first
    (`tools/cheats.py`, `tools/exefs_patches.py`, status in `build-cache/cheats.status` and `exefs.status`); a file the
-   tools refuse costs those cheats or patches, not the install.
+   tools refuse costs those cheats or patches, not the install. The patch files' writes into main's data go to
+   `build-cache/exefs.data`, which the game stores into main at start; after the lift they are checked against the
+   bytes the lifted code read as constants (`<gen>/main/folded.txt`, its jump tables).
 5. **Compile** the generated C, `modules.c`, `romfs.c`, `patches.c` and `ov_zstd.c` (they bake the per-install layout, RomFS
    hashes and the game version's patch page) and link
    them with the prebuilt runtime library for the host's level (`libtotkrt.a` or `libtotkrt-v2.a`) into `totk.exe`
@@ -72,8 +74,11 @@ loads the saves 1.4.3 wrote.
    writes the translations it makes in play as loose files; running the tool again with the game closed folds those in.
    The patches are written four times: `romfs-patches`, `romfs-patches-xbox` and `romfs-patches-xbox-label`
    (`--prompts`: Xbox button prompts for `prompts = xbox`, the first for `pad_layout = nintendo`, the second for `xbox`
-   and `auto`), and `romfs-patches-pc` (keyboard and mouse prompts for `prompts = pc`). The three prompt folders are
-   optional: when one cannot be made, the setup warns and finishes.
+   and `auto`), and `romfs-patches-pc` (keyboard and mouse prompts for `prompts = pc`). Each of the three prompt
+   folders is then laid out once more for each game resolution of another shape, `romfs-patches-<style>-<w>x<h>`
+   (`--prompts <style> --aspect`). Those differ from `romfs-patches-<style>` in the UI archive only, and `--reuse`
+   hard-links the other files from it, so the forty-five folders take seconds and a few MiB each. The prompt folders are optional: when one cannot be made, the
+   setup warns and finishes.
    When the graphics driver crashes while building the shaders, `prepare_shaders.py` runs that part again without the
    variants it was building ([building.md](building.md), step 6), and the setup finishes with a warning that names how
    many were left out.
@@ -155,7 +160,7 @@ runtime/              libtotkrt.a, libtotkrt-v2.a and the headers the generated
                       include
 bin/                  totk_xlate.exe, hactool.exe, SDL3.dll, libopus-0.dll, ninja.exe
 data/                 shader_list.tsv (MacNativeGames, MIT); pgo-<ver>.profdata + .json, the execution profile an
-                      update's code and the runtime libraries compile with (building.md, "Profile-guided build")
+                      update's code and the runtime libraries compile with (pgo.md)
 THIRD_PARTY_NOTICES.md
 totk.ini.example, VERSION  the default ini; the release string (the repository's VERSION plus the commit)
 ```

docs/mods.md

@@ -71,6 +71,10 @@ The rebuild needs the setup folder the game was installed from. When that folder
 A code patch over one of the port's own patches (the 60 FPS patch, the resolution patch and the other entries under
 **Built into the port**) turns that built-in patch off while the mod is on. The mod's row names it.
 
+A patch file may also change the game's data (constants and tables in its `.rodata` and `.data`). Those bytes are
+stored into the loaded game at start, before it runs. When two mods write the same bytes, the higher one wins,
+as with files.
+
 ## Cheats
 
 A cheat file is part of a mod: `mods\<mod>\cheats\<build id>.txt`, switched cheat by cheat on the Cheats tab. Cheats
@@ -87,6 +91,21 @@ The port keys a mod by its folder name. A renamed folder counts as a new mod:
 
 Rename mod folders before you set them up in the menu, not after.
 
+## HD texture packs
+
+Texture packs made for the emulators replace the game's textures with larger ones, so the heap the world's textures
+stream into must grow. Such a pack grows it in its own `System/HeapSize/Product.Nin_NX_NVN.HeapDef.byml.zs`, sized for
+the emulators' 8 GB memory option. The port reads that file at start: the texture heap and the game's memory pool grow
+by the pack's amount, and the game gets the port's own heap table with that growth (the log says
+`resolution: mod <name> ships a HeapDef with TextureHeap ...`). Nothing needs to be set. For a pack without such a file
+that still loops the panic Blood Moon, set `tex_heap_extra_mb` under `[mods]` in `totk.ini` (0 to 4096 MiB).
+
+Tested: **HDPacks.com Tears of the Kingdom 4K Texture Pack v1.0** (`4kTextures`, 10,407 textures, 6.5 GB) on game 1.4.3
+and 1.2.1, RTX 4080 SUPER: it loads into the game from a save and runs at the same frame rate, with about 3.3 GB more of
+textures in memory (4.5 GB of game memory pools against 1.2 GB without it). Other packs are not tested and come with no
+guarantee. Such a pack needs a lot of video memory; on a GPU with less than 8 GB, `astc_bc = 1` under `[performance]`
+keeps its textures at a quarter of the size.
+
 ## What does not work
 
 | On the Mods tab | Why | What to do |

docs/pgo.md

@@ -0,0 +1,120 @@
+# Profile-guided build
+
+A profile-guided build compiles the generated code and the runtime at -O2 with an execution profile: a record of
+which functions and branches ran while someone played. Hot paths are laid out straight, hot calls are inlined, and code
+that never ran is compiled for size. On 1.4.3 a trained fight ran 12% faster per frame and an untrained fight about
+6%; on 1.2.1 the trained fight ran 11% faster.
+
+Each game version has its own profile, `pgo-v121.profdata` and `pgo-v143.profdata`, trained on that version's code.
+
+## How a build uses a profile
+
+- `build_game.py --pgo <file>.profdata` compiles with that profile. Every compile depends on the file, so a new
+  profile rebuilds everything.
+- `tools/pgo.py merge` writes the profile to `build\pgo\pgo-<ver>.profdata`, next to its record `pgo-<ver>.json`. The
+  record holds the git tree of `recomp/` it was trained on and the digests of the runtime headers the generated C
+  includes (`cpu.h`, `simd.h`, `simd_fp.h`).
+- `make_release.py` ships every profile in `build\pgo` in the release's `data\` folder. It warns when a record names
+  another `recomp/` tree than the one released, or a header digest differs. The runtime libraries of the release
+  compile with the newest version's profile.
+- The installer builds a player's game with `data\pgo-<ver>.profdata` when the release has one for that version, and
+  at -O1 without it.
+
+A profile only fits the code it was trained on. clang matches each function by name and control-flow hash, and a
+function whose code changed loses its counts without a warning (`build_game.py` passes `-Wno-backend-plugin`, which
+hides the "hash mismatch" messages). The inline helpers of `cpu.h`, `simd.h` and `simd_fp.h` become part of every
+lifted function, so a change to them invalidates the profile as surely as a generator change: one change to the
+floating-point helpers left a profile with a fifth of its weight and 2% of its gain.
+
+## When to retrain
+
+- After a change to the lifter or the generator (`recomp/`) that changes the generated C. Patch table changes
+  (`recomp/patches-<ver>.txt`) touch only the patched functions and can wait for the next retrain.
+- After a change to `runtime/cpu.h`, `runtime/simd.h` or `runtime/simd_fp.h`.
+- Before a release whose `make_release.py` run warns about a profile.
+
+Train both versions from the same commit, from a clean `recomp/`.
+
+## Train
+
+### 1. Instrumented build
+
+MSYS2's clang has no profile runtime, so the instrumented build uses llvm-mingw (unpack it under `toolchain\`):
+
+```powershell
+$llvm = "toolchain\llvm-mingw-<date>\llvm-mingw-<date>-ucrt-x86_64"
+.venv\Scripts\python tools\build_game.py --version v143 --out build\v143-pgogen --toolchain $llvm --pgo gen
+```
+
+Regenerate `generated\<ver>` first if it is older than the commit you train on. If `totk.exe` stops at start with
+`0xc000007b`, copy `libiconv-2.dll` from `C:\msys64\ucrt64\bin` next to it.
+
+`totk.exe` writes `build\v143-pgogen\pgo\default_<id>.profraw` every minute and at exit. Each run of the same exe adds
+to that file.
+
+### 2. Shaders
+
+Copy your `totk.ini` into the build folder and prepare the shaders (`tools\prepare_shaders.py build\v143-pgogen
+--version v143`). Shader preparation runs `totk.exe` too, so move `pgo\*.profraw` out of the build folder afterwards:
+its counts describe shader building, not play.
+
+Keep the default settings, with the GPU command recorder on (`vk_recorder = 1`, the default).
+
+### 3. Training scenes
+
+The instrumented build runs slower than a normal one. Count each scene's time by the game clock, not by the wall
+clock. Spend 90 to 120 seconds on each of the first four scenes, where most frames are spent, and about 60 seconds on
+each of the others, which cover code the first four do not reach.
+
+| Place | What to do | Time |
+| --- | --- | --- |
+| Lookout Landing | Stand in the middle of the town and turn the camera slowly all the way round, then walk through the crowd. | 120 s |
+| Kakariko Village | The same: a slow camera turn, then walk between the houses. | 100 s |
+| A fight with a Battle Talus | Fight it with weapons and the bow. | 110 s |
+| The Depths, near a Lightroot | Turn the camera over the lit area, throw a Brightbloom Seed. | 90 s |
+| The Depths, away from any light | Walk and turn the camera in the dark. | 60 s |
+| The edge of a sky island | Run off the edge, skydive, open the paraglider. | 60 s |
+| Outdoors in the rain | Walk and look around while it rains. | 60 s |
+| A cave with Brightbloom plants | Walk in and fight the Moblins and Bokoblins inside. | 60 s |
+| Inside a shrine | Walk through the first rooms. | 60 s |
+| A Moblin camp | Fight several Moblins at once. | 60 s |
+| Next to Yunobo | Use his ability: roll him into something. | 60 s |
+| Hyrule Field, by the standing stone pillars | Run across the field, also backwards past the pillars. | 60 s |
+
+Keep two scenes out of the training, so that you can measure on scenes the profile has not seen: Hyrule Field in tall
+grass, and a fight with Moblins and Zonai constructs at once.
+
+With a save at a scene, `tools\run_game.py` plays it without you: `--save <backup> --script <input file> --seconds
+<game time>` ([agents/running.md](agents/running.md)). Without a save, play the scenes by hand.
+
+### 4. Free play
+
+Then play for 15 to 25 minutes and touch every kind of thing the game does, so that no common path is left without
+counts:
+
+- Run, sprint, climb, swim and glide; ride a horse; jump from a tower and skydive.
+- Fight with one-handed and two-handed weapons, a spear and the bow; shield parry, flurry rush, fused arrows.
+- Use every ability: Ultrahand (build a vehicle with a fan and wheels and drive it), Fuse, Ascend, Recall, Autobuild.
+- Call the sages and use their abilities: Tulin, Riju, Sidon, Yunobo, Mineru.
+- Open the inventory, the map and the camera; cook at a pot; talk to people in a town.
+- Travel quickly to a few places, so that areas load; enter and leave a shrine and the Depths.
+- Play in daytime and at night, in clear weather and in rain or snow.
+
+### 5. Merge
+
+```powershell
+.venv\Scripts\python tools\pgo.py merge build\v143-pgogen --version v143
+```
+
+writes `build\pgo\pgo-v143.profdata` and its record. Merge right after the training, from the checkout and commit the
+instrumented build was made in: the record reads the `recomp/` tree from git and the header digests from the build's
+Ninja deps log.
+
+Repeat steps 1 to 5 with `v121` for 1.2.1.
+
+## Measure
+
+Build the same code three ways in three folders with llvm-mingw, as the installer does: with the new profile, with
+the old one and with none. Prepare the shaders in each folder. Then run the held-out scenes and one trained scene
+in the order new, old, none, none, old, new, and compare the mean frame time over the same stretch of game time. One
+run of a fight can vary by up to 15% from the next, so a profile counts as better only when it wins both pairs.

docs/pr-review-kit/01-rules.md

@@ -0,0 +1,91 @@
+# Rules for taking a PR
+
+## 1. What to review first
+
+1. Fixes before features. A PR that fixes a bug, a glitch or a crash goes first; feature PRs wait.
+2. Order fixes by reach: how many players the bug hits and how badly. Windows and default settings first, rare
+   hardware and unusual setups later.
+3. Scale the depth of the checks to the reach. A crash many players hit and anything on the frame path get the full
+   checks. A rare case (a CPU without AVX, an unusual setup) gets the minimal fix, one targeted check and a short
+   note. A crash fix on rare hardware is still worth taking: cut the checks around it, not the fix.
+4. Before testing, find out where the bug reproduces: OS, GPU vendor and generation, game version, settings. If the
+   symptom cannot appear on the hardware at hand (an NVIDIA paging bug on an AMD machine), say so and do not run
+   tests that cannot decide it.
+5. Check a feature PR against the designs in `docs/milestones/`. A feature with a design (mods, cheats) goes in as
+   part of that design.
+6. Several PRs on one topic: compare them before taking either. Another PR may already do it right.
+
+## 2. How to read the diff
+
+- The author's claim is a hypothesis. Read the whole diff: races, wrong semantics, edge cases, leaks, lock order.
+- Check the behaviour against evidence: how yuzu, Ryujinx, LAS Recomp, SavageWindRecomp, nxnative and MacNativeGames
+  do the same thing in `refs/` (`tools/fetch_refs.sh` recreates it); the Vulkan spec; hardware docs
+  ([docs/maxwell.md](../../docs/maxwell.md)); upstream issues.
+- Weigh two things for every change: its cost on the hot path, and how it fails (device lost, hang, stale data, weak
+  or non-AVX CPUs, other GPU vendors, Wine/Proton).
+- Read the code a hypothesis depends on before running the game: the game's functions in Ghidra or
+  `generated/<ver>`, our path in `runtime/`. A run confirms what the code could not rule out. It is one run, not a
+  series of variants.
+- Order the work so the cheapest check that could refute the PR's idea runs first. Refuted: stop, write down why,
+  no build, no benches.
+
+## 3. What a PR needs
+
+- **Proof.** The bug reproduced before the fix, the same repro after, both results side by side: a screenshot, a log
+  line, a probe, a differential test. A fix without a reproduced failure is not proof. Where a repro is impossible
+  on the hardware at hand, say so; the fix is then taken only on strong static evidence in the code. Where a repro is
+  risky (device lost or a TDR can hang the machine), write the recipe and let the owner decide.
+- **Frame-path changes** get whole runs of the heavy scene with the switch off and on, at least two each
+  ([02-performance.md](02-performance.md)). Zero new slow frames is a hard bar.
+- **A switch.** A change on the hot path or in rendering sits behind an env or `totk.ini` switch, so a player can turn
+  it off and we can compare.
+- **Nothing changes for players outside the feature.** The DLSS PRs touched shared code; they went in once every
+  path that runs without DLSS was gated on DLSS being on.
+- **A fix for one setup keeps the default for everyone.** The PRs that raised the low memory block from 6 to 10 GiB for
+  all players were rejected; the right shape is 6 GiB plus the extra where it is needed.
+- **Parts we cannot run** (Linux-only code, an HDR display, RTX-only paths) are gated so they cannot touch anyone
+  else, or wait for a report from someone with that hardware. Note them as unrun.
+- **Project tests** from [03-correctness.md](03-correctness.md): lifter differential tests, probes, `--selftest`,
+  `tools/check_content.py --staged`.
+- **Text the player sees** goes into the catalog: `runtime/ui_text.h` ids and `runtime/ui_text.c` entries in every
+  language the menu has (English, Russian, Brazilian Portuguese, Simplified Chinese), and `installer/text/*.json` for
+  the installer. No hardcoded sentences, no second string table.
+- **A new setting** is described in `totk.ini.example` and [docs/settings.md](../../docs/settings.md).
+- **Licences.** MIT, BSD, ISC, zlib and Apache-2.0 code may be copied, with its header and a row in
+  `THIRD_PARTY_NOTICES.md`. yuzu, suyu and other GPL code is read for behaviour only; we write our own.
+
+## 4. Reasons to reject or rework
+
+- No test or repro for a change that is not trivial.
+- Slow: a state copy per draw, a global atomic counter on the hot path, synchronous file I/O or a GPU allocation on a
+  game or render thread, new slow frames.
+- Changes behaviour for everyone to serve one setup, or is on by default without a run in a scene where it acts.
+- "Fixes" a crash by turning a feature off for a vendor. A crash that goes away with a feature off points at the bug;
+  the feature gets fixed.
+- Breaks another feature (a preview that loses anti-aliasing).
+- Fake features: labelled FSR4 or frame generation, and not that.
+- Wrong measurements: percentiles over window means, counters over the wrong set.
+- Generated C edited by hand instead of the generator (`recomp/`).
+- A disallowed licence or pasted GPL code.
+
+## 5. Finishing a good PR
+
+A good PR that misses the project's scheme (hardcoded text, conflicts, layout, comment style) is brought to it while
+merging, not sent back to the author for another round. When only part of a PR is good, take that part and note what
+was left out and why.
+
+A bug found outside the PR under review is not fixed on the spot: search the open PRs on that topic, report the bug
+with them, and let the owner decide.
+
+## 6. Merging, branches, commits
+
+- Each PR or wave of PRs gets its own branch `task/<tag>` from a fresh `dev` and its own build dir. It merges into
+  `dev`.
+- Once the review is done, the symptom is gone on one paired shot and the probes pass, merge into `dev` and hand the
+  build to players: they test in game faster and wider than our routes. The wider checks (other routes, validation,
+  fps cost) follow as a separate diff. Speed never skips the review of the diff itself.
+- `main` only with the owner's explicit yes. Push only when told to.
+- A commit message is a plain English sentence, no `feat:`/`fix:` prefixes.
+- The version lives in the repo-root `VERSION` file; the owner decides when it moves.
+- After a merge, record which PRs went in, what was reworked, what was rejected and why. The reason for a rejection
+  goes into the PR as a plain English comment.

docs/pr-review-kit/02-performance.md

@@ -0,0 +1,97 @@
+# Performance: no lost frames on weak hardware
+
+## Who we tune for
+
+- The goal is many more fps on modest hardware, and Steam Deck later (Zen 2 4c/8t APU, shared LPDDR5, the exe runs
+  through Proton/Wine).
+- The developer's machine (Ryzen 9 7900X + RX 9070) is fast and its numbers do not carry over. A laptop with an
+  i7-9750H and a GTX 1050 Ti is a sample of a weak machine, not the target to fit constants to.
+- What weak hardware measured:
+  - GTX 1050 Ti (4 GiB, 214 MiB BAR, no ReBAR): the GPU read imported host pools over PCIe, 110-120 ms a frame
+    against 4.3 ms on the RX 9070. The VRAM mirror brought it to 10-11 ms; that machine is now CPU-bound at 25-45 fps,
+    replay 25-30 ms a frame.
+  - The same write-watch scan costs about 3x the CPU time there.
+  - NVIDIA and desktop AMD GPUs have no ASTC: textures decode in a compute shader. CPU decodes before that made
+    0.4-0.7 s load frames on the 1050 Ti.
+- Wine emulates `GetWriteWatch` with `mprotect` and a fault per first write. Anything relying on cheap OS memory
+  tricks (write-watch, `VirtualProtect` freezing, fault handlers) is checked against Wine's implementation.
+- The port uses about half of even a weak CPU, so the frame's critical path is the limit, not throughput. TotK keys
+  per-core data by `GetCurrentProcessorNumber` without locks, so the game's threads share three virtual cores on any
+  host. In a fight the main thread and five job workers keep 2.6 of the 3 busy: guest code speed sets the fps.
+- Perf work builds and measures 1.4.3. A 1.2.1 build is for a change about to merge with a large gain.
+
+## Gain ceiling first
+
+Before writing or taking a perf change, compute its ceiling: the share of the code it removes times the share of the
+frame that code holds on the critical path, on weak hardware. A ceiling of a few percent or less is not worth it.
+
+- Use the critical path (`TOTK_CRIT` + `tools/crit_path.py`, heavy frames only), not one thread's profile. On the weak
+  laptop the submit thread is 55% replay, yet the critical path of heavy frames is 67% guest code, 15%
+  `nvnSyncWait`, 13% core waits and wake latency.
+- Changes that can move the frame by 10% or more are usually design changes. Polishing a hot function from a flat
+  profile almost always gives zero.
+
+## A neutral safe change stays
+
+An optimization that measures neutral on the fast machine stays on by default when:
+
+- upstream (`refs/`, other ports) reports a gain for it, especially on weak CPUs or GPUs;
+- the static read leaves no hazard, and the probe or differential test passes;
+- whole heavy-scene runs off/on x2 show no fps loss and no new slow frames;
+- a switch turns it off.
+
+"No gain here" is not a reason to drop a safe change with upstream evidence. A regression or a hazard nobody ruled
+out is.
+
+## How to measure
+
+- **By game time, never by frame counts.** Run length, analysis windows, A/B blocks and input scripts go by the world
+  clock (`--seconds`, `TOTK_CRIT=w<lo>-<hi>`, `TOTK_PROFILE_FRAMES=w<lo>-<hi>`, `TOTK_SHOTS=w<lo>-<hi>:<step>`). The
+  same frame count at another fps is another scene.
+- **Check a scripted run reached the world.** Title menus on a slow boot eat button presses; confirm the run is in the
+  world before trusting its numbers.
+- **Whole runs off and on, two each at least.** An in-run A/B on a moving scene lies: the A and B blocks land on the
+  same game seconds every run, so a content difference repeats. One in-run A/B showed "-11% replay"; whole runs gave
+  5.84/5.73 against 5.76/5.73 ms, nothing. In-run A/B only on a static scene (standing still).
+- **Spread.** One script on near-identical builds gave 1% lows from 31.5 to 50 fps. Compare medians of three runs,
+  never one run against one run.
+- **A quiet machine.** `tools/run_game.py --quiet` takes the machine-wide lock; no other `totk.exe`, build or test runs
+  meanwhile. One game at a time.
+- **A warm shader cache, prepared properly.** Shader and pipeline caches are not copied between build dirs:
+  `totk.exe --prepare-shaders` or `--warm-shaders` in that dir. The AMD driver cache is keyed to the exe path and
+  limited in size: preparing another build dir can evict this one's libraries (40-220 ms per new effect in play).
+- **A heavy scene.** A fight with a large enemy and a crowd, fps limit at the maximum (`TOTK_FPS_LIMIT=360`). Compare
+  one shot with the scene's reference before trusting numbers.
+- **No base runs by habit.** Run the candidate alone and judge it against the targets: max fps, zero slow frames
+  (`perf: slow frame`), no black or flat frames, no new validation errors. Glitches are found by looking at shots and
+  frame dumps, not by diffing metrics. Run a base build only when a specific question needs a side-by-side.
+
+| What | How |
+|---|---|
+| fps, worst frame, render-thread buckets, GPU busy, VRAM | `--env TOTK_TRACE=perf,sched`, then `python tools/perf_summary.py <log> <lo> <hi>` |
+| the frame's critical path | `TOTK_CRIT=w<lo>-<hi>` + `TOTK_CRIT_OUT=<path>`, then `python tools/crit_path.py <file>` |
+| sampling profiler | `TOTK_PROFILE=<path>` + `TOTK_PROFILE_FRAMES=w<lo>-<hi>`, then `tools/prof_report.py <profile> <totk.map>` |
+| A/B on a static scene | `TOTK_AB=<seconds>`, then `tools/ab_summary.py --skip <seconds>` |
+| VRAM pressure on a large card | `TOTK_VRAM_LIMIT_MIB=<MiB>` caps the budget the port sees |
+
+## What the hot path must not do (Windows host)
+
+- **Files.** The game threads (main, job workers) and the render thread never open, read, write or flush a file, and
+  never wait for a thread that does. Opening a 20-byte file in a 139k-file folder: 0.1 ms idle, 11-40 ms in play while
+  the translator, the pipeline cache and the save wrote to the same volume. A save's `FlushFileBuffers` and
+  `MOVEFILE_WRITE_THROUGH`: 93-135 ms. Saving the pipeline cache under its lock on the present thread froze the replay
+  every 20 s. File I/O goes to a background thread; results come back in memory.
+- **Waits.** Short waits spin on the counter; waits over 1 ms use the high-resolution timer. A waitable timer wakes
+  0.3-0.5 ms late; the SDK's 20 us sleeps took 460-515 us. A 1 ms timer thread does not help: it gave more slow frames.
+- **GPU allocations.** VMA's `vkAllocateMemory` of a 256 MiB block under its lock: 8-23 ms. A new block on the render
+  thread is a hitch.
+- **128-bit division.** `(unsigned __int128)t * num / den` compiles to the library call `__udivti3`; counter
+  conversions that way took a tenth of the game threads' time in a fight. Use the precomputed scale (`platform.h`
+  `PlatScale`).
+- **Per-draw copies and counters.** A pipeline-state copy per draw, a global atomic counter, a scan of every page each
+  frame: on a weak CPU each is frames.
+- **Memory kept per frame.** Anything allocated per texture initialization or per frame must be freed. One copy kept
+  per texture init grew host memory by ~44 MiB a game minute until a commit failure after hours (#670). Check private
+  memory over a few idle minutes for a PR that stores per-object state.
+- **Scheduling.** Any timing or scheduling change is checked for slow frames. A preempt check at every host call made
+  one bug rarer but added 50-75 ms CPU hitches and was reverted.

docs/pr-review-kit/03-correctness.md

@@ -0,0 +1,92 @@
+# Correctness: checks that keep behaviour intact
+
+## Lifter and generated C
+
+- Generated C is never edited by hand: change the generator in `recomp/`.
+- `recomp/layout.py` alone decides module bases.
+- Every change to the lifter or the CPU headers (`runtime/cpu.h`, `runtime/simd.h`) passes the differential tests
+  against Unicorn, all run with `.venv`:
+
+| Test | Checks |
+|---|---|
+| `tests\diff\lift.py` | instructions sampled from the modules; `--coverage` lists the words the lifter rejects |
+| `tests\diff\simd.py` | the whole FP/SIMD encoding space |
+| `tests\diff\atomic.py` | exclusive load/store and ARMv8.1 atomics |
+| `tests\diff\func.py` | whole functions as the generator emits them: loops, jump tables, calls |
+| `tests\flags_check.py` | the function-local flag helpers against the packed ones |
+| `tests\excl_mt.py` | the exclusive monitor under contention and ABA, on several threads |
+| `tests\jumptables_check.py` | `recomp/jumptables.py` against Unicorn: 0 disagreements on 1.2.1 and 1.4.3 |
+
+- A `--cpu v2` build (no AVX2) passes the same tests with `--cflags=-march=x86-64-v2`. Players without AVX exist.
+- A compile error in generated C on a line the generator could not have written is a bit flip on disk or in memory.
+  Regenerate and compare hashes; leave `recomp/` alone.
+
+## Runtime
+
+- From the build dir: `totk.exe --selftest`, `--probe-kernel`, `--probe-vkexec`, `--probe-nvn-tex`, `--probe-shader`
+  pass.
+- Settings: `totk.exe --probe-cfg` and `python tests\settings_doc_check.py`.
+- Button prompts: `.venv-py314\Scripts\python tests\romfs_prompts_test.py`. Build dirs carry only `romfs-patches`; to
+  test PC or Xbox prompts in game, generate the overlay first:
+  `.venv-py314\Scripts\python tools\make_romfs_patches.py --version <ver> --prompts pc --out build\<dir>\romfs-patches-pc`,
+  then restart the game.
+- Installer: `tests\installer_gui_test.py` (event model, form, catalogs, cancel, display scale).
+- Shaders: `tools\shader_census.py --version <ver> --translate`, `tools\xlate_smoke.py`. A translator patch
+  (`tools/xlate/patches/`) needs `build/xlate` rebuilt and changes shader hashes: anything keyed to a shader hash
+  breaks with it.
+
+## GPU portability
+
+- New Vulkan code queries the feature, extension, format or limit it uses and keeps a working path or a logged
+  fallback without it (provoking vertex and subgroup size pinning in `vk_pipeline.c` are examples).
+- Never hardcode what one GPU reports: alignments, subgroup size (RDNA4 may run wave64, old AMD Vega is wave64 only),
+  depth formats, `maxPushDescriptors`.
+- One queue: every submit and present takes `vk_queue_lock`. Presenting from its own thread on a second queue gave
+  device lost on GTX 10-series in another port.
+- RDNA1/2 bugs do not reproduce on RDNA4 (another driver branch); NVIDIA bugs do not reproduce on AMD. A Ryzen 7000
+  iGPU is RDNA2 and runs the RDNA1/2 driver branch: a cheap testbed for those reports.
+- Validation: a `--validation` run in the world shows no new messages.
+
+## Maxwell / NVN behaviour
+
+The port must do what the Switch GPU does, not what Vulkan does by default. The full list with sources:
+[docs/maxwell.md](../../docs/maxwell.md), section 7. The traps a PR breaks most easily:
+
+- Provoking vertex: Vulkan takes the first vertex, NVN the last.
+- Unwritten vertex outputs read (0, 0, 0, 1).
+- A colour target outside the fragment shader's output map keeps its texel (write mask 0). AMD writes garbage there.
+- Feedback loops: the G-buffer pass reads its own targets. It needs the feedback layout and a by-region barrier.
+- Depth bias argument order, depth mode, depth clamp: checked against deko3d.
+- Report timestamps: the engine's GPU time comes from them, and dynamic resolution and game speed follow it.
+
+## Timers and the real-time step
+
+- With `realtime` set, the game steps by a measured ratio, not whole 30 Hz units. A countdown remainder in
+  (0, FLT_EPSILON] is never stepped again, and a consumer testing `> 0` sees it active forever. That was "weapons
+  cannot be changed after a Fuse until a reload" (#18).
+- Symptoms of this class: stuck until a reload or teleport, locked after an action, works sometimes.
+- A PR touching the frame step, the pacer or the fps limit is checked for this class.
+
+## Threads and scheduling
+
+- ARMv8 `stlr` then `ldar` is ordered (RCsc); C11 release/acquire lets x86 reorder that store and load. Changes to
+  the lifter's barriers or atomics pass `tests\diff\atomic.py` and `tests\excl_mt.py`.
+- A scheduler change is judged from one `TOTK_TRACE=sched` run by how long a ready, better thread waits for a core it
+  may use, plus the slow-frame count. Not by statistics over many runs.
+- A crash that looks like a race: rule out the save first. One reported crash was the save itself; it crashed the
+  same way on a real Switch.
+
+## Files and caches
+
+- A crash leaves zero-filled files. Caches are validated on load, not trusted because they exist.
+- The pipeline cache is dropped when vendor, device or `pipelineCacheUUID` change.
+- Unicode paths: W APIs with UTF-8 to UTF-16 conversion.
+- A failure that happens once and does not come back: rerun before debugging it as a code bug. Broken caches and bit
+  flips happen.
+
+## Game content boundary
+
+- The repository is public. No game files, and nothing extracted or generated outside `build/` and `generated/`.
+- Keys never appear in logs, commands, files, issues or comments.
+- Every file the generator emits carries the `GENERATED_MARKER` defined in `tools/check_content.py`.
+- `python tools/check_content.py --staged` before every commit. CI runs the same check.

docs/pr-review-kit/04-knowledge.md

@@ -0,0 +1,97 @@
+# Knowledge base
+
+Before investigating a symptom, or trusting a PR that "fixes" it, look for it here. Search by what the player sees
+(black, magenta, flicker, slow motion, missing, stale, streaks) and by the object (sky, grass, shadow, fog, HUD).
+
+## Invariants of recent changes
+
+A PR that touches these areas must keep them.
+
+- **Queue order.** When anything is submitted, every earlier part is already on the queue. Present, uploads, readbacks
+  from guest threads and the `vkDeviceWaitIdle` sites rely on it. The recorder thread now begins, ends and submits
+  the parts it records; the submitting thread still waits for that submission, so the order holds.
+  `TOTK_REC_END=0` restores the old split.
+- **GPU-written memory is never frozen.** Storage-buffer stores, copies and fills land in imported guest memory
+  without a fault or a write-watch bit, so a region only the GPU writes looks idle. Such regions are never frozen
+  read-only, and a frozen region is thawed before a GPU write. LAS Recomp never changes the protection of imported
+  memory because of NVIDIA. A suspected cause of NVIDIA "write invalid" device faults (#788, #802), not proven.
+- **VRAM pressure.** Under pressure, released textures whose storage the game has given to another texture go first:
+  30 s after release and once no finalized slot has bound them for 600 submits. Clean mirrored entries wait 5 s.
+  Both stop once use is back under the line. A PR touching texture lifetime is tested with `TOTK_VRAM_LIMIT_MIB`.
+- **Texture pins are freed.** Each pin counts the recording copies naming it and is freed 1200 submits after its last
+  use unless the replay holds it as a render target. Before that, pins leaked ~44 MiB a game minute (#670).
+
+## Fixed bug classes
+
+If a PR touches this code, confirm the fix still holds.
+
+| Symptom | Cause | Where the fix is |
+|---|---|---|
+| lime silhouettes in grass; rectangles in the sky | a target outside the output map must keep its texel; zero-filling it wiped the half-res depth | `vk_pipeline.c` `target_wmask` |
+| stale grass maps in the far terrain for a second after a hit | write-back of a texture the CPU had rewritten | `vk_texture.c` `write_back` |
+| cut grass stays standing; fire does not spread (#132) | CPU reads of GPU-rendered textures without an invalidate | `vk_texture.c` `vk_texture_write_back_small`, `nvn_ext.c` `nvn_pool_invalidate` |
+| cloud and fog volumes empty for the session | a one-shot draw skipped instead of waiting for its pipeline | `vk_exec_draw.c` `prepare_draw` |
+| tile rectangles | storage buffers bound by Ryujinx first-use order, not the c0 slot | `vk_exec_draw.c` `ssbo_slot` |
+| black dialog background and loading map | 3D targets rejected, the grading LUT never written | `vk_exec.c` `resolve_target` |
+| streaks on cliffs | primitive restart ignored | `vk_exec_draw.c` `restart_for` |
+| streaky map screen | the newest texture picked at shared storage | `vk_texture.c` `vk_texture_by_id` |
+| dark 16x16 blocks on the ground, sage spirit not drawn | G-buffer feedback loop | `vk_exec_draw.c`, `TOTK_FEEDBACK` |
+| magenta cave walls, brown sky | CPU-filled textures without a flush | write-watched mirrors, `vk_memory.c`, `vk_texture.c` |
+| black 3D scene | deferred texture binds | `nvn_api_cmd.c` `Bind*Deferred` |
+| HUD prompt boxes, black stamina wheel | `nvnTextureCompare` ignored the channel mapping | `nvn_api_obj.c` |
+| crash on world load | `GetCommandMemoryUsed` did not follow the driver's sizes; the game sizes model heaps from it | `nvn_api_cmd.c` |
+| 20 fps | a yield back-off starting before 2 ms | `kernel.c` SVC 0x0B: no back-off |
+| weapons locked after a Fuse (#18) | timer remainder in (0, FLT_EPSILON] with the real-time step | patch under the `realtime` setting |
+| magenta/blue garbage or device lost on RDNA1/2 (#5) | writes into 3D BC block views | in `dev` |
+| black window after a driver update | a pipeline cache from another driver | `vk_pipeline.c` drops the cache by UUID |
+
+## Known open risks
+
+A PR touching these needs stronger proof.
+
+- Buffer textures are not replayed and bind a stand-in: light and decal lists may read 0.
+- New GPU-only colour images start undefined; feedback effects may show garbage.
+- Clears with a partial channel or stencil mask are skipped (`W_CLEAR_MASK`, `W_STENCIL_MASK`).
+- 8-bit indices are widened on the CPU at record time; a copy in the same submission gives stale indices.
+- `UpdateUniformBuffer` is a copy between full barriers and ends the render pass (fps outdoors).
+- Report timestamps carry host time at fence completion: the engine's GPU time is wrong.
+- The game's SVC-free spin loops burn a host core (843 in `main`).
+- Images aliased onto one GPU-only pool allocation do not see each other's data on AMD.
+
+## Past PR decisions
+
+| PR | Decision | Why |
+|---|---|---|
+| #690 shrine snapshots | rejected | a pass restart per copy, a `VkPipeState` copy per draw, broken `bind_sig` skip; fps loss |
+| #674 | rejected | a global STXR counter on the hot path |
+| #660 | rejected | the preview loses anti-aliasing |
+| #713, #576 | rejected | low memory block 10 GiB for everyone; should be 6 GiB plus the extra where needed |
+| #573 | rejected | dynamic resolution off by default for everyone, the 1.4.3 site never run |
+| #568, #569 | rejected | fake FSR4 and frame generation |
+| #487 | rejected | wrong metrics: percentiles over window means, counters over the wrong set |
+| #451, #380 | rejected | not painless to take |
+| #566, #594, #534 DLSS | taken with changes | #594's shared paths gated on DLSS being on; nothing changes without DLSS |
+| #489 | partly taken | the runtime part checked by a frame dump; the installer's DLL download was the owner's call, its pinned hash re-checked |
+| #519, #582 amiibo | rewritten by us | took the data-format fixes; left out the features and the Linux event rework |
+| #483 vs #585 | #585's mapping kept | #483's stick glyph mapping was rewritten unasked while #585 already had it right |
+| #528 rain | taken after a check | code review fine, merged only after a run on a rain save |
+| #767 | taken with our fix | the default was skipped with `prompts = xbox|pc` when those folders did not exist |
+| #625 frame queue | taken, off by default | a new frame queue, one field run with it on |
+| #743 UB overlay | waiting | +25% in a Depths fight on an RTX 4080; needs review, fight runs x2 and shots on AMD |
+
+## Our mistakes, not to repeat
+
+- **"-11%" was an A/B artifact** of a moving scene. Whole runs off/on showed zero.
+- **Micro-optimizations from a flat profile** (1-3% of samples each) gave 26.2 against 26.4 ms on the weak laptop:
+  nothing. Nobody had computed a ceiling.
+- **A scheduler patch merged on run statistics** added 50-75 ms hitches. Slow frames are a hard bar.
+- **Ten game runs on lock theories** where one Ghidra read of the game's code narrowed it at once.
+- **45 minutes of builds and benches** on an idea its own first thread census had refuted: the game pins 53 of 56
+  threads itself. The cheapest refuting check goes first; when it refutes, stop and write the idea down.
+- **Tests of a bug that could not appear on the hardware at hand** (NVIDIA paging on an AMD machine): 245k tokens and
+  37 minutes for nothing. Ask first where the bug reproduces.
+- **Turning the recorder off for a vendor** instead of finding its bug. Stopped: the feature gets fixed, not silenced.
+- **Bisecting code when every build crashed the same way at once.** The cause was a bad page of a game file in the OS
+  file cache. When all builds fail identically, suspect inputs and caches first.
+- **A duplicate issue closed with a bare "Same as #N"** lost the logs and links. Move everything useful to the issue
+  that stays open first: hardware, driver, version, when it happens, workarounds, links from comments.

docs/pr-review-kit/05-checklist.md

@@ -0,0 +1,49 @@
+# Per-PR checklist
+
+## Before reading
+
+- [ ] Fix or feature? Fixes first, ordered by how many players they reach.
+- [ ] Where does the bug reproduce: OS, GPU vendor and generation, game version, settings? Can the hardware at hand
+      show it?
+- [ ] Is the symptom in [04-knowledge.md](04-knowledge.md) or `docs/maxwell.md`?
+- [ ] Another open PR on the same topic? Compare them.
+- [ ] Feature: is there a design in `docs/milestones/`?
+
+## Reading the diff
+
+- [ ] The whole diff read, not the description: races, edge cases, lock order, leaks.
+- [ ] Behaviour checked against `refs/` (Ryujinx, yuzu, LAS, SavageWind, nxnative, MacNativeGames), the Vulkan spec,
+      `docs/maxwell.md`.
+- [ ] Hot-path cost: no file I/O, GPU allocations, waits, per-draw copies or global atomics on game or render threads.
+- [ ] No memory kept per frame or per object without a free.
+- [ ] Failure modes: device lost, hang, stale data, CPUs without AVX, NVIDIA/Intel/old AMD, Wine/Proton.
+- [ ] New Vulkan code queries the feature and has a fallback; nothing hardcoded for one GPU.
+- [ ] Invariants kept: queue order, GPU-written memory never frozen, VRAM pressure path, pins freed.
+- [ ] Players without the feature see no change. The default for everyone stays for one setup's sake.
+- [ ] New behaviour behind a switch. A new setting described in `totk.ini.example` and `docs/settings.md`.
+- [ ] Player-visible text in the catalogs, in every menu language (`runtime/ui_text.c`, `installer/text/*.json`).
+- [ ] Copied code: allowed licence, header kept, row in `THIRD_PARTY_NOTICES.md`. No GPL code.
+- [ ] Generated C not edited by hand.
+- [ ] Parts we cannot run (Linux, HDR, RTX-only) gated away from everyone else and noted as unrun.
+
+## Proof
+
+- [ ] Repro before the fix and the same repro after, side by side.
+- [ ] Lifter or CPU headers: differential tests against Unicorn; `jumptables_check.py` 0 disagreements.
+- [ ] Runtime: `--selftest` and the four probes pass.
+- [ ] Settings: `--probe-cfg` and `tests\settings_doc_check.py`. Prompts: `romfs_prompts_test.py`, overlay generated
+      for a game check.
+- [ ] Frame path: whole heavy-scene runs off/on, two each at least, by game time, quiet machine, warm cache prepared
+      in that build dir. No fps loss, zero new slow frames.
+- [ ] Rendering: shots or a frame dump in a scene where the change acts; no black or flat frames; no new validation
+      errors.
+- [ ] Perf PR: gain ceiling computed for weak hardware.
+
+## Decision
+
+- [ ] Taken, partly taken or rejected, with the reason written down and posted to the PR.
+- [ ] What missed the project's scheme brought to it while merging.
+- [ ] Bugs found outside the PR reported, not fixed on the spot.
+- [ ] Merged into `dev` from `task/<tag>`. `main` only with the owner's yes.
+- [ ] `python tools/check_content.py --staged` before the commit.
+- [ ] Commit message: a plain English sentence, no `feat:`/`fix:`.

docs/pr-review-kit/README.md

@@ -0,0 +1,32 @@
+# PR review kit
+
+Rules and background for a tester or reviewer who takes community PRs into `dev`. One goal: no PR breaks the
+game's behaviour or costs frames on weak hardware.
+
+## The short version
+
+1. A PR description and "works for me" prove nothing. Read the diff yourself and check it against `refs/`, the
+   Vulkan spec and upstream issues.
+2. Almost every change needs a test: a repro before and after, a probe, a differential test, or whole runs with the
+   switch off and on.
+3. The developer's machine is fast; players run weak machines. "Neutral here" says nothing about them, in either
+   direction.
+4. A player without the problem sees no change: the new path sits behind a switch and the old path stays as it was.
+5. Nothing goes into `main` without the project owner's explicit yes. Accepted work lives in `dev` first.
+
+## Files
+
+| File | Covers |
+|---|---|
+| [01-rules.md](01-rules.md) | triage order, what a PR needs, reasons to reject, merging and commits |
+| [02-performance.md](02-performance.md) | target hardware, gain ceiling, how to measure, what the hot path must not do |
+| [03-correctness.md](03-correctness.md) | lifter, tests and probes, GPU portability, Maxwell behaviour, timers, threads, game content |
+| [04-knowledge.md](04-knowledge.md) | invariants of recent changes, fixed bug classes, open risks, past PR decisions, our mistakes |
+| [05-checklist.md](05-checklist.md) | the per-PR checklist |
+
+The public docs this kit builds on: [docs/architecture.md](../../docs/architecture.md),
+[docs/building.md](../../docs/building.md) ("Checks"), [docs/maxwell.md](../../docs/maxwell.md),
+[docs/settings.md](../../docs/settings.md), [docs/tester-guide.md](../../docs/tester-guide.md), `AGENTS.md`.
+Working with a coding agent: give it this folder and `AGENTS.md` before the first review.
+
+State as of 2026-11-06. The code and `docs/` win over this text where they differ.

docs/settings.md

@@ -64,6 +64,7 @@ Many rows below are developer tools that only act together with another setting
 
 | Setting | What it does | Effect of changing it | Safety |
 | --- | --- | --- | --- |
+| `vram_limit_mib` | Pretends the device-local memory budget is at most this many MiB, for the paths that react to the budget (`tex_vram_pressure`, mirror refusal); the driver knows nothing of it. | Lets a large card drive those paths; a low value drops released textures and refuses mirrors that would have fit. | Caution |
 | `background_window` | `1` keeps the window behind the user's windows, never fullscreen at start, and F11 keeps its place; SDL windows ignore it. | The window does not come to the front or flash, so it is easy to miss; meant for unattended runs. | Caution |
 | `gpu` | `vulkan` or `null`: the graphics back end; `null` has no GPU, completes every submit at once and returns zeros for texture reads. | `null` shows no picture; any other value stops the start with an error. | Caution |
 | `input` | `none` reports no input from the keyboard, mouse or controllers. | The game gets no controller input and the menu cursor stops following the mouse; `input_script` still applies. | Caution |
@@ -73,8 +74,10 @@ Many rows below are developer tools that only act together with another setting
 | `menu_test_set` | `<frame>:<key>=<value>[,...][;...]` changes settings live at that frame, with or without the menu open; `SAVE=1` also writes `totk.ini`. | Settings change during play without input, and `SAVE=1` rewrites the file. | Caution |
 | `probe_movie` | Text that picks the movie for `totk.exe --probe-movie`: the first file whose path contains it. | Only used by the movie decoder probe; ignored in normal play. | Safe |
 | `probe_movie_shot` | Path of a `.ppm` file that the movie probe writes the middle frame to. | Only used by the movie decoder probe. | Safe |
+| `probe_movie_bench` | `1` makes the movie probe also time the resampler that fits a movie to a game resolution of another shape, and print the time per frame. | Only used by the movie decoder probe. | Safe |
 | `swkbd_answer` | `<text>`, `<cancel>` or `<initial>`: answers the on-screen keyboard (for example a horse name) without opening a window. | The game gets that answer every time it asks for text. | Caution |
 | `swkbd_test` | `ok` or `cancel` opens the keyboard window and presses OK or Cancel on the first timer tick. | The keyboard window closes by itself. | Caution |
+| `test_oom` | `<site>:<n>[,...]` fails the first n allocations at a site: `commit` (guest memory), `staging` (upload buffers), `image` (texture images), `chunk` (draw data buffers). | Each forced failure takes the out-of-memory path: a memory release, a wait of 10 ms per retry and log lines; a large n stalls up to 30 s and then ends the game. | Caution |
 | `test_pad` | `<lx>,<ly>,<rx>,<ry>` attaches a virtual gamepad whose sticks rest at these raw SDL axis values (-32768 to 32767). | The game sees a controller with that drift or deflection; a malformed value is logged and ignored. | Caution |
 | `test_prepare_shaders` | `1` translates every recorded shader on all usable CPUs during start. | Start takes longer and fills the shader cache folder; the CPU is fully used while it runs. | Caution |
 
@@ -119,8 +122,11 @@ Many rows below are developer tools that only act together with another setting
 | Setting | What it does | Effect of changing it | Safety |
 | --- | --- | --- | --- |
 | `block_views_3d` (default `0`) | `1` gives 3D block-compressed images block views too. | On some AMD cards a GPU write past the end of the image's memory loses the Vulkan device, and the game stops. | Risky |
+| `deswizzle` (default `auto`) | Where block-linear textures are deswizzled: `auto` or `gpu` uses a compute pass where the texture's storage is an imported pool and the CPU elsewhere, `cpu` always uses the CPU, `check` uses the GPU and also deswizzles every upload on the CPU and compares the two. | `cpu` brings back the long frames on entering new areas (the replay thread deswizzles every texture); `check` waits for the GPU at every upload and is for testing only. The log names the mode in use. | Caution |
+| `dyn_delta` (default `1`) | Records only the pipeline dynamic states (cull mode, depth and stencil tests, vertex input, blend enables, equations and write masks, ...) whose values changed since the last draw in the command buffer, also across pipeline changes; `0` records the full set at every change of pipeline state, as before; `ab` alternates with the `ab` blocks. | Fewer commands per pipeline change on the render and recorder threads; a driver that loses dynamic state across pipeline binds would draw with stale state. `TOTK_TRACE=perf` prints the calls and commands per submission. | Caution |
 | `feedback` | `0` samples a target that an earlier draw of the same pass wrote with no barrier; `restart` ends and restarts the pass behind a full barrier; otherwise the feedback-loop layout is used when the GPU supports it. | `0` brings back the old behaviour that the barrier fixed, so the picture can break in passes that read a target they draw into; `restart` costs extra passes. | Risky |
 | `flush_span` (default `0`) | `1` copies one span from the lowest to the highest flushed address of a pool, as an older version did. | Copies far more data to the GPU in crowded towns: an estimated 30-50 ms of GPU time per large copy over PCIe 3.0 x8. | Caution |
+| `shadow_snap` (default `1`) | Shadow storage (Linux, Wine): `1` syncs the pools from per-submission write-watch snapshots, each 2 MiB region queried at most once per submission and shared with the texture checks; `0` queries each pool at every part of a submission, as before. | `1` cut the replay thread's write tracking from about 4.4 to 1.3 ms a frame on a Steam Deck; `0` is for comparison runs. Under Wine/Proton the default is `0` until it has been measured there; `1` turns it on. No effect where shadow storage is off (Windows by default). | Caution |
 | `freeze_idle` (default `60`) | Number of submissions a 2 MiB region stays unwritten before it is made read-only; `0` turns the freeze off. | A larger value keeps more regions writable and costs more write tracking; a write to a frozen region costs about 60 microseconds. | Caution |
 | `gpl_draw_build` (default `1`) | `0` leaves the missing shader libraries of a waiting draw to its background job instead of building them in parallel at once. | Draws whose pipeline is not ready are skipped meanwhile, until the job finishes. | Caution |
 | `lib_prebuild_keep` (default `0`) | Number of recorded graphics shader library variants that stay in memory after the boot prebuild. | Each kept library costs 140-460 KB of memory; the driver cache already makes a rebuild cheap. | Caution |
@@ -131,9 +137,13 @@ Many rows below are developer tools that only act together with another setting
 | `pcss_k` (default `0.5`) | Depth-to-penumbra factor of soft shadows at 100 percent, for tuning. | Changes how wide soft shadows blur; has no effect while `pcss` is `0`. | Caution |
 | `pipe_sync` | Any value other than `0` or empty stops graphics pipelines from being built in the background. | Pipelines are built on the render path, which causes stutter when new effects appear. | Caution |
 | `push_stage` | `fs` pushes the descriptor set from the fragment stage instead of the vertex stage. | The log names the stage in use; the code does not say what changes in the picture. | Unknown |
-| `rec_end` (default `1`) | `0` lets the submitting thread end a recorded part again, as an older version did. | On some AMD cards the GPU hangs or the scene turns green and then white when a submission is split. | Risky |
-| `tex_vram_pressure` (default `90`) | Percent of the device-local memory budget above which unused released textures are dropped, 0 to 100; `0` is off. | A low value drops textures sooner and re-uploads them; `0` keeps them and lets a 4 GiB card spill into system memory. Also a slider in the menu. | Caution |
+| `rec_async` (default `1`) | `0` makes the submitting thread wait until the recorder thread has recorded and submitted each part; `ab` alternates with the `ab` blocks. | With `0` the submitting thread waits for every part (about 0.3 ms a frame in a town on a Ryzen 7900X). | Caution |
+| `rec_end` (default `1`) | `0` lets the submitting thread begin, end and submit a recorded part again, as older versions did. | On some AMD cards the GPU hangs or the scene turns green and then white when a submission is split. | Risky |
+| `rec_spin_us` (default `60`) | Microseconds the recorder thread polls for the next packets before it sleeps; `0` polls 4096 pauses instead; `ab` polls only in the B blocks of `ab`. With fewer than 6 logical processors it always polls 4096 pauses. | A longer poll saves an OS wake-up per packet and takes more CPU time; a shorter one sleeps sooner. | Caution |
+| `tex_vram_pressure` (default `90`) | Percent of the device-local memory budget above which released textures are dropped, 0 to 100; `0` is off. First those the game has overwritten with a later texture and left alone for 30 s, then those that can be uploaded again. | A low value drops textures sooner and re-uploads them; `0` keeps them and lets the card spill into system memory. Also a slider in the menu. | Caution |
 | `txd_transpose` (default `1`) | `0` turns off the swap of gradient components for gradient texture samples. | Terrain textures turned across the screen show crawling patches again. | Caution |
+| `ub_overlay` (default `1`) | `0` copies each uniform update inside a pass into its buffer at once, ending the pass with a full barrier; `ab` alternates with the `ab` blocks. | With `0` scenes with many updated blocks (the terrain, the sky islands) take longer on the GPU; `TOTK_TRACE=perf` prints how many updates stayed in their pass. | Caution |
+| `ub_overlay_out` (default `1`) | `0` copies each uniform update between passes into its buffer at once, behind a full barrier of its own; `ab` alternates with the `ab` blocks. Needs `ub_overlay`. | With `0` a run of updates before a pass costs a full GPU barrier each; `TOTK_TRACE=perf` prints how many updates between passes were overlaid. | Caution |
 | `ubo_snapshot` (default `1`) | `0` binds live memory for uniform blocks; `ab` alternates with the `ab` blocks. | Without snapshots late draws read another object's matrix, so objects can be drawn in the wrong place. | Risky |
 | `vk_no_float64` (default `0`) | `1` clears the 64-bit float feature, so the translator lowers FP64 to FP32. | Shaders change, so the shader cache entries for it are made again; precision of FP64 shaders drops. | Caution |
 | `vk_ssbo_align` (default unset) | Power of two that storage buffer offsets are treated as needing, to test the unaligned path. | The translator and the draw code round storage offsets down to it; shaders are translated again. | Caution |
@@ -175,11 +185,21 @@ Many rows below are developer tools that only act together with another setting
 | --- | --- | --- | --- |
 | `app_pool` | Size in bytes of the guest application memory pool; accepts decimal or `0x` hex. | The default depends on the resolution settings; `0` stops the start with a message, and too small a pool leaves the game without memory. | Risky |
 | `audio_off` | Any value mixes audio without opening an output device. | The game runs in silence and reports two channels; same as `audio = none`. | Caution |
+| `kspin_us` (default `20`) | Microseconds a guest wait or a wait for a virtual core polls before it sleeps; `0` turns the poll off; `ab` polls only in the B blocks of `ab`. Off with fewer than 6 logical processors. | A wake within the poll costs no OS wake-up; a longer poll takes more CPU time from other threads. | Caution |
+| `mem_reserve_mib` (default `1536`) | MiB of system memory kept free for Windows, other programs and the game's own heap when the renderer sizes what it keeps in device memory, where device memory takes system memory (integrated GPUs, and discrete GPUs whose driver charges it to the process). | A larger value keeps fewer released textures and spare blocks when the system runs short; a smaller one lets the game use more before it holds back. | Caution |
 | `one_per_core` (default `0`) | `1` keeps the process on the first logical processor of each physical core; `ab` switches it with the `ab` blocks. | Busy threads stop sharing a core with a sibling thread, which can help or hurt on the CPU; it only applies when the CPU has hyperthreads. | Caution |
+| `pcore` (default `1`) | On a hybrid CPU (performance and efficiency cores), the game's main and render threads and the replay, recorder and present threads prefer the performance cores; `0` turns that off. | With `0` Windows may run those threads on efficiency cores; no effect on a CPU with one kind of core. The log says which cores were found. | Caution |
 | `reexec` | Counter of address-space retries that the runtime sets itself when it starts the process again. | The counter is cleared when the window is recreated; a value of 20 or more stops the start with a fatal error. | Risky |
+| `replay_chunks` (default `0`) | Most chunks a submission is cut into when `replay_workers` is set; `0` uses twice the number of recording threads. | Fewer chunks leave threads idle at the end of a submission, more add scan and submit work per chunk; no effect while `replay_workers` is `0`. | Caution |
+| `replay_chunk_draws` (default `128`) | Least draws a chunk of the chunked replay holds before it may end at a pass. | Small values make many short chunks, large ones leave big submissions on few threads; no effect while `replay_workers` is `0`. | Caution |
+| `replay_state_check` (default `0`) | `1` compares the replay state after the last chunk of a chunked submission with the state of the scan and logs differences. | Adds a comparison per submission and log lines on a mismatch; no effect while `replay_workers` is `0`. | Caution |
+| `replay_skew_us` (default `0`) | Debug: the even chunks of a chunked submission wait this many microseconds before they record, so the chunk after each one records first. | Shows a recording that depends on which thread gets to a chunk first in every submission; slows the replay; no effect with fewer than 2 `replay_workers`. | Caution |
 | `restart_wait` | Set by the runtime when it restarts itself; names the old process by id and creation time so the new process waits for it to exit. | Meant for the runtime's own use; the code does not describe a manual value. | Unknown |
 | `spin` (default `1`) | `0` makes a guest thread that polls memory keep its virtual core. | Polling threads can starve the thread whose item they wait for, so frames can stall. | Caution |
 | `spin_handoff_us` (default `50`) | Microseconds a polling wait lasts before it gives its virtual core to a waiting thread. | A shorter value hands cores over sooner; a longer one keeps them longer. | Caution |
+| `tex_block_mib` (default `64`) | Block size in MiB of the pool that holds the game's textures; images of a quarter block or more get memory of their own. `0` puts textures in the default pools with the buffers. | Larger blocks hold more memory that no texture uses; smaller ones mean more allocations from the driver. | Caution |
+| `tex_released_mib` (default `256`) | MiB of textures the game released that are kept for reuse; older ones beyond it are dropped. `0` keeps up to 2048 of them, whatever their size. | A larger value keeps more device memory and process commit; a smaller one makes the game create more textures again. | Caution |
+| `vma_block_mib` (default `64`) | Block size in MiB that the Vulkan memory allocator takes from the driver for buffers and other images on heaps over 1 GiB; larger allocations than half a block get memory of their own. `0` uses the allocator's 256. | Larger blocks hold more memory that nothing uses; smaller ones mean more allocations from the driver. | Caution |
 
 ## Generated-code patches
 
@@ -188,6 +208,7 @@ These names are code patch sets built into the generated game code (`recomp/patc
 | Setting | What it does | Effect of changing it | Safety |
 | --- | --- | --- | --- |
 | `aspect_patch` (default `0`) | Patch set that feeds the camera the output's aspect ratio when it is not 16:9. | The resolution code switches this set on or off itself at start, so a value here is replaced; the code does not settle what a manual value does. | Unknown |
+| `fixed_res_patch` (default `0`) | Patch set that holds the game's dynamic resolution at 100 percent (the scene tier). | The resolution code switches this set from `dynamic_res` (on at `0`); a manual value is replaced. | Unknown |
 | `fov_patch` (default `0`) | Patch set that scales the camera's vertical view angle by the field-of-view setting. | The resolution code switches this set itself whenever the field of view is not 100 percent; a manual value is replaced. | Unknown |
 | `mouse_direct_patch` (default `1`) | Patch set of the hooks in the game's camera code behind `mouse_mode = direct` on game 1.4.3. | `0` stops `mouse_mode = direct` from taking effect; other mouse modes are unaffected. | Safe |
 | `patch_output` (default `0`) | Patch set that makes the game use the configured output size for its swapchain and display configuration. | The resolution code switches this set whenever the output differs from the base size; a manual value is replaced. | Unknown |

installer/steps.py

@@ -8,7 +8,8 @@ with the bundled clang (hostcheck.c).
 
 Install folder: exefs/, romfs/ (romfs.bin, index.json), profile/, shared-fonts/ (Chinese and Korean text), totk.exe
 and its DLLs, nvngx_dlss.dll (with --dlss), romfs-index.bin, romfs-patches/, romfs-patches-xbox/,
-romfs-patches-xbox-label/, romfs-patches-pc/, totk_xlate.exe, the
+romfs-patches-xbox-label/, romfs-patches-pc/, romfs-patches-<style>-<w>x<h>/ (each of the three for each shaped size),
+totk_xlate.exe, the
 shader state of totk.exe --prepare-shaders, totk.ini, totk-install.json.
 Temp folder (<temp>/<dest name>-<hash>): check/ (ExeFS of both inputs, the host check), unpacked/ (the NSP of an NSZ
 or XCZ input until the extraction is done), gen/ (C), build/ (objects), logs/. The cleanup keeps gen/, build/ and
@@ -16,9 +17,10 @@ logs/: the next run lifts into the same gen/, where recomp/gen.py rewrites only
 in build/ recompiles only those. A change of the mods' code cheats then costs the lift (about a minute) and the chunks
 it touches, not a full compile.
 
-Install folder build-cache/: cheats.status (tools/cheats.py), exefs.status (tools/exefs_patches.py) and rebuild.txt, the command that starts this setup in
-its rebuild mode on the install folder (installer/gui.py --rebuild), one argument per line; totk.exe runs it at start
-when an enabled mod has code cheats the build lacks (runtime/cheats.c).
+Install folder build-cache/: cheats.status (tools/cheats.py), exefs.status and exefs.data (tools/exefs_patches.py) and
+rebuild.txt, the command that starts this setup in its rebuild mode on the install folder (installer/gui.py --rebuild),
+one argument per line; totk.exe runs it at start when an enabled mod has code cheats the build lacks
+(runtime/cheats.c).
 """
 import hashlib
 import http.client
@@ -167,7 +169,8 @@ def run_tool(ctx, step, script, args, parse=None, cwd=None):
 def free_bytes(path):
     """Free bytes on the disk of path (its nearest existing parent) and that parent's device."""
     p = Path(path)
-    while not p.exists():
+    # An unavailable drive or share has no existing ancestor: let disk_usage report its OS error at the root.
+    while not p.exists() and p.parent != p:
         p = p.parent
     return shutil.disk_usage(p).free, os.stat(p).st_dev
 
@@ -855,6 +858,12 @@ def exefs_status(ctx):
     return ctx.dest / 'build-cache' / 'exefs.status'
 
 
+def exefs_data(ctx):
+    """The data writes of the mods' ExeFS patch files (tools/exefs_patches.py --data), which runtime/mods.c stores into
+    main before the game runs."""
+    return ctx.dest / 'build-cache' / 'exefs.data'
+
+
 def cheats_status(ctx):
     """The build's cheat status file (tools/cheats.py --status), which runtime/cheats.c reads: in build-cache, not in a
     mod's own cheats folder, which the player (or TKMM's merge) may empty."""
@@ -869,7 +878,7 @@ class Lift(Step):
         # fingerprint takes a file's name and content only; the paths under mods/ go in too, because tools/cheats.py
         # and tools/exefs_patches.py name each patch set after its mod folder, so a renamed folder needs a lift
         files = cheat_files(ctx) + exefs_files(ctx)
-        return fingerprint('lift-4', ctx.keys['extract'], ctx.layout.recomp, ctx.layout.patches(game_version(ctx)),
+        return fingerprint('lift-5', ctx.keys['extract'], ctx.layout.recomp, ctx.layout.patches(game_version(ctx)),
                            ctx.layout.tools / 'cheats.py', ctx.layout.tools / 'exefs_patches.py', *files,
                            *(f.relative_to(ctx.dest / 'mods').as_posix() for f in files))
 
@@ -890,36 +899,54 @@ class Lift(Step):
             if m:
                 rep.progress('lift', 'emit', int(m.group(1)), int(m.group(2)), 'chunks')
 
-        # The code cheats of the mods' cheat files and the mods' ExeFS patch files become patch sets (tools/cheats.py,
-        # tools/exefs_patches.py); their status files tell the runtime what the build has. Both are the player's
-        # downloads, so files the tools refuse cost those cheats or patches, not the install: the game is built
-        # without them and the setup warns.
+        # The code cheats of the mods' cheat files and the mods' ExeFS patch files become patch sets and data writes
+        # (tools/cheats.py, tools/exefs_patches.py); their status files tell the runtime what the build has. Both are
+        # the player's downloads, so files the tools refuse cost those cheats or patches, not the install: the game is
+        # built without them and the setup warns.
         plain = layout.patches(game_version(ctx))
-        stages = [('cheats.py', 'patches-cheats.txt', cheats_status(ctx), cheat_files(ctx), 'warn.cheats_skipped'),
-                  ('exefs_patches.py', 'patches-exefs.txt', exefs_status(ctx), exefs_files(ctx), 'warn.exefs_skipped')]
+        # (tool, patch file it writes, its outputs for the runtime: the status first, then the data file, the files it
+        # reads, the warning when it fails)
+        stages = [('cheats.py', 'patches-cheats.txt', [cheats_status(ctx)], cheat_files(ctx), 'warn.cheats_skipped'),
+                  ('exefs_patches.py', 'patches-exefs.txt', [exefs_status(ctx), exefs_data(ctx)], exefs_files(ctx),
+                   'warn.exefs_skipped')]
         patches, skipped = plain, {}
-        for tool, out, status, inputs, warn in stages:
-            status.unlink(missing_ok=True)   # no status: the runtime treats every such file as not built
+
+        def drop(outputs):   # no status: the runtime treats every such file as not built
+            for f in outputs:
+                f.unlink(missing_ok=True)
+
+        for tool, out, outputs, inputs, warn in stages:
+            drop(outputs)
             if not inputs:
                 continue
             code, log = run_tool(ctx, 'lift', layout.tools / tool,
                                  ['--exefs', ctx.dest / 'exefs', '--mods', ctx.dest / 'mods', '--patches', patches,
-                                  '--out', ctx.temp / out, '--status', status])
+                                  '--out', ctx.temp / out, '--status', outputs[0],
+                                  *(['--data', outputs[1]] if len(outputs) > 1 else [])])
             if code:
-                status.unlink(missing_ok=True)
+                drop(outputs)
                 skipped[warn] = log
             else:
                 patches = ctx.temp / out
         args = ['--exefs', ctx.dest / 'exefs', '--out', gen]
         code, log = run_tool(ctx, 'lift', layout.recomp / 'gen.py', args + ['--patches', patches], parse)
         if code and patches != plain:
-            for tool, out, status, inputs, warn in stages:
+            for tool, out, outputs, inputs, warn in stages:
                 if inputs and warn not in skipped:
-                    status.unlink(missing_ok=True)
+                    drop(outputs)
                     skipped[warn] = log
             code, log = run_tool(ctx, 'lift', layout.recomp / 'gen.py', args + ['--patches', plain], parse)
         if code:
             raise InstallError('lift.failed', code=code, log=str(log))
+        # The mods' data writes must stay off the bytes the lifted code reads as constants (its jump tables), which only
+        # the lift knows: gen.py lists them in folded.txt, and a file that writes them becomes unsupported
+        if exefs_data(ctx).is_file():
+            code, log = run_tool(ctx, 'lift', layout.tools / 'exefs_patches.py',
+                                 ['--exefs', ctx.dest / 'exefs', '--status', exefs_status(ctx), '--data',
+                                  exefs_data(ctx), '--folded', gen / 'main' / 'folded.txt'])
+            if code:
+                drop([exefs_status(ctx), exefs_data(ctx)])
+                skipped.setdefault('warn.exefs_skipped', log)
         for warn, log in skipped.items():
             rep.warning(warn, log=str(log))
         (gen / STAMP).write_text(ctx.keys['lift'] + '\n')
@@ -1072,7 +1099,7 @@ class Prepare(Step):
         # input_prompts_folder). Both are written on every install, so the menu choices work without another run of
         # the setup. Optional: when one fails, the game keeps the Switch prompts with that layout and the menu says the
         # files are missing, so the setup warns once and goes on. romfs-patches-pc likewise, with keyboard and mouse
-        # prompts for prompts = pc (16:9 only: another shape takes make_romfs_patches.py --prompts pc --aspect).
+        # prompts for prompts = pc.
         failed = None
         for style in ('xbox', 'xbox-label', 'pc'):
             out = dest / f'romfs-patches-{style}'
@@ -1083,6 +1110,25 @@ class Prepare(Step):
             if code:
                 remove(out)
                 failed = failed or (code, log)
+        # Each prompts' folder once more for each shape of ASPECT_SIZES, romfs-patches-<style>-<w>x<h> with that
+        # shape's UI layout (--prompts <style> --aspect), which runtime/romfs.c overlay_init loads in place of
+        # romfs-patches-<w>x<h>: without it a player with Xbox or PC prompts on a screen that is not 16:9 chose between
+        # the prompts and the screen's shape. Only the Common archive differs from romfs-patches-<style>; --reuse links
+        # the fonts, messages and size table from there instead of compressing them again, so each folder takes
+        # seconds and a few MiB. Optional like romfs-patches-<style>, and skipped without it.
+        for style in ('xbox', 'xbox-label', 'pc'):
+            plain = dest / f'romfs-patches-{style}'
+            for size in ASPECT_SIZES:
+                out = dest / f'romfs-patches-{style}-{size}'
+                remove(out)
+                if not (plain / 'manifest.tsv').is_file():
+                    continue
+                code, log = run_tool(ctx, f'prepare-{style}-{size}', layout.tools / 'make_romfs_patches.py',
+                                     ['--version', version_tag(game_version(ctx)), '--romfs', dest / 'romfs', '--out',
+                                      out, '--prompts', style, '--aspect', size, '--reuse', plain])
+                if code:
+                    remove(out)
+                    failed = failed or (code, log)
         if failed:
             rep.warning('warn.xbox_prompts_failed', code=failed[0], log=str(failed[1]))
 

recomp/gen.py

@@ -9,6 +9,9 @@ Output, under --out:
   <mod>/<mod>_NNNN.c           one `void f_<addr>(Cpu *c)` per guest function, about --chunk-bytes of C per file
   <mod>/<mod>_table.c          sorted <mod>_fn_addrs[] / <mod>_fn_ptrs[] / <mod>_fn_count for rt_dispatch
   <mod>/manifest.json          chunk list, function count, statistics and timings
+  <mod>/folded.txt             the module's bytes the generated code took as constants: what the jump tables read at
+                               lift time, as module offsets [start, end); tools/exefs_patches.py keeps a mod's data
+                               writes off them
   patches.h                    code patch sets for the runtime (recomp/patches.py; no sets without --patches)
 
 Function discovery runs in this process, vectorised over the module's .text words. Function starts are the module
@@ -555,7 +558,8 @@ def _retry(fn, *args):
 
 # ------------------------------------------------------------------------------------------------ workers
 
-_W = {}  # worker state: 'mods' {name: (path, base)}, 'starts' {name: sorted array}, 'emitters' {name: Emitter}
+_W = {}  # worker state: 'mods' {name: (path, base)}, 'starts' {name: sorted array}, 'emitters' {name: Emitter},
+         # 'reads' {name: [(start, end)]} module bytes read since the last chunk
 
 
 def _worker_init(mods, starts, table, opts, jobs):
@@ -565,6 +569,7 @@ def _worker_init(mods, starts, table, opts, jobs):
     _W['patches'] = table
     _W['opts'] = opts   # Emitter keyword arguments
     _W['emitters'] = {}
+    _W['reads'] = {}
 
 
 def _emitter(name):
@@ -572,12 +577,31 @@ def _emitter(name):
     if e is None:
         path, base = _W['mods'][name]
         mod = Module(name, path, base, 'c')
+        # Module.read is how the jump tables (recomp/jumptables.py _eval) read .text and .rodata at lift time: the
+        # bytes it returns become part of the generated code, so <mod>/folded.txt lists them
+        read, reads = mod.read, _W['reads'].setdefault(name, [])
+
+        def recorded(a, n, signed=False):
+            reads.append((a - base, a - base + n))
+            return read(a, n, signed=signed)
+        mod.read = recorded
         e = _W['emitters'][name] = Emitter(mod, set(_W['starts'][name].tolist()),
                                            _W['patches'].for_module(name, base),
                                            spins=_W['patches'].spins_for(name, base), **_W['opts'])
     return e
 
 
+def merge_ranges(ranges):
+    """Sorted [start, end) ranges with overlapping and touching ones joined."""
+    out = []
+    for s, e in sorted(ranges):
+        if out and s <= out[-1][1]:
+            out[-1][1] = max(out[-1][1], e)
+        else:
+            out.append([s, e])
+    return out
+
+
 def _emit_chunk(task):
     """Worker: write one chunk. Returns its statistics."""
     name, number, path, fns = task
@@ -595,9 +619,12 @@ def _emit_chunk(task):
         written = write_if_changed(Path(path), text)
     except OSError as ex:
         written, error = False, f'{path}: {ex}'
+    reads = _W['reads'].get(name, [])
+    folded = merge_ranges(reads)
+    reads.clear()
     return {'module': name, 'number': number, 'file': Path(path).name, 'functions': len(fns), 'bytes': len(text),
             'written': written, 'error': error, 'seconds': time.perf_counter() - t0, 'stats': dict(e.stats),
-            'unsupported': dict(e.unsupported)}
+            'unsupported': dict(e.unsupported), 'folded': folded}
 
 
 # ------------------------------------------------------------------------------------------------ driver
@@ -613,6 +640,14 @@ def table_text(name, addrs):
     return ''.join(lines)
 
 
+def folded_text(name, ranges):
+    """<mod>/folded.txt: the module offsets [start, end) of the bytes the jump tables read at lift time, in hex."""
+    lines = [f'# {GENERATED_MARKER}: {name} bytes the generated code took as constants (jump tables), module offsets '
+             f'[start, end), generated by recomp/gen.py from game code. Do not edit or commit.']
+    lines += [f'{s:x} {e:x}' for s, e in ranges]
+    return '\n'.join(lines) + '\n'
+
+
 def remove_stale(moddir, name, nchunks, errors):
     """Delete chunks an earlier run wrote beyond this run's count, and temporary files an interrupted run left."""
     pat = re.compile(re.escape(name) + r'_(\d{4})\.c')
@@ -714,6 +749,7 @@ def main():
     # Largest chunks first, so the pool does not end on a long tail.
     tasks.sort(key=lambda t: -t[0])
     results = {name: [] for name in info}
+    folded = {name: [] for name in info}
     done_at, errors = {}, []
     t_emit = time.perf_counter()
     ctx = mp.get_context('spawn')
@@ -724,9 +760,11 @@ def main():
         print(f'emitting with {jobs} workers ({why})', file=sys.stderr, flush=True)
         try:
             # Chunk writes are atomic and idempotent, so a chunk lost with a failed pool is simply emitted again.
+            # Its folded bytes come with its result: a chunk emitted again reads them again in its new worker.
             with ctx.Pool(jobs, initializer=_worker_init, initargs=(paths, starts_abs, table, opts, jobs)) as pool:
                 for r in pool.imap_unordered(_emit_chunk, pending, chunksize=1):
                     results[r['module']].append(r)
+                    folded[r['module']] += r['folded']
                     if r['error']:
                         errors.append(r['error'])
                     done_at[r['module']] = time.perf_counter() - t_emit
@@ -753,6 +791,7 @@ def main():
             unsupported.update(r['unsupported'])
         t = time.perf_counter()
         table_written = write_if_changed(moddir / f'{name}_table.c', table_text(name, d['emitted']))
+        write_if_changed(moddir / 'folded.txt', folded_text(name, merge_ranges(folded[name])))
         removed = remove_stale(moddir, name, d['chunks'], errors)
         manifest = {
             'module': name,

recomp/patches-v121.txt

@@ -55,6 +55,21 @@ site cloth main 8222f0 9401d7cf @page:x8 bd400104 1e240842 1e240863 9401d7cf   #
 # 100; compare +0x28 * +0x38 (30 / time scale) instead, the same value at 30 Hz and independent of the frame rate.
 site bounce main bda798 bd402900 bd402900 bd403901 1e210800   # ldr s0, [x8, #0x28]; ldr s1, [x8, #0x38]; fmul s0, s0, s1
 
+# Climbing (issues #35, #276, #1001): the climb controller's velocity step (main+0x1918220, the same code as 1.4.3
+# main+0x1f49754) multiplies the requested velocity by a factor that makes up for a movement loss growing with the
+# frame rate: 1.272 - 0.362 (r - 1) clamped to [0.5, 1.5] (r = VFR +0, the 30 Hz ratio times the time scale). Right at
+# 30 fps, 78% of the speed at 60 and 30% at 240. Below r = 1 use 1.4.3's factor, 1.272 + 1.189 (1 - r) at most 2
+# (Nintendo's 60 fps fit), and below r = 0.5 the 1.4.3 set climb's g(r) = 0.66265 / r + 0.4982 + 0.0848 r, at most 8.
+# Without flags: h = max(min(1.4.3 factor, 2), g - 16 max(r - 0.5, 0)), s0 = max(s0, h), s1 = max(1.5, h), and the
+# following fmin s1, s0, s1 gives max(min(s0, 1.5), h). For r >= 1 h is at most the shipping value, so 30 fps and
+# slower keep the factor bit for bit. x8 is dead after the load at 0x19182c0; s5-s7 and s16 are written before any
+# later read in this function.
+# g takes u = max(page +0, r): the patch page's 30 Hz frame ratio without the time scale (runtime/frame_rate.c), so
+# slow motion at 30 or 60 fps keeps the shipping factor and only a real frame rate above 60 raises it; the runtime
+# switches the set off without the real-time step, where the page is not written.
+set climb    TOTK_CLIMB_FIX      1
+site climb main 19182cc 1e2f1001 @page:x8 bd400111 1e296a31 1e2e1005 1e3138a5 52862508 72a7e308 1e270106 1e2608a5 1e2528a5 529a1ca8 72a7f448 1e270106 1e2628a5 1e201006 1e2678a5 52946dc8 72a7e528 1e270106 1e3118c6 52828248 72a7dfe8 1e270107 1e2728c6 529573e8 72a7b5a8 1e270107 1f1118e6 1e283a27 2f00e410 1e3068e7 1e261010 1f1098e6 1e241010 1e3078c6 1e2668a5 1e256800 1e2f1001 1e256821   # fmov s1, #1.5 -> s5 = h(r); s0 = max(s0, s5); s1 = max(1.5, s5)
+
 # Cursor and aim steps (upstream game_patches.c patch_cursor_speed, 363a6eb). These updates read the game's 30 Hz frame
 # ratio for their smoothing but leave it out of the step itself, so the map cursor and the vertical Ultrahand aim moved
 # 2x as fast at 60 fps and 4x at 120 as at the shipping 30 fps (issue #44). Each site multiplies the step by that ratio,
@@ -65,6 +80,22 @@ site cursor main ec3d9c bd400361 f94002e9 bd400930 1e300800 bd400361   # map pan
 site cursor main 1d48d64 1e2a198c 1e2a198c bd400a90 1e30098c   # fdiv s12, s12, s10, then s12 *= [x20 + 8]: vertical Ultrahand aim
 site cursor main 1d484e4 1e200923 1e200923 bd400a90 1e300863   # fmul s3, s9, s0, then s3 *= [x20 + 8]: vertical gyro aim (Recall)
 
+# Gear rotation check (issues #330, #984). The update of ActorCheckRotateModule (main+0x21350c0; vtable +0x110, called
+# by the AI node dispatchers main+0x11f5a40 / 0x2adbcf8 with x1 = the update context) reports a gear or water wheel as
+# turning when the angle it turned since the previous update is above RotateStartDeg / 30, the angle of one 30 Hz
+# frame. Above 30 fps a gear turns less per update than that, reads as stopped, and the shrine doors that wait on it
+# (Riogok, Turakamik, the Wind Temple) never open. 1.4.3 (main+0x273d080) divides the angle by the context's +8, the
+# frame delta in 30 Hz frames (the AI root sets it from its owner's virtual +0x30, main+0x106acc4, and the frame
+# counter node SelectorGenericFrameCounter, main+0x10b66f8, adds it up per update). 1.2.1 gets the same context and
+# ignores it: the update overwrites x1 with its first instruction. The first site keeps ctx+8 in s12 (the prologue has
+# saved d12, the function first writes it at the second site, and callees preserve it); the second multiplies the
+# threshold by it, which is 1.4.3's test with both sides multiplied by the delta. s4 is free after the fdiv: the next
+# read follows a write in the callee main+0x11c2014. The context follows the measured step with the set realtime and is
+# the whole-vsync ratio without it (0.25 at 120 fps, 0.5 at 60 fps measured), so the set works either way.
+set gears    TOTK_HIGHFPS_FIX    1
+site gears main 21350e0 f9402001 bd40082c f9402001   # ldr x1, [x0, #0x40], after ldr s12, [x1, #8] (frame delta)
+site gears main 21352d0 1e24180c 1e241804 1e2c088c   # fdiv s12, s0, s4 -> fdiv s4, s0, s4; fmul s12, s4, s12
+
 # Output size (upstream games/totk/runtime/game_patches.c patch_output_res; runtime/resolution.c switches the set on).
 # The swapchain size nnMain writes before NvInitializeGraphics comes from page +48 ({w, h} as one 64-bit value), and
 # the output size the renderer lays out its canvas, crop and UI in (main+0xcc3f00, read from the graphics parameters
@@ -81,6 +112,14 @@ site scene main cc3f68 5e21d900 @page:x9    # scvtf s0, s8 -> adrp x9, page
 site scene main cc3f70 bd0cc660 2d420520    # str s0, [x19, #0xcc4] -> ldp s0, s1, [x9, #16]
 site scene main cc3f74 5e21d921 bd0cc660    # scvtf s1, s9 -> str s0, [x19, #0xcc4]
 
+# Fixed scene size (MacNativeGames game_patches_v143.c patch_fixed_res, 1.4.3 main+0x5848a8; runtime/resolution.c
+# res_dynres_live switches the set on unless TOTK_DYNAMIC_RES=1). The game's dynamic resolution controller
+# (main+0xcc26f4, called once per frame from the renderer at main+0xcc0fa0) steps the scene through tiers of 100, 90 and
+# 80 % by a cost model that grows with the scene width squared (runtime/resolution.c res_dynres_live), and the renderer stores scene width * tier / output width at +0xccc (main+0xcc1150).
+# Every path leaves through one epilogue, where the tier's scale moves into the result: 1.0 there makes every tier 100 %.
+set fixres   TOTK_FIXED_RES_PATCH 0
+site fixres main cc2c00 2a1703e9 52a7f009   # mov w9, w23 -> mov w9, #0x3f800000 (1.0f)
+
 # Camera aspect ratio for a game resolution that is not 16:9 (upstream game_patches.c, the 1.2.1 sites of aspect_install;
 # runtime/resolution.c res_init writes the output's width / height to page +56 and switches the set on). The same four
 # sites as patches-v143.txt, with the same original words: two camera sites (update and setup) load the camera's copy of

recomp/patches-v143.txt

@@ -82,6 +82,13 @@ site scene main 585e50 bd0cc660 2d420460    # str s0, [x19, #0xcc4] -> ldp s0, s
 site scene main 585e54 1e220140 bd0cc660    # scvtf s0, w10 -> str s0, [x19, #0xcc4]
 site scene main 585e60 bd0cca60 bd0cca61    # str s0, [x19, #0xcc8] -> str s1, [x19, #0xcc8]
 
+# Fixed scene size (MacNativeGames game_patches_v143.c patch_fixed_res; runtime/resolution.c res_dynres_live switches
+# the set on unless TOTK_DYNAMIC_RES=1). The dynamic resolution controller (main+0x584300, 1.2.1 0xcc26f4) steps the
+# scene through tiers of 100, 90 and 80 % by a cost model that grows with the scene width squared; at its one epilogue the tier's scale moves into
+# the result, and 1.0 there makes every tier 100 %. The original word is MacNativeGames' check against 1.4.3.
+set fixres   TOTK_FIXED_RES_PATCH 0
+site fixres main 5848a8 2a1703e9 52a7f009   # mov w9, w23 -> mov w9, #0x3f800000 (1.0f)
+
 # Camera aspect ratio for a game resolution that is not 16:9 (upstream game_patches_v143.c and aspect_patch.h
 # aspect_install / aspect_stub; runtime/resolution.c res_init writes the output's width / height to page +56 and
 # switches the set on). Each site leaves the ratio sead's perspective projection (+0xb0) takes in s0: two camera sites
@@ -111,6 +118,8 @@ site mouse_direct main 874cb4 bd042289 94c9acc7
 site mouse_direct main 874d7c 6d40a3e9 94c9ac91
 site mouse_direct main 18e444 f9400408 94e546db
 site mouse_direct main 18f108 bd4047ec 94e543a6
+# Keep native camera-distance easing when direct mouse orbit bypasses the stick integrators.
+site mouse_direct main 18ea34 b9418a88 94e5454f   # orbit state: ldr w8, [x20, #0x188]
 site mouse_direct main 1d59460 b9004be8 94761acc
 # The aiming, scope, ability, horse and Purah Pad camera cameras: per axis, the float load the stick turn is computed
 # from (the gamepad's part only) and the load of the angle it is added to (plus the mouse's degrees), each a bl to page
@@ -1077,3 +1086,19 @@ site cascade main 6fabb8 52a83408 @page:x8 b9404108   # mov w8, #20.0  -> ldr w8
 site cascade main 6fac40 52a84408 @page:x8 b9404508   # mov w8, #40.0  -> ldr w8, [page+68]
 site cascade main 6facc8 52a85408 @page:x8 b9404908   # mov w8, #80.0  -> ldr w8, [page+72]
 site cascade main 6fad4c 52a86408 @page:x8 b9404d08   # mov w8, #160.0 -> ldr w8, [page+76]
+
+# Climbing (issues #35, #276, #1001): the climb controller's velocity step (main+0x1f49754, shared by wall and ladder
+# climbing, climb jumps and climb-overs) multiplies the requested velocity by a factor that makes up for a movement loss
+# growing with the frame rate. Below ratio 1 the factor is 1.272 + 1.189 (1 - r) clamped to [1, 2] (r = VFR +0, the
+# 30 Hz ratio times the time scale): Nintendo's fit for 30 and 60 fps, flat at 2 from about 77 fps, so Link climbs at
+# 63% of the speed at 120 fps and 34% at 240. Below r = 0.5 take g(r) = 0.66265 / r + 0.4982 + 0.0848 r instead,
+# fitted to the replayed ladder climb at 120 and 240 fps and equal to the shipping 1.8665 at r = 0.5; at most 8. The
+# site computes g with 16 * max(r - 0.5, 0) subtracted, so g stays under the shipping factor for r >= 0.5, and folds it
+# into the existing clamp without flags: s0 = max(s0, g) (the value clamped), s1 = max(2, g) (the bound). r >= 0.5
+# keeps the shipping factor bit for bit. x8 is dead after the constant load at 0x1f497e8, s2-s4 are loaded again at
+# 0x1f49844 before any read.
+# g takes u = max(page +0, r): the patch page's 30 Hz frame ratio without the time scale (runtime/frame_rate.c), so
+# slow motion at 30 or 60 fps keeps the shipping factor and only a real frame rate above 60 raises it; the runtime
+# switches the set off without the real-time step, where the page is not written.
+set climb    TOTK_CLIMB_FIX      1
+site climb main 1f497f4 1e201001 @page:x8 bd400104 1e286884 52946dc8 72a7e528 1e270102 1e241842 52828248 72a7dfe8 1e270103 1e232842 529573e8 72a7b5a8 1e270103 1f040862 1e2a3883 2f00e404 1e246863 1e261004 1f048862 1e241004 1e247842 1e226800 1e201001 1e226821   # fmov s1, #2.0 -> s2 = g(u); s0 = max(s0, s2); s1 = max(2.0, s2)

runtime/bl_deswizzle_spv.h

@@ -0,0 +1,111 @@
+// Generated by tools/build_shaders.py from runtime/shaders/bl_deswizzle.comp; do not edit.
+// source sha256 bd2065290e315e85d2fcdb609d812c15ba9a109d7506b3105850a5860677b0ef
+#pragma once
+#include <stdint.h>
+static const uint32_t bl_deswizzle_spv[] = {
+    0x07230203, 0x00010300, 0x000d000b, 0x000000d0, 0x00000000, 0x00020011, 0x00000001, 0x0006000b,
+    0x00000001, 0x4c534c47, 0x6474732e, 0x3035342e, 0x00000000, 0x0003000e, 0x00000000, 0x00000001,
+    0x0007000f, 0x00000005, 0x00000004, 0x6e69616d, 0x00000000, 0x0000000b, 0x0000001f, 0x00060010,
+    0x00000004, 0x00000011, 0x00000040, 0x00000001, 0x00000001, 0x00040047, 0x0000000b, 0x0000000b,
+    0x0000001a, 0x00030047, 0x00000010, 0x00000002, 0x00050048, 0x00000010, 0x00000000, 0x00000023,
+    0x00000000, 0x00050048, 0x00000010, 0x00000001, 0x00000023, 0x00000004, 0x00050048, 0x00000010,
+    0x00000002, 0x00000023, 0x00000008, 0x00050048, 0x00000010, 0x00000003, 0x00000023, 0x0000000c,
+    0x00050048, 0x00000010, 0x00000004, 0x00000023, 0x00000010, 0x00050048, 0x00000010, 0x00000005,
+    0x00000023, 0x00000014, 0x00050048, 0x00000010, 0x00000006, 0x00000023, 0x00000018, 0x00050048,
+    0x00000010, 0x00000007, 0x00000023, 0x0000001c, 0x00050048, 0x00000010, 0x00000008, 0x00000023,
+    0x00000020, 0x00050048, 0x00000010, 0x00000009, 0x00000023, 0x00000024, 0x00050048, 0x00000010,
+    0x0000000a, 0x00000023, 0x00000028, 0x00040047, 0x0000001f, 0x0000000b, 0x0000001b, 0x00040047,
+    0x000000ad, 0x00000006, 0x00000010, 0x00030047, 0x000000ae, 0x00000002, 0x00040048, 0x000000ae,
+    0x00000000, 0x00000019, 0x00050048, 0x000000ae, 0x00000000, 0x00000023, 0x00000000, 0x00030047,
+    0x000000b0, 0x00000019, 0x00040047, 0x000000b0, 0x00000021, 0x00000001, 0x00040047, 0x000000b0,
+    0x00000022, 0x00000000, 0x00040047, 0x000000c1, 0x00000006, 0x00000010, 0x00030047, 0x000000c2,
+    0x00000002, 0x00040048, 0x000000c2, 0x00000000, 0x00000018, 0x00050048, 0x000000c2, 0x00000000,
+    0x00000023, 0x00000000, 0x00030047, 0x000000c4, 0x00000018, 0x00040047, 0x000000c4, 0x00000021,
+    0x00000000, 0x00040047, 0x000000c4, 0x00000022, 0x00000000, 0x00040047, 0x000000cd, 0x0000000b,
+    0x00000019, 0x00020013, 0x00000002, 0x00030021, 0x00000003, 0x00000002, 0x00040015, 0x00000006,
+    0x00000020, 0x00000000, 0x00040017, 0x00000009, 0x00000006, 0x00000003, 0x00040020, 0x0000000a,
+    0x00000001, 0x00000009, 0x0004003b, 0x0000000a, 0x0000000b, 0x00000001, 0x0004002b, 0x00000006,
+    0x0000000c, 0x00000001, 0x00040020, 0x0000000d, 0x00000001, 0x00000006, 0x000d001e, 0x00000010,
+    0x00000006, 0x00000006, 0x00000006, 0x00000006, 0x00000006, 0x00000006, 0x00000006, 0x00000006,
+    0x00000006, 0x00000006, 0x00000006, 0x00040020, 0x00000011, 0x00000009, 0x00000010, 0x0004003b,
+    0x00000011, 0x00000012, 0x00000009, 0x00040015, 0x00000013, 0x00000020, 0x00000001, 0x0004002b,
+    0x00000013, 0x00000014, 0x0000000a, 0x00040020, 0x00000015, 0x00000009, 0x00000006, 0x0004002b,
+    0x00000006, 0x00000019, 0x00000000, 0x0004002b, 0x00000006, 0x0000001d, 0x00000040, 0x0004003b,
+    0x0000000a, 0x0000001f, 0x00000001, 0x0004002b, 0x00000013, 0x00000024, 0x00000002, 0x00020014,
+    0x00000027, 0x0004002b, 0x00000013, 0x0000002d, 0x00000006, 0x0004002b, 0x00000013, 0x00000032,
+    0x00000007, 0x0004002b, 0x00000006, 0x00000037, 0x00000020, 0x0004002b, 0x00000006, 0x00000048,
+    0x00000005, 0x0004002b, 0x00000006, 0x0000004c, 0x0000001f, 0x0004002b, 0x00000013, 0x0000004f,
+    0x00000008, 0x0004002b, 0x00000013, 0x00000052, 0x00000009, 0x0004002b, 0x00000006, 0x0000006a,
+    0x00000004, 0x0004002b, 0x00000006, 0x0000006f, 0x00000002, 0x0004002b, 0x00000006, 0x00000078,
+    0x00000008, 0x0004002b, 0x00000006, 0x00000083, 0x00000003, 0x0004002b, 0x00000013, 0x00000093,
+    0x00000003, 0x0004002b, 0x00000013, 0x0000009b, 0x00000004, 0x0004002b, 0x00000013, 0x000000a4,
+    0x00000005, 0x00040017, 0x000000ac, 0x00000006, 0x00000004, 0x0003001d, 0x000000ad, 0x000000ac,
+    0x0003001e, 0x000000ae, 0x000000ad, 0x00040020, 0x000000af, 0x0000000c, 0x000000ae, 0x0004003b,
+    0x000000af, 0x000000b0, 0x0000000c, 0x0004002b, 0x00000013, 0x000000b1, 0x00000000, 0x0004002b,
+    0x00000013, 0x000000b2, 0x00000001, 0x0003001d, 0x000000c1, 0x000000ac, 0x0003001e, 0x000000c2,
+    0x000000c1, 0x00040020, 0x000000c3, 0x0000000c, 0x000000c2, 0x0004003b, 0x000000c3, 0x000000c4,
+    0x0000000c, 0x00040020, 0x000000c9, 0x0000000c, 0x000000ac, 0x0006002c, 0x00000009, 0x000000cd,
+    0x0000001d, 0x0000000c, 0x0000000c, 0x00050036, 0x00000002, 0x00000004, 0x00000000, 0x00000003,
+    0x000200f8, 0x00000005, 0x000300f7, 0x000000ce, 0x00000000, 0x000300fb, 0x00000019, 0x000000cf,
+    0x000200f8, 0x000000cf, 0x00050041, 0x0000000d, 0x0000000e, 0x0000000b, 0x0000000c, 0x0004003d,
+    0x00000006, 0x0000000f, 0x0000000e, 0x00050041, 0x00000015, 0x00000016, 0x00000012, 0x00000014,
+    0x0004003d, 0x00000006, 0x00000017, 0x00000016, 0x00050084, 0x00000006, 0x00000018, 0x0000000f,
+    0x00000017, 0x00050041, 0x0000000d, 0x0000001a, 0x0000000b, 0x00000019, 0x0004003d, 0x00000006,
+    0x0000001b, 0x0000001a, 0x00050080, 0x00000006, 0x0000001c, 0x00000018, 0x0000001b, 0x00050084,
+    0x00000006, 0x0000001e, 0x0000001c, 0x0000001d, 0x00050041, 0x0000000d, 0x00000020, 0x0000001f,
+    0x00000019, 0x0004003d, 0x00000006, 0x00000021, 0x00000020, 0x00050080, 0x00000006, 0x00000022,
+    0x0000001e, 0x00000021, 0x00050041, 0x00000015, 0x00000025, 0x00000012, 0x00000024, 0x0004003d,
+    0x00000006, 0x00000026, 0x00000025, 0x000500ae, 0x00000027, 0x00000028, 0x00000022, 0x00000026,
+    0x000300f7, 0x0000002a, 0x00000000, 0x000400fa, 0x00000028, 0x00000029, 0x0000002a, 0x000200f8,
+    0x00000029, 0x000200f9, 0x000000ce, 0x000200f8, 0x0000002a, 0x00050041, 0x00000015, 0x0000002e,
+    0x00000012, 0x0000002d, 0x0004003d, 0x00000006, 0x0000002f, 0x0000002e, 0x000500c4, 0x00000006,
+    0x00000030, 0x0000000c, 0x0000002f, 0x00050041, 0x00000015, 0x00000033, 0x00000012, 0x00000032,
+    0x0004003d, 0x00000006, 0x00000034, 0x00000033, 0x000500c4, 0x00000006, 0x00000035, 0x0000000c,
+    0x00000034, 0x00050080, 0x00000006, 0x0000003c, 0x0000002f, 0x00000034, 0x000500c4, 0x00000006,
+    0x0000003d, 0x00000037, 0x0000003c, 0x00050086, 0x00000006, 0x00000041, 0x00000022, 0x0000003d,
+    0x00050089, 0x00000006, 0x00000045, 0x00000022, 0x0000003d, 0x000500c2, 0x00000006, 0x00000049,
+    0x00000045, 0x00000048, 0x000500c7, 0x00000006, 0x0000004d, 0x00000045, 0x0000004c, 0x00050041,
+    0x00000015, 0x00000050, 0x00000012, 0x0000004f, 0x0004003d, 0x00000006, 0x00000051, 0x00000050,
+    0x00050041, 0x00000015, 0x00000053, 0x00000012, 0x00000052, 0x0004003d, 0x00000006, 0x00000054,
+    0x00000053, 0x00050084, 0x00000006, 0x00000055, 0x00000051, 0x00000054, 0x00050086, 0x00000006,
+    0x00000059, 0x00000041, 0x00000055, 0x00050089, 0x00000006, 0x0000005d, 0x00000041, 0x00000055,
+    0x00050086, 0x00000006, 0x00000062, 0x0000005d, 0x00000051, 0x00050089, 0x00000006, 0x00000067,
+    0x0000005d, 0x00000051, 0x00050084, 0x00000006, 0x0000006b, 0x00000067, 0x0000006a, 0x000500c2,
+    0x00000006, 0x0000006d, 0x0000004d, 0x0000006a, 0x000500c7, 0x00000006, 0x0000006e, 0x0000006d,
+    0x0000000c, 0x00050084, 0x00000006, 0x00000070, 0x0000006e, 0x0000006f, 0x00050080, 0x00000006,
+    0x00000071, 0x0000006b, 0x00000070, 0x000500c2, 0x00000006, 0x00000073, 0x0000004d, 0x0000000c,
+    0x000500c7, 0x00000006, 0x00000074, 0x00000073, 0x0000000c, 0x00050080, 0x00000006, 0x00000075,
+    0x00000071, 0x00000074, 0x00050084, 0x00000006, 0x00000079, 0x00000062, 0x00000078, 0x00050084,
+    0x00000006, 0x0000007b, 0x00000079, 0x00000030, 0x00050089, 0x00000006, 0x0000007e, 0x00000049,
+    0x00000030, 0x00050084, 0x00000006, 0x0000007f, 0x0000007e, 0x00000078, 0x00050080, 0x00000006,
+    0x00000080, 0x0000007b, 0x0000007f, 0x000500c2, 0x00000006, 0x00000082, 0x0000004d, 0x0000006f,
+    0x000500c7, 0x00000006, 0x00000084, 0x00000082, 0x00000083, 0x00050084, 0x00000006, 0x00000085,
+    0x00000084, 0x0000006f, 0x00050080, 0x00000006, 0x00000086, 0x00000080, 0x00000085, 0x000500c7,
+    0x00000006, 0x00000088, 0x00000045, 0x0000000c, 0x00050080, 0x00000006, 0x00000089, 0x00000086,
+    0x00000088, 0x00050084, 0x00000006, 0x0000008d, 0x00000059, 0x00000035, 0x00050086, 0x00000006,
+    0x00000090, 0x00000049, 0x00000030, 0x00050080, 0x00000006, 0x00000091, 0x0000008d, 0x00000090,
+    0x00050041, 0x00000015, 0x00000094, 0x00000012, 0x00000093, 0x0004003d, 0x00000006, 0x00000095,
+    0x00000094, 0x000500ae, 0x00000027, 0x00000096, 0x00000075, 0x00000095, 0x000400a8, 0x00000027,
+    0x00000097, 0x00000096, 0x000300f7, 0x00000099, 0x00000000, 0x000400fa, 0x00000097, 0x00000098,
+    0x00000099, 0x000200f8, 0x00000098, 0x00050041, 0x00000015, 0x0000009c, 0x00000012, 0x0000009b,
+    0x0004003d, 0x00000006, 0x0000009d, 0x0000009c, 0x000500ae, 0x00000027, 0x0000009e, 0x00000089,
+    0x0000009d, 0x000200f9, 0x00000099, 0x000200f8, 0x00000099, 0x000700f5, 0x00000027, 0x0000009f,
+    0x00000096, 0x0000002a, 0x0000009e, 0x00000098, 0x000400a8, 0x00000027, 0x000000a0, 0x0000009f,
+    0x000300f7, 0x000000a2, 0x00000000, 0x000400fa, 0x000000a0, 0x000000a1, 0x000000a2, 0x000200f8,
+    0x000000a1, 0x00050041, 0x00000015, 0x000000a5, 0x00000012, 0x000000a4, 0x0004003d, 0x00000006,
+    0x000000a6, 0x000000a5, 0x000500ae, 0x00000027, 0x000000a7, 0x00000091, 0x000000a6, 0x000200f9,
+    0x000000a2, 0x000200f8, 0x000000a2, 0x000700f5, 0x00000027, 0x000000a8, 0x0000009f, 0x00000099,
+    0x000000a7, 0x000000a1, 0x000300f7, 0x000000aa, 0x00000000, 0x000400fa, 0x000000a8, 0x000000a9,
+    0x000000aa, 0x000200f8, 0x000000a9, 0x000200f9, 0x000000ce, 0x000200f8, 0x000000aa, 0x00050041,
+    0x00000015, 0x000000b3, 0x00000012, 0x000000b2, 0x0004003d, 0x00000006, 0x000000b4, 0x000000b3,
+    0x00050041, 0x00000015, 0x000000b6, 0x00000012, 0x0000009b, 0x0004003d, 0x00000006, 0x000000b7,
+    0x000000b6, 0x00050084, 0x00000006, 0x000000b8, 0x00000091, 0x000000b7, 0x00050080, 0x00000006,
+    0x000000ba, 0x000000b8, 0x00000089, 0x00050084, 0x00000006, 0x000000bd, 0x000000ba, 0x00000095,
+    0x00050080, 0x00000006, 0x000000be, 0x000000b4, 0x000000bd, 0x00050080, 0x00000006, 0x000000c0,
+    0x000000be, 0x00000075, 0x00050041, 0x00000015, 0x000000c5, 0x00000012, 0x000000b1, 0x0004003d,
+    0x00000006, 0x000000c6, 0x000000c5, 0x00050080, 0x00000006, 0x000000c8, 0x000000c6, 0x00000022,
+    0x00060041, 0x000000c9, 0x000000ca, 0x000000c4, 0x000000b1, 0x000000c8, 0x0004003d, 0x000000ac,
+    0x000000cb, 0x000000ca, 0x00060041, 0x000000c9, 0x000000cc, 0x000000b0, 0x000000b1, 0x000000c0,
+    0x0003003e, 0x000000cc, 0x000000cb, 0x000200f9, 0x000000ce, 0x000200f8, 0x000000ce, 0x000100fd,
+    0x00010038,
+};

runtime/boot.c

@@ -307,6 +307,7 @@ void rt_frame_presented(const tex_t *t, uint64_t n) {
     fflush(stderr);
     fs_save_flush();
     vk_pipeline_save();   // no-op without the Vulkan back end
+    playtime_save();
     prof_write();
     fflush(stderr);
     plat_exit(0);

runtime/cfg.c

@@ -38,29 +38,29 @@ static const char *const known[] = {
     "BIND_PLUS", "BIND_R", "BIND_RCLICK", "BIND_RIGHT", "BIND_RSTICK_DOWN", "BIND_RSTICK_LEFT", "BIND_RSTICK_RIGHT", "BIND_RSTICK_UP",
     "BIND_SHAKE",
     "BIND_SKIP", "BIND_STATS", "BIND_UP", "BIND_X", "BIND_Y",
-    "BIND_ZL", "BIND_ZR", "BLOCK_VIEWS_3D", "CLOTH_REALTIME", "COLOR_BRIGHTNESS", "COLOR_CONTRAST", "COLOR_GAMMA",
+    "BIND_ZL", "BIND_ZR", "BLOCK_VIEWS_3D", "CLIMB_FIX", "CLOTH_REALTIME", "COLOR_BRIGHTNESS", "COLOR_CONTRAST", "COLOR_GAMMA",
     "COLOR_PRESET", "COLOR_SATURATION", "CONTACT_DEBUG", "CONTACT_SHADOWS", "CORE_DIAG", "CRIT", "CRIT_OUT",
-    "DESC_TEMPLATE", "DEVICE_PROFILE", "DISCORD_APP_ID", "DISCORD_IMAGE", "DISCORD_RPC", "DLAA", "DLAA_JITTER", "DLAA_MV", "DLAA_MASKDEBUG", "DLAA_MVDEBUG", "DLAA_OBJMV", "DLAA_SURVEY", "DLAA_TIMES", "DLSS", "DLSS_CHAR_MASK", "DLSS_PRESET", "DLSS_REACTIVE", "DUMP", "DUMP_BUF", "DUMP_DIR",
-    "DUMP_LOG", "EXEFS", "FEEDBACK", "FIX_C0", "FIX_ZERO", "FLIGHT", "FLIGHT_AT", "FLIGHT_DIR", "FLIGHT_FRAMES",
+    "DESC_TEMPLATE", "DESWIZZLE", "DEVICE_PROFILE", "DISCORD_APP_ID", "DISCORD_IMAGE", "DISCORD_RPC", "DLAA", "DLAA_JITTER", "DLAA_MV", "DLAA_MASKDEBUG", "DLAA_MVDEBUG", "DLAA_OBJMV", "DLAA_SURVEY", "DLAA_TIMES", "DLSS", "DLSS_CHAR_MASK", "DLSS_PRESET", "DLSS_REACTIVE", "DUMP", "DUMP_BUF", "DUMP_DIR",
+    "DUMP_LOG", "DYNAMIC_RES", "DYN_DELTA", "EXEFS", "FEEDBACK", "FIXED_RES_PATCH", "FIX_C0", "FIX_ZERO", "FLIGHT", "FLIGHT_AT", "FLIGHT_DIR", "FLIGHT_FRAMES",
     "FLIGHT_SPIKE_MS", "FLIGHT_THUMBS", "FLUSH_SPAN", "FOV", "FOV_PATCH", "FPS60", "FPS_LIMIT", "FRAME_QUEUE",
     "FREEZE_IDLE", "FREE_CORES",
     "FULLSCREEN", "FXAA", "GPL", "GPL_DRAW_BUILD", "GPU", "GPU_INDEX", "GPU_NAME", "HDR", "HDR_GAME_NITS", "HDR_HUE",
-    "HDR_PEAK_NITS", "HDR_UI_NITS", "HIGHFPS_FIX", "INPUT", "INPUT_RECORD", "INPUT_SCRIPT", "LANG", "LIB_PREBUILD",
+    "HDR_PEAK_NITS", "HDR_UI_NITS", "HIGHFPS_FIX", "INPUT", "INPUT_RECORD", "INPUT_SCRIPT", "KSPIN_US", "LANG", "LIB_PREBUILD",
     "LIB_PREBUILD_KEEP", "LIB_PREBUILD_THREADS", "LOCALE", "LOD_GUARD", "LSTICK_DEADZONE", "MEMORY", "MENU_LANG", "MENU_TEST",
-    "MENU_CURSOR", "MENU_CURSOR_LOG", "MENU_TEST_DIR", "MENU_TEST_SET", "MINIMAL_LAYOUT", "MIRROR_GAP", "MODS_DIR", "MODS_DISABLED", "MODS_ORDER", "MODS_REBUILD", "MOTION", "MOUSE_AIM_SCALE", "MOUSE_MODE", "MOUSE_DIRECT_PATCH", "MOUSE_SENS", "MOVIE_ASPECT", "MOVIE_HOST", "MOVIE_PACE",
+    "MEM_RESERVE_MIB", "MENU_CURSOR", "MENU_CURSOR_LOG", "MENU_TEST_DIR", "MENU_TEST_SET", "MINIMAL_LAYOUT", "MIRROR_GAP", "MODS_DIR", "MODS_DISABLED", "MODS_ORDER", "MODS_REBUILD", "MOTION", "MOUSE_AIM_SCALE", "MOUSE_MODE", "MOUSE_DIRECT_PATCH", "MOUSE_SENS", "MOVIE_ASPECT", "MOVIE_HOST", "MOVIE_PACE",
     "MVOBJ_ALL", "MVOBJ_BANKS", "MVOBJ_EQUAL", "MVOBJ_EXACT", "MVOBJ_EXEC", "MVOBJ_NOC4", "MVOBJ_ONLY", "MVOBJ_TIMES", "MVPREV_CHECK", "MVPREV_GROW", "MVPREV_SKIP", "MVPREV_TRUST", "MV_CHECK",
     "MV_SURVEY",
-    "ONE_PER_CORE", "OPERATION_MODE", "OUTPUT_RES", "PAD_LAYOUT", "PASS_CRUMBS", "PATCH_OUTPUT", "PATCH_SCENE", "PCSS", "PCSS_K", "PERF_OVERLAY", "PERF_SLOW_MS", "PIPELINE_CACHE", "PIPE_MEMO", "PIPE_PREBUILD", "PIPE_SKIP", "PIPE_SYNC", "PIPE_WAIT",
+    "ONE_PER_CORE", "OPERATION_MODE", "OUTPUT_RES", "PAD_LAYOUT", "PASS_CRUMBS", "PATCH_OUTPUT", "PATCH_SCENE", "PCORE", "PCSS", "PCSS_K", "PERF_OVERLAY", "PERF_SLOW_MS", "PIPELINE_CACHE", "PIPE_MEMO", "PIPE_PREBUILD", "PIPE_SKIP", "PIPE_SYNC", "PIPE_WAIT",
     "PREWARM_XLATE", "PROBE_MOVIE", "PROBE_MOVIE_BENCH", "PROBE_MOVIE_SHOT", "PROFILE", "PROFILE_CHAINS", "PROFILE_DIR", "PROFILE_FRAMES",
-    "PROFILE_MAIN", "PROFILE_RENDER", "PROMPTS", "PUSH_SKIP", "PUSH_STAGE", "RDOC", "RDOC_DIR", "RDOC_DLL", "REALTIME", "REC_END", "REEXEC",
-    "RENDER_SCALE", "REPLAY_ASYNC",
+    "PROFILE_MAIN", "PROFILE_RENDER", "PROMPTS", "PUSH_SKIP", "PUSH_STAGE", "RDOC", "RDOC_DIR", "RDOC_DLL", "REALTIME", "REC_ASYNC", "REC_END", "REC_SPIN_US", "REEXEC",
+    "RENDER_SCALE", "REPLAY_ASYNC", "REPLAY_CHUNKS", "REPLAY_CHUNK_DRAWS", "REPLAY_SKEW_US", "REPLAY_STATE_CHECK", "REPLAY_WORKERS",
     "REPLAY_CHECK", "RESOLVE_CHECK", "RESTART_WAIT", "ROMFS", "ROMFS_INDEX", "ROMFS_MMAP", "ROMFS_MODS", "ROMFS_PATCHES",
     "ROMFS_PROOF_DIR", "ROMFS_VERIFY", "RSTICK_DEADZONE", "RUMBLE", "RUMBLE_CURVE", "RUMBLE_STRENGTH", "RUMBLE_TRIGGERS", "SHADER_CACHE",
-    "SHADER_MEMO", "SHADOW_RANGE", "SHADOW_RES", "SHADOW_SCALE", "SHARED_FONTS", "SHARPNESS", "SHOTS", "SHOTS_DIR",
+    "SHADER_MEMO", "SHADOW_DEFER", "SHADOW_RANGE", "SHADOW_RES", "SHADOW_SCALE", "SHADOW_SNAP", "SHARED_FONTS", "SHARPNESS", "SHOTS", "SHOTS_DIR",
     "SHOTS_FULL", "SPEED_PROBE", "SPIN", "SPIN_HANDOFF_US", "SPLIT_DRAWS", "SPLIT_US", "STICK_DEADZONE", "STICK_SENS",
-    "SWKBD_ANSWER",
-    "SWKBD_TEST", "TEST_PAD", "TEST_PREPARE_SHADERS", "TEX_ALIAS", "TEX_MEMO", "TEX_VRAM_PRESSURE", "TIMER_1MS", "TRACE", "TXD_TRANSPOSE", "UBO_SNAPSHOT", "UPSCALER", "VK_DYN_A2C", "VK_DYN_VERTEX", "VK_NO_FLOAT64",
-    "VK_PRESENT", "VK_RECORDER", "VK_SSBO_ALIGN", "VK_SUBGROUP", "VK_SUBGROUP_PIN", "VK_VALIDATION", "VOLUME", "VRAM_HEADROOM", "VS_OUTS",
+    "SWKBD_ANSWER", "SYNC_AHEAD",
+    "SWKBD_TEST", "TEST_OOM", "TEST_PAD", "TEST_PREPARE_SHADERS", "TEX_ALIAS", "TEX_BLOCK_MIB", "TEX_HEAP_EXTRA_MB", "TEX_MEMO", "TEX_RELEASED_MIB", "TEX_VRAM_PRESSURE", "TIMER_1MS", "TRACE", "TXD_TRANSPOSE", "UBO_SNAPSHOT", "UB_OVERLAY", "UB_OVERLAY_OUT", "UPSCALER", "VB_USED", "VK_DYN_A2C", "VK_DYN_VERTEX", "VK_NO_FLOAT64",
+    "VK_PRESENT", "VK_RECORDER", "VK_SSBO_ALIGN", "VK_SUBGROUP", "VK_SUBGROUP_PIN", "VK_VALIDATION", "VMA_BLOCK_MIB", "VOLUME", "VRAM_HEADROOM", "VRAM_LIMIT_MIB", "VS_OUTS",
     "WALK_KEY", "WALK_SCALE", "WB_EXACT", "WB_FENCE_LIMIT", "WB_MERGE", "WINDOW_SIZE", "XLATE", "XLATE_ID", "XLATE_KEEP", "XLATE_PROCS",
     "ZSTD_CHECK", "ZSTD_NATIVE",
 };
@@ -69,9 +69,10 @@ static const char *const known[] = {
 static const struct { const char *key, *section; } sections[] = {
     {"FPS_LIMIT", "frame rate"}, {"FPS60", "frame rate"}, {"FRAME_QUEUE", "frame rate"},
     {"REALTIME", "frame rate"}, {"CLOTH_REALTIME", "frame rate"}, {"HIGHFPS_FIX", "frame rate"},
+    {"CLIMB_FIX", "frame rate"},
     {"FULLSCREEN", "display"}, {"WINDOW_SIZE", "display"}, {"PERF_OVERLAY", "display"}, {"VK_PRESENT", "display"},
     {"GPU_INDEX", "display"}, {"OUTPUT_RES", "display"}, {"RENDER_SCALE", "display"}, {"UPSCALER", "display"},
-    {"SHARPNESS", "display"}, {"ASPECT", "display"}, {"MOVIE_ASPECT", "display"}, {"FOV", "display"}, {"ANISO", "display"},
+    {"SHARPNESS", "display"}, {"DYNAMIC_RES", "display"}, {"ASPECT", "display"}, {"MOVIE_ASPECT", "display"}, {"FOV", "display"}, {"ANISO", "display"},
     {"COLOR_PRESET", "display"},
     {"COLOR_BRIGHTNESS", "display"}, {"COLOR_CONTRAST", "display"}, {"COLOR_SATURATION", "display"},
     {"COLOR_GAMMA", "display"}, {"HDR", "display"}, {"HDR_PEAK_NITS", "display"}, {"HDR_GAME_NITS", "display"},
@@ -82,6 +83,7 @@ static const struct { const char *key, *section; } sections[] = {
     {"SHADOW_SCALE", "display"}, {"SHADOW_RANGE", "display"}, {"CONTACT_SHADOWS", "display"},
     {"PCSS", "display"},
     {"MODS_DIR", "mods"}, {"MODS_ORDER", "mods"}, {"MODS_DISABLED", "mods"}, {"MODS_REBUILD", "mods"},
+    {"TEX_HEAP_EXTRA_MB", "mods"},
     {"MENU_LANG", "language"}, {"LANG", "language"}, {"OPERATION_MODE", "language"},
     {"VOLUME", "audio"}, {"AUDIO_CHANNELS", "audio"},
     {"MOTION", "input"}, {"RUMBLE_STRENGTH", "input"}, {"RUMBLE_CURVE", "input"}, {"RUMBLE_TRIGGERS", "input"},
@@ -94,13 +96,13 @@ static const struct { const char *key, *section; } sections[] = {
     {"BIND_RIGHT", "input"}, {"BIND_DOWN", "input"}, {"BIND_LSTICK_LEFT", "input"}, {"BIND_LSTICK_UP", "input"},
     {"BIND_LSTICK_RIGHT", "input"}, {"BIND_LSTICK_DOWN", "input"}, {"BIND_RSTICK_LEFT", "input"},
     {"BIND_RSTICK_UP", "input"}, {"BIND_RSTICK_RIGHT", "input"}, {"BIND_RSTICK_DOWN", "input"}, {"BIND_SHAKE", "input"},
-    {"ASYNC_PREEMPT", "performance"}, {"REPLAY_ASYNC", "performance"}, {"FREE_CORES", "performance"},
+    {"ASYNC_PREEMPT", "performance"}, {"REPLAY_ASYNC", "performance"}, {"REPLAY_WORKERS", "performance"}, {"FREE_CORES", "performance"},
     {"TIMER_1MS", "performance"}, {"PIPE_WAIT", "performance"}, {"PIPE_SKIP", "performance"},
     {"LIB_PREBUILD", "performance"}, {"LIB_PREBUILD_THREADS", "performance"}, {"PIPE_PREBUILD", "performance"},
     {"PREWARM_XLATE", "performance"}, {"GPL", "performance"}, {"ASTC", "performance"}, {"MEMORY", "performance"},
-    {"TEX_ALIAS", "performance"}, {"TEX_VRAM_PRESSURE", "performance"}, {"ASTC_BC", "performance"}, {"LOD_GUARD", "performance"}, {"FIX_C0", "performance"}, {"FIX_ZERO", "performance"},
-    {"BIND_SKIP", "performance"}, {"SHADER_MEMO", "performance"}, {"PIPE_MEMO", "performance"}, {"TEX_MEMO", "performance"}, {"DESC_TEMPLATE", "performance"},
-    {"PUSH_SKIP", "performance"}, {"SPLIT_DRAWS", "performance"}, {"SPLIT_US", "performance"}, {"VK_VALIDATION", "performance"},
+    {"TEX_ALIAS", "performance"}, {"TEX_VRAM_PRESSURE", "performance"}, {"TEX_RELEASED_MIB", "performance"}, {"TEX_BLOCK_MIB", "performance"}, {"VMA_BLOCK_MIB", "performance"}, {"ASTC_BC", "performance"}, {"LOD_GUARD", "performance"}, {"FIX_C0", "performance"}, {"FIX_ZERO", "performance"},
+    {"BIND_SKIP", "performance"}, {"SHADER_MEMO", "performance"}, {"PIPE_MEMO", "performance"}, {"TEX_MEMO", "performance"}, {"VB_USED", "performance"}, {"DESC_TEMPLATE", "performance"},
+    {"SYNC_AHEAD", "performance"}, {"SHADOW_DEFER", "performance"}, {"PUSH_SKIP", "performance"}, {"SPLIT_DRAWS", "performance"}, {"SPLIT_US", "performance"}, {"VK_VALIDATION", "performance"},
     {"VK_DYN_VERTEX", "performance"}, {"VK_DYN_A2C", "performance"}, {"VK_SUBGROUP_PIN", "performance"},
     {"TRACE", "diagnostics"}, {"FLIGHT", "diagnostics"}, {"PASS_CRUMBS", "diagnostics"}, {"FLIGHT_SPIKE_MS", "diagnostics"},
     {"WB_FENCE_LIMIT", "diagnostics"}, {"WB_MERGE", "diagnostics"}, {"WB_EXACT", "diagnostics"},
@@ -118,6 +120,7 @@ static const CfgMeta metas[] = {
     {"VK_PRESENT", "mailbox", "mailbox|fifo|immediate", LIVE}, {"GPU_INDEX", "", NULL, RESTART},
     {"PERF_OVERLAY", "off", "off|fps|full", LIVE},
     {"OUTPUT_RES", "1920x1080", NULL, RESTART}, {"RENDER_SCALE", "game", "game|50|67|75|100|125|150|200", RESTART},
+    {"DYNAMIC_RES", "1", "0|1", LIVE},   // resolution.c res_dynres_live
     {"UPSCALER", "fsr", "fsr|bilinear", LIVE}, {"SHARPNESS", "80", "0..100", LIVE},
     {"DLAA", "0", "0|1", RESTART}, {"DLSS", "dlaa", "dlaa|quality|balanced|performance|ultra_performance", LIVE},
     {"DLSS_PRESET", "default", "default|k|j", LIVE},
@@ -140,7 +143,7 @@ static const CfgMeta metas[] = {
     {"FPS_LIMIT", "60", "30|40|45|60|90|120|144|display", LIVE}, {"FPS60", "1", "0|1", LIVE}, {"FRAME_QUEUE", "0", "0|1", LIVE},
     {"REALTIME", "1", "0|1", RESTART}, {"CLOTH_REALTIME", "1", "0|1", RESTART},
     {"MODS_REBUILD", "auto", "auto|off", RESTART},
-    {"HIGHFPS_FIX", "1", "0|1", RESTART},
+    {"HIGHFPS_FIX", "1", "0|1", RESTART}, {"CLIMB_FIX", "1", "0|1", RESTART},
     {"LANG", "", "ja|en-US|fr|de|it|es|zh-CN|ko|nl|pt|ru|zh-TW|en-GB|fr-CA|es-419|zh-Hans|zh-Hant|pt-BR", RESTART},
     {"OPERATION_MODE", "docked", "docked|handheld", RESTART},
     {"ASSUME_FOCUS", "1", "0|1", LIVE},   // read on every focus change (ov_system.c)
@@ -162,6 +165,10 @@ static const CfgMeta metas[] = {
     {"DISCORD_IMAGE", "totk_512x512", NULL, LIVE},
     // Advanced: scheduling
     {"ASYNC_PREEMPT", "1", "0|1|2", RESTART}, {"REPLAY_ASYNC", "0", "0|1", RESTART},
+    {"REPLAY_WORKERS", "0", "0..16", RESTART}, {"REPLAY_CHUNKS", "0", "0..64", RESTART},
+    {"REPLAY_CHUNK_DRAWS", "128", "1..100000", RESTART}, {"REPLAY_STATE_CHECK", "0", "0|1", RESTART},
+    {"REPLAY_SKEW_US", "0", "0..100000", RESTART},
+    {"SYNC_AHEAD", "0", "0|1", RESTART}, {"SHADOW_DEFER", "1", "0|1", RESTART},
     {"FREE_CORES", "0", "0|1", RESTART}, {"TIMER_1MS", "1", "0|1", RESTART},
     // shaders and pipelines
     {"PIPE_WAIT", "default", "default|draw", RESTART}, {"PIPE_SKIP", "fullscreen", "strict|any|fullscreen", RESTART},
@@ -169,10 +176,12 @@ static const CfgMeta metas[] = {
     {"PIPE_PREBUILD", "1", "0|1", RESTART}, {"PREWARM_XLATE", "", "1..64", RESTART}, {"GPL", "1", "0|1", RESTART},
     // textures and memory
     {"ASTC", "auto", "auto|gpu|native|cpu", RESTART}, {"MEMORY", "auto", "auto|import|mirror|verify", RESTART},
-    {"TEX_ALIAS", "1", "0|1", RESTART}, {"TEX_VRAM_PRESSURE", "90", "0..100", RESTART}, {"ASTC_BC", "", "0|1", RESTART}, {"ZSTD_NATIVE", "1", "0|1", RESTART}, {"ZSTD_CHECK", "0", "0|1", RESTART},
+    {"TEX_ALIAS", "1", "0|1", RESTART}, {"TEX_VRAM_PRESSURE", "90", "0..100", RESTART},
+    {"TEX_RELEASED_MIB", "256", "0..65536", RESTART}, {"TEX_BLOCK_MIB", "64", "0..1024", RESTART}, {"VMA_BLOCK_MIB", "64", "0..1024", RESTART}, {"ASTC_BC", "", "0|1", RESTART}, {"ZSTD_NATIVE", "1", "0|1", RESTART}, {"ZSTD_CHECK", "0", "0|1", RESTART},
     // render fixes
     {"LOD_GUARD", "1", "0|1", RESTART}, {"FIX_C0", "1", "0|1", RESTART}, {"FIX_ZERO", "1", "0|1", RESTART},
     {"BIND_SKIP", "1", "0|1", RESTART}, {"SHADER_MEMO", "1", "0|1", RESTART}, {"PIPE_MEMO", "1", "0|1", RESTART}, {"TEX_MEMO", "1", "0|1", RESTART},
+    {"VB_USED", "1", "0|1", RESTART},
     {"DESC_TEMPLATE", "1", "0|1", RESTART},
     {"PUSH_SKIP", "1", "0|1", RESTART}, {"SPLIT_DRAWS", "64", "0..1000", RESTART},
     {"SPLIT_US", "1000", "0..20000", RESTART},
@@ -335,8 +344,10 @@ static const char *raw_get(const char *name) {
 // picture between black bars. The sizes are the ones the installer lays the HUD out for (installer/steps.py
 // ASPECT_SIZES, ui/menu.cpp row_output_res): of the display's shape within 1%, the smallest at least 1080 high that fits
 // the display, else the largest that fits. A 16:9 display, an unknown one or one of another shape keeps 1920x1080.
-// Xbox or keyboard prompts keep 1920x1080 too: the setup builds their folders for 16:9 only (romfs.c overlay_init), so
-// a default of another shape would trade the prompts the player chose for the Switch ones.
+// Xbox and keyboard prompts have a folder for each of these sizes (romfs-patches-xbox-<w>x<h>, -xbox-label-<w>x<h>,
+// -pc-<w>x<h>, installer/steps.py Prepare): they keep 1920x1080 only when the one for the display's size is missing,
+// as in a source build without make_romfs_patches.py --prompts <style> --aspect or an install by an older setup,
+// where a default of another shape would trade the prompts the player chose for the Switch ones.
 static const unsigned SHAPED[][2] = {
     {2560, 1080}, {3440, 1440}, {3840, 1600}, {5120, 2160},             // 21:9
     {3840, 1080}, {5120, 1440},                                         // 32:9
@@ -344,10 +355,22 @@ static const unsigned SHAPED[][2] = {
     {1920, 1280}, {2160, 1440}, {2256, 1504}, {2880, 1920}, {3000, 2000}, // 3:2
 };
 static char screen_res[16];   // set once by cfg_init, "" for none
+// romfs-patches-xbox-, romfs-patches-xbox-label- and romfs-patches-pc-<screen_res> have a manifest (set once by
+// cfg_init)
+static const char *const screen_folders[3] = {"romfs-patches-xbox", "romfs-patches-xbox-label", "romfs-patches-pc"};
+static int screen_prompts[3];
 
-static int switch_prompts(void) {
+// Whether the prompts in effect have their folder at screen_res. The same picks as input_map.c input_prompts_pick and
+// input_prompts_folder: an unknown or empty value is the Nintendo prompts, an unset one Xbox while the pad layout is
+// "xbox_prompts"; the Xbox prompts take romfs-patches-xbox with the Nintendo pad layout (or "xbox_prompts") and
+// romfs-patches-xbox-label with any other.
+static int shape_keeps_prompts(void) {
     const char *p = raw_get(PREFIX "PROMPTS"), *l = raw_get(PREFIX "PAD_LAYOUT");
-    return (!p || !strcasecmp(p, "nintendo")) && !(l && !strcasecmp(l, "xbox_prompts"));
+    if (p && *p && !strcasecmp(p, "pc")) return screen_prompts[2];
+    int xbox = p && *p ? !strcasecmp(p, "xbox") : l && !strcasecmp(l, "xbox_prompts");
+    if (!xbox) return 1;
+    int by_position = l && (!strcasecmp(l, "nintendo") || !strcasecmp(l, "xbox_prompts"));
+    return screen_prompts[by_position ? 0 : 1];
 }
 
 static void screen_default(void) {
@@ -365,7 +388,9 @@ static void screen_default(void) {
     if (best < 0) best = fit;
     if (best < 0) return;
     snprintf(screen_res, sizeof screen_res, "%ux%u", SHAPED[best][0], SHAPED[best][1]);
-    if (!raw_get(PREFIX "OUTPUT_RES") && switch_prompts())
+    for (int i = 0; i < 3; i++)
+        screen_prompts[i] = romfs_shaped_prompts_ready(screen_folders[i], SHAPED[best][0], SHAPED[best][1]);
+    if (!raw_get(PREFIX "OUTPUT_RES") && shape_keeps_prompts())
         rt_log("cfg: the display is %dx%d: output_res defaults to %s, its shape (totk.ini output_res sets another)", sw,
                sh, screen_res);
 }
@@ -374,7 +399,8 @@ const char *cfg_get(const char *name) {
     const char *v = raw_get(name);
     if (v || strncmp(name, PREFIX, PREFIX_LEN)) return v;
     if (!profile_of())
-        return screen_res[0] && !strcasecmp(name + PREFIX_LEN, "OUTPUT_RES") && switch_prompts() ? screen_res : NULL;
+        return screen_res[0] && !strcasecmp(name + PREFIX_LEN, "OUTPUT_RES") && shape_keeps_prompts() ? screen_res
+                                                                                                         : NULL;
     for (size_t i = 0; i < sizeof DECK / sizeof *DECK; i++)
         if (!strcasecmp(DECK[i].key, name + PREFIX_LEN)) return DECK[i].value;
     return NULL;

runtime/cfg_probe.c

@@ -230,6 +230,16 @@ int probe_cfg(void) {
     dn[KEY_MOUSE_R] = dn[KEY_Z] = 1;
     CHECK(input_map_buttons(dn) == (1ull << B_Y | 1ull << B_ZL), "keys: right mouse button and Z both press ZL");
     memset(dn, 0, sizeof dn);
+    dn[KEY_T] = 1;
+    CHECK(input_map_buttons(dn) == 1ull << B_Y, "keys: T is Y too (the left button is A in the menus)");
+    plat_setenv("TOTK_BIND_X", "key:t");
+    input_map_configure();
+    CHECK(input_map_buttons(dn) == 1ull << B_X, "bind: bind_X = key:t takes T from Y's defaults");
+    dn[KEY_T] = 0, dn[KEY_MOUSE_L] = 1;
+    CHECK(input_map_buttons(dn) == 1ull << B_Y, "bind: Y keeps the left mouse button");
+    plat_setenv("TOTK_BIND_X", NULL);
+    input_map_reset();
+    memset(dn, 0, sizeof dn);
     dn[KEY_X] = dn[KEY_TAB] = dn[KEY_M] = dn[KEY_LCTRL] = dn[KEY_3] = 1;
     CHECK(input_map_buttons(dn) == (1ull << B_ZR | 1ull << B_PLUS | 1ull << B_MINUS | 1ull << B_STICK_L | 1ull << B_RIGHT),
           "keys: X ZR, Tab Plus, M Minus, Left Ctrl stick click, 3 D-pad right");

runtime/cheats.c

@@ -10,8 +10,9 @@
 // (tools/cheats.py has the format). A cheat whose status line is missing or has another content hash was added or
 // edited after the build: a data cheat runs anyway, a code cheat waits for the setup to run again.
 //
-// Data cheats run in a VM after Atmosphere's dmnt cheat VM (opcodes 0-A; the 0xC.. and 0xF.. extensions are not
-// supported), every presented frame while their switch is on, with registers cleared before each cheat. Memory is the
+// Data cheats run in a VM after Atmosphere's dmnt cheat VM (opcodes 0-A and C0-C3; C4 and the 0xF.. extensions are not
+// supported), every presented frame while their switch is on, with registers and saved registers cleared before each
+// cheat; the static registers (C3) stay, shared by all cheats, as dmnt keeps them. Memory is the
 // guest's, mapped at its own addresses: main's image is written with its pages made writable for the write (read-only
 // data included, as dmnt writes through the debugger), everything else after mem_query says the guest may read or write
 // it. Buttons are input_poll's HidNpadButton bits, the same as the cheat format's key masks; they read 0 while the menu
@@ -53,6 +54,7 @@ static int g_n, g_cap, g_vm;   // g_vm: some cheat needs cheats_frame
 static char g_file[64];        // "<build id>.txt"
 static char g_dir[PLAT_PATH_MAX];   // the folder of totk.exe, with a trailing separator
 static int g_files;            // mods whose file was read
+static uint64_t g_static[256]; // C3's static registers: 0x00-0x7F loaded by cheats, 0x80-0xFF stored by them
 static uint64_t g_main, g_main_size, g_code_end;   // main's image; its code with the padding up to .rodata
 
 static uint32_t fnv(const char *s, size_t n) {
@@ -96,6 +98,11 @@ static int op_len(const uint32_t *w, int left) {
     case 5: case 7: n = 2; break;
     case 9: n = 1 + (((f >> 8) & 0xF) ? (width == 8 ? 2 : 1) : 0); break;
     case 0xA: n = ((f >> 8) & 0xF) == 2 || ((f >> 8) & 0xF) == 4 || ((f >> 8) & 0xF) == 5 ? 2 : 1; break;
+    case 0xC:   // C0TcSX..: an address word for operand types 0 and 2, the value (two words at width 8) for 4
+        if (((f >> 24) & 0xF) != 0) n = 1;
+        else if (((f >> 8) & 0xF) == 0 || ((f >> 8) & 0xF) == 2) n = 2;
+        else n = ((f >> 8) & 0xF) == 4 ? (((f >> 20) & 0xF) == 8 ? 3 : 2) : 1;
+        break;
     default: n = 1; break;
     }
     return n <= left ? n : 0;
@@ -105,12 +112,15 @@ static int op_len(const uint32_t *w, int left) {
 static const char *vm_unsupported(const uint32_t *w, char *buf, int n) {
     uint32_t f = w[0];
     int t = f >> 28, width = (f >> 24) & 0xF;
-    if (t > 0xA || (t == 5 && ((f >> 12) & 0xF) > 1) || (t == 0xA && ((f >> 8) & 0xF) > 5)) {
+    int ext = f >> 24;   // 0xC0-0xC3 run; C0's width is at bits 20-23 and its operand type (at most 5) at bits 8-11
+    if (ext == 0xC0) t = -1, width = (f >> 20) & 0xF;
+    if ((t > 0xA && (ext < 0xC1 || ext > 0xC3)) || (t == -1 && ((f >> 8) & 0xF) > 5) ||
+        (t == 5 && ((f >> 12) & 0xF) > 1) || (t == 0xA && ((f >> 8) & 0xF) > 5)) {
         snprintf(buf, n, "opcode %08X is not supported", f);
         return buf;
     }
     if ((t <= 1 || t == 5 || t == 6 || t == 7 || t == 9 || t == 0xA) && width != 1 && width != 2 && width != 4 &&
-        width != 8) {
+        width != 8) {   // C0 too (t -1)
         snprintf(buf, n, "opcode %08X has a bad width", f);
         return buf;
     }
@@ -227,7 +237,9 @@ static int compare(int cond, uint64_t a, uint64_t b) {
 }
 
 // From i (after a failed condition, or an else reached while running), the opcode to continue at: after the
-// matching else (else_ok) or end.
+// matching else (else_ok) or end. Conditions 1, 8 and C0 open a block, as in Atmosphere's dmnt
+// (stratosphere/dmnt/source/cheat/impl/dmnt_cheat_vm.cpp, CheatVirtualMachine::SkipConditionalBlock), which steps
+// over whole opcodes so that an operand word reading 20000000 ends nothing.
 static int skip(const uint32_t *w, int n, int i, int else_ok) {
     int depth = 0;
     while (i < n) {
@@ -236,17 +248,41 @@ static int skip(const uint32_t *w, int n, int i, int else_ok) {
         uint32_t f = w[i];
         int t = f >> 28;
         i += len;
-        if (t == 1 || t == 8) depth++;
-        else if (t == 2 && ((f >> 24) & 0xF) == 0 && depth-- == 0) return i;
-        else if (t == 2 && depth == 0 && else_ok) return i;
+        if (t == 1 || t == 8 || f >> 24 == 0xC0) depth++;
+        else if (t == 2 && ((f >> 24) & 0xF) == 1) {   // else; any other 2x000000 ends, as dmnt decodes is_else
+            if (depth == 0 && else_ok) return i;
+        } else if (t == 2 && depth-- == 0) {
+            return i;
+        }
     }
     return n;
 }
 
 static uint64_t imm64(const uint32_t *w) { return (uint64_t)w[0] << 32 | w[1]; }
 
+// C0TcSX..: if register S <cond c> operand X, all cut to width T; the operand as Atmosphere's dmnt reads it
+// (stratosphere/dmnt/source/cheat/impl/dmnt_cheat_vm.cpp, CheatVirtualMachine::DecodeNextOpcode and Execute,
+// BeginRegisterConditionalBlock). X: 0 region M + a, 1 region M + register r, 2 register R + a, 3 register R +
+// register r, 4 the value V, 5 register R. M, R at bits 4-7, r and the top bits of a at bits 0-3. A read the guest
+// cannot make compares as 0: dmnt ignores the read's result and compares the zero it started with.
+static int reg_cond(const uint32_t *w, const uint64_t *r) {
+    uint32_t f = w[0];
+    int width = (f >> 20) & 0xF, cond = (f >> 16) & 0xF, a4 = (f >> 4) & 0xF, a0 = f & 0xF;
+    uint64_t src = masked(width, r[(f >> 12) & 0xF]), b = 0, addr;
+    switch ((f >> 8) & 0xF) {   // w[1] exists for types 0, 2 and 4 only (op_len)
+    case 0: addr = region(a4) + (((uint64_t)a0 << 32) | w[1]); break;
+    case 1: addr = region(a4) + r[a0]; break;
+    case 2: addr = r[a4] + (((uint64_t)a0 << 32) | w[1]); break;
+    case 3: addr = r[a4] + r[a0]; break;
+    case 4: return compare(cond, src, width == 8 ? imm64(w + 1) : masked(width, w[1]));
+    default: return compare(cond, src, masked(width, r[a4]));
+    }
+    if (!rd(addr, width, &b)) b = 0;
+    return compare(cond, src, b);
+}
+
 static void run(const Cheat *c, uint64_t keys) {
-    uint64_t r[16] = {0};
+    uint64_t r[16] = {0}, saved[16] = {0};
     int loop[16] = {0};
     const uint32_t *w = c->words;
     int n = c->n, budget = OPS_PER_RUN;
@@ -323,6 +359,44 @@ static void run(const Cheat *c, uint64_t keys) {
             if (z) r[y] += (uint64_t)width;
             break;
         }
+        case 0xC:   // C1-C3 as dmnt's Execute runs SaveRestoreRegister(Mask) and ReadWriteStaticRegister
+            switch (f >> 24) {
+            case 0xC0:
+                if (!reg_cond(w + i, r)) {
+                    i = skip(w, n, i + len, 1);
+                    continue;
+                }
+                break;
+            case 0xC1: {   // C10D0Sx0: x 0 register D = saved S, 1 saved D = register S, 2 saved D = 0,
+                           // 3 register D = 0
+                int d = (f >> 16) & 0xF, s = (f >> 8) & 0xF, op = (f >> 4) & 0xF;
+                if (op == 1) saved[d] = r[s];
+                else if (op == 2) saved[d] = 0;
+                else if (op == 3) r[d] = 0;
+                else r[d] = saved[s];   // dmnt restores for any other x too
+                break;
+            }
+            case 0xC2: {   // C2x0XXXX: for each register in the mask X, x as C1 with D = S = that register
+                int op = (f >> 20) & 0xF;
+                for (int k = 0; k < 16; k++) {
+                    if (!(f >> k & 1)) continue;
+                    if (op == 1) saved[k] = r[k];
+                    else if (op == 2) saved[k] = 0;
+                    else if (op == 3) r[k] = 0;
+                    else r[k] = saved[k];
+                }
+                break;
+            }
+            case 0xC3: {   // C3000XXx: static register XX < 0x80 into register x, else register x into it
+                int s = (f >> 4) & 0xFF;
+                if (s < 0x80) r[f & 0xF] = g_static[s];
+                else g_static[s] = r[f & 0xF];
+                break;
+            }
+            default:
+                return;
+            }
+            break;
         default:
             return;
         }

runtime/frame_rate.c

@@ -91,6 +91,11 @@ void frame_step_init(void) {
         rt_patch_disable(cloth);
         rt_log("frame rate: cloth clock off, it needs the real-time step (TOTK_REALTIME=0)");
     }
+    int climb = rt_patch_set("climb");   // reads the page's frame ratio, written only with the real-time step
+    if (climb >= 0 && !rt_patch_enabled(realtime) && rt_patch_enabled(climb)) {
+        rt_patch_disable(climb);
+        rt_log("frame rate: climb speed fix off, it needs the real-time step (TOTK_REALTIME=0)");
+    }
     if (rt_patch_enabled(realtime)) {
         step_word = (_Atomic uint64_t *)rt_patch_page();
         step_interval = (_Atomic uint32_t *)((uint8_t *)rt_patch_page() + 8);

runtime/gfx_present.c

@@ -33,9 +33,20 @@
 #define VSYNC_HZ 60
 
 // ---- queue lock
+// The recorder queues the parts it records and submits them later (vk_rec.c rec_submit): the first lock of a thread
+// waits until the parts queued before it are submitted, so whatever it puts on the queue comes after them, as when
+// the submitting thread submitted them itself. Not inside a held lock: the recorder takes this lock to submit.
 static PlatRMutex g_ql;
-void vk_queue_lock(void) { plat_rmutex_lock(&g_ql); }
-void vk_queue_unlock(void) { plat_rmutex_unlock(&g_ql); }
+static _Thread_local int ql_depth;
+void vk_queue_lock(void) {
+    if (!ql_depth) rec_queue_wait();
+    plat_rmutex_lock(&g_ql);
+    ql_depth++;
+}
+void vk_queue_unlock(void) {
+    ql_depth--;
+    plat_rmutex_unlock(&g_ql);
+}
 
 // ---- window state shared with the back end
 static VkSurfaceKHR g_surface;
@@ -72,6 +83,7 @@ void gfx_window_closed(void) {
     rt_log("gfx: window closed; %s", restart ? "restarting" : "exiting");
     fs_save_flush();
     vk_pipeline_save();
+    playtime_save();
     prof_write();   // plat_exit runs no atexit handlers
     if (restart) {
         plat_setenv("TOTK_REEXEC", NULL);   // the new process counts its own address-space retries (memory.c)
@@ -235,6 +247,7 @@ static void window_settings_live(void) {
     if (started && v == seen) return;
     seen = v;
     res_fov_live();
+    res_dynres_live();
     const char *e = cfg_value("TOTK_VK_PRESENT");
     if (!started || strncmp(e, present, sizeof present - 1)) {
         snprintf(present, sizeof present, "%s", e);

runtime/host.h

@@ -36,6 +36,10 @@ void perf_end(int kind, uint64_t t0);           // counts render-thread time onl
 void perf_replay(uint64_t t0);                  // replay (NVN -> Vulkan recording) time on any thread, waits inside it included
 void perf_replay_thread(void);                  // the calling thread replays for the render thread: perf_end counts its waits
 void perf_main_thread(void);                    // the calling thread is the guest main thread (TOTK_PROFILE_MAIN)
+// The calling thread is one every frame goes through: on a hybrid CPU it prefers the P-cores (main.c, TOTK_PCORE);
+// the first thread of each role logs it.
+enum { PCORE_MAIN, PCORE_RENDER, PCORE_REPLAY, PCORE_RECORDER, PCORE_PRESENT, PCORE_N };
+void perf_core_thread(int role);
 // Events per 60-frame window: graphics pipelines finished by the background workers, draws skipped while pending,
 // first texture uploads and re-uploads after CPU writes, draws skipped while a shader translates in the background,
 // draws that waited for their pending pipeline and got it, draws into a target the previous frame did not render that
@@ -171,6 +175,23 @@ int mem_guest_readable(uint64_t a, uint64_t n);            // [a, a+n) can be re
                                                            // module or the committed heap answered from the layout,
                                                            // anything else by plat_mem_readable (an OS query)
 
+// ---- short out-of-memory spells (oom.c)
+enum { RT_OOM_COMMIT, RT_OOM_STAGING, RT_OOM_IMAGE, RT_OOM_CHUNK, RT_OOM_SITES };
+extern volatile int32_t rt_oom_req;                        // set by rt_oom_request; vk_texture_new_submit takes it
+void rt_oom_request(void);                                 // asks the Vulkan back end to give memory back at its next
+                                                           // submit (released textures, headroom blocks); any thread,
+                                                           // any lock held
+int rt_oom_wait(const char *what, uint64_t bytes, int (*retry)(void *arg), void *arg);
+                                                           // after a failed allocation: requests a release, then calls
+                                                           // retry every 10 ms for up to 30 s; 1 once retry returned 1,
+                                                           // 0 after the wait (the caller ends the game). Logs the
+                                                           // first 8 with the commit numbers
+void *rt_realloc_wait(void *old, size_t n, const char *what);   // realloc, NULL only after rt_oom_wait gave up
+int rt_oom_forced(int site);                               // 1: fail this attempt at site (TOTK_TEST_OOM=<site>:<n>,
+                                                           // sites commit, staging, image, chunk; or rt_oom_force)
+void rt_oom_force(int site, int n);                        // probes: fail the next n attempts at site
+int rt_oom_forced_left(int site);                          // forced failures still to come at site
+
 // ---- modules (loader.c)
 typedef struct {
     const char *name; uint64_t base, size, text_end, data_off;   // text_end, data_off: offsets from base
@@ -260,6 +281,12 @@ void cam_pack_served(void);                                // romfs.c serves the
 // ---- game resolution (resolution.c): TOTK_OUTPUT_RES, TOTK_RENDER_SCALE
 uint64_t res_heap_extra_bytes(void);                       // added to the application pool (mem_init), 2 MiB steps
 uint64_t res_heap_extra_kib(void);                         // added to the HeapDef texture heap and its parent; 0 = none
+uint64_t res_tex_heap_extra_bytes(void);                   // TOTK_TEX_HEAP_EXTRA_MB: added to the application pool and
+                                                           // the heap cap (mem_init), 2 MiB steps; 0 = none
+int res_heapdef_tex_grown(const void *in, size_t n);       // the HeapDef table (.byml.zs) grows TextureHeap over the
+                                                           // shipped size (an HD texture pack's)
+int res_heapdef_plain(const void *in, size_t n);           // a readable HeapDef table whose TextureHeap is not
+                                                           // grown (a content mod's)
 int res_heapdef_build(const void *in, size_t n, void **out, size_t *out_n);   // the HeapDef table (.byml.zs) with
                                                            // the grown heaps, malloc'd; 0 = cannot (logged)
 void res_heapdef_served(void);                             // romfs.c serves the grown table
@@ -272,8 +299,13 @@ void res_sizes(unsigned out[2], unsigned scene[2]);      // the output and scene
                                                            // until res_init switched the sets on)
 int res_is_16x9(unsigned w, unsigned h);                   // within 1% of 16:9 (the test res_init and romfs.c share)
 int res_wants_wide_overlay(const unsigned out[2]);         // not 16:9 on a game version with the camera aspect patch
+int romfs_shaped_ready(unsigned w, unsigned h);            // romfs-patches-<w>x<h> has a manifest (romfs.c; works
+                                                           // before fs_init)
+int romfs_shaped_prompts_ready(const char *folder, unsigned w, unsigned h);   // the same for <folder>-<w>x<h>, a
+                                                           // prompts' folder (romfs-patches-xbox, -xbox-label, -pc)
 void res_wanted(unsigned out[2]);                          // the output size TOTK_OUTPUT_RES asks for
 void res_fov_live(void);                                   // TOTK_FOV to the camera (set fov); on a settings change
+void res_dynres_live(void);                                // TOTK_DYNAMIC_RES to the game (set fixres); on a settings change
 // ---- code patches (patches.c) and the frame step (frame_rate.c)
 void rt_patches_init(void);                                // after ldr_load: switches from the environment, the patch
                                                            // page committed; logs the sets
@@ -316,6 +348,10 @@ int probe_cfg(void);                                       // cfg_probe.c
 int probe_dpi(void);                                       // dpi_probe_win.c (Windows only)
 int probe_mods(void);                                      // mods_probe.c
 void discord_init(void);                                   // discord_rpc.c: the Rich Presence thread, game runs only
+void playtime_init(void);                                  // playtime.c: the play time thread, game runs only
+void playtime_save(void);                                  // playtime.c: writes <profile>/playtime.txt; at exit
+int playtime_get(uint64_t *total_s, uint64_t *session_s); // playtime.c: all runs and this one; 0 without a profile
+int probe_playtime(void);                                  // playtime.c: the file, the ticks and the save
 int probe_discord(void);                                   // discord_rpc.c
 int probe_loader(void), probe_kernel(void), probe_ipc(void), probe_services(void), probe_fs(void);
 int probe_gfx_abi(void);                                   // gfx_abi_probe.c

runtime/hostfn.c

@@ -35,7 +35,7 @@ typedef struct {
     const char *name;
     GuestFn fn;
     uint64_t original;   // observers: the guest definition rt_original calls
-    uint64_t calls;
+    uint64_t calls;      // with TOTK_TRACE=host only (hostfn_report)
     int blocking;        // runs with the virtual core given up (hostfn_blocking)
 } Slot;
 static Slot slots[MAX_HOSTFN];
@@ -44,6 +44,21 @@ static RtHostEntry added[MAX_ADDED];
 static int nadded;
 static int sealed;
 
+// The call count (hostfn_calls): one counter per calling thread, each on its own cache line, written only by that
+// thread. A locked add on a per-function counter beside the read-mostly slot fields made the game threads share and
+// bounce those lines on every host call (1.4.3 town: about 1% of the game threads' time in hostfn_entry).
+#define MAX_COUNTERS 4096
+typedef struct { _Alignas(64) uint64_t n; } Counter;
+static Counter counters[MAX_COUNTERS];
+static uint32_t ncounters;
+static uint64_t shared_calls;   // the threads past MAX_COUNTERS, counted with an atomic add
+static __thread uint64_t *thread_calls;
+
+__attribute__((noinline)) static uint64_t *counter_new(void) {
+    uint32_t i = __atomic_fetch_add(&ncounters, 1, __ATOMIC_RELAXED);
+    return thread_calls = i < MAX_COUNTERS ? &counters[i].n : &shared_calls;
+}
+
 static GuestFn table_find(const RtHostEntry *start, const RtHostEntry *stop, const char *name) {
     __asm__("" : "+r"(start), "+r"(stop));   // separate objects: keep the compiler from reasoning about them
     for (const RtHostEntry *p = start; p < stop; p++)
@@ -146,8 +161,11 @@ static void hostfn_entry(Cpu *c) {
     uint64_t i = (c->pc - RT_HOSTFN_BASE) >> 4;
     if (__builtin_expect((c->pc & 15) || i >= (uint64_t)nslots, 0)) bad_entry(c);
     Slot *s = &slots[i];
-    uint64_t n = __atomic_fetch_add(&s->calls, 1, __ATOMIC_RELAXED);
-    if (__builtin_expect(!n && (rt_trace & RT_TRACE_HOSTCALL), 0)) {
+    uint64_t *k = thread_calls;
+    if (__builtin_expect(!k, 0)) k = counter_new();
+    if (__builtin_expect(k == &shared_calls, 0)) __atomic_fetch_add(k, 1, __ATOMIC_RELAXED);
+    else __atomic_store_n(k, *k + 1, __ATOMIC_RELAXED);
+    if (__builtin_expect(rt_trace & RT_TRACE_HOSTCALL, 0) && !__atomic_fetch_add(&s->calls, 1, __ATOMIC_RELAXED)) {
         char lr[96];
         rt_log("host call %s from %s", s->name, ldr_describe(c->x[30] - 4, lr, sizeof lr));
     }
@@ -178,8 +196,9 @@ void hostfn_seal(void) {
 }
 
 uint64_t hostfn_calls(void) {
-    uint64_t n = 0;
-    for (int i = 0; i < nslots; i++) n += __atomic_load_n(&slots[i].calls, __ATOMIC_RELAXED);
+    uint32_t m = __atomic_load_n(&ncounters, __ATOMIC_RELAXED);
+    uint64_t n = __atomic_load_n(&shared_calls, __ATOMIC_RELAXED);
+    for (uint32_t i = 0; i < m && i < MAX_COUNTERS; i++) n += __atomic_load_n(&counters[i].n, __ATOMIC_RELAXED);
     return n;
 }
 

runtime/input.c

@@ -623,17 +623,22 @@ static void poll_merged(uint64_t *buttons, int32_t *lx, int32_t *ly, int32_t *rx
         if (ikb_foreground()) {
             uint8_t down[KEY_COUNT] = {0};
             // Mouse buttons count only while the cursor is captured, and a button already held when the capture began
-            // (the click that refocused the window) is ignored until it is released.
+            // (the click that refocused the window) is ignored until it is released. While a game menu has the pointer
+            // (menu_cursor.c) and it is over the window, the left and right buttons are that menu's A and B, and the
+            // middle and side buttons keep their bindings: bound to L and R they turn the menu's tabs.
             static int was_captured;
             static uint8_t mouse_ignore[KEY_COUNT];
             int cap = ikb_captured();
+            float pointer_x, pointer_y;
+            int menu_pointer = !cap && menu_cursor_pointer() && ikb_pointer(&pointer_x, &pointer_y);
             for (int k = 1; k < KEY_COUNT; k++) {
                 down[k] = ikb_down(k);
                 kbm_used |= down[k];
                 if (k < KEY_MOUSE_L || k > KEY_MOUSE_X2) continue;
                 if (cap && !was_captured) mouse_ignore[k] = down[k];
                 if (!down[k]) mouse_ignore[k] = 0;
-                if (!cap || mouse_ignore[k]) down[k] = 0;
+                int counts = cap || (menu_pointer && k != KEY_MOUSE_L && k != KEY_MOUSE_R);
+                if (!counts || mouse_ignore[k]) down[k] = 0;
             }
             was_captured = cap;
             b |= input_map_buttons(down);

runtime/input_map.c

@@ -16,7 +16,7 @@
 #define DEFAULT_KEYS \
     {KEY_W, B_SL_UP}, {KEY_A, B_SL_LEFT}, {KEY_S, B_SL_DOWN}, {KEY_D, B_SL_RIGHT},                  /* move */ \
     {KEY_UP, B_SR_UP}, {KEY_LEFT, B_SR_LEFT}, {KEY_DOWN, B_SR_DOWN}, {KEY_RIGHT, B_SR_RIGHT},       /* camera, second source */ \
-    {KEY_MOUSE_L, B_Y}, {KEY_MOUSE_R, B_ZL}, {KEY_Z, B_ZL}, {KEY_F, B_ZR}, {KEY_X, B_ZR}, \
+    {KEY_MOUSE_L, B_Y}, {KEY_T, B_Y}, {KEY_MOUSE_R, B_ZL}, {KEY_Z, B_ZL}, {KEY_F, B_ZR}, {KEY_X, B_ZR}, \
     {KEY_SPACE, B_X}, {KEY_LSHIFT, B_B}, {KEY_E, B_A}, {KEY_Q, B_L}, {KEY_R, B_R}, \
     {KEY_LCTRL, B_STICK_L}, {KEY_C, B_STICK_R}, \
     {KEY_1, B_LEFT}, {KEY_2, B_UP}, {KEY_3, B_RIGHT}, {KEY_4, B_DOWN}, \
@@ -31,7 +31,7 @@
     {PAD_ZR, B_ZR}
 #define NINTENDO_PADS DEFAULT_PADS(B_B, B_A, B_Y, B_X)
 #define XBOX_PADS DEFAULT_PADS(B_A, B_B, B_X, B_Y)
-enum { DEFAULT_KEY_COUNT = 27, DEFAULT_PAD_COUNT = 16 };
+enum { DEFAULT_KEY_COUNT = 28, DEFAULT_PAD_COUNT = 16 };
 _Static_assert(sizeof (Binding[]){DEFAULT_KEYS} / sizeof (Binding) == DEFAULT_KEY_COUNT, "DEFAULT_KEY_COUNT");
 
 static const InputMap defaults = {{DEFAULT_KEYS}, DEFAULT_KEY_COUNT, {{NINTENDO_PADS}, {XBOX_PADS}},
@@ -166,11 +166,27 @@ int input_map_bind(const char *button, const char *list) {
 
 void input_map_configure(void) {
     input_map_reset();
+    uint8_t lined[ACTION_COUNT] = {0};
     for (int a = 0; a < ACTION_COUNT; a++) {
         char var[32];
         snprintf(var, sizeof var, "TOTK_BIND_%s", action_names[a]);
         const char *v = cfg_get(var);
-        if (v && *v && input_map_bind(action_names[a], v)) rt_log("input: %s = %s", var + 5, v);
+        if (v && *v && input_map_bind(action_names[a], v)) lined[a] = 1, rt_log("input: %s = %s", var + 5, v);
+    }
+    // A key a line binds leaves the defaults of the actions without a line, as the remapping rows move a key: with
+    // bind_X = key:t, T presses X only, not also Y, whose defaults hold it.
+    uint8_t bound[KEY_COUNT] = {0}, list[INPUT_BINDING_MAX];
+    for (int a = 0; a < ACTION_COUNT; a++) {
+        if (!lined[a]) continue;
+        int n = input_map_keys_of(&input_map, a, list, INPUT_BINDING_MAX);
+        for (int i = 0; i < n; i++) bound[list[i]] = 1;
+    }
+    for (int a = 0; a < ACTION_COUNT; a++) {
+        if (lined[a]) continue;
+        int n = input_map_keys_of(&input_map, a, list, INPUT_BINDING_MAX), w = 0;
+        for (int i = 0; i < n; i++)
+            if (!bound[list[i]]) list[w++] = list[i];
+        if (w < n) keys_put(&input_map, a, list, w);
     }
 }
 

runtime/input_map.h

@@ -80,6 +80,7 @@ const char *input_pad_name(int pad);
 // skipped; when none is valid the action keeps its bindings. 1 when applied.
 int input_map_bind(const char *button, const char *list);
 // Applies the TOTK_BIND_<action> settings (cfg_get) over the defaults: at start, and again when one of them changes.
+// A key a line binds leaves the defaults of the actions that have no line.
 void input_map_configure(void);
 
 // The layout for a TOTK_PAD_LAYOUT value: "nintendo" or "xbox" (any case); "xbox_prompts", 0.1.9's name for the Nintendo

runtime/kernel.c

@@ -246,6 +246,44 @@ static void wake_sync_waiters(void) {
     }
 }
 
+// ---- a short poll before a sleep
+// Most guest waits and waits for a virtual core end within microseconds. The thread polls for TOTK_KSPIN_US (default
+// 20) with G released before it sleeps on its condition variable: a wake in that time costs no OS wake-up (4-12 us
+// on a desktop Ryzen, more from a deep C-state) and the waker's signal finds nobody asleep. Off with fewer than 6 logical
+// processors, where the poll would take a CPU that a running thread needs; TOTK_KSPIN_US=0 turns it off. Savage Wind
+// Recomp polls the same way before it parks (runtime/ksync.c cond_wait). TOTK_KSPIN_US=ab polls 20 us in the
+// B blocks of TOTK_AB only.
+static int64_t g_kspin_ns = -1;
+static int g_kspin_ab;
+
+static uint64_t kspin_ns(void) {
+    int64_t v = __atomic_load_n(&g_kspin_ns, __ATOMIC_RELAXED);
+    if (v < 0) {
+        const char *e = cfg_get("TOTK_KSPIN_US");
+        g_kspin_ab = e && !strcmp(e, "ab");
+        long long us = e && *e && !g_kspin_ab ? atoll(e) : 20;
+        uint32_t cpus = plat_cpu_usable();
+        v = us <= 0 || cpus < 6 ? 0 : us * 1000;
+        __atomic_store_n(&g_kspin_ns, v, __ATOMIC_RELAXED);
+        rt_log("kernel: waits poll %lld us before they sleep%s (TOTK_KSPIN_US, %u logical processors)",
+               (long long)(v / 1000), g_kspin_ab ? " in the B blocks" : "", cpus);
+    }
+    return v && (!g_kspin_ab || rt_ab()) ? (uint64_t)v : 0;
+}
+
+// Holding G: releases it and polls *p until it reaches at_least or ns pass, then takes G again. 1 if it was reached.
+static int kspin(const int *p, int at_least, uint64_t ns) {
+    uint64_t end = kern_now_ns() + ns;
+    int ok = 0;
+    unlock();
+    for (unsigned i = 1; !(ok = __atomic_load_n(p, __ATOMIC_ACQUIRE) >= at_least); i++) {
+        plat_pause();
+        if (!(i & 31) && kern_now_ns() >= end) break;
+    }
+    lock();
+    return ok;
+}
+
 // SleepConditionVariableSRW counts milliseconds: round up (late is legal, early is not) and re-check the deadline
 // against the performance counter, because the wait may also end early.
 static uint32_t block_inner(RtThread *t, int kind, int64_t ns) {
@@ -262,6 +300,8 @@ static uint32_t block_inner(RtThread *t, int kind, int64_t ns) {
     uint64_t now = kern_now_ns();
     int forever = ns < 0 || (uint64_t)ns > UINT64_MAX - now;
     uint64_t deadline = forever ? 0 : now + (uint64_t)ns;
+    uint64_t poll = kspin_ns();
+    if (poll) kspin(&t->woke, 1, forever || poll < (uint64_t)ns ? poll : (uint64_t)ns);
     while (!t->woke) {
         if (forever) {
             plat_cond_wait(&t->cv, &G);
@@ -991,6 +1031,9 @@ static void core_take(RtThread *t) {
         while (__atomic_load_n(&t->cur_core, __ATOMIC_RELAXED) < 0 && kern_now_ns() < end) plat_pause();
         lock();
         if (t->cur_core >= 0) g_ap_spin_ok++;
+    } else if (t->cur_core < 0) {   // a holder may block or yield any moment (kspin_ns)
+        uint64_t poll = kspin_ns();
+        if (poll) kspin(&t->cur_core, 0, poll);
     }
     int sampled = 0;
     while (t->cur_core < 0) {
@@ -1518,6 +1561,7 @@ uint32_t kern_main_thread(Cpu **cpu) {
     t->cpu->sp = t->stack_top;
     t->tid = plat_thread_id();
     perf_main_thread();
+    perf_core_thread(PCORE_MAIN);
     PlatIntr *self = ap_open_self();
     lock();
     t->hthr = self;

runtime/kernel_probe.c

@@ -10,6 +10,7 @@
 #include "cfg.h"
 #include "host.h"
 #include "kernel.h"
+#include "layout.h"
 #include "platform/platform.h"
 
 #define STACK 0x10000
@@ -682,6 +683,25 @@ static int cmp_u64(const void *a, const void *b) {
     return x < y ? -1 : x > y;
 }
 
+// A refused commit waits and retries (memory.c commit, oom.c rt_oom_wait): three forced refusals, then the page maps.
+static void test_oom_commit(void) {
+    const RtGuestLayout *g = mem_layout();
+    uint64_t a = g->alias.base + g->alias.size - 0x10000;
+    rt_oom_force(RT_OOM_COMMIT, 3);
+    uint64_t t0 = plat_now_ms();
+    uint32_t r = mem_map_physical(a, 0x1000);
+    uint64_t ms = plat_now_ms() - t0;
+    CHECK(r == 0 && rt_oom_forced_left(RT_OOM_COMMIT) == 0 && ms >= 20,
+          "MapPhysicalMemory after 3 refused commits: result %#x, %d refusals left, %llu ms", r,
+          rt_oom_forced_left(RT_OOM_COMMIT), (unsigned long long)ms);
+    rt_oom_force(RT_OOM_COMMIT, 0);
+    if (r == 0) {
+        *(volatile uint64_t *)(uintptr_t)a = 1;
+        CHECK(mem_unmap_physical(a, 0x1000) == 0, "UnmapPhysicalMemory after the refused commits");
+    }
+    printf("oom: commit waited %llu ms through 3 refusals\n", (unsigned long long)ms);
+}
+
 static void test_platform(void) {
     PlatThread *t = plat_thread_new(plat_worker, (void *)(intptr_t)1, "probe-kernel plat");
     CHECK(t, "plat_thread_new");
@@ -719,6 +739,90 @@ static void test_platform(void) {
     CHECK(over[N / 2] < 500000, "plat_sleep_ns(1 ms) median overshoot %.3f ms (want < 0.5)", over[N / 2] / 1e6);
 }
 
+// ---- crash report (main.c crash_handler): a child faults on a guest thread with known values in rbx and x19; the
+// report must hold the host registers, the faulting instruction's bytes and the memory both values point at.
+#define CRASH_CHILD_ENV "TOTK_PROBE_KERNEL_CRASH_CHILD"
+static uint64_t crash_host_words[6] = {0, 0, 0x5ca1ab1e0c0de001ull, 0, 0, 0};
+static uint64_t crash_guest_words[6] = {0, 0, 0x5ca1ab1e0c0de002ull, 0, 0, 0};
+
+// The compound shape note (main.c crash_compound_shape, 1.2.1): a shape of 3 records whose child 0 has no shape, child
+// 1 has one but its live bit is clear, child 2 is live and filled.
+static uint64_t crash_shape[0x1a0 / 8], crash_records[3 * 0x78 / 8], crash_child_shape;
+
+static void crash_shape_fill(Cpu *c) {
+    uint8_t *s = (uint8_t *)crash_shape, *r = (uint8_t *)crash_records;
+    static const char *const names[3] = {"PartA", "PartB", "PartC"};
+    for (int i = 0; i < 3; i++) {
+        uint64_t name = (uint64_t)(uintptr_t)(r + i * 0x78 + 0x14);
+        memcpy(r + i * 0x78 + 8, &name, 8);
+        memcpy(r + i * 0x78 + 0x14, names[i], strlen(names[i]) + 1);
+    }
+    uint64_t child = (uint64_t)(uintptr_t)&crash_child_shape, vt = MOD_MAIN_BASE + 0x45e1218;
+    uint64_t rec = (uint64_t)(uintptr_t)crash_records;
+    uint32_t n = 3, bits = 0x5;
+    memcpy(r + 0x78, &child, 8);
+    memcpy(r + 2 * 0x78, &child, 8);
+    memcpy(s, &vt, 8);
+    memcpy(s + 0x78, &n, 4);
+    memcpy(s + 0x80, &rec, 8);
+    memcpy(s + 0x88, &bits, 4);
+    memcpy(s + 0x190, &n, 4);
+    c->x[20] = (uint64_t)(uintptr_t)crash_shape;
+    c->x[21] = 0x1000038;
+    c->x[30] = MOD_MAIN_BASE + 0xe056d4;
+}
+
+static void crash_child_main(Cpu *c) {
+    const char *mode = getenv(CRASH_CHILD_ENV);
+    if (mode && !strcmp(mode, "shape")) crash_shape_fill(c);
+    c->x[19] = (uint64_t)(uintptr_t)&crash_guest_words[2];
+    c->sp -= 0x100;   // a frame on the guest stack, as a crashing function has
+    __asm__ volatile("movq %0, %%rbx\n\tmovq 0x10, %%rax" : : "r"(&crash_host_words[2]) : "rax", "rbx", "memory");
+}
+
+// The report line that holds needle, NULL when none does
+static const char *report_line(const char *out, const char *needle, char *line, size_t cap) {
+    const char *p = strstr(out, needle);
+    if (!p) return NULL;
+    while (p > out && p[-1] != '\n') p--;
+    size_t n = strcspn(p, "\r\n");
+    snprintf(line, cap, "%.*s", (int)(n < cap ? n : cap - 1), p);
+    return line;
+}
+
+static void test_crash_report(void) {
+    static char out[65536];
+    char line[512], want[64];
+    int code = plat_run_self(CRASH_CHILD_ENV, "1", out, sizeof out, 30000);
+    CHECK(code == 71 && strstr(out, "FATAL: access violation") && strstr(out, "reading address 0x10"),
+          "crash child: exit code %d, no access violation at 0x10 reported", code);
+    snprintf(want, sizeof want, "rbx %016llx", (unsigned long long)(uintptr_t)&crash_host_words[2]);
+    CHECK(strstr(out, want), "crash report: no host register line with %s", want);
+    CHECK(report_line(out, "host code at rip-", line, sizeof line) && strstr(line, "| 48 8b 04 25 10 00 00 00"),
+          "crash report: the faulting instruction's bytes are missing");
+    const char *h = report_line(out, "*5ca1ab1e0c0de001", line, sizeof line);
+    CHECK(h && strstr(h, "rbx"), "crash report: no memory line for rbx (%s)", h ? h : "none");
+    const char *g = report_line(out, "*5ca1ab1e0c0de002", line, sizeof line);
+    CHECK(g && strstr(g, "x19"), "crash report: no memory line for x19 (%s)", g ? g : "none");
+    CHECK(strstr(out, "guest stack sp+0x00:"), "crash report: no guest stack words");
+    CHECK(report_line(out, "  memory commit: this process ", line, sizeof line) && !strstr(line, "process -1 MB"),
+          "crash report: no memory commit line");
+    printf("crash report: exit code %d, host registers, code bytes, memory at rbx and x19, guest stack, commit %s\n", code,
+           failures ? "checked with failures" : "present");
+    if (rt_game_version != 121) return;
+    code = plat_run_self(CRASH_CHILD_ENV, "shape", out, sizeof out, 30000);
+    CHECK(report_line(out, "  compound shape ", line, sizeof line) && strstr(line, "3 children, 3 records") &&
+              strstr(line, "live bits: 00000005"),
+          "crash report: no compound shape line (%s)", strstr(out, "compound shape") ? line : "none");
+    CHECK(report_line(out, "child   0:", line, sizeof line) && strstr(line, "shape 0 \"PartA\""),
+          "crash report: child 0 should have no shape (%s)", line);
+    CHECK(report_line(out, "child   1:", line, sizeof line) && strstr(line, "(off) \"PartB\""),
+          "crash report: child 1 should be off (%s)", line);
+    CHECK(report_line(out, "child   2:", line, sizeof line) && !strstr(line, "(off)") && strstr(line, "\"PartC\""),
+          "crash report: child 2 should be live (%s)", line);
+    printf("crash report, compound shape: exit code %d, %s\n", code, failures ? "checked with failures" : "children listed");
+}
+
 int probe_kernel(void) {
     a_lock_waiter = hostfn_register("probe-kernel lock waiter", lock_waiter);
     a_cv_waiter = hostfn_register("probe-kernel condvar waiter", cv_waiter);
@@ -742,7 +846,15 @@ int probe_kernel(void) {
         printf("probe-kernel: FAILED (no PLT stub for %s in main)\n", TICK_SYMBOL);
         return 1;
     }
+    const char *child = getenv(CRASH_CHILD_ENV);
+    if (child && *child) {
+        kern_run_on_guest_main(crash_child_main);
+        printf("crash child: returned without a fault\n");
+        return 1;
+    }
     test_platform();
+    test_oom_commit();
+    test_crash_report();
     kern_run_on_guest_main(probe_main);
     RtStats s = rt_stats();
     printf("stats: %llu SVCs, %llu threads, %llu host calls\n", (unsigned long long)s.svcs,

runtime/main.c

@@ -7,6 +7,7 @@
 //     --probe-loader    load and relocate the modules, check them against the measured 1.2.1 values
 //     --probe-kernel, --probe-ipc, --probe-services, --probe-fs    host probes
 //     --probe-discord  the Rich Presence IPC exchange; skips without a Discord client; no game files
+//     --probe-playtime  the play time file, its ticks and saves (playtime.c); needs no game files
 //     --probe-dpi       the process DPI awareness (per-monitor v2) and the primary monitor's size; no game files
 //     --probe-gfx-abi   NVN object layouts against nvn.h's reserved sizes; needs no game files
 //     --probe-nvn, --probe-nvn-tex, --probe-window, --probe-shader, --probe-vkmem, --probe-vkexec    graphics probes
@@ -20,7 +21,8 @@
 //
 // Also here: the runtime services the generated code calls besides rt_dispatch and rt_svc (rt_brk, rt_udf,
 // rt_ticks, the function profiler hook), logging, SHA-256, and the crash reporter (a vectored exception handler that
-// names the lifted function, the guest registers and the guest frame-pointer chain).
+// names the lifted function, the host and guest registers, the machine code around the fault, the guest frame-pointer
+// chain and the memory the registers point at).
 #include <limits.h>
 #include <stdio.h>
 #include <stdlib.h>
@@ -28,6 +30,7 @@
 
 #include "cfg.h"
 #include "host.h"
+#include "layout.h"
 #include "platform/platform.h"
 #include "sha256.h"
 
@@ -187,8 +190,8 @@ static int by_host(const void *a, const void *b) {
 }
 
 // The lifted function that contains a host code address, by the nearest function start below it; 0 outside the lifted
-// code.
-static uint64_t lifted_function(uintptr_t rip) {
+// code. *host, when given, gets the function's host start.
+static uint64_t lifted_function_at(uintptr_t rip, uintptr_t *host) {
     if (!fn_rev) {
         static RtModule mods[MAX_MODULES];
         int n = rt_linked_modules(mods, MAX_MODULES);
@@ -210,11 +213,113 @@ static uint64_t lifted_function(uintptr_t rip) {
         if (fn_rev[mid].host <= rip) lo = mid + 1;
         else hi = mid;
     }
+    if (lo && host) *host = fn_rev[lo - 1].host;
     return lo ? fn_rev[lo - 1].guest : 0;
 }
 
+static uint64_t lifted_function(uintptr_t rip) { return lifted_function_at(rip, NULL); }
+
 static int readable(uint64_t a, uint64_t n) { return plat_mem_readable(a, n); }
 
+// The machine code around the faulting instruction: 32 bytes before rip, then 32 from rip. The user's totk.exe is built
+// on their machine, so its offsets need not match ours; these bytes and the offset into the lifted function find the
+// same instruction in a local build of that version, and with it the guest instruction and the operands' registers.
+static void crash_code_bytes(uintptr_t rip) {
+    uint8_t b[64];
+    int before = 32;
+    if (!plat_mem_read(rip - 32, b, sizeof b)) {
+        before = 0;   // rip near the start of a mapping: the bytes from rip only
+        if (!plat_mem_read(rip, b + 32, 32)) return;
+    }
+    rt_logf("  host code at rip-%d:", before);
+    for (int i = 32 - before; i < 64; i++) rt_logf("%s%02x", i == 32 ? " | " : " ", b[i]);
+    rt_log_raw("\n", 1);
+}
+
+// 64 bytes around each register value that points at readable memory, as 8-byte words from 16 bytes below the value,
+// the word holding the value marked with *. Host and guest registers that land on the same window share a line. A
+// guest pointer is a host address here (the guest address space is mapped one to one), so both kinds are read alike.
+// The guest registers come from the Cpu, which the lifted code writes back only at calls and returns, so the host
+// registers carry the faulting function's live values.
+typedef struct { uint64_t start; char names[56]; uint64_t w[8]; } CrashWin;
+
+static void crash_win_add(CrashWin *wins, int *n, int cap, const char *name, uint64_t v) {
+    if (v < 0x10000 || v > 0x00007fffffffffffull) return;   // small integers and non-canonical values
+    uint64_t start = (v & ~7ull) - 16;
+    for (int i = 0; i < *n; i++)
+        if (wins[i].start == start) {
+            size_t len = strlen(wins[i].names);
+            if (len + strlen(name) + 2 < sizeof wins[i].names)
+                snprintf(wins[i].names + len, sizeof wins[i].names - len, " %s", name);
+            return;
+        }
+    if (*n == cap || !plat_mem_read(start, wins[*n].w, sizeof wins[*n].w)) return;
+    wins[*n].start = start;
+    snprintf(wins[*n].names, sizeof wins[*n].names, "%s", name);
+    (*n)++;
+}
+
+static void crash_memory(const PlatCrash *x, const Cpu *c) {
+    static CrashWin wins[64];   // the crash handler runs once (busy flag), so static storage needs no allocation
+    static const char *const gpr_names[16] = PLAT_GPR_NAMES;
+    int n = 0;
+    char name[8];
+    if (x->has_regs)
+        for (int i = 0; i < 16; i++) crash_win_add(wins, &n, 64, gpr_names[i], x->gpr[i]);
+    if (c) {
+        for (int i = 0; i < 31; i++) snprintf(name, sizeof name, "x%d", i), crash_win_add(wins, &n, 64, name, c->x[i]);
+        crash_win_add(wins, &n, 64, "sp", c->sp);
+    }
+    if (!n) return;
+    rt_logf("  memory at the register values (8-byte words from value-16, * the value's word):\n");
+    for (int i = 0; i < n; i++) {
+        rt_logf("  %016llx %-12s", (unsigned long long)wins[i].start, wins[i].names);
+        for (int k = 0; k < 8; k++) rt_logf(" %s%016llx", k == 2 ? "*" : "", (unsigned long long)wins[i].w[k]);
+        rt_log_raw("\n", 1);
+    }
+    // The guest stack above sp: the crashing function's spills and locals, and the callers' saved registers
+    uint64_t s[16];
+    if (c && plat_mem_read(c->sp, s, sizeof s))
+        for (int i = 0; i < 16; i += 4)
+            rt_logf("  guest stack sp+0x%02x: %016llx %016llx %016llx %016llx\n", i * 8, (unsigned long long)s[i],
+                    (unsigned long long)s[i + 1], (unsigned long long)s[i + 2], (unsigned long long)s[i + 3]);
+}
+
+// 1.2.1: a fault right after a compound physics shape was asked for its first child's material (the call returning to
+// main+0xe056d4 reads the result without a check). The shape (vtable main+0x45e1218, built by main+0x7359fc) is in x20
+// there. It returns null when no child slot is both live (bitmap at +0x88) and filled (shape pointer at +0 of the
+// 0x78-byte records at +0x80). Every child is filled when the shape is built, so the list tells children removed later
+// (null shape) from children switched off (clear bit), and the names name the object. Runs only in the crash handler.
+static void crash_compound_shape(const Cpu *c) {
+    if (!c || rt_game_version != 121 || c->x[30] != MOD_MAIN_BASE + 0xe056d4) return;
+    uint64_t shape = c->x[20];
+    if (!readable(shape, 0x1a0) || LD64(shape) != MOD_MAIN_BASE + 0x45e1218) {
+        rt_logf("  compound shape: x20 %#llx is not one\n", (unsigned long long)shape);
+        return;
+    }
+    uint32_t children = LD32(shape + 0x190), records = LD32(shape + 0x78);
+    uint64_t rec = LD64(shape + 0x80);
+    rt_logf("  compound shape %#llx (holder %#llx): %u children, %u records at %#llx, live bits:",
+            (unsigned long long)shape, (unsigned long long)(c->x[21] - 0x38), children, records,
+            (unsigned long long)rec);
+    uint32_t n = children > 0x400 ? 0x400 : children;   // the bitmap holds 0x400 bits
+    for (uint32_t w = 0; w < (n + 31) / 32; w++) rt_logf(" %08x", LD32(shape + 0x88 + 4 * w));
+    rt_log_raw("\n", 1);
+    if (records > 0x400 || !readable(rec, (uint64_t)records * 0x78)) return;
+    for (uint32_t i = 0; i < records; i++) {
+        uint64_t r = rec + (uint64_t)i * 0x78, child = LD64(r), s = LD64(r + 8);
+        char name[33];
+        size_t k = 0;
+        for (; k < 32 && readable(s + k, 1); k++) {
+            name[k] = *(const char *)(uintptr_t)(s + k);
+            if (!name[k]) break;
+        }
+        name[k] = 0;
+        int live = LD32(shape + 0x88 + 4 * (i >> 5)) >> (i & 31) & 1;
+        rt_logf("    child %3u: shape %#llx%s \"%s\"\n", i, (unsigned long long)child, live ? "" : " (off)", name);
+    }
+}
+
 static void crash_handler(const PlatCrash *x) {
     static volatile int32_t busy;
     if (plat_xchg32(&busy, 1)) plat_exit(71);
@@ -227,23 +332,48 @@ static void crash_handler(const PlatCrash *x) {
         rt_logf(", %s address %#llx", x->access == 1 ? "writing" : x->access == 8 ? "executing" : "reading",
                 (unsigned long long)x->addr);
     rt_log_raw("\n", 1);
-    uint64_t f = x->module ? 0 : lifted_function(rip);
-    if (f) rt_logf("  in lifted function f_%llx = %s\n", (unsigned long long)f, ldr_describe(f, d, sizeof d));
-    // abort and invalid parameter: where the host code called it; a fault in another module: the host calls that led
-    // into it
+    if (x->cxx_type)
+        rt_logf("  C++ type %s%s%s\n", x->cxx_type, x->cxx_what ? ", what(): " : "", x->cxx_what ? x->cxx_what : "");
+    // A driver that runs out of memory faults or throws (std::bad_alloc) instead of returning an error: the commit
+    // tells that apart from a driver bug, as rt_fatal's line does for our own failed allocations.
+    double proc, used, limit;
+    plat_commit_info(&proc, &used, &limit);
+    rt_logf("  memory commit: this process %.0f MB, the system %.0f of %.0f MB\n", proc, used, limit);
+    uintptr_t fstart = 0;
+    uint64_t f = x->module ? 0 : lifted_function_at(rip, &fstart);
+    if (f)
+        rt_logf("  in lifted function f_%llx = %s, host +%#llx into it\n", (unsigned long long)f,
+                ldr_describe(f, d, sizeof d), (unsigned long long)(rip - fstart));
+    if (x->has_regs) {
+        static const char *const gpr_names[16] = PLAT_GPR_NAMES;
+        for (int i = 0; i < 16; i++)
+            rt_logf("  %-3s %016llx%s", gpr_names[i], (unsigned long long)x->gpr[i], i % 4 == 3 ? "\n" : "");
+        crash_code_bytes(rip);
+    }
+    // abort and invalid parameter: where the host code called it; a fault in another module: the driver code and the
+    // host calls that led into it
     uintptr_t exe = plat_exe_base();
     for (int i = 0; i < x->nframes; i++) {
+        const char *m = x->frame_module[i];
+        if (m) {
+            rt_logf("  host frame %2d: %s+%#llx\n", i, *m ? m : "outside every module",
+                    (unsigned long long)x->frame_offset[i]);
+            continue;
+        }
         uint64_t g = lifted_function(x->frames[i]);
         rt_logf("  host frame %2d: totk+%#llx", i, (unsigned long long)(x->frames[i] - exe));
         if (g) rt_logf(" in lifted function f_%llx", (unsigned long long)g);
         rt_log_raw("\n", 1);
     }
+    if (x->frames_left_out) rt_logf("  (%d frames in other modules left out)\n", x->frames_left_out);
     Cpu *c = kern_cpu();
     if (c) {
         for (int i = 0; i < 31; i++)
             rt_logf("  x%-2d %016llx%s", i, (unsigned long long)c->x[i], i % 4 == 3 ? "\n" : "");
-        rt_logf("  sp  %016llx\n  pc  %016llx  lr %s\n", (unsigned long long)c->sp, (unsigned long long)c->pc,
-                ldr_describe(c->x[30], d, sizeof d));
+        // pc: the guest code running, or the host function a guest call entered (its name tells the host call apart)
+        rt_logf("  sp  %016llx\n  pc  %016llx", (unsigned long long)c->sp, (unsigned long long)c->pc);
+        if (hostfn_name(c->pc)) rt_logf(" (%s)", ldr_describe(c->pc, d, sizeof d));
+        rt_logf("  lr %s\n", ldr_describe(c->x[30], d, sizeof d));
         uint64_t fp = c->x[29];
         for (int n = 0; n < 32 && fp && !(fp & 7) && readable(fp, 16); n++) {
             uint64_t next = LD64(fp), lr = LD64(fp + 8);
@@ -254,6 +384,8 @@ static void crash_handler(const PlatCrash *x) {
     } else {
         rt_logf("  not on a guest thread\n");
     }
+    crash_memory(x, c);
+    crash_compound_shape(c);
     kern_free_cores_note();
     fflush(stderr);
     plat_exit(71);
@@ -396,7 +528,9 @@ static void check_memory(void) {
     CHECK(q.state == 0 && q.base == g->alias.base && q.size == g->alias.size, "QueryMemory on the free alias region");
     CHECK(mem_map_physical(a, 0x1000) == 0 && LD64(a + 0x10) == 0 && mem_unmap_physical(a, 0x1000) == 0,
           "remapped physical memory is not zeroed");
-    CHECK(mem_map_physical(a, 0x100000000ull) == 0x10801, "MapPhysicalMemory beyond the pool is not LimitReached");
+    // past the pool whatever its size (TOTK_APP_POOL, TOTK_TEX_HEAP_EXTRA_MB): 4 GiB fits a grown one
+    CHECK(mem_map_physical(a, (g->pool + 0x100000) & ~0xfffffull) == 0x10801,
+          "MapPhysicalMemory beyond the pool is not LimitReached");
     mem_query(RT_HOSTFN_BASE + 0x40, &q);
     CHECK(q.state == 0x10 && q.base == RT_HOSTFN_BASE, "QueryMemory on the host-function window");
     mem_query(g->alias.base + g->alias.size + 0x1000, &q);
@@ -463,7 +597,7 @@ static int probe_crash(int null) {
 
 static int usage(const char *argv0) {
     fprintf(stderr, "usage: %s [--exefs DIR] [--romfs DIR] [--profile DIR] [--selftest | --probe-loader | "
-                    "--probe-cfg | --probe-dpi | --probe-mods | --probe-kernel | --probe-ipc | --probe-services | --probe-fs | --probe-gfx-abi | "
+                    "--probe-cfg | --probe-dpi | --probe-mods | --probe-playtime | --probe-kernel | --probe-ipc | --probe-services | --probe-fs | --probe-gfx-abi | "
                     "--probe-nvn | --probe-nvn-tex | --probe-window | --probe-shader | --probe-vkmem | "
                     "--probe-vkexec | --probe-astc | --probe-movie | --probe-crash-call | --probe-crash-null | --probe-discord | "
                     "--prepare-shaders | --warm-shaders | --boot |--seconds S [--shot PNG]]\n", argv0);
@@ -518,6 +652,20 @@ int main(int argc, char **argv) {
     // Timed guest waits (kernel.c block_inner) sleep in whole milliseconds; at the default 15.6 ms tick a 1 ms wait
     // costs a full tick. TOTK_TIMER_1MS=0 keeps the default for comparison.
     if (cfg_bool("TOTK_TIMER_1MS")) plat_timer_1ms();
+    int power = plat_power_full();
+    if (power >= 0)
+        rt_log("power: %s", power ? "OS power throttling off (EcoQoS, timer resolution in the background)"
+                                  : "OS power throttling: the opt-out was refused");
+    // Hybrid CPUs: the threads every frame goes through prefer the P-cores (perf_core_thread). TOTK_PCORE=0: off.
+    if (rt_knob("PCORE", "1")) {
+        char why[160];
+        int n = plat_perf_core_setup(why, sizeof why);
+        if (n) rt_log("power: %s; the game's main and render threads and the vk replay, recorder and present threads "
+                      "prefer the %d P-core ones (TOTK_PCORE=0 turns it off)", why, n);
+        else rt_log("power: %s: no P-core preference", why);
+    } else {
+        rt_log("power: TOTK_PCORE=0: no P-core preference");
+    }
     cfg = (RtConfig){.exefs = rt_default_exefs, .romfs = rt_default_romfs, .profile = rt_default_profile,
                      .gpu = cfg_get("TOTK_GPU")};
     portable_paths(&cfg);   // command-line paths below still win
@@ -553,6 +701,7 @@ int main(int argc, char **argv) {
     if (!mode) {
         cfg.boot = cfg.seconds > 0;   // --seconds starts the game like --boot, with its checkpoints
         discord_init();   // the Rich Presence thread (discord_rpc.c); it reads totk.ini itself and idles while off
+        playtime_init();  // the play time thread (playtime.c): the Game tab's play time across runs
         rt_boot();
         return 0;
     }
@@ -566,6 +715,7 @@ int main(int argc, char **argv) {
 #endif
     if (!strcmp(mode, "--probe-mods")) return probe_mods();
     if (!strcmp(mode, "--probe-discord")) return probe_discord();
+    if (!strcmp(mode, "--probe-playtime")) return probe_playtime();
     if (!strcmp(mode, "--probe-kernel")) return probe_kernel();
     if (!strcmp(mode, "--probe-ipc")) return probe_ipc();
     if (!strcmp(mode, "--probe-services")) return probe_services();

runtime/memory.c

@@ -20,11 +20,11 @@
 #define PAGE 0x1000ull
 #define GiB (1ull << 30)
 #define LOW_SEARCH_FROM 0x100000000ull
-#define LOW_SIZE (6 * GiB)
+#define LOW_BASE (6 * GiB)    // the stack and heap block without a texture heap extra
 #define STACK_OFF 0x10000000ull
 #define STACK_SIZE 0x10000000ull
 #define HEAP_OFF (STACK_OFF + STACK_SIZE + 0x10000000ull)
-#define HEAP_CAP (4 * GiB)
+#define HEAP_CAP (4 * GiB)    // without TOTK_TEX_HEAP_EXTRA_MB, which raises it by its own size (mem_init)
 #define ALIAS_SIZE (64 * GiB)
 #define ALIAS_ALIGN (4 * GiB)
 #define ALIAS_PAGES (ALIAS_SIZE / PAGE)
@@ -48,6 +48,9 @@ enum { ATTR_PERMISSION_LOCKED = 16 };
 
 static RtGuestLayout L;
 static uint64_t mods_lo, mods_hi;   // fixed module window
+// The stack and heap block: LOW_BASE, plus the texture heap extra (tex_heap_extra_mb or a mod's HeapDef,
+// res_tex_heap_extra_bytes) rounded up to a GiB, so only a player who grows the texture heap reserves more (mem_init).
+static uint64_t low_size = LOW_BASE;
 static PlatMutex lock = PLAT_MUTEX_INIT;
 static uint64_t heap_size, alias_bytes;
 static uint8_t *alias_pages;        // 1 per mapped 4 KiB page of the alias region
@@ -78,7 +81,7 @@ static volatile int64_t frz_count[4];   // freezes; thaws by a write fault, by m
 
 static int watched(uint64_t addr, uint64_t size) {
     const RtGuestLayout *g = &L;
-    return (addr >= g->aslr.base && size <= LOW_SIZE && addr - g->aslr.base <= LOW_SIZE - size) ||
+    return (addr >= g->aslr.base && size <= low_size && addr - g->aslr.base <= low_size - size) ||
            (addr >= g->alias.base && size <= ALIAS_SIZE && addr - g->alias.base <= ALIAS_SIZE - size);
 }
 
@@ -172,12 +175,29 @@ static int thaw_fault(uint64_t a) {
 }
 
 // Mapping changes thaw the regions they touch and run under frz_lock, so no freeze sees them half done.
-static void commit(uint64_t a, uint64_t n, const char *what) {
+typedef struct { uint64_t a, n; } CommitTry;
+static int commit_try(void *arg) {
+    const CommitTry *t = arg;
+    if (rt_oom_forced(RT_OOM_COMMIT)) return 0;
     plat_mutex_lock(&frz_lock);
-    thaw_range_locked(a, n, 3);
-    int ok = !n || plat_mem_commit(a, n);
+    thaw_range_locked(t->a, t->n, 3);
+    int ok = !t->n || plat_mem_commit(t->a, t->n);
     plat_mutex_unlock(&frz_lock);
-    if (!ok) rt_fatal("memory: commit %s %#llx+%#llx failed", what, (unsigned long long)a, (unsigned long long)n);
+    return ok;
+}
+
+// A refused commit waits for memory (rt_oom_wait): the system's commit ran out for a moment (a page file growing, the
+// driver's device memory charged to this process), and the Vulkan back end gives memory back at its next submit. The
+// wait holds the caller's lock (lock or klock: only memory SVCs and kernel-region allocations wait behind it) but not
+// frz_lock, which write faults on frozen regions need. Every caller waits, the boot's ones too: there a refusal has the
+// same cause, and the wait only delays an end the game would reach anyway.
+static void commit(uint64_t a, uint64_t n, const char *what) {
+    CommitTry t = {a, n};
+    if (commit_try(&t)) return;
+    char w[64];
+    snprintf(w, sizeof w, "memory: commit of %s at %#llx", what, (unsigned long long)a);
+    if (!rt_oom_wait(w, n, commit_try, &t))
+        rt_fatal("memory: commit %s %#llx+%#llx failed", what, (unsigned long long)a, (unsigned long long)n);
 }
 
 static void decommit(uint64_t a, uint64_t n, const char *what) {
@@ -221,10 +241,15 @@ void mem_init(void) {
 
     // Stack and heap block, then the alias region above it, both searched below the module window.
     uint64_t low = 0, alias = 0;
-    for (uint64_t at = LOW_SEARCH_FROM; at + LOW_SIZE <= mods_lo && !low; at += GiB)
-        if (reserve_at(at, LOW_SIZE, 1)) low = at;
-    if (!low) reexec_or_fail("a 6 GiB stack and heap block below 2^39");
-    for (uint64_t at = align_up(low + LOW_SIZE, ALIAS_ALIGN); at + ALIAS_SIZE <= mods_lo && !alias; at += ALIAS_ALIGN)
+    low_size = LOW_BASE + align_up(res_tex_heap_extra_bytes(), GiB);
+    for (uint64_t at = LOW_SEARCH_FROM; at + low_size <= mods_lo && !low; at += GiB)
+        if (reserve_at(at, low_size, 1)) low = at;
+    if (!low) {
+        char what[64];
+        snprintf(what, sizeof what, "a %llu GiB stack and heap block below 2^39", (unsigned long long)(low_size / GiB));
+        reexec_or_fail(what);
+    }
+    for (uint64_t at = align_up(low + low_size, ALIAS_ALIGN); at + ALIAS_SIZE <= mods_lo && !alias; at += ALIAS_ALIGN)
         if (reserve_at(at, ALIAS_SIZE, 1)) alias = at;
     if (!alias) reexec_or_fail("a 64 GiB alias region below 2^39");
 
@@ -238,11 +263,11 @@ void mem_init(void) {
     L = (RtGuestLayout){
         .aslr = {low, RT_ADDRESS_LIMIT - low},
         .stack = {low + STACK_OFF, STACK_SIZE},
-        .heap = {low + HEAP_OFF, HEAP_CAP},
+        .heap = {low + HEAP_OFF, HEAP_CAP + res_tex_heap_extra_bytes()},
         .alias = {alias, ALIAS_SIZE},
         .kernel = {RT_KERNEL_BASE, RT_KERNEL_SIZE},
         .hostfn = {RT_HOSTFN_BASE, RT_HOSTFN_SIZE},
-        .pool = pool ? strtoull(pool, NULL, 0) : DEFAULT_POOL + res_heap_extra_bytes(),   // resolution.c
+        .pool = pool ? strtoull(pool, NULL, 0) : DEFAULT_POOL + res_heap_extra_bytes() + res_tex_heap_extra_bytes(),
     };
     if (!L.pool) rt_fatal("memory: TOTK_APP_POOL is 0");
 }
@@ -276,7 +301,7 @@ static int inside(uint64_t a, uint64_t n, RtRange r) { return a >= r.base && n <
 // ---- 0x01 SetHeapSize
 uint32_t mem_set_heap_size(uint64_t size, uint64_t *addr) {
     plat_mutex_lock(&lock);
-    if (size > HEAP_CAP || size % 0x200000 || size + image_bytes() > L.pool) {
+    if (size > L.heap.size || size % 0x200000 || size + image_bytes() > L.pool) {
         plat_mutex_unlock(&lock);
         return R_INVALID_STATE;
     }
@@ -506,7 +531,7 @@ static void taken_gap(uint64_t a, uint64_t *base, uint64_t *size) {
     uint64_t lo = 0, hi = RT_ADDRESS_LIMIT;
     RtRange known[3 + MOD_COUNT + 2];
     int n = 0;
-    known[n++] = (RtRange){L.aslr.base, LOW_SIZE};
+    known[n++] = (RtRange){L.aslr.base, low_size};
     known[n++] = L.alias;
     known[n++] = L.kernel;
     known[n++] = L.hostfn;
@@ -549,14 +574,14 @@ uint32_t mem_query(uint64_t a, RtMemoryInfo *out) {
         r.base = 0;
         r.size = low;
         r.state = ST_FREE;
-    } else if (a < low + LOW_SIZE) {
+    } else if (a < low + low_size) {
         uint64_t heap = L.heap.base;
         if (a >= heap && a < heap + heap_size) {
             r = (RtMemoryInfo){.base = heap, .size = heap_size, .state = ST_NORMAL, .perm = PERM_RW};
         } else if (a < heap) {
             r = (RtMemoryInfo){.base = low, .size = heap - low, .state = ST_FREE};
         } else {
-            r = (RtMemoryInfo){.base = heap + heap_size, .size = low + LOW_SIZE - heap - heap_size, .state = ST_FREE};
+            r = (RtMemoryInfo){.base = heap + heap_size, .size = low + low_size - heap - heap_size, .state = ST_FREE};
         }
     } else if (a - L.alias.base < L.alias.size) {
         uint64_t p = (a - L.alias.base) / PAGE;

runtime/mods.c

@@ -15,7 +15,9 @@
 // when its build id is main's (an IPS file's name, a pchtxt file's @nsobid line); others are another version's and
 // counted. The setup builds every mod's files into patch sets and writes build-cache/exefs.status next to totk.exe
 // (tools/exefs_patches.py has the format): a file whose line is missing or has another content hash waits for the
-// rebuild at start (cheats.c), and its mod stays off until then.
+// rebuild at start (cheats.c), and its mod stays off until then. A file's writes into main's data (.rodata, .data,
+// .bss) are lines of build-cache/exefs.data, stored into the loaded main before the game runs (mods_exefs_data), as
+// emulators apply a patch file to the module before its dynamic linker runs.
 #include <ctype.h>
 #include <stdio.h>
 #include <stdlib.h>
@@ -170,22 +172,26 @@ static int same_build(const char *hex, size_t n) {
     return 1;
 }
 
-// build-cache/exefs.status next to totk.exe, read once; "" when there is none.
-static const char *exefs_status(void) {
-    static char *text;
-    if (text) return text;
+// build-cache/<name> next to totk.exe, read once into *text; "" when there is none (mods_probe_cache sets the text).
+static char *status_text, *data_text;
+static const char *build_cache(const char *name, char **text) {
+    if (*text) return *text;
     char path[2100];
     uint64_t n = 0;
     if (plat_exe_path(path, sizeof path)) {
         char *cut = strrchr(path, '/'), *back = strrchr(path, '\\');
         if (back > cut) cut = back;
-        if (cut) snprintf(cut + 1, sizeof path - (size_t)(cut + 1 - path), "build-cache/exefs.status");
-        text = (char *)plat_file_read(path, &n);
+        if (cut && snprintf(cut + 1, sizeof path - (size_t)(cut + 1 - path), "build-cache/%s", name) <
+                       (int)(sizeof path - (size_t)(cut + 1 - path)))
+            *text = (char *)plat_file_read(path, &n);
     }
-    if (!text) text = calloc(1, 1);
-    return text;
+    if (!*text) *text = calloc(1, 1);
+    return *text;
 }
 
+static const char *exefs_status(void) { return build_cache("exefs.status", &status_text); }
+static const char *exefs_data(void) { return build_cache("exefs.data", &data_text); }
+
 // The status line of key: its kind, built-in sets and note, NUL-terminated copies; 0 when none.
 static int status_line(const char *key, const char *content, char *kind, char *over, char *note) {
     char want[20];
@@ -194,8 +200,9 @@ static int status_line(const char *key, const char *content, char *kind, char *o
         const char *end = strchr(line, '\n');
         size_t len = end ? (size_t)(end - line) : strlen(line);
         if (len > 18 && !memcmp(line, want, 18)) {
-            char copy[600];
-            snprintf(copy, sizeof copy, "%.*s", (int)(len < sizeof copy ? len : sizeof copy - 1), line + 18);
+            char copy[600];   // the line after the two hashes, without the next line
+            size_t rest = len - 18;
+            snprintf(copy, sizeof copy, "%.*s", (int)(rest < sizeof copy ? rest : sizeof copy - 1), line + 18);
             char *t1 = strchr(copy, '\t'), *t2 = t1 ? strchr(t1 + 1, '\t') : NULL;
             if (!t1 || !t2) return 0;
             *t1 = *t2 = 0;
@@ -269,9 +276,14 @@ static int exefs_file_cb(const char *leaf, int kind, uint64_t size, void *ctx) {
         m->sets[m->nsets++] = (uint32_t)strtoul(key, NULL, 16);
         size_t k = strlen(m->overrides);
         if (*over) snprintf(m->overrides + k, sizeof m->overrides - k, "%s%s", k ? "," : "", over);
+    } else if (!strcmp(kind_s, "data") && m->ndata < (int)(sizeof m->data / sizeof *m->data)) {
+        m->data[m->ndata++] = (uint32_t)strtoul(key, NULL, 16);
     } else if (!strcmp(kind_s, "other")) {
         m->other_version++;
     }
+    unsigned long bytes;   // a code or data file's data writes: exefs.data must hold them all (mods_exefs_data)
+    if ((!strcmp(kind_s, "code") || !strcmp(kind_s, "data")) && sscanf(note, "data %lu", &bytes) == 1)
+        m->data_bytes += (uint32_t)bytes;
     return 1;
 }
 
@@ -326,34 +338,62 @@ static int scan_cb(const char *name, int kind, uint64_t size, void *ctx) {
     if (m->cheats[0]) plat_fs_list(m->cheats, cheats_cb, &m->kind);
     int unsupported = exefs_check(m, path);
     // a mod whose code part the port cannot use is off as a whole: its RomFS files may expect that code
-    m->problem = strchr(m->name, '|')                                ? MOD_BAD_NAME
-               : m->kind & MOD_SUBSDK                                ? MOD_PLUGIN
-               : unsupported                                         ? MOD_EXEFS_UNSUPPORTED
-               : m->pending                                          ? MOD_EXEFS_LATER
-               : m->nsets || m->kind & (MOD_ROMFS | MOD_CHEATS)     ? MOD_OK
-               : m->kind & MOD_EXEFS                                 ? MOD_EXEFS_OTHER
-                                                                     : MOD_EMPTY;
+    m->problem = strchr(m->name, '|')                                       ? MOD_BAD_NAME
+               : m->kind & MOD_SUBSDK                                       ? MOD_PLUGIN
+               : unsupported                                                ? MOD_EXEFS_UNSUPPORTED
+               : m->pending                                                 ? MOD_EXEFS_LATER
+               : m->nsets || m->ndata || m->kind & (MOD_ROMFS | MOD_CHEATS) ? MOD_OK
+               : m->kind & MOD_EXEFS                                        ? MOD_EXEFS_OTHER
+                                                                            : MOD_EMPTY;
     m->enabled = !listed(disabled, m->name);
     nmods++;
     return 1;
 }
 
-void mods_scan(void) {
-    static int done;
-    if (done) return;
-    done = 1;
+// The mods folder into out: TOTK_MODS_DIR, else <dir of totk.exe>/mods. 0 when the exe's path is unknown.
+static int mods_folder(char *out, size_t cap) {
     const char *d = cfg_get("TOTK_MODS_DIR");
-    if (d && *d) {
-        snprintf(dir, sizeof dir, "%s", d);
-    } else {
-        char exe[2048];
-        if (!plat_exe_path(exe, sizeof exe)) return;
-        char *cut = strrchr(exe, '/'), *back = strrchr(exe, '\\');
-        if (back > cut) cut = back;
-        if (!cut) return;
-        *cut = 0;
-        snprintf(dir, sizeof dir, "%s/mods", exe);
-    }
+    if (d && *d) return snprintf(out, cap, "%s", d) < (int)cap;
+    char exe[2048];
+    if (!plat_exe_path(exe, sizeof exe)) return 0;
+    char *cut = strrchr(exe, '/'), *back = strrchr(exe, '\\');
+    if (back > cut) cut = back;
+    if (!cut) return 0;
+    *cut = 0;
+    return snprintf(out, cap, "%s/mods", exe) < (int)cap;
+}
+
+typedef struct {
+    const char *folder, *disabled;
+    void (*cb)(const char *name, const char *romfs, void *ctx);
+    void *ctx;
+} Peek;
+
+static int peek_cb(const char *name, int kind, uint64_t size, void *ctx) {
+    (void)size;
+    const Peek *pk = ctx;
+    if (kind != PLAT_FS_DIR || listed(pk->disabled, name)) return 1;
+    char path[2100], romfs[2048];
+    if (snprintf(path, sizeof path, "%s/%s", pk->folder, name) >= (int)sizeof path) return 1;
+    find_root(path, "romfs", romfs, sizeof romfs);
+    if (!romfs[0]) find_root(path, "romfslite", romfs, sizeof romfs);
+    if (romfs[0]) pk->cb(name, romfs, pk->ctx);
+    return 1;
+}
+
+void mods_peek_romfs(void (*cb)(const char *name, const char *romfs, void *ctx), void *ctx) {
+    char folder[2048];
+    if (!mods_folder(folder, sizeof folder) || !plat_dir_exists(folder)) return;
+    Peek pk = {folder, cfg_get("TOTK_MODS_DISABLED"), cb, ctx};
+    plat_fs_list(folder, peek_cb, &pk);
+}
+
+static int scanned;
+
+void mods_scan(void) {
+    if (scanned) return;
+    scanned = 1;
+    if (!mods_folder(dir, sizeof dir)) return;
     if (!plat_dir_exists(dir)) {
         rt_log("mods: no folder %s", dir);
         return;
@@ -376,18 +416,171 @@ void mods_scan(void) {
     rt_log("mods: %d folders in %s", nmods, dir);
 }
 
-void mods_exefs_switch(void) {
-    mods_scan();
-    for (int i = 0; i < nmods; i++) {
-        const ModInfo *m = &mods[i];
-        if (!start_enabled[i] || m->problem != MOD_OK || !m->nsets) continue;
-        for (int k = 0; k < m->nsets; k++) {
-            char name[32];
-            snprintf(name, sizeof name, "mod_%08x", m->sets[k]);
-            int set = rt_patch_set(name);
-            if (set >= 0) rt_patch_switch(set, 1);
-            else rt_log("mods: %s: patch set %s is not in this build", m->name, name);
+// ---- ExeFS data writes (build-cache/exefs.data; tools/exefs_patches.py data_line has the format)
+
+// 1 when key names one of m's usable ExeFS patch files: one with a patch set, or one with data writes only.
+static int owns(const ModInfo *m, uint32_t key) {
+    for (int k = 0; k < m->nsets; k++)
+        if (m->sets[k] == key) return 1;
+    for (int k = 0; k < m->ndata; k++)
+        if (m->data[k] == key) return 1;
+    return 0;
+}
+
+static int hex_digit(char c) {
+    return c >= '0' && c <= '9'   ? c - '0'
+         : c >= 'a' && c <= 'f'   ? c - 'a' + 10
+         : c >= 'A' && c <= 'F'   ? c - 'A' + 10
+                                  : -1;
+}
+
+// One line: "<file hash>\t<content hash>\t<offset in main>\t<bytes>[\tbase]", all hex. 0 when malformed.
+typedef struct { char key[9], content[9]; uint64_t off; uint8_t bytes[256]; int n, base; } DataRun;
+static int parse_run(const char *line, size_t len, DataRun *r) {
+    char copy[1100], *f[6];
+    if (len >= sizeof copy) return 0;
+    memcpy(copy, line, len);
+    copy[len] = 0;
+    int nf = 0;
+    for (char *p = copy; p && nf < 6; nf++) {
+        f[nf] = p;
+        p = strchr(p, '\t');
+        if (p) *p++ = 0;
+    }
+    if (nf < 4 || nf > 5 || strlen(f[0]) != 8 || strlen(f[1]) != 8) return 0;
+    memcpy(r->key, f[0], 9);
+    memcpy(r->content, f[1], 9);
+    char *end;
+    r->off = strtoull(f[2], &end, 16);
+    size_t hex = strlen(f[3]);
+    if (!*f[2] || *end || !hex || hex % 2 || hex / 2 > sizeof r->bytes) return 0;
+    r->n = (int)(hex / 2);
+    for (int i = 0; i < r->n; i++) {
+        int hi = hex_digit(f[3][2 * i]), lo = hex_digit(f[3][2 * i + 1]);
+        if (hi < 0 || lo < 0) return 0;
+        r->bytes[i] = (uint8_t)(hi << 4 | lo);
+    }
+    r->base = nf == 5;
+    return nf == 4 || (!strcmp(f[4], "base") && r->n == 8);
+}
+
+// Stores n bytes into the loaded main at a: a read-only page (.rodata, loader.c seal) is writable for its part of the
+// copy only, as cheats.c wr_main does. 0 when a page is neither readable nor writable.
+static int store_main(uint64_t a, const uint8_t *from, int n) {
+    while (n > 0) {
+        uint64_t page = a & ~0xFFFull, end;
+        int k = (int)(page + 0x1000 - a);
+        if (k > n) k = n;
+        int prot = plat_mem_state(page, page + 0x1000, &end);
+        if (prot != PLAT_PROT_READ && prot != PLAT_PROT_RW) return 0;
+        if (prot == PLAT_PROT_READ && !plat_mem_protect(page, 0x1000, PLAT_PROT_RW)) return 0;
+        memcpy((void *)(uintptr_t)a, from, (size_t)k);
+        if (prot == PLAT_PROT_READ) plat_mem_protect(page, 0x1000, PLAT_PROT_READ);
+        a += (uint64_t)k;
+        from += k;
+        n -= k;
+    }
+    return 1;
+}
+
+long mods_exefs_data(const ModInfo *m, uint64_t base, uint64_t size, uint64_t code_end,
+                     int (*store)(uint64_t addr, const uint8_t *bytes, int n)) {
+    if (!store) store = store_main;
+    long total = 0, found = 0;
+    for (int pass = 0; pass < 2; pass++) {   // pass 0 checks every line, pass 1 stores
+        // exefs.data must hold every byte the status lines count: one missing or cut short would leave the code
+        // patches on without their data
+        if (pass && found != (long)m->data_bytes) {
+            rt_log("mods: %s: build-cache/exefs.data holds %ld of the %u bytes its ExeFS patch files write", m->name,
+                   found, m->data_bytes);
+            return -1;
         }
+        for (const char *line = exefs_data(), *next; *line; line = next) {
+            const char *nl = strchr(line, '\n');
+            size_t len = nl ? (size_t)(nl - line) : strlen(line);
+            next = nl ? nl + 1 : line + len;
+            if (len && line[len - 1] == '\r') len--;
+            char key[9];
+            snprintf(key, sizeof key, "%.*s", (int)(len < 8 ? len : 8), line);
+            if (!len || *line == '#' || !owns(m, (uint32_t)strtoul(key, NULL, 16))) continue;
+            DataRun r;
+            char kind[16], over[256], note[256];
+            // a line whose file the status does not list as built (another content) or that leaves main's data: the
+            // two build-cache files and this build disagree
+            if (!parse_run(line, len, &r) || !status_line(r.key, r.content, kind, over, note) ||
+                (strcmp(kind, "code") && strcmp(kind, "data")) || r.off < code_end || r.off > size ||
+                (uint64_t)r.n > size - r.off) {
+                rt_log("mods: %s: a line of build-cache/exefs.data does not fit this build: %.*s", m->name, (int)len,
+                       line);
+                return -1;
+            }
+            if (!pass) {
+                found += r.n;
+                continue;
+            }
+            if (r.base) {   // a DT_RELR word holds its addend; the dynamic linker adds the module's base
+                uint64_t w;
+                memcpy(&w, r.bytes, 8);
+                w += base;
+                memcpy(r.bytes, &w, 8);
+            }
+            if (!store(base + r.off, r.bytes, r.n)) {
+                rt_log("mods: %s: cannot write main+%#llx", m->name, (unsigned long long)r.off);
+                return -1;
+            }
+            total += r.n;
+        }
+    }
+    return total;
+}
+
+static char *dup_text(const char *s) {
+    if (!s) return NULL;
+    size_t n = strlen(s) + 1;
+    char *d = malloc(n);
+    if (d) memcpy(d, s, n);
+    return d;
+}
+
+void mods_probe_cache(const char *status, const char *data) {
+    free(status_text);
+    free(data_text);
+    status_text = dup_text(status);
+    data_text = dup_text(data);
+}
+
+void mods_rescan(void) {
+    free(mods);
+    mods = NULL;
+    nmods = 0;
+    scanned = 0;
+}
+
+// Switches the patch set mod_<key> on; 0 when this build lacks it (logged).
+static int set_on(const ModInfo *m, uint32_t key) {
+    char name[32];
+    snprintf(name, sizeof name, "mod_%08x", key);
+    int set = rt_patch_set(name);
+    if (set >= 0) rt_patch_switch(set, 1);
+    else rt_log("mods: %s: patch set %s is not in this build", m->name, name);
+    return set >= 0;
+}
+
+void mods_exefs_apply(uint64_t base, uint64_t size, uint64_t code_end,
+                      int (*store)(uint64_t addr, const uint8_t *bytes, int n)) {
+    // bottom first: the top mod's data writes land last and win, as its RomFS files do. Patch sets are only switches
+    // (rt_patch_switch), so their order does not matter.
+    for (int i = nmods - 1; i >= 0; i--) {
+        const ModInfo *m = &mods[i];
+        if (!start_enabled[i] || m->problem != MOD_OK || !(m->nsets || m->ndata)) continue;
+        // the data first: a mod whose data writes cannot all be stored keeps its code patches off too, as half of a
+        // patch file is worse than none
+        long stored = size ? mods_exefs_data(m, base, size, code_end, store) : 0;
+        if (stored < 0) {
+            rt_log("mods: %s: its ExeFS patch files stay off: their data writes could not all be stored", m->name);
+            continue;
+        }
+        for (int k = 0; k < m->nsets; k++) set_on(m, m->sets[k]);
         char over[256];
         snprintf(over, sizeof over, "%s", m->overrides);
         for (char *s = strtok(over, ","); s; s = strtok(NULL, ",")) {
@@ -395,10 +588,20 @@ void mods_exefs_switch(void) {
             if (set >= 0) rt_patch_switch(set, 0);
             rt_log("mods: %s replaces the port's patch set %s; it stays off", m->name, s);
         }
-        rt_log("mods: %s: %d ExeFS patch files on", m->name, m->nsets);
+        rt_log("mods: %s: %d ExeFS patch files on, %ld bytes of main's data stored", m->name, m->nsets + m->ndata,
+               stored);
     }
 }
 
+void mods_exefs_switch(void) {
+    mods_scan();
+    const RtLoaded *main_mod = NULL;
+    for (int i = 0; ldr_module(i); i++)
+        if (!strcmp(ldr_module(i)->name, "main")) main_mod = ldr_module(i);
+    if (main_mod) mods_exefs_apply(main_mod->base, main_mod->size, main_mod->text_end, NULL);
+    else mods_exefs_apply(0, 0, 0, NULL);
+}
+
 int mods_exefs_pending(uint32_t *key) {
     int n = 0;
     for (int i = 0; i < nmods; i++) {

runtime/mods.h

@@ -2,7 +2,8 @@
 // them: <mod>/romfs, <mod>/exefs and <mod>/cheats, the Atmosphere layout (atmosphere/contents/<title id>/romfs,
 // atmosphere/exefs_patches/<name>/*.ips) or TKMM's merged output (romfs or romfslite, exefs, cheats). ExeFS patch
 // files (.ips, .pchtxt) for the game's main are patch sets the setup builds into the game (tools/exefs_patches.py),
-// switched with their mod (mods_exefs_switch). The list is scanned once at start (romfs.c overlay_init);
+// switched with their mod (mods_exefs_switch), and data writes stored into main before the game runs (mods_exefs_data).
+// The list is scanned once at start (romfs.c overlay_init);
 // its order and switches live in totk.ini [mods] (TOTK_MODS_ORDER: names top first, separated by '|';
 // TOTK_MODS_DISABLED: names switched off). The top mod wins a file two mods replace.
 #pragma once
@@ -36,10 +37,17 @@ typedef struct {
     uint32_t pending, pending_key;          // ExeFS patch files the build lacks; FNV-1a of their names and contents
     int nsets;
     uint32_t sets[16];                      // the built patch sets mod_<hash> of its ExeFS patch files
+    int ndata;
+    uint32_t data[16];                      // the hashes of its ExeFS patch files with data writes only (no set)
+    uint32_t data_bytes;                    // the bytes its usable ExeFS patch files write into main's data, as their
+                                            // status lines count them ("data <bytes>")
     char overrides[256];                    // the port's own patch sets its patches replace, comma separated
 } ModInfo;
 
 void mods_scan(void);              // once; later calls return at once
+// Every switched-on mod folder's RomFS root, without the rest of the scan (no ExeFS checks, no order): for mem_init,
+// which runs before the paths mods_scan needs are set (resolution.c mod_tex_heap_kib). cb gets the name and root.
+void mods_peek_romfs(void (*cb)(const char *name, const char *romfs, void *ctx), void *ctx);
 const char *mods_dir(void);        // the folder scanned (default: <dir of totk.exe>/mods)
 int mods_count(void);
 ModInfo *mods_at(int i);           // 0 = top of the order; NULL out of range
@@ -48,8 +56,25 @@ ModInfo *mods_at(int i);           // 0 = top of the order; NULL out of range
 void mods_set_enabled(int i, int on);
 void mods_move(int i, int delta);  // -1 up, +1 down; nothing at the ends
 int mods_changed(void);            // 1 when the order or a switch differs from the start
-// After rt_patches_init: the patch sets of the enabled mods' ExeFS patches on, the port's own sets they replace off.
+// After ldr_load and rt_patches_init, before any game code: the patch sets of the enabled mods' ExeFS patches on, the
+// port's own sets they replace off, and their data writes stored into main (mods_exefs_data).
 void mods_exefs_switch(void);
+// What mods_exefs_switch does with main's image at base (size bytes, code below code_end; size 0: no data stored), for
+// every enabled mod from the bottom one up, so that the top mod's data writes win where two mods write the same bytes.
+// store as for mods_exefs_data.
+void mods_exefs_apply(uint64_t base, uint64_t size, uint64_t code_end,
+                      int (*store)(uint64_t addr, const uint8_t *bytes, int n));
+// Stores the data writes of mod m's ExeFS patch files (build-cache/exefs.data) into main's image at base: size bytes,
+// code below code_end. A DT_RELR word gets base added, as the dynamic linker adds it. Every line is checked before the
+// first store: one that falls outside [code_end, size) or does not belong to a usable file of m keeps them all out.
+// store writes n bytes at a guest address, 1 on success; NULL: into the loaded main, whose .rodata is read-only.
+// Returns the bytes stored, -1 when a line kept them out or a store failed. --probe-mods passes a buffer and memcpy.
+long mods_exefs_data(const ModInfo *m, uint64_t base, uint64_t size, uint64_t code_end,
+                     int (*store)(uint64_t addr, const uint8_t *bytes, int n));
+// --probe-mods: the texts of build-cache/exefs.status and exefs.data instead of the files (NULL: read the file), and a
+// fresh scan at the next mods_scan.
+void mods_probe_cache(const char *status, const char *data);
+void mods_rescan(void);
 // The ExeFS patch files of the enabled mods that the build lacks, their FNV-1a mixed into *key (cheats.c starts the
 // rebuild for them).
 int mods_exefs_pending(uint32_t *key);

runtime/mods_probe.c

@@ -39,6 +39,93 @@ static int remove_cb(const char *name, int kind, uint64_t size, void *ctx) {
     return 1;
 }
 
+static uint32_t fnv(const char *p, size_t n) {
+    uint32_t h = 0x811C9DC5u;
+    for (size_t i = 0; i < n; i++) h = (h ^ (uint8_t)p[i]) * 0x01000193u;
+    return h;
+}
+
+static ModInfo *by_name(const char *name) {
+    for (int i = 0; mods_at(i); i++)
+        if (!strcmp(mods_at(i)->name, name)) return mods_at(i);
+    return NULL;
+}
+
+// mods_exefs_data's store into a buffer standing in for main
+static int store_buffer(uint64_t a, const uint8_t *p, int n) {
+    memcpy((void *)(uintptr_t)a, p, (size_t)n);
+    return 1;
+}
+
+// A second scan of a folder of its own: data writes of ExeFS patch files (build-cache/exefs.data, given as text).
+static void probe_data(const char *root) {
+    remove_tree(root);
+    plat_file_mkdir_p(root);
+    const char *bid = *mods_main_build_id() ? mods_main_build_id() : "0";
+    char n_ips[160], p_ips[160], q_ips[160], x_ips[160], y_ips[160];
+    snprintf(n_ips, sizeof n_ips, "N/exefs/%s.ips", bid);
+    snprintf(p_ips, sizeof p_ips, "P/exefs/%s.ips", bid);
+    snprintf(q_ips, sizeof q_ips, "Q/exefs/%s.ips", bid);
+    snprintf(x_ips, sizeof x_ips, "X/exefs/%s.ips", bid);
+    snprintf(y_ips, sizeof y_ips, "Y/exefs/%s.ips", bid);
+    const char *files[] = {n_ips, p_ips, q_ips, x_ips, y_ips};
+    for (size_t i = 0; i < sizeof files / sizeof *files; i++) put(root, files[i]);
+    uint32_t kn = fnv(n_ips, strlen(n_ips)), kp = fnv(p_ips, strlen(p_ips)), kq = fnv(q_ips, strlen(q_ips));
+    uint32_t kx = fnv(x_ips, strlen(x_ips)), ky = fnv(y_ips, strlen(y_ips));
+    uint32_t c = fnv("probe", 5);
+    char status[512], data[768];
+    snprintf(status, sizeof status,
+             "%08X\t%08X\tdata\t-\tdata 11\n%08X\t%08X\tdata\t-\tdata 1\n%08X\t%08X\tdata\t-\tdata 4\n"
+             "%08X\t%08X\tdata\t-\tdata 1\n%08X\t%08X\tdata\t-\tdata 1\n",
+             kn, c, kp, c, kq, c, kx, c, ky, c);
+    snprintf(data, sizeof data,
+             "# file hash, content hash, offset in main, bytes[, base]\n"
+             "%08X\t%08X\t10\t414243\n"                   // N: three bytes of data
+             "%08X\t%08X\t18\t0800000000000000\tbase\r\n"  // N: a DT_RELR word, 8 plus main's base (a CRLF line)
+             "DEADBEEF\t%08X\t20\t99\n"                   // another file's: not N's
+             "%08X\t%08X\t4\t77\n"                         // P: below main's data
+             "%08X\t%08X\t30\t66\n"                        // Q: 1 of the 4 bytes its status counts
+             "%08X\t%08X\t28\tBB\n"                        // X: the byte Y writes too
+             "%08X\t%08X\t28\tAA\n",                       // Y, above X in the order: its byte stays
+             kn, c, kn, c, c, kp, c, kq, c, kx, c, ky, c);
+    mods_probe_cache(status, data);
+    cfg_set("TOTK_MODS_DIR", root);
+    cfg_set("TOTK_MODS_ORDER", "Y");   // Y on top, the rest by name: Y N P Q X
+    cfg_set("TOTK_MODS_DISABLED", "");
+    mods_rescan();
+    mods_scan();
+    ModInfo *n = by_name("N"), *p = by_name("P"), *q = by_name("Q");
+    CHECK(n && p && q && by_name("X") && by_name("Y"), "second scan: five mods");
+    if (n && p && q) {
+        CHECK(n->kind == MOD_EXEFS && n->problem == MOD_OK && n->ndata == 1 && !n->nsets,
+              "an ExeFS patch file with data writes only makes a usable mod");
+        uint8_t img[0x40];
+        memset(img, 0, sizeof img);
+        uint64_t base = (uint64_t)(uintptr_t)img, word;
+        long stored = mods_exefs_data(n, base, sizeof img, 8, store_buffer);
+        memcpy(&word, img + 0x18, 8);
+        CHECK(stored == 11 && !memcmp(img + 0x10, "ABC", 3) && word == base + 8 && !img[0x20],
+              "data writes: bytes as they are, a DT_RELR word rebased, other files' lines left");
+        memset(img, 0x5A, sizeof img);
+        int untouched = 1;
+        CHECK(mods_exefs_data(p, base, sizeof img, 8, store_buffer) == -1, "a line below main's data keeps all out");
+        CHECK(n->data_bytes == 11 && q->data_bytes == 4, "the status lines count the data bytes");
+        CHECK(mods_exefs_data(q, base, sizeof img, 8, store_buffer) == -1,
+              "a data file holding fewer bytes than the status counts keeps all out");
+        for (size_t i = 0; i < sizeof img; i++) untouched &= img[i] == 0x5A;
+        CHECK(untouched, "nothing stored when a line does not fit or bytes are missing");
+        // every enabled mod as at start: X and Y write the same byte, and Y, the higher one, wins as its RomFS files
+        // would; P and Q stay out as above
+        memset(img, 0, sizeof img);
+        CHECK(EQ(mods_at(0)->name, "Y") && EQ(mods_at(4)->name, "X"), "mods_order puts Y on top and X at the bottom");
+        mods_exefs_apply(base, sizeof img, 8, store_buffer);
+        CHECK(img[0x28] == 0xAA, "two mods write the same byte: the top mod's byte stays");
+        CHECK(!memcmp(img + 0x10, "ABC", 3) && !img[0x30], "the other mods' data as each one stores it alone");
+    }
+    mods_probe_cache(NULL, NULL);
+    remove_tree(root);
+}
+
 // s ends in tail, either path separator matching either.
 static int ends_with(const char *s, const char *tail) {
     size_t n = strlen(s), t = strlen(tail);
@@ -126,7 +213,7 @@ int probe_mods(void) {
         CHECK(EQ(cfg_get("TOTK_MODS_DISABLED"), "A"), "switching A off writes it");
         CHECK(mods_changed(), "changes seen");
     }
-    remove_tree(root);
+    probe_data(root);
     printf("mods: %s\n", fails ? "FAIL" : "ok");
     int table = restbl_probe();
     return fails || table ? 1 : 0;

runtime/mouse_direct.c

@@ -119,6 +119,7 @@ void input_direct_mouse_row(int32_t dx, float sensitivity, uint64_t now, InputMo
 #define HOST_MOUSE_ORBIT_BEGIN (RT_PATCH_PAGE + 0xfb0)
 #define HOST_MOUSE_ORBIT_ANGLES (RT_PATCH_PAGE + 0xfa0)
 #define HOST_MOUSE_HAND (RT_PATCH_PAGE + 0xf90)
+#define HOST_MOUSE_ORBIT_STATE (RT_PATCH_PAGE + 0xf70)
 static _Thread_local uint64_t mouse_input_object;
 
 // Where a game version keeps what the hooks read and write (recomp/patches-v143.txt, patches-v121.txt). The camera
@@ -162,8 +163,8 @@ static void sample_mark_taken(uint64_t generation) {
 static _Thread_local struct { uint64_t object; MouseSample mouse; } orbit;
 typedef struct {
     uint64_t object, generation, input, seen;
-    float pitch, original_target;
-    int hold_pitch, target_saved, resume;
+    float pitch;
+    int hold_pitch, resume;
 } OrbitTrack;
 static OrbitTrack orbit_seen[16];
 
@@ -268,19 +269,17 @@ static uint32_t player_state_logged(void) {
     return f;
 }
 
-// main+0x191d78 builds the pitch target from +0x15c and +0x11c; the orbit update filters +0x108 toward it and adds
-// (1 - +0x88) * +0x10c at output. Updating only +0x108 makes the camera spring back to the unchanged target.
+// The orbit output adds (1 - +0x88) * +0x10c to its filtered pitch at +0x108.
+// Hold the final angle in orbit_angles. Do not overwrite the configured target
+// at +0x15c: main+0x191534 also uses it as the reference for camera distance.
+// Replacing that reference with the held pitch changes the native return from
+// aiming and collision close-ups, even though no distance field is written.
 static float orbit_pitch_offset(uint64_t object) {
     return (1 - guest_float(object + 0x88)) * guest_float(object + 0x10c);
 }
 
-static void orbit_pitch_target(uint64_t object, float pitch) {
-    guest_float_store(object + 0x15c, pitch - orbit_pitch_offset(object) - guest_float(object + 0x11c));
-}
-
 static void orbit_pitch_release(OrbitTrack *track) {
-    if (track->target_saved) guest_float_store(track->object + 0x15c, track->original_target);
-    track->hold_pitch = track->target_saved = track->resume = 0;
+    track->hold_pitch = track->resume = 0;
 }
 
 // The pitch the camera has now (the filtered pitch plus its vertical offset), as orbit_angles leaves it
@@ -288,8 +287,8 @@ static float orbit_pitch_now(uint64_t object) {
     return wrap_degrees(guest_float(object + 0x108) + orbit_pitch_offset(object));
 }
 
-// The camera stays where it is now: the mouse holds the pitch the camera has (orbit_begin writes its target), so
-// neither the game's default target nor the mouse's target from before an aim or a lock-on pulls it back.
+// Resume from the pitch the camera reached during the handoff. orbit_angles
+// holds this final angle without replacing the native configuration reference.
 static void orbit_pitch_settle(OrbitTrack *track) {
     track->pitch = orbit_pitch_now(track->object);
     track->hold_pitch = 1, track->resume = 0;
@@ -471,7 +470,7 @@ static void orbit_begin(Cpu *c) {
             memcpy(&override, (void *)(uintptr_t)(input + 0x15c), 1);
             plat_mutex_lock(&sample_lock);
             OrbitTrack *track = orbit_track(object);
-            if (track->input != input) track->hold_pitch = track->target_saved = track->resume = 0;
+            if (track->input != input) track->hold_pitch = track->resume = 0;
             uint64_t now = plat_now_ns();
             // A gamepad's vertical input gives the pitch back to the game's own target. The game's lock-on or override
             // (aiming) drives the camera itself: the mouse lets go without restoring the target it had before, and
@@ -484,7 +483,6 @@ static void orbit_begin(Cpu *c) {
             track->seen = now;
             take = moved || track->hold_pitch;
             if (take) track->input = input;
-            if (track->hold_pitch) orbit_pitch_target(object, track->pitch);
             plat_mutex_unlock(&sample_lock);
             if (moved) {   // the mouse's part of this step's stick goes to the final angles instead
                 guest_float_store(input + 0x420, m.pad[0]);
@@ -500,6 +498,42 @@ static void orbit_begin(Cpu *c) {
     c->x[8] = guest_pointer(object + 8);   // displaced ldr x8, [x0, #8]
 }
 
+// The native manual-orbit state (4) integrates both pitch and distance from
+// the stick. Direct mouse orbit bypasses those integrators, so entering that
+// state during an aim return can latch the close-up's radius indefinitely.
+// Select the native automatic-distance state (3) at the existing state decision
+// point, before its transition initializer runs (main+0x18ea34, 1.4.3).
+// This retains the game's own radius curve, transition easing and collision.
+static void orbit_state(Cpu *c) {
+    uint64_t object = c->x[20];
+    uint32_t override;
+    memcpy(&override, (void *)(uintptr_t)(object + 0x188), sizeof override);
+    if (rt_game_version == 143 && direct_mode() && (uint32_t)c->x[10] == 4 && !override) {
+        MouseSample m = sample_read();
+        uint64_t input = guest_pointer(object + 0x20);
+        if (m.object == input && m.camera_enabled && plat_now_ns() - m.when < 150000000ull &&
+            m.pad[0] == 0 && m.pad[1] == 0 &&
+            !(input_buttons_last() & ((1ull << B_ZR) | (1ull << B_R)))) {
+            uint64_t held = guest_pointer(input + 0x148);
+            uint8_t flags;
+            memcpy(&flags, (void *)(uintptr_t)(input + 0x15c), 1);
+            plat_mutex_lock(&sample_lock);
+            OrbitTrack *track = orbit_track(object);
+            int owned = track->input == input && track->hold_pitch;
+            plat_mutex_unlock(&sample_lock);
+            if (owned && !(held & 0x20) && !(flags & 0x40)) {
+                c->x[10] = 3;
+                static int logged;
+                if (!logged) {
+                    logged = 1;
+                    rt_log("mouse: native distance transition retained for direct orbit");
+                }
+            }
+        }
+    }
+    c->x[8] = override; // displaced ldr w8, [x20, #0x188]
+}
+
 // The final pitch and yaw are on the stack (1.4.3 sp + 0x44 / 0x48, 1.2.1 sp + 0x3c / 0x40)
 static void orbit_angles(Cpu *c) {
     const MouseLayout *ml = mouse_layout();
@@ -526,9 +560,7 @@ static void orbit_angles(Cpu *c) {
             guest_float_store(yaw_at, wrap_degrees(yaw + yaw_delta));
             guest_float_store(object + 0x108, wrap_degrees(new_pitch - orbit_pitch_offset(object)));
             guest_float_store(object + 0x120, wrap_degrees(guest_float(object + 0x120) + yaw_delta));
-            if (!track->target_saved) track->original_target = guest_float(object + 0x15c), track->target_saved = 1;
             track->pitch = new_pitch; track->input = orbit.mouse.object; track->hold_pitch = 1;
-            orbit_pitch_target(object, new_pitch);
             static int logged;
             if (!logged) {
                 logged = 1;
@@ -1062,6 +1094,7 @@ __attribute__((constructor)) static void mouse_direct_register(void) {
     rt_register_host(HOST_MOUSE_DIRECT, camera_input);
     rt_register_host(HOST_MOUSE_ACTIVITY, camera_activity);
     rt_register_host(HOST_MOUSE_ORBIT_BEGIN, orbit_begin);
+    if (rt_game_version == 143) rt_register_host(HOST_MOUSE_ORBIT_STATE, orbit_state);
     rt_register_host(HOST_MOUSE_ORBIT_ANGLES, orbit_angles);
     rt_register_host(HOST_MOUSE_HAND, hand_input);
     rt_register_host(HOST_MOUSE_LOAD, camera_load);

runtime/nvn_api.h

@@ -43,6 +43,15 @@ _Static_assert(sizeof(nvn_mapping_t) == 0x28, "NVNmappingRequest stride");
 
 const pool_t *nvn_pool_find(uint64_t addr, uint64_t len);  // GPU-visible pool holding [addr, addr+len], NULL if none
 uint64_t nvn_new_serial(void);                             // next tex_t / prog_t serial, from 1
+// TOTK_SYNC_AHEAD (nvn_api_core.c): nvnQueuePresentTexture notes the newest fence at each present; the newest
+// previous-frame value the game was told is done although it may not be (0: none). The back end's shadow copies
+// wait for it or are queued for the GPU (vk_sync.c prologue), so the GPU never reads CPU writes the game made under
+// that assumption too early.
+void nvn_sync_ahead_present(void);
+uint64_t nvn_sync_ahead_guard(void);
+int vk_mem_shadow_on(void);   // vk_memory.c: 1 with shadow storage on (valid once the Vulkan back end is up)
+void vk_mem_shadow_fence(void);   // vk_memory.c: returns once every shadow sync that read the old guard is done
+void gfx_replay_wait_value(uint64_t v);   // vk_backend.c: the replay thread submitted value v
 uint64_t nvn_fence_last(void);                             // fence value of the newest submit (0 before the first)
 int nvn_depth_minus_w(void);                               // 1 after DeviceSetDepthMode(NEAR_IS_MINUS_W), else 0
 void nvn_guest_written(uint64_t addr, uint64_t size);      // the host or the game wrote pool storage: back end re-reads

runtime/nvn_api_cmd.c

@@ -141,10 +141,14 @@ static void pool_drain(void) {
 // system-managed page file, so the same request succeeds a moment later, and a stall of some frames is better than
 // ending the game. Frees the pooled buffers first. NULL only after about 30 s without memory; the caller ends the game.
 // Never called with a lock held.
+static void big_drain(void);
+
 static void *realloc_wait(void *old, size_t n, const char *what) {
     void *p = realloc(old, n);
     if (p) return p;
     pool_drain();
+    big_drain();
+    rt_oom_request();   // the Vulkan back end gives memory back at its next submit (oom.c)
     static uint32_t reports;
     int report = __atomic_fetch_add(&reports, 1, __ATOMIC_RELAXED) < 8;
     if (report) {
@@ -406,8 +410,10 @@ void nvn_pins_stats(uint64_t *live, uint64_t *freed) {
 // adding control memory again releases the copies recorded there without walking every live copy. 4 KiB pages cost
 // about 500 lookups per AddControlMemory under the exclusive lock (1.4.3 field: 53 calls a frame, 2 MiB ranges, 3.7
 // copies per page), and the recording threads waited on it.
-// deferred: 0 unknown, 1 none, 2 some; pins: texture pins its ops name, counted in their refs (pin_refs)
-typedef struct { uint64_t magic, bytes, made_at, used_at, deferred, pins; } copy_prefix;
+// deferred: 0 unknown, 1 none, 2 some; pins: texture pins its ops name, counted in their refs; pin_v: those pins, in
+// record order, or NULL (then pin_refs walks the records); cap: bytes of the block from the large-block cache
+// (copy_block), 0 for a plain heap block
+typedef struct { uint64_t magic, bytes, made_at, used_at, deferred, pins, cap; const tex_t **pin_v; } copy_prefix;
 _Static_assert(sizeof(copy_prefix) % 8 == 0, "records stay 8-byte aligned");
 typedef struct { uintptr_t lo, hi; uint8_t *copy; } cmd_copy;
 typedef struct { cmd_copy *v; uint32_t n, cap; } copy_bucket;
@@ -429,15 +435,117 @@ static void zombie_add(copy_prefix *pf) {   // copy_lock held
     zombies[nzombies++] = pf;
 }
 
-static void copy_free(copy_prefix *pf) {   // copy_lock held
-    if (pf->pins) pin_refs((const uint8_t *)(pf + 1), -1);
+// Blocks of large copies are kept for the next ones. 1.4.3 makes two copies of 0.5-2 MiB a frame (the per-frame
+// recordings that inline many others); above the heap's direct-allocation threshold (about 512 KiB) every malloc mapped
+// fresh zero-filled pages and every free unmapped them, so the copy ran at half the speed of a small one and the free
+// held copy_lock for tens of microseconds. A block is reused for a copy that fills at least half of it.
+#define BIG_MIN ((size_t)256 << 10)        // smaller blocks come from the heap and go back to it
+#define BIG_MAX ((size_t)16 << 20)         // a larger block is never kept
+#define BIG_SLOTS 8
+static PlatMutex big_lock = PLAT_MUTEX_INIT;   // lock order: copy_lock, then big_lock
+static struct { copy_prefix *p; size_t cap; } big[BIG_SLOTS];
+static uint32_t nbig;
+
+static void big_drain(void) {
+    plat_mutex_lock(&big_lock);
+    while (nbig) free(big[--nbig].p);
+    plat_mutex_unlock(&big_lock);
+}
+
+// A block for a copy of n bytes after its prefix; prefix.cap says where it goes back (copy_block_free). wait: a failed
+// allocation waits for memory (realloc_wait), which is never done with a lock held; else it ends the game at once.
+static copy_prefix *copy_block(size_t n, int wait, const char *what) {
+    size_t total = sizeof(copy_prefix) + n, cap = 0;
+    copy_prefix *pf = NULL;
+    if (total >= BIG_MIN && total <= BIG_MAX) {
+        plat_mutex_lock(&big_lock);
+        uint32_t best = BIG_SLOTS;
+        for (uint32_t i = 0; i < nbig; i++)
+            if (big[i].cap >= total && big[i].cap / 2 <= total && (best == BIG_SLOTS || big[i].cap < big[best].cap))
+                best = i;
+        if (best < BIG_SLOTS) {
+            pf = big[best].p;
+            cap = big[best].cap;
+            big[best] = big[--nbig];
+        }
+        plat_mutex_unlock(&big_lock);
+        if (!pf) cap = (total + total / 8 + 0xffff) & ~(size_t)0xffff;   // room for the next frame's slightly larger one
+    }
+    if (!pf) pf = wait ? realloc_wait(NULL, cap ? cap : total, what) : malloc(cap ? cap : total);
+    if (!pf) rt_fatal("nvn: %s allocation (%zu bytes)", what, cap ? cap : total);
+    pf->cap = cap;
+    return pf;
+}
+
+static void copy_block_free(copy_prefix *pf) {
+    size_t cap = pf->cap;
+    if (cap) {
+        plat_mutex_lock(&big_lock);
+        if (nbig < BIG_SLOTS) {
+            big[nbig].p = pf;
+            big[nbig++].cap = cap;
+            pf = NULL;
+        } else {
+            uint32_t small = 0;
+            for (uint32_t i = 1; i < nbig; i++)
+                if (big[i].cap < big[small].cap) small = i;
+            if (big[small].cap < cap) {
+                copy_prefix *drop = big[small].p;
+                big[small].p = pf;
+                big[small].cap = cap;
+                pf = drop;
+            }
+        }
+        plat_mutex_unlock(&big_lock);
+    }
     free(pf);
 }
 
-static void zombie_sweep(uint64_t replayed) {   // copy_lock held
+// Counts (delta 1) or uncounts (-1) the pins a copy names: from its list, without walking its records. A copy of a
+// per-frame recording is 0.5-2 MiB of records naming a few pins; walking them again at every free cost about 40 us.
+static void copy_pins_ref(const copy_prefix *pf, int delta) {
+    if (!pf->pins) return;
+    if (!pf->pin_v) {
+        pin_refs((const uint8_t *)(pf + 1), delta);
+        return;
+    }
+    uint64_t now = __atomic_load_n(&nsubmits, __ATOMIC_RELAXED);
+    for (uint64_t i = 0; i < pf->pins; i++) pin_ref(pf->pin_v[i], delta, now);
+}
+
+// A copy in no map and no zombie list: nothing else can reach it, so copy_lock need not be held.
+static void copy_free(copy_prefix *pf) {
+    copy_pins_ref(pf, -1);
+    free((void *)pf->pin_v);
+    copy_block_free(pf);
+}
+
+// Copies retired under copy_lock and freed after it is released: the frees, the unpinning walk over their records and
+// the large blocks kept the lock exclusive for most of release_copies, while the game's other recording threads waited
+// for it in copy_touch (1.4.3 town: three threads, 0.6 ms a frame of waiting).
+typedef struct { copy_prefix **v; uint32_t n, cap; } dead_list;
+static __thread dead_list dead;
+
+static void dead_add(copy_prefix *pf) {
+    if (dead.n == dead.cap) {
+        uint32_t cap = dead.cap ? dead.cap * 2 : 256;
+        copy_prefix **v = realloc(dead.v, sizeof *v * cap);
+        if (!v) { copy_free(pf); return; }   // out of memory: free it here, under the lock, as before
+        dead.v = v;
+        dead.cap = cap;
+    }
+    dead.v[dead.n++] = pf;
+}
+
+static void dead_free(void) {   // copy_lock not held
+    for (uint32_t i = 0; i < dead.n; i++) copy_free(dead.v[i]);
+    dead.n = 0;
+}
+
+static void zombie_sweep(uint64_t replayed) {   // copy_lock held; the swept copies go to the dead list
     for (uint32_t i = 0; i < nzombies;) {
         if (zombies[i]->used_at <= replayed) {
-            copy_free(zombies[i]);
+            dead_add(zombies[i]);
             zombies[i] = zombies[--nzombies];
         } else {
             i++;
@@ -477,13 +585,16 @@ static uint64_t copy_touch(uint64_t handle, uint64_t now, const char *what) {
 }
 
 // The recording [p, p + n), OP_BEGIN to OP_END (the recording buffer is one block: recording emits no OP_JUMP), copied
-// into one host block of its exact size. One allocation and one copy: growing the copy record by record cost the
-// game's main thread about 1 ms a frame in reallocations and small copies (1.4.3, field).
+// into one host block (copy_block: its exact size, or a kept large block). One allocation and one copy: growing the
+// copy record by record cost the game's main thread about 1 ms a frame in reallocations and small copies (1.4.3,
+// field). The pins the records name are counted and listed (copy_prefix.pin_v) on the way.
 static uint8_t *flatten_recording(const uint8_t *p, size_t n) {
-    uint8_t *out = realloc_wait(NULL, sizeof(copy_prefix) + n, "command copy");
-    if (!out) rt_fatal("nvn: command copy allocation");
+    copy_prefix *block = copy_block(n, 1, "command copy");
+    uint8_t *out = (uint8_t *)block;
     memcpy(out + sizeof(copy_prefix), p, n);
+    static __thread struct { const tex_t **v; uint64_t cap; } named;   // the pins of this recording, reused
     uint64_t pins_named = 0;
+    int listed = 1;   // 0 after a failed growth: the copy goes without a list
     for (size_t at = 0;;) {   // the records stay checked: a corrupt one stops here, not in the replay
         op_hdr h;
         memcpy(&h, p + at, sizeof h);
@@ -493,6 +604,14 @@ static uint8_t *flatten_recording(const uint8_t *p, size_t n) {
         const tex_t *pinned[9];
         int k = op_pins(h.op, out + sizeof(copy_prefix) + at + 8, pinned);
         for (int i = 0; i < k; i++) pin_ref(pinned[i], 1, 0);
+        if (k && listed && pins_named + (uint64_t)k > named.cap) {
+            uint64_t cap = named.cap ? named.cap * 2 : 64;
+            while (cap < pins_named + (uint64_t)k) cap *= 2;
+            const tex_t **v = realloc((void *)named.v, sizeof *v * cap);
+            if (v) named.v = v, named.cap = cap;
+            else listed = 0;
+        }
+        if (k && listed) memcpy((void *)(named.v + pins_named), pinned, sizeof *pinned * (size_t)k);
         pins_named += (uint64_t)k;
         at += h.size;
         if (h.op == OP_END) {
@@ -500,7 +619,10 @@ static uint8_t *flatten_recording(const uint8_t *p, size_t n) {
             break;
         }
     }
-    *(copy_prefix *)out = (copy_prefix){.magic = COPY_MAGIC, .bytes = n, .pins = pins_named};
+    const tex_t **pin_v = NULL;
+    if (pins_named && listed && (pin_v = malloc(sizeof *pin_v * pins_named)))
+        memcpy((void *)pin_v, (const void *)named.v, sizeof *pin_v * pins_named);
+    *block = (copy_prefix){.magic = COPY_MAGIC, .bytes = n, .pins = pins_named, .cap = block->cap, .pin_v = pin_v};
     return out + sizeof(copy_prefix);
 }
 
@@ -550,7 +672,7 @@ static void release_copies(uintptr_t lo, uintptr_t hi) {
             copy_bytes -= pf->bytes;
             pf->magic = 0;
             if (pf->used_at > replayed) zombie_add(pf);   // a queued submission still reads it
-            else copy_free(pf);
+            else dead_add(pf);
             b->v[i] = b->v[--b->n];
         }
         if (!b->n) {
@@ -560,6 +682,7 @@ static void release_copies(uintptr_t lo, uintptr_t hi) {
         }
     }
     plat_mutex_unlock(&copy_lock);
+    dead_free();
 }
 
 // ---------- CommandBuffer: object and memory
@@ -1182,11 +1305,14 @@ void nvn_submit_prepare_async(const uint64_t *handles, uint64_t *out, int n) {
         }
         out[i] = handles[i];
         if (pf->used_at > replayed && pf->used_at != now && pf->deferred != 1) {
-            size_t sz = sizeof *pf + pf->bytes;
-            copy_prefix *cl = malloc(sz);
-            if (!cl) rt_fatal("nvn: out of memory (recording clone, %zu bytes)", sz);
-            memcpy(cl, pf, sz);
-            if (cl->pins) pin_refs((const uint8_t *)(cl + 1), 1);
+            copy_prefix *cl = copy_block(pf->bytes, 0, "recording clone");   // copy_lock held
+            size_t cap = cl->cap;
+            memcpy(cl, pf, sizeof *pf + pf->bytes);
+            cl->cap = cap;
+            cl->pin_v = NULL;
+            if (pf->pin_v && (cl->pin_v = malloc(sizeof *cl->pin_v * pf->pins)))
+                memcpy((void *)cl->pin_v, (const void *)pf->pin_v, sizeof *cl->pin_v * pf->pins);
+            copy_pins_ref(cl, 1);
             cl->magic = 0;   // not a live handle: the game never sees it
             cl->used_at = now;
             zombie_add(cl);
@@ -1197,6 +1323,7 @@ void nvn_submit_prepare_async(const uint64_t *handles, uint64_t *out, int n) {
         }
     }
     plat_mutex_unlock(&copy_lock);
+    dead_free();
     for (int i = 0; i < n; i++) {
         int nd = resolve_deferred((const uint8_t *)(uintptr_t)out[i], 0);
         if (out[i] == handles[i]) ((copy_prefix *)(uintptr_t)out[i] - 1)->deferred = nd ? 2 : 1;

runtime/nvn_api_core.c

@@ -471,6 +471,49 @@ NVN_API(nvnQueueFenceSync) {
     s->fenced = 1;
     s->fence = nvn_fence_last();
 }
+// TOTK_SYNC_AHEAD=1: one more frame in flight. Each frame the game waits (one nvnSyncWait, frame_wait_lr) for
+// the frame it presented last; on the Switch that work is long done by then, here it is still in the replay and on the
+// GPU (about 11 ms from the submit), and the wait sets the frame time. With the switch on, that wait, and only that
+// one, waits for the frame before it instead. What the game reads back of GPU results after the wait can then be a
+// frame old. The game's CPU writes to memory the GPU still reads reach the GPU only through shadow copies, which are
+// queued for the GPU or wait (nvn_sync_ahead_guard). The wait still lasts until the replay thread submitted the asked
+// value, so no copy of that value itself picks up those writes. pa_fence: the newest fence at the latest present and
+// the one before, present thread only for writes. pa_guard: the newest value the game was told is done early.
+static uint64_t pa_fence[2], pa_guard;
+// The return address of that wait. 1.2.1: 0x7000b23dfc. 1.4.3: 0x70002db99c, the second of the two nvnSyncWait calls
+// in f_70002db8ac; like on 1.2.1 its sync comes from a two-entry ring indexed per frame, and the first call waits on
+// one object's sync behind a mode check. 0: no site known, the switch stays off.
+static uint64_t frame_wait_lr(void) {
+    return rt_game_version == 121 ? 0x7000b23dfcull : rt_game_version == 143 ? 0x70002db99cull : 0;
+}
+static int sync_ahead(void) {
+    static int on = -1;
+    if (on < 0) {
+        on = cfg_bool("TOTK_SYNC_AHEAD");
+        // Without shadow storage the GPU reads the game's own memory, which the game writes ahead: nothing guards
+        // that. The Vulkan back end sets the mode while it starts, before the game's first present or wait; the null
+        // back end never does.
+        if (on && !vk_mem_shadow_on()) {
+            rt_log("nvn: TOTK_SYNC_AHEAD needs shadow storage (TOTK_SHADOW=1 on native Windows); "
+                   "the game waits for the GPU as before");
+            on = 0;
+        }
+        if (on && !frame_wait_lr()) {
+            rt_log("nvn: TOTK_SYNC_AHEAD: no frame wait site is known for this game version; the game waits for the "
+                   "GPU as before");
+            on = 0;
+        }
+        if (on) rt_log("nvn: TOTK_SYNC_AHEAD: a wait for the frame presented last waits for the frame before it");
+    }
+    return on;
+}
+void nvn_sync_ahead_present(void) {
+    if (!sync_ahead()) return;
+    __atomic_store_n(&pa_fence[1], __atomic_load_n(&pa_fence[0], __ATOMIC_RELAXED), __ATOMIC_RELEASE);
+    __atomic_store_n(&pa_fence[0], nvn_fence_last(), __ATOMIC_RELEASE);
+}
+uint64_t nvn_sync_ahead_guard(void) { return __atomic_load_n(&pa_guard, __ATOMIC_ACQUIRE); }
+
 // NVNsyncWaitResult nvnSyncWait(const NVNsync *s, uint64_t timeout_ns): a sync never fenced reports a timeout at once;
 // a fenced one is released only after the back end completed that fence value
 NVN_API(nvnSyncWait) {
@@ -478,13 +521,34 @@ NVN_API(nvnSyncWait) {
     uint64_t timeout = arg_x(c, 1);
     if (!s->fenced) { ret_x(c, NVN_SYNC_WAIT_RESULT_TIMEOUT_EXPIRED); return; }
     const GfxBackend *b = gfx_backend();
-    if (b->fence_done() >= s->fence) { b->cpu_synced(s->fence); ret_x(c, NVN_SYNC_WAIT_RESULT_ALREADY_SIGNALED); return; }
+    uint64_t want = s->fence;
+    // Only the game's per-frame wait lags, and not a poll (timeout 0).
+    if (timeout && c->x[30] == frame_wait_lr() && sync_ahead() && b->fence_done() < want) {
+        uint64_t p0 = __atomic_load_n(&pa_fence[0], __ATOMIC_ACQUIRE);
+        uint64_t p1 = __atomic_load_n(&pa_fence[1], __ATOMIC_ACQUIRE);
+        if (want <= p0 && want > p1) want = p1;
+    }
+    if (want != s->fence) {   // the replay of the asked value is done first: the guard then covers every copy of it
+        {
+            NVN_BLOCKING();
+            gfx_replay_wait_value(s->fence);
+        }
+        uint64_t g = __atomic_load_n(&pa_guard, __ATOMIC_RELAXED);
+        while (s->fence > g &&
+               !__atomic_compare_exchange_n(&pa_guard, &g, s->fence, 1, __ATOMIC_RELEASE, __ATOMIC_RELAXED)) {}
+        vk_mem_shadow_fence();   // a shadow sync that read the old guard finishes before the game writes again
+    }
+    if (b->fence_done() >= want) {
+        b->cpu_synced(want == s->fence ? s->fence : b->fence_done());
+        ret_x(c, NVN_SYNC_WAIT_RESULT_ALREADY_SIGNALED);
+        return;
+    }
     int ok;
     {
         NVN_BLOCKING();
-        ok = b->fence_wait(s->fence, timeout);
+        ok = b->fence_wait(want, timeout);
     }
-    if (ok) b->cpu_synced(s->fence);
+    if (ok) b->cpu_synced(want == s->fence ? s->fence : b->fence_done());
     ret_x(c, ok ? NVN_SYNC_WAIT_RESULT_CONDITION_SATISFIED : NVN_SYNC_WAIT_RESULT_TIMEOUT_EXPIRED);
 }
 // void nvnEventBuilderSetDefaults(NVNeventBuilder *b)

runtime/nvn_api_obj.c

@@ -713,6 +713,7 @@ NVN_API(nvnQueuePresentTexture) {
     const tex_t *t = (const tex_t *)o->tex[idx];
     uint64_t period = present_period(o), now = kern_now_ns();
     uint64_t slot = now + pace(&o->pace, now, period);
+    nvn_sync_ahead_present();
     gfx_backend()->present(t, o->crop, slot);
     o->ticket[idx] = o->presents;
     o->avail[idx] = cfg_bool("TOTK_FRAME_QUEUE") ? slot : slot + period;

runtime/oom.c

@@ -0,0 +1,106 @@
+// Short out-of-memory spells: a failed allocation waits for memory instead of ending the game.
+//
+// Windows refuses commit beyond the current limit while it grows a system-managed page file, and on some drivers device
+// memory counts as process commit (vk_memory.c vk_mem_host_probe), so for a moment every allocation of the process can
+// fail and the same request succeeds a little later. The failure sites (memory.c commit, vk_memory.c vk_staging,
+// vk_texture.c create_tex, vk_exec_draw.c transient chunks, nvn_api_cmd.c realloc_wait) ask the Vulkan back end to give
+// memory back (rt_oom_request: released textures and headroom blocks, at its next submit, where its locks are free) and
+// retry for up to OOM_WAIT_MS before they end the game.
+//
+// TOTK_TEST_OOM=<site>:<n>[,<site>:<n>...] (sites commit, staging, image, chunk) fails the first n attempts at a site, so
+// the wait can be tested without running out of memory; the probes arm it with rt_oom_force.
+#include <stdlib.h>
+#include <string.h>
+
+#include "cfg.h"
+#include "host.h"
+#include "platform/platform.h"
+
+#define OOM_WAIT_MS 30000u
+#define OOM_STEP_MS 10u
+#define OOM_REPORTS 8
+#define OOM_REQUEST_EVERY 100   // retries between two release requests: a second, since a release only frees what is old enough
+
+volatile int32_t rt_oom_req;
+
+static const char *const site_name[RT_OOM_SITES] = {"commit", "staging", "image", "chunk"};
+static volatile int32_t force_left[RT_OOM_SITES];
+static volatile int32_t forced_parsed;
+static PlatOnce force_once = PLAT_ONCE_INIT;
+
+static void force_parse(void *arg) {
+    (void)arg;
+    const char *s = cfg_get("TOTK_TEST_OOM");
+    while (s && *s) {
+        const char *colon = strchr(s, ':'), *end = strchr(s, ',');
+        if (!end) end = s + strlen(s);
+        if (colon && colon < end) {
+            for (int i = 0; i < RT_OOM_SITES; i++)
+                if ((size_t)(colon - s) == strlen(site_name[i]) && !strncmp(s, site_name[i], (size_t)(colon - s))) {
+                    int n = atoi(colon + 1);
+                    __atomic_store_n(&force_left[i], n > 0 ? n : 0, __ATOMIC_RELAXED);
+                    rt_log("oom: TOTK_TEST_OOM fails the first %d %s attempts", n, site_name[i]);
+                }
+        }
+        s = *end ? end + 1 : end;
+    }
+    __atomic_store_n(&forced_parsed, 1, __ATOMIC_RELEASE);
+}
+
+int rt_oom_forced(int site) {
+    if (!__atomic_load_n(&forced_parsed, __ATOMIC_ACQUIRE)) plat_once(&force_once, force_parse, NULL);
+    if (__atomic_load_n(&force_left[site], __ATOMIC_RELAXED) <= 0) return 0;
+    return plat_dec32(&force_left[site]) >= 0;
+}
+
+void rt_oom_force(int site, int n) {
+    if (!__atomic_load_n(&forced_parsed, __ATOMIC_ACQUIRE)) plat_once(&force_once, force_parse, NULL);
+    __atomic_store_n(&force_left[site], n, __ATOMIC_RELAXED);
+}
+
+int rt_oom_forced_left(int site) {
+    int n = __atomic_load_n(&force_left[site], __ATOMIC_RELAXED);
+    return n > 0 ? n : 0;
+}
+
+void rt_oom_request(void) { __atomic_store_n(&rt_oom_req, 1, __ATOMIC_RELEASE); }
+
+int rt_oom_wait(const char *what, uint64_t bytes, int (*retry)(void *arg), void *arg) {
+    rt_oom_request();
+    static volatile int32_t reports;
+    int report = plat_inc32(&reports) <= OOM_REPORTS;
+    double proc, used, limit;
+    if (report) {
+        plat_commit_info(&proc, &used, &limit);
+        rt_log("%s of %llu bytes failed (memory commit: this process %.0f MB, the system %.0f of %.0f MB); waiting up "
+               "to %u s for memory", what, (unsigned long long)bytes, proc, used, limit, OOM_WAIT_MS / 1000);
+    }
+    uint64_t t0 = plat_now_ms();
+    for (int n = 1; plat_now_ms() - t0 < OOM_WAIT_MS; n++) {
+        plat_sleep_ms(OOM_STEP_MS);
+        if (retry(arg)) {
+            if (report)
+                rt_log("%s of %llu bytes succeeded after %llu ms", what, (unsigned long long)bytes,
+                       (unsigned long long)(plat_now_ms() - t0));
+            return 1;
+        }
+        if (n % OOM_REQUEST_EVERY == 0) rt_oom_request();
+    }
+    plat_commit_info(&proc, &used, &limit);
+    rt_log("%s of %llu bytes still failed after %u s (memory commit: this process %.0f MB, the system %.0f of %.0f MB)",
+           what, (unsigned long long)bytes, OOM_WAIT_MS / 1000, proc, used, limit);
+    return 0;
+}
+
+typedef struct { void *old, *p; size_t n; } Realloc;
+static int realloc_try(void *arg) {
+    Realloc *r = arg;
+    return (r->p = realloc(r->old, r->n)) != NULL;
+}
+
+void *rt_realloc_wait(void *old, size_t n, const char *what) {
+    void *p = realloc(old, n);
+    if (p || !n) return p;
+    Realloc r = {old, NULL, n};
+    return rt_oom_wait(what, n, realloc_try, &r) ? r.p : NULL;
+}

runtime/ov_probe.c

@@ -929,8 +929,14 @@ static void probe_menu_text(void) {
     cfg_set("TOTK_MENU_LANG", "zh-Hans");
     CHECK(menu_text(UI_MENU_TITLE) == ui_text_in("zh-Hans", UI_MENU_TITLE), "menu_text ignores MENU_LANG=zh-Hans");
     cfg_set("TOTK_MENU_LANG", "");
+    // the system language as ui_text.c resolve_menu_lang takes it: the tag, else its base language (ru-RU: ru), else en
     const char *sys_lang = ui_text_lang(plat_ui_language());
-    const char *auto_lang = ui_text_defined(sys_lang, UI_MENU_TITLE) ? sys_lang : "en";
+    char sys_base[16] = "";
+    size_t base_n = strcspn(sys_lang, "-");
+    if (sys_lang[base_n] && base_n < sizeof sys_base) memcpy(sys_base, sys_lang, base_n);
+    const char *auto_lang = ui_text_defined(sys_lang, UI_MENU_TITLE) ? sys_lang
+                            : *sys_base && ui_text_defined(sys_base, UI_MENU_TITLE) ? sys_base
+                                                                                      : "en";
     CHECK(menu_text(UI_MENU_CONTINUE) == ui_text_in(auto_lang, UI_MENU_CONTINUE), "menu_text auto ignores the system language");
 }
 

runtime/perf.c

@@ -62,6 +62,15 @@ uint64_t perf_begin(void) { return kern_now_ns(); }
 void perf_replay_thread(void) { prof_replay_tid = plat_thread_id(); }
 void perf_main_thread(void) { prof_main_tid = plat_thread_id(); }
 
+void perf_core_thread(int role) {
+    static const char *const names[] = {"game main", "game render", "vk replay", "vk recorder", "vk present"};
+    static int32_t said;
+    int r = plat_thread_perf_core();
+    if (r < 0 || role < 0 || role >= PCORE_N) return;
+    if (__atomic_fetch_or(&said, 1 << role, __ATOMIC_RELAXED) & (1 << role)) return;
+    rt_log("power: the %s thread %s the P-cores", names[role], r ? "prefers" : "could not select");
+}
+
 void perf_end(int kind, uint64_t t0) {
     if (!t0) return;
     uint32_t tid = plat_thread_id();

runtime/platform/linux.c

@@ -481,6 +481,14 @@ uint64_t plat_counter_hz(void) { return 1000000000u; }
 // inherit it from the thread that creates them.
 void plat_timer_1ms(void) { prctl(PR_SET_TIMERSLACK, 1000ul, 0, 0, 0); }
 
+int plat_power_full(void) { return -1; }
+
+int plat_perf_core_setup(char *why, size_t n) {
+    snprintf(why, n, "not done on Linux");
+    return 0;
+}
+int plat_thread_perf_core(void) { return -1; }
+
 // clock_nanosleep wakes within the timer slack (50 us by default) of its deadline; the last PLAT_SPIN_NS are spent
 // yielding so the deadline is met closely, as on Windows.
 #define PLAT_SPIN_NS 100000u
@@ -949,6 +957,10 @@ static void fault_handler(int sig, siginfo_t *si, void *uc) {
         }
         if (sig == SIGSEGV || sig == SIGBUS)
             c.access = (err & 16) ? 8 : (err & 2) ? 1 : 0, c.addr = (uint64_t)(uintptr_t)si->si_addr;
+        static const int gr[16] = {REG_RAX, REG_RCX, REG_RDX, REG_RBX, REG_RSP, REG_RBP, REG_RSI, REG_RDI,
+                                   REG_R8,  REG_R9,  REG_R10, REG_R11, REG_R12, REG_R13, REG_R14, REG_R15};
+        for (int i = 0; i < 16; i++) c.gpr[i] = (uint64_t)u->uc_mcontext.gregs[gr[i]];
+        c.has_regs = 1;
         crash_fn(&c);
     }
     signal(sig, SIG_DFL);   // returns into the faulting instruction, which faults again with the default action
@@ -1456,6 +1468,41 @@ int plat_open_folder(const char *path) {
 void *plat_lib_open(const char *name) { return dlopen(name, RTLD_NOW | RTLD_LOCAL); }
 void *plat_lib_sym(void *lib, const char *name) { return lib ? dlsym(lib, name) : NULL; }
 
+void plat_lib_error(char *out, size_t cap) {
+    const char *e = dlerror();
+    snprintf(out, cap, "%s", e ? e : "unknown error");
+}
+
+int plat_ngx_core_path(char *out, size_t cap) { return snprintf(out, cap, "libnvidia-ngx.so.1") < (int)cap; }
+
+// UTF-8 to UTF-32 wchar_t, without the C library's locale (the runtime never sets one, so mbstowcs would stop at the
+// first non-ASCII byte). Returns 0 on malformed input or when out has no room.
+static int utf8_to_wide(const char *in, wchar_t *out, size_t cap) {
+    const unsigned char *u = (const unsigned char *)in;
+    size_t n = 0;
+    while (*u) {
+        unsigned c = *u++, more = c < 0x80 ? 0 : c >= 0xF0 ? 3 : c >= 0xE0 ? 2 : c >= 0xC0 ? 1 : 4;
+        if (more == 4) return 0;
+        if (more) c &= 0x3F >> more;
+        for (unsigned k = 0; k < more; k++, u++) {
+            if ((*u & 0xC0) != 0x80) return 0;
+            c = c << 6 | (*u & 0x3F);
+        }
+        if (n + 1 >= cap) return 0;
+        out[n++] = (wchar_t)c;
+    }
+    out[n] = 0;
+    return 1;
+}
+
+int plat_exe_dir_wide(wchar_t *out, size_t cap) {
+    char dir[PLAT_PATH_MAX];
+    char *cut = plat_exe_path(dir, sizeof dir) ? plat_last_sep(dir) : NULL;
+    if (!cut) return 0;
+    cut[1] = 0;
+    return utf8_to_wide(dir, out, cap);
+}
+
 // ---- machine load: /proc/stat and /proc/self/stat for the CPU, hwmon for the temperature, the amdgpu sysfs files of
 // the render device for the GPU.
 
@@ -1626,6 +1673,17 @@ void plat_commit_info(double *proc_mb, double *used_mb, double *limit_mb) {
     if (private_bytes) *proc_mb = (double)(private_bytes >> 20);
 }
 
+void plat_mem_room(uint64_t *commit_room, uint64_t *phys_room) {
+    *commit_room = *phys_room = UINT64_MAX;
+    char line[128];
+    unsigned long long k;
+    FILE *f = fopen("/proc/meminfo", "r");
+    if (!f) return;
+    while (fgets(line, sizeof line, f))
+        if (sscanf(line, "MemAvailable: %llu kB", &k) == 1) *phys_room = (uint64_t)k << 10;
+    fclose(f);
+}
+
 // ---- child processes
 
 struct PlatChild {

runtime/platform/platform.h

@@ -99,6 +99,12 @@ uint64_t plat_now_ns(void);               // monotonic, high resolution, arbitra
 uint64_t plat_counter(void);              // the OS's raw monotonic counter (QPC on Windows), plat_counter_hz per second
 uint64_t plat_counter_hz(void);
 void plat_timer_1ms(void);                // asks the OS for timed waits accurate to 1 ms or better (call before threads)
+int plat_power_full(void);                // opts the process out of OS power throttling: 1 done, 0 refused, -1 no such thing
+// Hybrid CPUs: finds the performance-core logical processors the process may use, once, before any thread asks; returns
+// their count (0: none to prefer, why says which case). plat_thread_perf_core: the calling thread prefers them; 1 done,
+// 0 refused, -1 nothing selected.
+int plat_perf_core_setup(char *why, size_t n);
+int plat_thread_perf_core(void);
 // t * num / den for a fixed num / den without a 128-bit division per call (__udivti3 took 5% of the game's running
 // samples through plat_now_ns and rt_ticks): t * q + (t * f >> 64) with q = num / den and f = (num % den) * 2^64 / den.
 // Monotonic in t and at most one below the exact quotient while t * (num % den) / den fits 64 bits.
@@ -222,9 +228,10 @@ _Noreturn void plat_reexec(void);
 // - A fault in this executable's own code (an access violation, an illegal instruction, a division by zero, a stack
 //   overflow) is reported as it happens.
 // - A fault in another module (a GPU driver, a system library) is reported only when nothing handles it: on Windows
-//   from the unhandled-exception filter, with the return addresses in this executable on the faulting thread's stack,
-//   so the report shows which host call ran into the driver; on Linux from the signal handler, without frames. Drivers
-//   that raise and catch their own exceptions never reach fn.
+//   from the unhandled-exception filter, with the return addresses on the faulting thread's stack in every module, so
+//   the report shows which driver code failed and which host call ran into it; on Linux from the signal handler,
+//   without frames. Drivers that raise and catch their own exceptions never reach fn. For a C++ exception thrown by
+//   MSVC-built code (a driver) the report names its type and, for a std::exception, its what() text.
 // - On Windows abort() (a failed assert, std::terminate) and an invalid parameter to a C runtime function go there
 //   too, with the host frames in this executable: the C runtime ends the process for both with 0xc0000409 and no
 //   exception.
@@ -234,10 +241,19 @@ typedef struct {
     uintptr_t rip, offset;     // the faulting instruction, and its offset in its module
     int access;                // the data access that faulted: 0 read, 1 write, 8 execute; -1 when not an access
     uint64_t addr;             // its address when access >= 0
-    int nframes;               // return addresses in this executable, innermost first: for abort and invalid parameter
-    uintptr_t frames[16];      // (rip is frames[0]) and for a fault in another module; 0 for a fault in this one
+    int nframes;               // return addresses, innermost first: for abort and invalid parameter (rip is frames[0])
+    uintptr_t frames[32];      // and for a fault in another module; 0 for a fault in this one
+    const char *frame_module[32];   // the module of each frame: NULL for this executable, "" for none
+    uintptr_t frame_offset[32];     // each frame's offset in its module
+    int frames_left_out;       // frames in other modules past the limit the walk keeps, so this executable's still fit
     const char *module;        // the module holding rip: NULL for this executable, "" for none (a jump to address 0)
+    const char *cxx_type;      // a C++ exception from MSVC-built code: the thrown type's decorated name (.?AV...@@)
+    const char *cxx_what;      // and what() when the type derives from std::exception; NULL when unknown
+    int has_regs;              // gpr holds the faulting thread's registers (a fault; not abort or invalid parameter)
+    uint64_t gpr[16];          // rax rcx rdx rbx rsp rbp rsi rdi r8..r15, in the x86 encoding order
 } PlatCrash;
+#define PLAT_GPR_NAMES {"rax", "rcx", "rdx", "rbx", "rsp", "rbp", "rsi", "rdi", \
+                        "r8", "r9", "r10", "r11", "r12", "r13", "r14", "r15"}
 void plat_crash_hook(void (*fn)(const PlatCrash *c));
 int plat_mem_read(uint64_t addr, void *out, size_t n);    // copies n bytes, 0 instead of faulting on unmapped memory
 int plat_screen_size(int *w, int *h);                    // the primary display's current mode in pixels; 0 unknown
@@ -387,6 +403,16 @@ int plat_open_folder(const char *path);                   // shows the folder in
 // in the system directories only. NULL when missing.
 void *plat_lib_open(const char *name);
 void *plat_lib_sym(void *lib, const char *name);
+// Why the calling thread's last plat_lib_open failed, for a log line (the OS error code on Windows, dlerror on Linux).
+void plat_lib_error(char *out, size_t cap);
+
+// ---- NVIDIA NGX (vk_dlaa.c). The path of the driver's NGX core library into out: on Windows _nvngx.dll in the folder
+// the driver records in the registry (NGXCore FullPath), on Linux libnvidia-ngx.so.1, which the dynamic loader finds
+// where the driver put it. 0 when the driver records none.
+int plat_ngx_core_path(char *out, size_t cap);
+// The running executable's folder with a trailing separator, as the wchar_t string NGX takes for its application data
+// path (UTF-16 on Windows, UTF-32 code points on Linux). 0 on failure or when out has no room.
+int plat_exe_dir_wide(wchar_t *out, size_t cap);
 
 // ---- machine load for the performance overlay (perfmon.c's sampling thread; each call can take milliseconds). A number
 // the system does not give is -1.
@@ -407,6 +433,9 @@ void plat_mem_info(float *used_mb, float *total_mb, float *proc_mb);
 // Commit charge in MB: this process's private bytes, the system's commit in use and its limit (RAM plus page files).
 // An allocation fails when the system's reaches the limit, however much physical memory is free. -1 when unknown.
 void plat_commit_info(double *proc_mb, double *used_mb, double *limit_mb);
+// What the system can still give, in bytes: commit (Windows: limit minus charge; Linux, which overcommits: UINT64_MAX)
+// and physical memory available without paging. UINT64_MAX when unknown.
+void plat_mem_room(uint64_t *commit_room, uint64_t *phys_room);
 // This process's memory in bytes: resident (the working set) and private (Windows: committed private bytes; Linux: the
 // resident anonymous pages). 0 when unknown.
 void plat_process_memory(uint64_t *resident, uint64_t *private_bytes);

runtime/platform/win32.c

@@ -18,6 +18,13 @@
 
 #include "platform.h"
 
+// Hybrid laptops: the drivers read these exports of the executable when it starts and run it on the dedicated GPU.
+// NVIDIA Optimus can otherwise hide its GPU from a program it put on the integrated one, whatever the Windows and NVIDIA
+// settings say; AMD switchable graphics read the second one. TOTK_GPU_INDEX still picks another GPU. Savage Wind Recomp
+// exports the same two (runtime/platform/windows.c).
+__declspec(dllexport) DWORD NvOptimusEnablement = 1;
+__declspec(dllexport) int AmdPowerXpressRequestHighPerformance = 1;
+
 _Static_assert(sizeof(PlatMutex) == sizeof(SRWLOCK), "PlatMutex holds an SRWLOCK");
 _Static_assert(sizeof(PlatCond) == sizeof(CONDITION_VARIABLE), "PlatCond holds a CONDITION_VARIABLE");
 
@@ -289,6 +296,98 @@ uint32_t plat_cpu_usable(void) {
     return plat_cpu_count();
 }
 
+// ---- performance cores of a hybrid CPU
+// An Intel CPU with P-cores and E-cores (a Core 5 210H: 4 P-cores with two threads each, 4 E-cores) runs a thread on an
+// E-core at about 60% of a P-core's speed, and the frame waits for its slowest critical thread. The few threads every
+// frame goes through ask for the P-cores (plat_thread_perf_core): their selected CPU sets are the logical processors
+// above the lowest efficiency class that the process affinity allows. A thread's CPU sets bind it like an affinity
+// mask, except that processors the system reserves for itself are left out instead of failing. Nothing is selected
+// on a CPU with one efficiency class, or when the affinity already leaves only P-cores or only E-cores. LAS Recomp
+// does the same for its game, render and vsync threads (runtime/platform/windows.c plat_thread_perf_core).
+// Nothing is selected either below PC_MIN_CORES physical P-cores (an i5-1235U or Core 3 100U has 2): the five bound
+// threads (perf_core_thread) and a spinning recorder would crowd two cores and their hyperthreads while the E-cores
+// stay idle, which can be slower than letting the scheduler place them.
+#define PC_MIN_CORES 4
+static ULONG *pc_ids;   // the CPU set ids of the allowed P-core logical processors; NULL: none selected
+static ULONG pc_n;
+
+int plat_perf_core_setup(char *why, size_t n) {
+    ULONG len = 0;
+    GetSystemCpuSetInformation(NULL, 0, &len, GetCurrentProcess(), 0);
+    uint8_t *buf = len ? malloc(len) : NULL;
+    if (!buf || !GetSystemCpuSetInformation((PSYSTEM_CPU_SET_INFORMATION)buf, len, &len, GetCurrentProcess(), 0)) {
+        free(buf);
+        snprintf(why, n, "no CPU set information (error %lu)", GetLastError());
+        return 0;
+    }
+    // The process affinity mask covers its first processor group only: it filters on a single-group system.
+    DWORD_PTR proc = 0, sys = 0;
+    int filter = GetActiveProcessorGroupCount() == 1 && GetProcessAffinityMask(GetCurrentProcess(), &proc, &sys) && proc;
+    int lo = 255, hi = -1, all = 0;
+    for (ULONG off = 0; off < len;) {
+        SYSTEM_CPU_SET_INFORMATION *i = (SYSTEM_CPU_SET_INFORMATION *)(buf + off);
+        if (!i->Size) break;
+        if (i->Type == CpuSetInformation) {
+            int ec = i->CpuSet.EfficiencyClass;
+            lo = ec < lo ? ec : lo, hi = ec > hi ? ec : hi, all++;
+        }
+        off += i->Size;
+    }
+    if (!all || hi <= lo) {
+        free(buf);
+        snprintf(why, n, "not a hybrid CPU (%d logical processors, one efficiency class)", all);
+        return 0;
+    }
+    ULONG *ids = calloc((size_t)all, sizeof *ids);
+    uint16_t *cores = calloc((size_t)all, sizeof *cores);   // group << 8 | core index of each allowed physical P-core
+    int np = 0, ne = 0, ok_p = 0, ok_e = 0, nc = 0;
+    for (ULONG off = 0; ids && cores && off < len;) {
+        SYSTEM_CPU_SET_INFORMATION *i = (SYSTEM_CPU_SET_INFORMATION *)(buf + off);
+        if (!i->Size) break;
+        if (i->Type == CpuSetInformation) {
+            int perf = i->CpuSet.EfficiencyClass > lo;
+            int allowed = !filter || (i->CpuSet.Group == 0 && i->CpuSet.LogicalProcessorIndex < 64 &&
+                                      (proc >> i->CpuSet.LogicalProcessorIndex) & 1);
+            perf ? np++ : ne++;
+            if (allowed) perf ? ok_p++ : ok_e++;
+            if (allowed && perf) {
+                ids[pc_n++] = i->CpuSet.Id;
+                uint16_t key = (uint16_t)(i->CpuSet.Group << 8 | i->CpuSet.CoreIndex);
+                int k = 0;
+                while (k < nc && cores[k] != key) k++;
+                if (k == nc) cores[nc++] = key;
+            }
+        }
+        off += i->Size;
+    }
+    int counted = ids && cores;
+    free(buf);
+    free(cores);
+    if (!counted || !ok_p || !ok_e) {
+        free(ids);
+        pc_n = 0;
+        snprintf(why, n, "hybrid CPU, %d P-core + %d E-core logical processors; the affinity allows %d + %d, so it decides",
+                 np, ne, ok_p, ok_e);
+        return 0;
+    }
+    if (nc < PC_MIN_CORES) {
+        free(ids);
+        pc_n = 0;
+        snprintf(why, n, "hybrid CPU, %d P-core + %d E-core logical processors on %d P-cores, fewer than %d", np, ne, nc,
+                 PC_MIN_CORES);
+        return 0;
+    }
+    pc_ids = ids;
+    snprintf(why, n, "hybrid CPU, %d P-core + %d E-core logical processors%s", np, ne,
+             ok_p != np || ok_e != ne ? " (affinity-limited)" : "");
+    return (int)pc_n;
+}
+
+int plat_thread_perf_core(void) {
+    if (!pc_ids) return -1;
+    return SetThreadSelectedCpuSets(GetCurrentThread(), pc_ids, pc_n) ? 1 : 0;
+}
+
 // PLAT_PRIO_* are the THREAD_PRIORITY_* values.
 void plat_thread_priority(int prio) { SetThreadPriority(GetCurrentThread(), prio); }
 void plat_thread_set_priority(PlatThread *t, int prio) { SetThreadPriority(t->h, prio); }
@@ -319,6 +418,17 @@ uint64_t plat_counter_hz(void) { return qpc_freq(); }
 // The default timer tick is 15.6 ms; a 1 ms wait would cost a full tick.
 void plat_timer_1ms(void) { timeBeginPeriod(1); }
 
+// Windows 11 runs the threads of a process it deems in the background at EcoQoS (on the E-cores of a hybrid Intel CPU,
+// at a low clock) and ignores its timer resolution while its window is hidden. The game is always latency-bound, so it
+// opts out of both. Windows before 10 1709 does not know the class: 0.
+int plat_power_full(void) {
+    PROCESS_POWER_THROTTLING_STATE s = {
+        .Version = PROCESS_POWER_THROTTLING_CURRENT_VERSION,
+        .ControlMask = PROCESS_POWER_THROTTLING_EXECUTION_SPEED | PROCESS_POWER_THROTTLING_IGNORE_TIMER_RESOLUTION,
+        .StateMask = 0};
+    return SetProcessInformation(GetCurrentProcess(), ProcessPowerThrottling, &s, sizeof s) ? 1 : 0;
+}
+
 static PlatScale qpc_ns;   // set before main, so no thread reads it half written
 
 __attribute__((constructor)) static void qpc_ns_init(void) { qpc_ns = plat_scale(1000000000u, qpc_freq()); }
@@ -626,8 +736,21 @@ void plat_mem_fault_hook(int (*fn)(uint64_t addr)) {
 
 static void (*crash_fn)(const PlatCrash *c);
 
+// The faulting thread's general registers, for the report to tie the faulting instruction to its operands.
+static void regs_of(const CONTEXT *x, PlatCrash *c) {
+    const DWORD64 r[16] = {x->Rax, x->Rcx, x->Rdx, x->Rbx, x->Rsp, x->Rbp, x->Rsi, x->Rdi,
+                           x->R8,  x->R9,  x->R10, x->R11, x->R12, x->R13, x->R14, x->R15};
+    for (int i = 0; i < 16; i++) c->gpr[i] = r[i];
+    c->has_regs = 1;
+}
+
+static LONG WINAPI crash_filter(EXCEPTION_POINTERS *ep);
+
 static LONG CALLBACK crash_veh(EXCEPTION_POINTERS *ep) {
     DWORD code = ep->ExceptionRecord->ExceptionCode;
+    // Heap corruption is raised by the heap manager, which then ends the process itself: it never reaches the
+    // unhandled-exception filter, so it is reported from here, wherever the heap was called from.
+    if (code == 0xc0000374u) return crash_filter(ep);   // STATUS_HEAP_CORRUPTION
     const char *what = code == EXCEPTION_ACCESS_VIOLATION ? "access violation"
                      : code == EXCEPTION_ILLEGAL_INSTRUCTION ? "illegal instruction"
                      : code == EXCEPTION_INT_DIVIDE_BY_ZERO ? "integer division by zero"
@@ -643,6 +766,7 @@ static LONG CALLBACK crash_veh(EXCEPTION_POINTERS *ep) {
     PlatCrash c = {what, (uint32_t)code, rip, rip - (uintptr_t)mi.lpBaseOfDll, -1, 0};
     if (code == EXCEPTION_ACCESS_VIOLATION && ep->ExceptionRecord->NumberParameters >= 2)
         c.access = (int)ep->ExceptionRecord->ExceptionInformation[0], c.addr = ep->ExceptionRecord->ExceptionInformation[1];
+    regs_of(ep->ContextRecord, &c);
     crash_fn(&c);
     return EXCEPTION_CONTINUE_SEARCH;
 }
@@ -674,16 +798,20 @@ static const char *image_name(const char *base) {
     return base + ((const IMAGE_EXPORT_DIRECTORY *)(base + d->VirtualAddress))->Name;
 }
 
-// The return addresses in [lo, lo + size) on the crashed thread's stack, innermost first: unwinds the faulting context
+// The return addresses on the crashed thread's stack, innermost first, with their modules: unwinds the faulting context
 // with the x64 unwind tables; a function without a table entry (a leaf, or address 0 after a call through a null
 // pointer) has its return address on top of the stack. The walk stops at return address 0 (the thread's first frame)
-// and when the stack pointer leaves the thread's stack, so a damaged stack ends it instead of faulting.
-static int frames_in(const CONTEXT *at, uintptr_t lo, uintptr_t size, uintptr_t *out, int max) {
+// and when the stack pointer leaves the thread's stack, so a damaged stack ends it instead of faulting. A driver can
+// nest deeper than the report holds, so frames in other modules stop being kept after the first 12 (counted in
+// frames_left_out) and the frames in this executable, which name the host call that ran into the driver, still fit.
+static void frames_of(const CONTEXT *at, PlatCrash *x) {
+    enum { MAX = sizeof x->frames / sizeof *x->frames, OTHER_MAX = 12 };
     CONTEXT c = *at;
     ULONG_PTR low, high;
     GetCurrentThreadStackLimits(&low, &high);
-    int n = 0;
-    for (int i = 0; i < 256 && n < max && (c.Rip || !i) && c.Rsp >= low && c.Rsp + 8 <= high; i++) {
+    void *exe = GetModuleHandleW(NULL);
+    int other = 0;
+    for (int i = 0; i < 256 && x->nframes < MAX && (c.Rip || !i) && c.Rsp >= low && c.Rsp + 8 <= high; i++) {
         DWORD64 img;
         RUNTIME_FUNCTION *fe = RtlLookupFunctionEntry(c.Rip, &img, NULL);
         if (fe) {
@@ -694,9 +822,64 @@ static int frames_in(const CONTEXT *at, uintptr_t lo, uintptr_t size, uintptr_t
             c.Rip = *(const DWORD64 *)c.Rsp;
             c.Rsp += 8;
         }
-        if (c.Rip - lo < size) out[n++] = (uintptr_t)c.Rip;
+        if (!c.Rip) break;
+        void *base = NULL;
+        RtlPcToFileHeader((void *)(uintptr_t)c.Rip, &base);
+        if (base != exe && other++ >= OTHER_MAX) {
+            x->frames_left_out++;
+            continue;
+        }
+        int n = x->nframes++;
+        x->frames[n] = (uintptr_t)c.Rip;
+        x->frame_module[n] = base == exe ? NULL : base ? image_name(base) : "";
+        x->frame_offset[n] = (uintptr_t)c.Rip - (uintptr_t)base;
     }
-    return n;
+}
+
+// MSVC's C++ throw (0xe06d7363) passes the thrown object in ExceptionInformation[1], its ThrowInfo in [2] and the image
+// base the ThrowInfo's 32-bit offsets are relative to in [3] (magic 0x19930520..0x19930522 in [0]). The ThrowInfo lists
+// the types a catch may match, the thrown type first; each type descriptor holds a vtable pointer, a spare pointer and
+// the decorated name. A std::exception keeps its what() text in the pointer right after its vtable pointer (MSVC's
+// __std_exception_data). Every read is checked: the exception may come from a damaged heap.
+typedef struct { uint32_t attributes; int32_t unwind, forward_compat, catchable_types; } CxxThrowInfo;
+typedef struct { uint32_t properties; int32_t type, this_mdisp, this_pdisp, this_vdisp, size, copy; } CxxCatchableType;
+
+// Copies the string at p into buf (cut to n - 1 characters), checking each page before reading it; NULL when the first
+// byte cannot be read.
+static const char *copy_cstr(uintptr_t p, char *buf, size_t n) {
+    size_t i = 0;
+    for (; p && i + 1 < n; i++) {
+        if ((!i || !((p + i) & 4095)) && !plat_mem_readable(p + i, 1)) break;
+        if (!(buf[i] = ((const char *)p)[i])) return buf;
+    }
+    buf[i] = 0;
+    return i ? buf : NULL;
+}
+
+static void cxx_info(const EXCEPTION_RECORD *er, PlatCrash *x) {
+    if (er->NumberParameters < 4 || er->ExceptionInformation[0] - 0x19930520u > 2) return;
+    uintptr_t obj = er->ExceptionInformation[1], info = er->ExceptionInformation[2],
+              base = er->ExceptionInformation[3];
+    if (!info || !plat_mem_readable(info, sizeof(CxxThrowInfo))) return;
+    const CxxThrowInfo *ti = (const void *)info;
+    uintptr_t arr = base + (uintptr_t)(intptr_t)ti->catchable_types;
+    if (!ti->catchable_types || !plat_mem_readable(arr, 4)) return;
+    int32_t n = *(const int32_t *)arr;
+    if (n <= 0 || n > 64 || !plat_mem_readable(arr + 4, 4u * (uint64_t)n)) return;
+    static char type[128], what[256];   // the crash report runs once: crash_fn ends the process
+    int exception = 0;
+    for (int32_t i = 0; i < n; i++) {
+        uintptr_t ct = base + (uintptr_t)(intptr_t)((const int32_t *)(arr + 4))[i];
+        if (!plat_mem_readable(ct, sizeof(CxxCatchableType))) continue;
+        uintptr_t td = base + (uintptr_t)(intptr_t)((const CxxCatchableType *)ct)->type;
+        char name[64];
+        if (!copy_cstr(td + 16, i ? name : type, i ? sizeof name : sizeof type))   // past vtable and spare pointers
+            continue;
+        if (!i) x->cxx_type = type;
+        if (!strcmp(i ? name : type, ".?AVexception@std@@")) exception = 1;
+    }
+    if (exception && obj && plat_mem_readable(obj + 8, 8))
+        x->cxx_what = copy_cstr(*(const uintptr_t *)(obj + 8), what, sizeof what);
 }
 
 // The last-chance filter: an exception no handler took, in any module. crash_veh reports faults in this executable as
@@ -721,16 +904,15 @@ static LONG WINAPI crash_filter(EXCEPTION_POINTERS *ep) {
                    (uint32_t)code, rip, 0, -1, 0};
     if (code == EXCEPTION_ACCESS_VIOLATION && er->NumberParameters >= 2)
         c.access = (int)er->ExceptionInformation[0], c.addr = er->ExceptionInformation[1];
+    regs_of(ep->ContextRecord, &c);
     void *base = NULL;
     RtlPcToFileHeader((void *)rip, &base);
-    MODULEINFO mi;
     uintptr_t exe = (uintptr_t)GetModuleHandleW(NULL);
     c.offset = rip - (uintptr_t)base;
     if (!base) c.module = "";
     else if ((uintptr_t)base != exe) c.module = image_name(base);
-    if (c.module && GetModuleInformation(GetCurrentProcess(), (HMODULE)exe, &mi, sizeof mi))
-        c.nframes = frames_in(ep->ContextRecord, exe, mi.SizeOfImage, c.frames,
-                              (int)(sizeof c.frames / sizeof *c.frames));
+    if (c.module) frames_of(ep->ContextRecord, &c);
+    if (code == 0xe06d7363u) cxx_info(er, &c);
     crash_fn(&c);
     return EXCEPTION_CONTINUE_SEARCH;
 }
@@ -1389,6 +1571,27 @@ void *plat_lib_open(const char *name) {
 
 void *plat_lib_sym(void *lib, const char *name) { return lib ? (void *)GetProcAddress((HMODULE)lib, name) : NULL; }
 
+void plat_lib_error(char *out, size_t cap) { snprintf(out, cap, "error %lu", GetLastError()); }
+
+int plat_ngx_core_path(char *out, size_t cap) {
+    wchar_t dir[WPATH];
+    DWORD n = sizeof dir;
+    char d[PLAT_PATH_MAX];
+    if (RegGetValueW(HKEY_LOCAL_MACHINE, L"SOFTWARE\\NVIDIA Corporation\\Global\\NGXCore", L"FullPath", RRF_RT_REG_SZ,
+                     NULL, dir, &n) != ERROR_SUCCESS ||
+        !WideCharToMultiByte(CP_UTF8, 0, dir, -1, d, (int)sizeof d, NULL, NULL))
+        return 0;
+    return snprintf(out, cap, "%s\\_nvngx.dll", d) < (int)cap;
+}
+
+int plat_exe_dir_wide(wchar_t *out, size_t cap) {
+    DWORD len = GetModuleFileNameW(NULL, out, (DWORD)cap);
+    if (!len || len >= cap) return 0;
+    while (len && out[len - 1] != L'\\' && out[len - 1] != L'/') len--;
+    out[len] = 0;
+    return len != 0;
+}
+
 // ---- machine load: GetSystemTimes for the CPU, PDH (pdh.dll, loaded on first use) for the thermal zones and the GPU
 // engine and adapter memory counters that Task Manager reads.
 
@@ -1581,6 +1784,12 @@ void plat_commit_info(double *proc_mb, double *used_mb, double *limit_mb) {
         *proc_mb = (double)(pmc.PrivateUsage >> 20);
 }
 
+void plat_mem_room(uint64_t *commit_room, uint64_t *phys_room) {
+    *commit_room = *phys_room = UINT64_MAX;
+    MEMORYSTATUSEX ms = {.dwLength = sizeof ms};
+    if (GlobalMemoryStatusEx(&ms)) *commit_room = ms.ullAvailPageFile, *phys_room = ms.ullAvailPhys;
+}
+
 void plat_process_memory(uint64_t *resident, uint64_t *private_bytes) {
     PROCESS_MEMORY_COUNTERS_EX pm = {.cb = sizeof pm};
     *resident = *private_bytes = 0;

runtime/playtime.c

@@ -0,0 +1,214 @@
+// Play time: how long the player has spent in the game, across runs, shown on the Game tab of the settings menu
+// (ui/menu.cpp tab_game). The game itself keeps no play time a player can see, and the Switch's own play log is not
+// there to read, so the port counts it.
+//
+// One host thread (it never holds a virtual core, as discord_rpc.c's) wakes every second and adds the time since its
+// last wake when the window has the focus (gfx_window_focused) and the game presented at least one frame meanwhile
+// (gfx_stats). So the shader prewarm before the first frame, a window left in the background and a hung game do not
+// count; loading screens and the game's own menus do, as they present frames. One wake adds at most TICK_CAP_MS, so a
+// machine that slept with the game open adds two seconds, not the night.
+//
+// The total lives in <profile>/playtime.txt, next to account.uid and the saves, so it stays with them when the
+// install is updated or moved. It is plain text, "total_seconds <n>", written atomically (plat_file_write_atomic)
+// every SAVE_EVERY_MS while it grows and once more when the game exits (playtime_save from boot.c, svc.c and
+// gfx_present.c, next to prof_write); a crash loses at most SAVE_EVERY_MS. A file that does not read as that is kept
+// as playtime.txt.bad and the count starts again from zero. A scripted run (--seconds) reads the total and adds nothing.
+#include <inttypes.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "gfx.h"
+#include "host.h"
+#include "platform/platform.h"
+
+#define TICK_MS 1000
+#define TICK_CAP_MS 2000
+#define SAVE_EVERY_MS 30000
+#define TOTAL_CAP_S (100ull * 365 * 24 * 3600)   // a hundred years: anything larger is a damaged file
+
+static PlatMutex lock = PLAT_MUTEX_INIT;   // the totals and the file
+static uint64_t total_ms, session_ms, saved_ms;
+static int loaded;                         // 1 once the file was read (or found missing), -1 when there is no profile
+static char path[1100];
+
+// The seconds of a playtime.txt: "total_seconds <n>", a line end or nothing after it, a UTF-8 byte order mark before it
+// (an editor may add one). 1 when the file reads so and n is at most TOTAL_CAP_S.
+static int parse(const uint8_t *b, uint64_t n, uint64_t *seconds) {
+    if (n >= 3 && !memcmp(b, "\xEF\xBB\xBF", 3)) b += 3, n -= 3;
+    char text[64] = {0};
+    memcpy(text, b, n < sizeof text - 1 ? n : sizeof text - 1);
+    unsigned long long s = 0;
+    char tail = 0;
+    if (sscanf(text, "total_seconds %llu%c", &s, &tail) < 1 || (tail && tail != '\n' && tail != '\r') ||
+        s > TOTAL_CAP_S)
+        return 0;
+    *seconds = s;
+    return 1;
+}
+
+// Holding lock. dir: the profile directory.
+static void load(const char *dir) {
+    if (loaded) return;
+    if (!dir || !*dir || snprintf(path, sizeof path, "%s/playtime.txt", dir) >= (int)sizeof path) {
+        loaded = -1;
+        rt_log("playtime: no profile directory; play time is not counted");
+        return;
+    }
+    loaded = 1;
+    uint64_t n = 0, s = 0;
+    uint8_t *b = plat_file_read(path, &n);
+    if (!b) {
+        rt_log("playtime: no %s yet; counting from zero", path);
+        return;
+    }
+    if (parse(b, n, &s)) {
+        free(b);
+        total_ms = saved_ms = s * 1000;
+        rt_log("playtime: %llu h %02llu min played before this run (%s)", (unsigned long long)(s / 3600),
+               (unsigned long long)(s / 60 % 60), path);
+        return;
+    }
+    char bad[1110];
+    snprintf(bad, sizeof bad, "%s.bad", path);
+    int kept = plat_file_write_atomic(bad, b, (size_t)n);   // the whole file, for the player to recover the number
+    free(b);
+    rt_log("playtime: %s does not read as \"total_seconds <n>\"; %s, counting from zero", path,
+           kept ? "kept as playtime.txt.bad" : "could not keep a copy");
+}
+
+// Holding lock.
+static void save_locked(void) {
+    if (loaded != 1 || total_ms / 1000 == saved_ms / 1000) return;
+    char text[48];
+    int n = snprintf(text, sizeof text, "total_seconds %" PRIu64 "\n", total_ms / 1000);
+    if (plat_file_write_atomic(path, text, (size_t)n)) saved_ms = total_ms;
+    else rt_log("playtime: cannot write %s", path);
+}
+
+void playtime_save(void) {
+    plat_mutex_lock(&lock);
+    save_locked();
+    plat_mutex_unlock(&lock);
+}
+
+int playtime_get(uint64_t *total_s, uint64_t *session_s) {
+    plat_mutex_lock(&lock);
+    int ok = loaded == 1;
+    *total_s = total_ms / 1000;
+    *session_s = session_ms / 1000;
+    plat_mutex_unlock(&lock);
+    return ok;
+}
+
+// Holding lock: the time since the last tick, d ms, when the game was played meanwhile; the file every SAVE_EVERY_MS.
+static void tick_locked(uint64_t now, uint64_t d, int playing, uint64_t *last_save) {
+    if (d > TICK_CAP_MS) d = TICK_CAP_MS;
+    if (playing) total_ms += d, session_ms += d;
+    if (now - *last_save >= SAVE_EVERY_MS) save_locked(), *last_save = now;
+}
+
+static void playtime_main(void *arg) {
+    (void)arg;
+    plat_thread_priority(PLAT_PRIO_BELOW_NORMAL);
+    uint64_t last = plat_now_ms(), last_save = last, frames = gfx_stats().frames;
+    for (;;) {
+        plat_sleep_ms(TICK_MS);
+        uint64_t now = plat_now_ms(), f = gfx_stats().frames;
+        int playing = f != frames && gfx_window_focused();
+        plat_mutex_lock(&lock);
+        tick_locked(now, now - last, playing, &last_save);
+        plat_mutex_unlock(&lock);
+        last = now, frames = f;
+    }
+}
+
+// The file is read at once, so the Game tab shows the total from the first frame. A scripted run (--seconds: benches,
+// menu shots, input scripts) shows it but adds nothing: those runs are not play.
+void playtime_init(void) {
+    static int32_t started;
+    if (plat_xchg32(&started, 1)) return;
+    plat_mutex_lock(&lock);
+    load(rt_config()->profile);
+    int on = loaded == 1;
+    plat_mutex_unlock(&lock);
+    if (!on) return;
+    if (rt_config()->seconds > 0) {
+        rt_log("playtime: a scripted run (--seconds); play time is shown, not counted");
+        return;
+    }
+    if (!plat_thread_spawn(playtime_main, NULL, "playtime", 0))
+        rt_log("playtime: cannot start its thread; play time is not counted");
+}
+
+// --probe-playtime: the file's parsing, a missing, a valid and a damaged file, the periodic save and the tick's cap, in
+// a directory next to totk.exe. Needs no game files.
+static int fails;
+#define CHECK(c, what) do { if (!(c)) { fails++; fprintf(stderr, "FAIL playtime: %s\n", what); } } while (0)
+
+static void reset(void) { total_ms = session_ms = saved_ms = 0, loaded = 0, path[0] = 0; }
+
+static int file_is(const char *p, const char *want) {
+    uint64_t n = 0;
+    uint8_t *b = plat_file_read(p, &n);
+    int ok = b && n == strlen(want) && !memcmp(b, want, n);
+    free(b);
+    return ok;
+}
+
+int probe_playtime(void) {
+    uint64_t s = 0;
+    CHECK(parse((const uint8_t *)"total_seconds 42\n", 17, &s) && s == 42, "a line with LF");
+    CHECK(parse((const uint8_t *)"total_seconds 42\r\n", 18, &s) && s == 42, "a line with CRLF");
+    CHECK(parse((const uint8_t *)"total_seconds 42", 16, &s) && s == 42, "no line end");
+    CHECK(parse((const uint8_t *)"\xEF\xBB\xBFtotal_seconds 7\n", 19, &s) && s == 7, "a byte order mark");
+    CHECK(!parse((const uint8_t *)"total_seconds 42x", 17, &s), "a character after the number");
+    CHECK(!parse((const uint8_t *)"total_seconds -1\n", 17, &s), "a negative number");
+    CHECK(!parse((const uint8_t *)"total_seconds 3153600001\n", 25, &s), "over a hundred years");
+    CHECK(!parse((const uint8_t *)"seconds 42\n", 11, &s), "another key");
+    CHECK(!parse((const uint8_t *)"", 0, &s), "an empty file");
+
+    char dir[1024];
+    if (!plat_exe_path(dir, sizeof dir)) return 1;
+    char *cut = plat_last_sep(dir);
+    if (!cut) return 1;
+    strcpy(cut + 1, "playtime-probe");
+    plat_file_mkdir_p(dir);
+    char file[1100], bad[1110];
+    snprintf(file, sizeof file, "%s/playtime.txt", dir);
+    snprintf(bad, sizeof bad, "%s.bad", file);
+    plat_file_delete(file);
+    plat_file_delete(bad);
+
+    uint64_t total, session, last_save = 0;
+    reset();
+    load(dir);
+    CHECK(playtime_get(&total, &session) && total == 0, "a missing file counts from zero");
+    tick_locked(1000, 1000, 1, &last_save);
+    tick_locked(2000, 1000, 0, &last_save);
+    CHECK(plat_file_size(file) < 0, "nothing is written before SAVE_EVERY_MS");
+    tick_locked(2000 + 3600000, 3600000, 1, &last_save);   // a machine that slept: one tick adds TICK_CAP_MS, and saves
+    playtime_get(&total, &session);
+    CHECK(total == 3 && session == 3, "played ticks count, an idle one does not, a long one is capped");
+    CHECK(file_is(file, "total_seconds 3\n"), "the save after SAVE_EVERY_MS writes the total");
+
+    reset();
+    load(dir);
+    playtime_get(&total, &session);
+    CHECK(total == 3 && session == 0, "the next run reads the total back");
+
+    static const char damaged[] = "total_seconds twelve\n";
+    plat_file_write_atomic(file, damaged, sizeof damaged - 1);
+    reset();
+    load(dir);
+    playtime_get(&total, &session);
+    CHECK(total == 0, "a damaged file counts from zero");
+    CHECK(file_is(bad, damaged), "a damaged file is kept whole as playtime.txt.bad");
+
+    plat_file_delete(file);
+    plat_file_delete(bad);
+    reset();
+    fprintf(stderr, "playtime: %s (%d failures)\n", fails ? "FAILED" : "ok", fails);
+    return fails ? 1 : 0;
+}

runtime/resolution.c

@@ -14,10 +14,11 @@
 // 64-bit word, +40 the renderer (1.2.1) or display configuration (1.4.3) the patched code stores there.
 // +56 the camera's aspect ratio as a float when the output is not 16:9 (set aspect), +60 the field-of-view factor
 // (set fov, res_fov_live).
-// 1.4.3 scales the scene it reads by output width / 1920 (a 2560x1440 output renders its 1600x900 scene at 2128x1200;
-// MacNativeGames measured the same), so its page holds the wanted size divided by that factor and the game's 5/6 needs
-// no scene patch. 1.2.1 renders the size it reads (1600x900 on a 2560x1440 output): its page holds the wanted size, and
-// the scene set is on whenever the output differs from 1920x1080.
+// 1.4.3 scales the scene it reads by canvas width / the console mode's base width, 1920 docked and 1280 handheld (a
+// 2560x1440 output renders its 1600x900 scene at 2128x1200 docked; MacNativeGames measured the same), so its page holds
+// the wanted size divided by that factor (scene_scale) and the game's 5/6 needs no scene patch. 1.2.1 renders the size
+// it reads (1600x900 on a 2560x1440 output): its page holds the wanted size, and the scene set is on whenever the
+// output differs from 1920x1080.
 //
 // Heap: agl's DynamicTextureAllocator is one fixed heap, "DynamicTextureAllocatorMem" in
 // System/HeapSize/Product.Nin_NX_NVN.HeapDef.byml.zs (113664 KiB, carved from "Graphics", 201728 KiB). At 2560x1440
@@ -39,6 +40,8 @@
 
 #include "cfg.h"
 #include "host.h"
+#include "mods.h"
+#include "ov.h"
 #include "platform/platform.h"
 
 #define BASE_W 1920
@@ -46,7 +49,10 @@
 #define SCENE_W 1600
 #define SCENE_H 900
 #define DTA_KIB 113664u      // DynamicTextureAllocatorMem, shipped
-#define SCENE_MAX 4096       // per side, as MacNativeGames' TOTK_RENDER_RES
+#define TEX_HEAP_KIB 506880u // TextureHeap, shipped (1.2.1 and 1.4.3): what a mod's own HeapDef is measured against
+#define SCENE_MAX 8192       // per side
+#define SCENE_KEEP 4096      // per side: the heap bound shrinks no scene below this (scene_for)
+#define DTA_MAX_KIB 2031616u // DynamicTextureAllocatorMem grown at most to 1984 MiB, under 2 GiB (decide)
 #define SHADOW_BASE 960      // the sun shadow map's base size (its cascades: a D16 array of 4 layers)
 
 static struct {
@@ -89,12 +95,25 @@ static int dlss_dll(void) {
 #endif
 }
 
-// The scene size for a share k of the output, within SCENE_MAX per side.
-static void scene_for(double k, unsigned *w, unsigned *h) {
+static uint64_t shadow_kib(void);
+
+// The scene size for a share k of the output, within SCENE_MAX a side and within what a texture heap of DTA_MAX_KIB
+// holds next to the output and the shadow map (res_heap_extra_kib's growth). 1.2.1 already misplaced textures in a
+// pool past 2 GiB, where a signed 32-bit offset turns negative (recomp/patches-v121.txt, set pooloff); larger heaps
+// have not been run. The heap bound never takes the scene below 4096 a side, the size earlier builds gave: an output
+// near 8K grows the heap past 2 GiB for its own targets, and its scene stays what it was. Needs R.shadow. 1 when the
+// scene was made smaller than k asks.
+static int scene_for(double k, unsigned *w, unsigned *h) {
     double sw = R.ow * k, sh = R.oh * k;
     double fit = fmin(1.0, fmin(SCENE_MAX / sw, SCENE_MAX / sh));
+    double keep = fmin(1.0, fmin(SCENE_KEEP / sw, SCENE_KEEP / sh));
+    double room = (double)(DTA_MAX_KIB - shadow_kib()) / DTA_KIB - 1.0 -
+                  fmax(0.0, (double)R.ow * R.oh / (BASE_W * BASE_H) - 1.0);
+    double most = (1.0 + fmax(0.0, room)) * SCENE_W * SCENE_H;   // scene pixels that fit
+    if (sw * sh * fit * fit > most) fit = fmax(keep, sqrt(most / (sw * sh)));
     *w = (unsigned)(sw * fit + 0.5);
     *h = (unsigned)(sh * fit + 0.5);
+    return fit < 1.0;
 }
 
 // The scene's share of the output for a TOTK_DLSS mode; 0 for dlaa (the scene keeps base_k), while DLSS/DLAA is off
@@ -120,6 +139,25 @@ static double dlss_share(const char *m) {
     return 0;
 }
 
+// A game resolution that is not 16:9 needs the HUD's layout for its shape (romfs-patches-<w>x<h>, romfs.c
+// overlay_init), which the installer makes but a developer build only has after tools/make_romfs_patches.py --aspect.
+// Without it the camera would widen while the 16:9 HUD stretched across the picture, so the game draws the largest
+// 16:9 size that fits inside the asked one and the camera aspect patch stays off (res_init); gfx_present.c shows the
+// picture with bars (present setting aspect). Not when TOTK_ROMFS_PATCHES names the overlay itself or the game version
+// has no camera aspect patch. TOTK_OUTPUT_RES is restart-only (cfg.c, RESTART), so this one decision holds all run.
+static void shape_fallback(void) {
+    const char *own = cfg_get("TOTK_ROMFS_PATCHES");
+    unsigned out[2] = {R.ow, R.oh};
+    if (own || !res_wants_wide_overlay(out) || romfs_shaped_ready(R.ow, R.oh)) return;
+    unsigned h = R.oh, w = R.ow;
+    if ((uint64_t)w * 9 <= (uint64_t)h * 16) h = (unsigned)((uint64_t)w * 9 / 16) & ~1u;
+    w = (unsigned)((uint64_t)h * 16 / 9) & ~1u;
+    rt_log("resolution: no romfs-patches-%ux%u for the HUD at this shape; the game draws %ux%u (16:9) and the present "
+           "adds bars. Make the layout with: tools/make_romfs_patches.py --version <v121|v143> --aspect %ux%u --out "
+           "build/<version>/romfs-patches-%ux%u", R.ow, R.oh, w, h, R.ow, R.oh, R.ow, R.oh);
+    R.ow = w, R.oh = h;
+}
+
 static void decide(void) {
     if (R.decided) return;
     R.decided = 1;
@@ -131,6 +169,7 @@ static void decide(void) {
     if (sscanf(o, "%ux%u%c", &w, &h, &tail) == 2 && w >= 1280 && h >= 720 && w <= 7680 && h <= 4320) {
         R.ow = w & ~1u;
         R.oh = h & ~1u;
+        shape_fallback();
     } else {
         rt_log("resolution: TOTK_OUTPUT_RES=%s is not <width>x<height> in 1280x720..7680x4320; 1920x1080", o);
     }
@@ -146,6 +185,12 @@ static void decide(void) {
             rt_log("resolution: TOTK_RENDER_SCALE=%s is not game or a percentage 25..400; game", s);
         }
     }
+    const char *sr = cfg_value("TOTK_SHADOW_RES");
+    if (strcmp(sr, "game")) {
+        unsigned n = (unsigned)atoi(sr);
+        if (n >= 256 && n <= 4096 && !(n & 7)) R.shadow = n;
+        else rt_log("resolution: TOTK_SHADOW_RES=%s is not game or a size 256..4096 (a multiple of 8); game", sr);
+    }
     // A DLSS mode other than dlaa sets the scene size instead. On 1.4.3 with DLSS able to run (DLAA on, the DLL there)
     // the mode can change while the game runs (res_dlss_mode): the scene patch is then always on and the heap is sized
     // for the largest scene the modes ask for. Everyone else keeps the game's own scene size and its dynamic resolution
@@ -156,16 +201,14 @@ static void decide(void) {
     R.dlss_k = dlss_share(m);
     if (R.dlss_k) rt_log("resolution: DLSS %s: the scene at %.0f %% of the output (render_scale ignored)", m, R.dlss_k * 100);
     double sk = R.dlss_k ? R.dlss_k : k;
-    if (R.live || R.dlss_k || R.ow * sk > SCENE_MAX || R.oh * sk > SCENE_MAX) R.scene_set = 1;
-    scene_for(sk, &R.sw, &R.sh);
+    if (R.live || R.dlss_k) R.scene_set = 1;
+    if (scene_for(sk, &R.sw, &R.sh)) {
+        R.scene_set = 1;
+        rt_log("resolution: a %.0fx%.0f scene is larger than the port allows (8192 a side, a texture heap under 2 GiB "
+               "past 4096 a side); %ux%u", R.ow * sk, R.oh * sk, R.sw, R.sh);
+    }
     if (R.live) scene_for(fmax(k, 2.0 / 3.0), &R.hw, &R.hh);
     else R.hw = R.sw, R.hh = R.sh;
-    const char *sr = cfg_value("TOTK_SHADOW_RES");
-    if (strcmp(sr, "game")) {
-        unsigned n = (unsigned)atoi(sr);
-        if (n >= 256 && n <= 4096 && !(n & 7)) R.shadow = n;
-        else rt_log("resolution: TOTK_SHADOW_RES=%s is not game or a size 256..4096 (a multiple of 8); game", sr);
-    }
 }
 
 // Texture heap for a sun shadow map above the base size: the four layers of 2 bytes, three maps in flight (the engine
@@ -187,8 +230,46 @@ uint64_t res_heap_extra_kib(void) {
     return kib + ((shadow_kib() + 1023) & ~1023ull);   // whole MiB
 }
 
+// 1.4.3's factor between the scene size it reads from the page and the one it renders. The game's scene setter
+// (main+0x585e48 and the code before it) multiplies the page's size by the canvas width (display configuration +0x10)
+// over the current console mode's base width (+0xf4 of the mode's entry, mode 1 handheld: main+0x2dac1c stores
+// GetOperationMode != 1 at +0x546). The base is 1280 handheld and 1920 docked. The canvas is the output whenever the
+// scene patch is on: docked the game's own canvas is 1920, handheld res_init switches the output patch on (a 1280 base
+// with the game's 1280x720 handheld canvas would render R.ow / 1280 times the scene). 1.2.1 renders the size it reads.
+static double scene_scale(void) {
+    if (!RT_V143) return 1.0;
+    return (double)R.ow / (ov_handheld() ? 1280 : BASE_W);
+}
+
 uint64_t res_heap_extra_bytes(void) { return ((res_heap_extra_kib() << 10) + 0x1fffff) & ~0x1fffffull; }
 
+// TOTK_TEX_HEAP_EXTRA_MB (0 to 4096): more room in "TextureHeap" and its parent "ゲームグラフィックス", the heap the
+// world's textures stream into. HD texture packs made for emulators with 8 GB of DRAM grow it by about 3.3 GiB in their
+// own HeapDef; without the room the game runs out and starts its panic Blood Moon over and over. The pool and the heap
+// cap grow by the same amount (memory.c mem_init).
+static uint64_t mod_tex_heap_kib(void);
+
+static uint64_t cfg_tex_heap_kib(void) {
+    const char *v = cfg_get("TOTK_TEX_HEAP_EXTRA_MB");
+    if (!v || !*v) return 0;
+    char *end;
+    long mb = strtol(v, &end, 10);
+    if (end == v || *end || mb < 0 || mb > 4096) {
+        static int logged;
+        if (!logged++) rt_log("resolution: TOTK_TEX_HEAP_EXTRA_MB=%s is not 0..4096; 0", v);
+        return 0;
+    }
+    return (uint64_t)mb << 10;
+}
+
+// The larger of TOTK_TEX_HEAP_EXTRA_MB and what an enabled mod's own HeapDef adds to the texture heap (mod_tex_heap_kib).
+static uint64_t tex_heap_extra_kib(void) {
+    uint64_t c = cfg_tex_heap_kib(), m = mod_tex_heap_kib();
+    return c > m ? c : m;
+}
+
+uint64_t res_tex_heap_extra_bytes(void) { return ((tex_heap_extra_kib() << 10) + 0x1fffff) & ~0x1fffffull; }
+
 // ---- the HeapDef table. BYML version 2..7, little endian ("YB"): a hash node is {u8 0xC1, u24 count} and count
 // entries {u24 key index, u8 type, u32 value}; an int (type 0xD1) is stored in the entry, so a size changes in place.
 typedef struct { uint8_t *p; size_t n; uint32_t keys; } Byml;
@@ -226,28 +307,103 @@ static size_t heap_size_field(const Byml *b, uint32_t heaps, const char *heap) {
     return at && u32_at(b, at, &node) ? field(b, node, "Size", 0xd1) : 0;
 }
 
-int res_heapdef_build(const void *in, size_t n, void **out, size_t *out_n) {
-    uint64_t extra = res_heap_extra_kib();
-    *out = NULL;
-    if (!extra) return 0;
+// A HeapDef in b's place (zstd frame in[0..n) with its content size, no dictionary): b.p malloc'd and *heaps its
+// "Heaps" node; 0 when it is not one (b.p freed). why: what the log line says the caller keeps then.
+static int heapdef_open(const void *in, size_t n, Byml *b, uint32_t *heaps, const char *why) {
+    b->p = NULL;
     if (ZSTD_getDictID_fromFrame(in, n)) {
-        rt_log("resolution: the HeapDef table needs a zstd dictionary; the game keeps its own resolution");
+        rt_log("resolution: the HeapDef table needs a zstd dictionary; %s", why);
         return 0;
     }
     unsigned long long raw = ZSTD_getFrameContentSize(in, n);
     if (raw == ZSTD_CONTENTSIZE_UNKNOWN || raw == ZSTD_CONTENTSIZE_ERROR || raw > (64u << 20)) {
-        rt_log("resolution: the HeapDef table is not a zstd frame with a size; the game keeps its own resolution");
+        rt_log("resolution: the HeapDef table is not a zstd frame with a size; %s", why);
         return 0;
     }
-    Byml b = {malloc((size_t)raw), (size_t)raw, 0};
-    if (!b.p) return 0;
-    size_t got = ZSTD_decompress(b.p, b.n, in, n);
-    uint32_t root, heaps = 0;
+    *b = (Byml){malloc((size_t)raw), (size_t)raw, 0};
+    if (!b->p) return 0;
+    size_t got = ZSTD_decompress(b->p, b->n, in, n);
+    uint32_t root;
     uint16_t ver = 0;
-    if (!ZSTD_isError(got) && got == b.n && b.n >= 16) memcpy(&ver, b.p + 2, 2);
-    size_t heaps_at = 0, dta_at = 0, gfx_at = 0;
-    if (ver >= 2 && ver <= 7 && !memcmp(b.p, "YB", 2) && u32_at(&b, 4, &b.keys) && u32_at(&b, 12, &root) &&
-        (heaps_at = field(&b, root, "Heaps", 0xc1)) && u32_at(&b, heaps_at, &heaps)) {
+    if (!ZSTD_isError(got) && got == b->n && b->n >= 16) memcpy(&ver, b->p + 2, 2);
+    size_t heaps_at = 0;
+    if (ver >= 2 && ver <= 7 && !memcmp(b->p, "YB", 2) && u32_at(b, 4, &b->keys) && u32_at(b, 12, &root) &&
+        (heaps_at = field(b, root, "Heaps", 0xc1)) && u32_at(b, heaps_at, heaps))
+        return 1;
+    free(b->p);
+    b->p = NULL;
+    return 0;
+}
+
+// HD texture packs made for the emulators ship their own HeapDef with "TextureHeap" and its parent grown (HDPacks.com's
+// 4K pack for TotK: by 3,419,000 KiB, 3.3 GiB, for the emulators' 8 GB memory option). Served as it is, the game asks
+// for that room and the application pool has none: it crashed at start (issue #673). So before mem_init sizes the pool,
+// every switched-on mod's HeapDef is read (mods.c mods_peek_romfs: mods_scan needs paths main sets later) and the
+// largest TextureHeap growth over the shipped size counts as TOTK_TEX_HEAP_EXTRA_MB (the larger of the two, at most
+// 4 GiB). romfs.c heapdef_overlay then serves another table grown by that amount in the pack's place: the table of the
+// highest switched-on mod below it whose texture heap is not grown, else the game's own, so the other heaps keep the
+// sizes the content that runs needs.
+#define HEAPDEF_REL "System/HeapSize/Product.Nin_NX_NVN.HeapDef.byml.zs"
+static struct { int done; uint64_t kib; } MT;
+
+// "TextureHeap" of the HeapDef table in[0..n) in KiB; 0 when the table cannot be read or has no such size (the failure
+// is logged with why).
+static int32_t heapdef_tex_kib(const void *in, size_t n, const char *why) {
+    Byml b;
+    uint32_t heaps;
+    size_t th_at = 0;
+    int32_t th = 0;
+    if (!heapdef_open(in, n, &b, &heaps, why)) return 0;
+    if ((th_at = heap_size_field(&b, heaps, "TextureHeap"))) memcpy(&th, b.p + th_at, 4);
+    else rt_log("resolution: the HeapDef table has no TextureHeap size; %s", why);
+    free(b.p);
+    return th;
+}
+
+static void mod_heapdef(const char *name, const char *romfs, void *ctx) {
+    (void)ctx;
+    char path[2300], why[400];
+    if (snprintf(path, sizeof path, "%s/%s", romfs, HEAPDEF_REL) >= (int)sizeof path) return;
+    uint64_t n = 0;
+    uint8_t *in = plat_file_read(path, &n);
+    if (!in) return;
+    snprintf(why, sizeof why, "mod %s: its texture heap growth is left out", name);
+    int32_t th = heapdef_tex_kib(in, (size_t)n, why);
+    free(in);
+    if (!th) return;
+    int64_t grow = (int64_t)th - (int64_t)TEX_HEAP_KIB;
+    if (grow > (int64_t)(4096u << 10)) grow = 4096u << 10;
+    if (grow > 0 && (uint64_t)grow > MT.kib) MT.kib = (uint64_t)grow;
+    rt_log("resolution: mod %s ships a HeapDef with TextureHeap %d KiB (the game's %u KiB)%s", name, th, TEX_HEAP_KIB,
+           grow > 0 ? ": the port grows the texture heap and the pool by the difference" : "");
+}
+
+static uint64_t mod_tex_heap_kib(void) {
+    if (!MT.done) MT.done = 1, mods_peek_romfs(mod_heapdef, NULL);
+    return MT.kib;
+}
+
+// Only such a table is swapped for the game's own (romfs.c heapdef_overlay): a mod table that changes other heaps
+// keeps them. A table it cannot read stays the base (logged).
+int res_heapdef_tex_grown(const void *in, size_t n) {
+    return heapdef_tex_kib(in, n, "the mod's table stays the base") > (int32_t)TEX_HEAP_KIB;
+}
+
+// A table that can stand in for an HD pack's (romfs.c heapdef_overlay): readable, with TextureHeap at most the shipped
+// size. A content mod's table (The Second Quest grows RSDB and UI for its own files) is one.
+int res_heapdef_plain(const void *in, size_t n) {
+    int32_t th = heapdef_tex_kib(in, n, "it does not stand in for the texture pack's table");
+    return th > 0 && th <= (int32_t)TEX_HEAP_KIB;
+}
+
+int res_heapdef_build(const void *in, size_t n, void **out, size_t *out_n) {
+    uint64_t extra = res_heap_extra_kib(), tex = tex_heap_extra_kib();
+    *out = NULL;
+    if (!extra && !tex) return 0;
+    Byml b;
+    uint32_t heaps = 0;
+    size_t dta_at = 0, gfx_at = 0;
+    if (heapdef_open(in, n, &b, &heaps, "the game keeps its own resolution")) {
         dta_at = heap_size_field(&b, heaps, "DynamicTextureAllocatorMem");
         gfx_at = heap_size_field(&b, heaps, "Graphics");
     }
@@ -263,6 +419,26 @@ int res_heapdef_build(const void *in, size_t n, void **out, size_t *out_n) {
     int32_t dta2 = dta + (int32_t)extra, gfx2 = gfx + (int32_t)extra;
     memcpy(b.p + dta_at, &dta2, 4);
     memcpy(b.p + gfx_at, &gfx2, 4);
+    int32_t th = 0, th2 = 0, gg = 0, gg2 = 0;
+    if (tex) {   // TOTK_TEX_HEAP_EXTRA_MB: the world texture heap and its parent grow by the same amount
+        static const char parent[] = "\xe3\x82\xb2\xe3\x83\xbc\xe3\x83\xa0\xe3\x82\xb0\xe3\x83\xa9\xe3\x83\x95"
+                                     "\xe3\x82\xa3\xe3\x83\x83\xe3\x82\xaf\xe3\x82\xb9";   // ゲームグラフィックス
+        size_t th_at = heap_size_field(&b, heaps, "TextureHeap"), gg_at = heap_size_field(&b, heaps, parent);
+        if (th_at && gg_at) {
+            memcpy(&th, b.p + th_at, 4);
+            memcpy(&gg, b.p + gg_at, 4);
+            th2 = th + (int32_t)tex, gg2 = gg + (int32_t)tex;
+            memcpy(b.p + th_at, &th2, 4);
+            memcpy(b.p + gg_at, &gg2, 4);
+        } else {
+            rt_log("resolution: the HeapDef table has no TextureHeap sizes; TOTK_TEX_HEAP_EXTRA_MB left out");
+            tex = 0;
+            if (!extra) {
+                free(b.p);
+                return 0;
+            }
+        }
+    }
     size_t cap = ZSTD_compressBound(b.n);
     void *z = malloc(cap);
     size_t zn = z ? ZSTD_compress(z, cap, b.p, b.n, 9) : 0;
@@ -273,7 +449,12 @@ int res_heapdef_build(const void *in, size_t n, void **out, size_t *out_n) {
     }
     *out = z;
     *out_n = zn;
-    rt_log("resolution: HeapDef DynamicTextureAllocatorMem %d -> %d KiB, Graphics %d -> %d KiB", dta, dta2, gfx, gfx2);
+    if (extra)
+        rt_log("resolution: HeapDef DynamicTextureAllocatorMem %d -> %d KiB, Graphics %d -> %d KiB", dta, dta2, gfx,
+               gfx2);
+    if (tex)
+        rt_log("resolution: HeapDef TextureHeap %d -> %d KiB and its parent %d -> %d KiB (TOTK_TEX_HEAP_EXTRA_MB or a "
+               "mod's HeapDef)", th, th2, gg, gg2);
     return 1;
 }
 
@@ -336,15 +517,20 @@ void res_init(void) {
         return;
     }
     uint8_t *page = rt_patch_page();
-    double k = RT_V143 ? (double)R.ow / BASE_W : 1.0;   // 1.4.3's own scaling of the scene with the output width
+    double k = scene_scale();
     int scene_on = R.scene_set || (!RT_V143 && (R.ow != BASE_W || R.oh != BASE_H));
+    // Handheld, 1.4.3 draws a 1280x720 canvas unless the output patch is on, whatever the output setting (measured: a
+    // 1920x1080 output with the patch off composed the frame at 1280x720 and only the swapchain was 1920x1080). With
+    // the scene patch on the output patch is switched on too, so the canvas is the output size that scene_scale and
+    // the heap size (res_heap_extra_kib) assume.
+    int output_on = R.ow != BASE_W || R.oh != BASE_H || (RT_V143 && scene_on && ov_handheld());
     float sc[2] = {(float)(R.sw / k), (float)(R.sh / k)};
     uint64_t out = pack(R.ow, R.oh), none = 0;
     memcpy(page + 16, sc, sizeof sc);
     memcpy(page + 32, &out, 8);
     memcpy(page + 40, &none, 8);
     memcpy(page + 48, &out, 8);
-    rt_patch_switch(output, R.ow != BASE_W || R.oh != BASE_H);
+    rt_patch_switch(output, output_on);
     rt_patch_switch(scene, scene_on);
     R.on = 1;
     if (R.live) g_fnprof = 1;   // until the game sets its render size: res_fn_hit, or res_present gives up
@@ -413,6 +599,29 @@ void res_fov_live(void) {
     rt_log("resolution: field of view %d%% of the game's", pct);
 }
 
+// TOTK_DYNAMIC_RES = 0 holds the game's dynamic resolution at 100 % (recomp/patches-v121.txt and patches-v143.txt,
+// set fixres), so the scene renders at the size render_scale chose. The game picks its tier (100, 90 or 80 %) mostly
+// from a fixed cost model made for the Switch: one term grows with the object counts, one with (scene width / 1600)
+// squared. The GPU time it also measures (two NVN report timestamps, a frame apart) only lowers that prediction when the
+// GPU is slower than the model says, which a PC GPU never is, so the model alone decides. Measured on 1.2.1: a
+// 2867-wide scene stays at 100 %, one wider than about 2970 does not, and a 4267-wide one (5120x1440 at render_scale
+// 83) fits no tier and falls back to 80 % every frame, whatever the GPU's load. 1 (the default) leaves the game's own
+// stepping and the set off. Runs on a settings change (gfx_present.c window_settings_live), like res_fov_live.
+void res_dynres_live(void) {
+    static int set = -2, shown = -1;
+    if (set == -2) set = rt_patch_set("fixres");
+    int dyn = cfg_bool("TOTK_DYNAMIC_RES");
+    if (dyn == shown) return;
+    shown = dyn;
+    if (set < 0) {
+        if (!dyn) rt_log("resolution: this game version has no fixed-resolution patch; the game keeps its dynamic resolution");
+        return;
+    }
+    rt_patch_switch(set, !dyn);
+    rt_log("resolution: dynamic resolution %s",
+           dyn ? "on: the game's own 100, 90 or 80 %" : "off: the scene stays at 100 %");
+}
+
 void res_wanted(unsigned out[2]) {
     decide();
     out[0] = R.ow, out[1] = R.oh;
@@ -465,7 +674,7 @@ int res_dlss_mode(const char *m) {
     unsigned w, h;
     scene_for(k ? k : R.base_k, &w, &h);
     if (w == R.sw && h == R.sh) return 0;
-    double s = RT_V143 ? (double)R.ow / BASE_W : 1.0;
+    double s = scene_scale();
     float sc[2] = {(float)(w / s), (float)(h / s)};
     memcpy((uint8_t *)rt_patch_page() + 16, sc, sizeof sc);
     R.sw = w, R.sh = h;

runtime/romfs.c

@@ -533,6 +533,22 @@ static void added_init(void) {
     rt_log("RomFS overlay: %zu files added to the RomFS", nadded);
 }
 
+// RomFS file `rel` as the image holds it, whatever overlays replace it. malloc'd; NULL when it is missing, unreadable
+// or 16 MiB or more (logged with `keeps`, what the game does without it).
+static void *read_original(const char *rel, size_t *n, const char *keeps) {
+    uint64_t i = lower(rel);
+    if (i >= file_count || strcmp(entry_name(entry(i)), rel)) return NULL;
+    uint64_t size = entry(i).size;
+    void *in = size && size < (16u << 20) ? malloc((size_t)size) : NULL;
+    if (!in || read_at(image, in, (uint32_t)size, entry(i).offset) != (int64_t)size) {
+        free(in);
+        rt_log("RomFS: cannot read %s; %s", rel, keeps);
+        return NULL;
+    }
+    *n = (size_t)size;
+    return in;
+}
+
 // RomFS file `rel` as the game would read it: from an overlay that replaces it (*base), else from the image. malloc'd;
 // NULL when it is missing, unreadable or 16 MiB or more (logged with `keeps`, what the game does without it).
 static void *read_served(const char *rel, Overlay **base, size_t *n, const char *keeps) {
@@ -829,17 +845,55 @@ static void restbl_overlay(void) {
                members, packs, (now_s() - t0) * 1000);
 }
 
-// The HeapDef table a bigger game resolution needs (resolution.c): its texture heap and that heap's parent grow. The
-// base is an overlay that replaces the table (a mod's heap sizes stay), else the RomFS original; the grown table is
-// written to <profile>/resolution/ and served in the base's place.
+// The HeapDef table a bigger game resolution or TOTK_TEX_HEAP_EXTRA_MB needs (resolution.c): its texture heaps and
+// their parents grow. The base is an overlay that replaces the table (a mod's heap sizes stay), else the RomFS
+// original; the grown table is written to <profile>/resolution/ and served in the base's place. A served mod table
+// whose own TextureHeap is grown (an HD texture pack's, resolution.c res_heapdef_tex_grown) is not the base: its
+// growth is already in the amount (resolution.c mod_tex_heap_kib). In its place the table of the highest switched-on
+// mod below it that changes other heaps only grows by the amount (plain_mod_heapdef: with the pack above The Second
+// Quest, the default order for HDPacks.com's "4kTextures", the game's own sizes for RSDB and UI were too small for
+// that mod's files and the game crashed loading a save), else the RomFS original; either is served in the pack's
+// place. A mod table that changes other heaps only and is served itself stays the base.
 #define HEAPDEF "System/HeapSize/Product.Nin_NX_NVN.HeapDef.byml.zs"
+
+// The HeapDef table of the highest switched-on mod that changes other heaps only (malloc'd, *n, *who its name); NULL
+// when no mod has one. Mods above the served table ship none, so the first found is the highest below it.
+static void *plain_mod_heapdef(size_t *n, const char **who) {
+    for (int i = 0; i < mods_count(); i++) {
+        const ModInfo *m = mods_at(i);
+        char path[2300];
+        if (!m->enabled || !m->romfs[0] ||
+            snprintf(path, sizeof path, "%s/%s", m->romfs, HEAPDEF) >= (int)sizeof path)
+            continue;
+        uint64_t fn = 0;
+        uint8_t *f = plat_file_read(path, &fn);
+        if (!f) continue;
+        if (fn < (16u << 20) && res_heapdef_plain(f, (size_t)fn)) {
+            *n = (size_t)fn;
+            *who = m->name;
+            return f;
+        }
+        free(f);
+    }
+    return NULL;
+}
 static void heapdef_overlay(void) {
-    if (!res_heap_extra_kib()) return;
+    if (!res_heap_extra_kib() && !res_tex_heap_extra_bytes()) return;
     static const char keeps[] = "the game keeps its own resolution";
     Overlay *base;
     size_t n;
     void *in = read_served(HEAPDEF, &base, &n, keeps), *out = NULL;
     size_t out_n = 0;
+    if (in && base && base->mod && res_heapdef_tex_grown(in, n)) {
+        free(in);
+        const char *who = NULL;
+        if ((in = plain_mod_heapdef(&n, &who)))
+            rt_log("RomFS: %s: the HeapDef of %s with the texture heap growth in place of the mod's table",
+                   mods_at(base->mod - 1)->name, who);
+        else if ((in = read_original(HEAPDEF, &n, keeps)))
+            rt_log("RomFS: %s: the game's own HeapDef with the mod's texture heap growth in place of the mod's table",
+                   mods_at(base->mod - 1)->name);
+    }
     int ok = in && res_heapdef_build(in, n, &out, &out_n);
     free(in);
     if (ok && serve_generated(HEAPDEF, base, out, out_n, "resolution", "grown heaps", keeps)) res_heapdef_served();
@@ -865,12 +919,11 @@ static void camera_overlay(void) {
 // The button prompts' folders (input_prompts_folder) and whether each had a manifest next to romfs-index.bin at start;
 // the one the settings asked for at start, the one overlay_init loaded (NULL: the game's own prompts) and why the first
 // did not load (ROMFS_PROMPTS_*). The menu compares them with the settings in effect (ui/menu.cpp prompts_note).
-// romfs-patches-pc keeps that name, but at a game resolution of another shape than 16:9 its files come from pc_dir,
-// romfs-patches-pc-<w>x<h> (overlay_init).
-static struct { const char *name; int ready; } prompts_folders[3];
+// Each folder keeps its name, but at a game resolution of another shape than 16:9 its files come from dir,
+// <name>-<w>x<h> (overlay_init), and ready is that one's.
+static struct { const char *name; int ready; char dir[64]; } prompts_folders[3];
 static const char *prompts_wanted, *prompts_loaded;
 static int prompts_off;
-static char pc_dir[64];
 
 const char *romfs_prompts(void) { return prompts_loaded; }
 
@@ -910,15 +963,15 @@ static void overlay_init(void) {
     // the UI laid out for that shape (make_romfs_patches.py --aspect), when the game version has the camera aspect patch
     // the layout goes with (res_wants_wide_overlay).
     // With prompts = xbox, the Xbox prompts' folder for the pad layout in its place (input_prompts_folder): the same
-    // patches with Xbox button icons (make_romfs_patches.py --prompts). The game reads the archives once at boot, so
-    // the prompts stay until restart even though the layout itself changes live. Each folder replaces romfs-patches as
-    // a whole and the setup builds no folder of another shape with Xbox prompts, so a game resolution that is not 16:9
-    // keeps the Switch prompts. So does a RomFS mod (the TOTK_ROMFS_MODS manifest or a player mod) that replaces one of
-    // the prompts' fonts: romfs-patches loads in their place, so the mod's file is not served beside prompts files made
-    // for the game's own. A mod's resource size table is no clash: restbl_overlay merges it with the prompts' one.
-    // prompts = pc takes romfs-patches-pc whatever the pad layout, and at a game resolution of another shape
-    // romfs-patches-pc-<w>x<h> (make_romfs_patches.py --prompts pc --aspect): it carries that shape's UI layout too, so
-    // it replaces romfs-patches-<w>x<h> as romfs-patches-pc replaces romfs-patches.
+    // patches with Xbox button icons (make_romfs_patches.py --prompts); prompts = pc takes romfs-patches-pc whatever
+    // the pad layout. The game reads the archives once at boot, so the prompts stay until restart even though the
+    // layout itself changes live. Each folder replaces romfs-patches as a whole. At a game resolution of another shape
+    // the prompts come from <folder>-<w>x<h> (make_romfs_patches.py --prompts <style> --aspect): it carries that
+    // shape's UI layout too, so it replaces romfs-patches-<w>x<h> as the folder replaces romfs-patches. Without it the
+    // Switch prompts stay with the shape's layout. So they do when a RomFS mod (the TOTK_ROMFS_MODS manifest or a
+    // player mod) replaces one of the prompts' fonts: romfs-patches loads in their place, so the mod's file is not
+    // served beside prompts files made for the game's own. A mod's resource size table is no clash: restbl_overlay
+    // merges it with the prompts' one.
     char def[2100], path[2100];
     const char *patches = cfg_get("TOTK_ROMFS_PATCHES"), *mods = cfg_get("TOTK_ROMFS_MODS");
     int wide = 0;
@@ -937,45 +990,47 @@ static void overlay_init(void) {
     static const char *const layouts[2] = {"nintendo", "xbox"};   // one of each Xbox folder, then the PC one
     for (int i = 0; i < 3; i++) {
         const char *name = i < 2 ? input_prompts_folder(PROMPTS_XBOX, layouts[i]) : input_prompts_folder(PROMPTS_PC, 0);
-        const char *dir = name;
-        if (i == 2) {
-            if (wide) snprintf(pc_dir, sizeof pc_dir, "%s-%ux%u", name, out[0], out[1]);
-            else snprintf(pc_dir, sizeof pc_dir, "%s", name);
-            dir = pc_dir;
-        }
+        char *dir = prompts_folders[i].dir;
+        if (wide) snprintf(dir, sizeof prompts_folders[i].dir, "%s-%ux%u", name, out[0], out[1]);
+        else snprintf(dir, sizeof prompts_folders[i].dir, "%s", name);
         prompts_folders[i].name = name;
         prompts_folders[i].ready =
             snprintf(path, sizeof path, "%s/%s/manifest.tsv", index_dir, dir) < (int)sizeof path && plat_file_exists(path);
     }
     const char *layout = cfg_value("TOTK_PAD_LAYOUT");
-    int pick = input_prompts_pick(cfg_get("TOTK_PROMPTS"), layout), shaped = wide && pick == PROMPTS_PC;
+    int pick = input_prompts_pick(cfg_get("TOTK_PROMPTS"), layout);
     const char *style = pick == PROMPTS_PC ? "keyboard and mouse" : "Xbox";
     prompts_wanted = input_prompts_folder(pick, layout);
+    const char *wanted_dir = prompts_wanted;   // the folder its files come from
+    for (int i = 0; i < 3; i++)
+        if (prompts_wanted && !strcmp(prompts_folders[i].name, prompts_wanted)) wanted_dir = prompts_folders[i].dir;
     prompts_off = !prompts_wanted                         ? ROMFS_PROMPTS_OK
-                  : wide && !shaped                       ? ROMFS_PROMPTS_SHAPE
                   : patches && !wide                      ? ROMFS_PROMPTS_PATCHES
-                  : !romfs_prompts_ready(prompts_wanted)  ? (shaped ? ROMFS_PROMPTS_SHAPE : ROMFS_PROMPTS_MISSING)
+                  : !romfs_prompts_ready(prompts_wanted)  ? (wide ? ROMFS_PROMPTS_SHAPE : ROMFS_PROMPTS_MISSING)
                                                           : ROMFS_PROMPTS_OK;
     restbl_find();
     size_t first = noverlay;
     if (prompts_wanted && prompts_off == ROMFS_PROMPTS_OK) {
         // fits: its manifest path did
-        snprintf(path, sizeof path, "%s/%s", index_dir, shaped ? pc_dir : prompts_wanted);
+        snprintf(path, sizeof path, "%s/%s", index_dir, wanted_dir);
         if (overlay_load(path, 1, 0)) {
             prompts_loaded = prompts_wanted;
             rt_log("RomFS overlay: %s button prompts (%s)", style, path);
-            if (shaped) patches = NULL;   // in place of romfs-patches-<w>x<h>, whose UI layout it carries
+            if (wide) patches = NULL;   // in place of romfs-patches-<w>x<h>, whose UI layout it carries
         } else {
             prompts_off = ROMFS_PROMPTS_FAILED;
             rt_log("RomFS overlay: %s failed its check (run the setup again); romfs-patches loads instead", path);
         }
     } else if (prompts_wanted) {
-        rt_log("RomFS overlay: prompts = %s, but %s; Switch prompts stay", pick == PROMPTS_PC ? "pc" : "xbox",
-               prompts_off == ROMFS_PROMPTS_SHAPE     ? (shaped ? "the game resolution has no romfs-patches-pc-<w>x<h> "
-                                                                  "(tools/make_romfs_patches.py --prompts pc --aspect)"
-                                                             : "the game resolution takes its own UI layout")
-               : prompts_off == ROMFS_PROMPTS_PATCHES ? "TOTK_ROMFS_PATCHES is set"
-                                                      : "their folder has no manifest.tsv (run the setup again)");
+        char why[256];
+        if (prompts_off == ROMFS_PROMPTS_SHAPE)
+            snprintf(why, sizeof why, "the game resolution has no %s (tools/make_romfs_patches.py --prompts %s "
+                     "--aspect %ux%u)", wanted_dir, prompts_wanted + strlen("romfs-patches-"), out[0], out[1]);
+        else
+            snprintf(why, sizeof why, "%s", prompts_off == ROMFS_PROMPTS_PATCHES
+                                                ? "TOTK_ROMFS_PATCHES is set"
+                                                : "their folder has no manifest.tsv (run the setup again)");
+        rt_log("RomFS overlay: prompts = %s, but %s; Switch prompts stay", pick == PROMPTS_PC ? "pc" : "xbox", why);
     }
     size_t prompts_end = noverlay;
     char plain[2100];   // romfs-patches, when it has a manifest
@@ -994,7 +1049,7 @@ static void overlay_init(void) {
         overlay_drop(first, prompts_end);
         prompts_loaded = NULL;
         prompts_off = ROMFS_PROMPTS_MODS;
-        if (shaped) overlay_load(def, 0, 0);
+        if (wide) overlay_load(def, 0, 0);
         else if (plain_ok) overlay_load(plain, 0, 0);
     }
     restbl_collect();
@@ -1026,6 +1081,26 @@ static void index_path(char *out, size_t cap) {
     strcpy(out + k, "romfs-index.bin");
 }
 
+// 1 when <prefix>-<w>x<h> has a manifest next to romfs-index.bin. Works before fs_init: index_dir is not set that early
+// (cfg_init, res_init and resolution.c decide run first), so the index path is worked out again here.
+static int shaped_folder_ready(const char *prefix, unsigned w, unsigned h) {
+    char dir[2100], path[2200];
+    index_path(dir, sizeof dir);
+    char *sep = plat_last_sep(dir);
+    if (sep) *sep = 0;
+    else strcpy(dir, ".");
+    return snprintf(path, sizeof path, "%s/%s-%ux%u/manifest.tsv", dir, prefix, w, h) < (int)sizeof path &&
+           plat_file_exists(path);
+}
+
+// 1 when the game resolution w x h has the UI layout of its shape: romfs-patches-<w>x<h>, the folder overlay_init
+// looks for.
+int romfs_shaped_ready(unsigned w, unsigned h) { return shaped_folder_ready("romfs-patches", w, h); }
+
+// 1 when it has a prompts' folder (input_prompts_folder) with that layout too: <folder>-<w>x<h>, which overlay_init
+// loads in its place with those prompts.
+int romfs_shaped_prompts_ready(const char *folder, unsigned w, unsigned h) { return shaped_folder_ready(folder, w, h); }
+
 static void load_index(void) {
     char path[2100];
     index_path(path, sizeof path);

runtime/shaders/bl_deswizzle.comp

@@ -0,0 +1,37 @@
+#version 450
+// Block-linear to linear copy of one texture level and layer on the GPU (vk_deswizzle.c), the same layout as
+// tex_deswizzle.c nvn_bl_offset: GOB = 64 bytes x 8 rows; a block is 1 GOB wide, 2^th GOBs high and 2^td deep, its
+// GOBs ordered y first, then z; blocks fill the level x first, then y, then z.
+// One invocation per 16-byte segment of the block-linear storage, in storage order, so the reads (from the pool's
+// imported buffer: guest memory, over PCIe on a discrete GPU) are contiguous; the writes go to a device-local buffer
+// with rows of pitch16 segments. Segments outside the level's rows, slices or row width are padding and skipped.
+layout(local_size_x = 64) in;
+layout(std430, set = 0, binding = 0) readonly buffer Src { uvec4 src[]; };
+layout(std430, set = 0, binding = 1) writeonly buffer Dst { uvec4 dst[]; };
+layout(push_constant) uniform Push {
+    uint src_base;   // first segment of the level and layer in the src binding
+    uint dst_base;   // first segment of its linear copy in the dst binding
+    uint segs;       // segments to visit: whole blocks covering the level
+    uint pitch16;    // segments per linear row (row bytes rounded up to 16)
+    uint rows, slices, th, td;
+    uint gobs_x, blocks_y;
+    uint nx;         // workgroups along x (the dispatch is 2D when one dimension is not enough)
+} pc;
+
+void main() {
+    uint s = (gl_WorkGroupID.y * pc.nx + gl_WorkGroupID.x) * 64u + gl_LocalInvocationID.x;
+    if (s >= pc.segs) return;
+    uint bh = 1u << pc.th, bd = 1u << pc.td;
+    uint per_block = 32u << (pc.th + pc.td);   // 512-byte GOBs of 32 segments each
+    uint block = s / per_block, inb = s % per_block;
+    uint gob = inb >> 5u, ing = inb & 31u;
+    uint per_layer = pc.gobs_x * pc.blocks_y;
+    uint bz = block / per_layer, rem = block % per_layer;
+    uint by = rem / pc.gobs_x, bx = rem % pc.gobs_x;
+    // Inside a GOB, segment bits: 4 = x % 64 / 32, 3..2 = y % 8 / 2, 1 = x % 32 / 16, 0 = y % 2.
+    uint xseg = bx * 4u + ((ing >> 4u) & 1u) * 2u + ((ing >> 1u) & 1u);
+    uint y = by * 8u * bh + (gob % bh) * 8u + ((ing >> 2u) & 3u) * 2u + (ing & 1u);
+    uint z = bz * bd + gob / bh;
+    if (xseg >= pc.pitch16 || y >= pc.rows || z >= pc.slices) return;
+    dst[pc.dst_base + (z * pc.rows + y) * pc.pitch16 + xseg] = src[pc.src_base + s];
+}

runtime/simd_fp.h

@@ -205,19 +205,21 @@ SN T FN(slow)(Cpu *c, int op, T a, T b, T d) {
 // it to fcmp one, a single ucomiss + jne for scalars and one vcmpps for vectors).
 #define FOK(r) ((__builtin_elementwise_abs(r) < FN(fv)(MINN)) | (__builtin_elementwise_abs(r) > FN(fv)(MINN)))
 SN T FN(redo)(Cpu *c, int op, T a, T b, T d, T r) {
-    if (c->fpcr & FPCR_FZ) return FN(slow)(c, op, a, b, d);
+    if (c->fpcr & (FPCR_FZ | FPCR_RM)) return FN(slow)(c, op, a, b, d);
     return FBAD(op, r) ? FN(fix)(c, op, a, b, d) : r;
 }
 
-// Scalar FP op (constant op folds the selection). RMode = nearest runs on the host, with or without FZ.
+// Scalar FP op (constant op folds the selection). RMode = nearest runs on the host, with or without FZ. A directed
+// RMode is rare, so it shares the result check's branch: the host result, computed in round-to-nearest, is then
+// thrown away and redo recomputes in the FPCR mode. Min and max do not round, so RMode does not matter to them.
 SI T FN(op)(Cpu *c, int op, T a, T b, T d) {
-    if (simd_directed(c)) return FN(slow)(c, op, a, b, d);
     if (FMINMAX(op)) {
         if (c->fpcr & FPCR_FZ) a = FN(flush)(a), b = FN(flush)(b);
         return FN(minmax)(c, op, a, b);
     }
     T r = FCALC(op, a, b, d, K, FMA_S);
-    if (__builtin_expect(op == FOP_RSQRTS ? FREDO(op, r) : !FOK(r), 0)) r = FN(redo)(c, op, a, b, d, r);
+    if (__builtin_expect((op == FOP_RSQRTS ? FREDO(op, r) : !FOK(r)) | simd_directed(c), 0))
+        r = FN(redo)(c, op, a, b, d, r);
     return r;
 }
 
@@ -237,9 +239,9 @@ SI VT FN(vflush)(VT a) {
     return (VT)FN(vsel)(den, u & SGN, u);
 }
 
-// Vector FP op: native lanes; any lane FREDO flags (or FPCR.RMode) reruns all lanes through the scalar path.
+// Vector FP op: native lanes; any lane FREDO flags (or a directed FPCR.RMode, checked in the same branch) reruns all
+// lanes through the scalar path. Min and max ignore RMode, as in FN(op).
 SI VT FN(vop)(Cpu *c, int op, VT a, VT b, VT d) {
-    if (simd_directed(c)) return FN(vslow)(c, op, a, b, d);
     VT r;
     if (FMINMAX(op)) {
         if (c->fpcr & FPCR_FZ) a = FN(vflush)(a), b = FN(vflush)(b);
@@ -252,7 +254,8 @@ SI VT FN(vop)(Cpu *c, int op, VT a, VT b, VT d) {
         return r;
     }
     r = FCALC(op, a, b, d, VK, FMA_V);
-    if (__builtin_expect(op == FOP_RSQRTS ? __builtin_reduce_or(FREDO(op, r)) != 0 : __builtin_reduce_and(FOK(r)) == 0, 0))
+    if (__builtin_expect((op == FOP_RSQRTS ? __builtin_reduce_or(FREDO(op, r)) != 0 : __builtin_reduce_and(FOK(r)) == 0)
+                         | simd_directed(c), 0))
         return FN(vslow)(c, op, a, b, d);
     return r;
 }
@@ -260,13 +263,15 @@ SI VT FN(vop)(Cpu *c, int op, VT a, VT b, VT d) {
 SI VT FN(vfz)(Cpu *c, VT a) { return c->fpcr & FPCR_FZ ? FN(vflush)(a) : a; }
 
 // ---- compares
+// FPCR.FZ needs no flush here: rt_fpcr_sync (runtime/cpu.h) keeps MXCSR.DAZ equal to FPCR.FZ, and x86 compares
+// under DAZ read a denormal operand as a zero of its sign, as ARM compares do under FZ.
 SI uint64_t FN(cmp)(Cpu *c, T a, T b) { // FCMP NZCV
-    a = FN(fz)(c, a), b = FN(fz)(c, b);
+    (void)c;
     return a == b ? 0x60000000ull : a < b ? 0x80000000ull : a > b ? 0x20000000ull : 0x30000000ull;
 }
 // FCMEQ/FCMGE/FCMGT/FACGE/FACGT (op 0..4)
 SI VTU FN(vcmp)(Cpu *c, int op, VT a, VT b) {
-    a = FN(vfz)(c, a), b = FN(vfz)(c, b);
+    (void)c;
     if (op >= 3) a = __builtin_elementwise_abs(a), b = __builtin_elementwise_abs(b);
     switch (op) {
     case 0: return (VTU)(a == b);

runtime/svc.c

@@ -216,6 +216,7 @@ static void svc_body(Cpu *c, uint32_t imm, uint64_t pc) {
         rt_log("guest ExitProcess at %s; exiting", ldr_describe(pc, at, sizeof at));
         fs_save_flush();
         vk_pipeline_save();
+        playtime_save();
         prof_write();
         fflush(NULL);
         if (svc_test_exit_hook) return svc_test_exit_hook(0);

runtime/tex_probe.c

@@ -355,7 +355,7 @@ static int test_import(pool_t *pool) {
     GCHECK(n == 4 && pages[0] == a + 0x30000 && pages[3] == a + 0x33000, "CPU writes to 4 pages: %llu reported",
            (unsigned long long)n);
     vk_mem_resolve(a + 0x30000, 0x4000, &sp);
-    vk_mem_shadow_sync();
+    vk_mem_shadow_sync(0);
     copy_wait(sp.buffer, sp.offset, sb, 0, 0x4000);
     vmaInvalidateAllocation(c->vma, sa, 0, VK_WHOLE_SIZE);
     GCHECK(!memcmp(s, m + 0x30000, 0x4000), "GPU read of CPU-written heap differs");

runtime/ui/amiibo_page.cpp

@@ -36,6 +36,8 @@ static int g_selected_entry = -1;
 // The last action: a catalog id and the file or folder it names, formatted when drawn (the menu language can change)
 static UiTextId g_status_id = UI_MENU_AMIIBO_READY;
 static std::string g_status_arg;
+// Flag to request font rebuild (set when amiibo directory is refreshed with new files)
+static bool g_font_rebuild_requested = false;
 
 static void set_status(UiTextId id, const std::string &arg = std::string()) {
     g_status_id = id;
@@ -124,6 +126,9 @@ static void refresh_dir(void) {
         if (a.is_dir != b.is_dir) return a.is_dir;
         return a.name < b.name;
     });
+    
+    // Request font rebuild to include any new filenames
+    amiibo_request_font_rebuild();
 }
 
 void tab_amiibo(float scale) {
@@ -303,3 +308,44 @@ void tab_amiibo(float scale) {
 
     ImGui::EndChild();
 }
+
+// Collect amiibo filenames for font glyph building (called from menu.cpp fonts_up)
+void amiibo_collect_filenames(void (*callback)(void *, const char *), void *ctx) {
+    std::error_code ec;
+    fs::path dir_path(g_current_dir);
+    
+    if (!fs::exists(dir_path, ec) || !fs::is_directory(dir_path, ec)) {
+        return;
+    }
+    
+    for (const auto &entry : fs::directory_iterator(dir_path, fs::directory_options::skip_permission_denied, ec)) {
+        if (ec) break;
+        std::error_code status_ec;
+        bool is_dir = entry.is_directory(status_ec);
+        if (status_ec || is_dir) continue;
+        
+        std::string ext = entry.path().extension().string();
+        if (!iequals(ext, ".bin") && !iequals(ext, ".nfc")) {
+            continue;
+        }
+        
+        std::string filename = entry.path().filename().string();
+        if (!filename.empty()) {
+            callback(ctx, filename.c_str());
+        }
+    }
+}
+
+// Request font atlas rebuild on next frame
+void amiibo_request_font_rebuild(void) {
+    g_font_rebuild_requested = true;
+}
+
+// Check and clear font rebuild request flag
+extern "C" bool amiibo_check_font_rebuild(void) {
+    if (g_font_rebuild_requested) {
+        g_font_rebuild_requested = false;
+        return true;
+    }
+    return false;
+}

runtime/ui/amiibo_page.h

@@ -2,3 +2,9 @@
 #pragma once
 
 void tab_amiibo(float scale);
+
+// Collect amiibo filenames for font glyph building (called from menu.cpp fonts_up)
+void amiibo_collect_filenames(void (*callback)(void *, const char *), void *ctx);
+
+// Request font atlas rebuild on next frame (called when amiibo directory is refreshed)
+void amiibo_request_font_rebuild(void);

runtime/ui/menu.cpp

@@ -51,12 +51,15 @@ extern "C" {
 void rt_log(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
 void vk_queue_lock(void);
 void vk_queue_unlock(void);
+int vk_exec_replay_workers(void);
 void res_sizes(unsigned out[2], unsigned scene[2]);
 double frame_step_limit_hz(void);
 extern const int rt_game_version;
 extern const char rt_port_version[];
 void cheats_frame(uint64_t frame);   // cheats.c
 bool tab_cheats(float scale);        // ui/cheats_tab.cpp; true: restart into the rebuild
+int playtime_get(uint64_t *total_s, uint64_t *session_s);   // playtime.c
+bool amiibo_check_font_rebuild(void);  // ui/amiibo_page.cpp; check and clear font rebuild request
 }
 
 // Roboto Medium (Apache-2.0, ui/fonts/LICENSE-Roboto.txt) with Latin and Cyrillic glyphs.
@@ -352,6 +355,9 @@ static int fonts_up(float scale) {
             if (const CheatInfo *ci = cheats_info(k); ci && ci->name) b.AddText(ci->name);
         for (int k = 0; k < mods_count(); k++)
             if (const ModInfo *mi = mods_at(k); mi) b.AddText(mi->name);
+        // Add amiibo filenames so they display correctly in the menu
+        amiibo_collect_filenames(add_row_glyphs, &b);
+        
         b.BuildRanges(&ranges[i]);
         ImFontConfig mc;
         mc.MergeMode = true;
@@ -662,6 +668,7 @@ static void tab_screen(float scale) {
     opt_group(UI_MENU_GRAPHICS);
     row_output_res();
     row_render_scale();
+    opt_toggle("TOTK_DYNAMIC_RES", UI_MENU_DYNAMIC_RES, UI_MENU_DYNAMIC_RES_DESC);
     row_shadow_res();
     static const UiTextId up[] = {UI_MENU_UPSCALER_FSR, UI_MENU_UPSCALER_BILINEAR};
     opt_choice("TOTK_UPSCALER", UI_MENU_UPSCALER, up, 2, UI_MENU_UPSCALER_DESC);
@@ -698,6 +705,19 @@ static void tab_screen(float scale) {
     opt_end();
 }
 
+// Play time across runs (playtime.c), read only. Hidden while it is not counted (no profile directory).
+static void row_play_time(void) {
+    uint64_t total, session;
+    if (!playtime_get(&total, &session)) return;
+    char value[64], run[64], note[96];
+    snprintf(value, sizeof value, menu_text(UI_MENU_PLAY_TIME_HM), (unsigned long long)(total / 3600),
+             (unsigned)(total / 60 % 60));
+    snprintf(run, sizeof run, menu_text(UI_MENU_PLAY_TIME_HM), (unsigned long long)(session / 3600),
+             (unsigned)(session / 60 % 60));
+    snprintf(note, sizeof note, menu_text(UI_MENU_PLAY_TIME_SESSION), run);
+    opt_text_row("##play_time", menu_text(UI_MENU_PLAY_TIME), value, note, menu_text(UI_MENU_PLAY_TIME_DESC), false);
+}
+
 static void tab_game(float scale) {
     opt_begin("##game", scale);
     row_fps_limit();
@@ -706,6 +726,7 @@ static void tab_game(float scale) {
     static const UiTextId mode[] = {UI_MENU_MODE_DOCKED, UI_MENU_MODE_HANDHELD};
     opt_choice("TOTK_OPERATION_MODE", UI_MENU_OPERATION_MODE, mode, 2, UI_MENU_OPERATION_MODE_DESC);
     opt_toggle("TOTK_ASSUME_FOCUS", UI_MENU_ASSUME_FOCUS, UI_MENU_ASSUME_FOCUS_DESC);
+    row_play_time();
     opt_end();
 }
 
@@ -776,8 +797,8 @@ static const char *prompts_note(bool *restart) {
     const char *layout = cfg_value("TOTK_PAD_LAYOUT"), *used = romfs_prompts();
     const char *want = input_prompts_folder(input_prompts_pick(cfg_get("TOTK_PROMPTS"), layout), layout);
     if (want == used || (want && used && !strcmp(want, used))) return nullptr;
-    // A game resolution of another shape needs a folder of its own for the PC prompts (romfs-patches-pc-<w>x<h>), which
-    // the setup does not write: the shape is the reason then, not the setup.
+    // A game resolution of another shape needs a folder of its own for the prompts (<folder>-<w>x<h>). The setup writes
+    // one for each size the Screen tab offers; at any other size the shape is the reason, not the setup.
     int off = romfs_prompts_off(want);
     if (want && !romfs_prompts_ready(want) && off != ROMFS_PROMPTS_SHAPE) return menu_text(UI_MENU_PROMPTS_MISSING);
     switch (off) {
@@ -890,6 +911,23 @@ static void row_threads(const char *key, UiTextId label, UiTextId desc) {
     opt_choice_list(key, label, vals, names, N, desc);
 }
 
+// The chunked replay's recording threads: Off, then the counts; the note under the value is what this run uses, which
+// a change replaces only after a restart.
+static void row_replay_workers(void) {
+    static const char *const vals[] = {"0", "2", "3", "4", "6", "8", "12", "16"};
+    enum { N = sizeof vals / sizeof *vals };
+    const char *names[N];
+    names[0] = menu_text(UI_MENU_OFF);
+    for (int i = 1; i < N; i++) names[i] = vals[i];
+    static char now[64];
+    char n[16];
+    int w = vk_exec_replay_workers();
+    snprintf(n, sizeof n, "%d", w);
+    if (w >= 0) snprintf(now, sizeof now, menu_text(UI_MENU_REPLAY_WORKERS_NOW), w ? n : menu_text(UI_MENU_OFF));
+    opt_choice_list("TOTK_REPLAY_WORKERS", UI_MENU_REPLAY_WORKERS, vals, names, N, UI_MENU_REPLAY_WORKERS_DESC, -1,
+                    w >= 0 ? now : nullptr);
+}
+
 // A free-text key shown as Off (unset) / On (value set): On writes on_value, any set value reads as On.
 static void row_set_or_not(const char *key, UiTextId label, UiTextId desc, const char *on_value) {
     const char *vals[] = {"", on_value}, *names[] = {menu_text(UI_MENU_OFF), menu_text(UI_MENU_ON)};
@@ -939,6 +977,7 @@ static void tab_advanced(float scale) {
     static const UiTextId preempt[] = {UI_MENU_PREEMPT_OFF, UI_MENU_PREEMPT_ON, UI_MENU_PREEMPT_INHERIT};   // 0|1|2
     opt_choice("TOTK_ASYNC_PREEMPT", UI_MENU_ASYNC_PREEMPT, preempt, 3, UI_MENU_ASYNC_PREEMPT_DESC);
     opt_toggle("TOTK_REPLAY_ASYNC", UI_MENU_REPLAY_ASYNC, UI_MENU_REPLAY_ASYNC_DESC);
+    row_replay_workers();
     opt_toggle("TOTK_TIMER_1MS", UI_MENU_TIMER_1MS, UI_MENU_TIMER_1MS_DESC);
 
     opt_group(UI_MENU_GROUP_SHADERS);
@@ -1066,6 +1105,7 @@ static void mods_builtin_rows(void) {
     opt_toggle("TOTK_REALTIME", UI_MENU_REALTIME, UI_MENU_REALTIME_DESC);
     opt_toggle("TOTK_CLOTH_REALTIME", UI_MENU_CLOTH_REALTIME, UI_MENU_CLOTH_REALTIME_DESC);
     opt_toggle("TOTK_HIGHFPS_FIX", UI_MENU_HIGHFPS_FIX, UI_MENU_HIGHFPS_FIX_DESC);
+    opt_toggle("TOTK_CLIMB_FIX", UI_MENU_CLIMB_FIX, UI_MENU_CLIMB_FIX_DESC);
     opt_button("##mods_quickmenu", UI_MENU_MODS_QUICKMENU, UI_MENU_MODS_QUICKMENU_DESC,
                menu_text(romfs_patches_loaded() ? UI_MENU_ON : UI_MENU_OFF));
 }
@@ -1094,8 +1134,8 @@ static void tab_mods(float scale) {
         char code[512] = "", problem[512] = "";
         if (m->nsets && m->overrides[0])
             snprintf(code, sizeof code, menu_text(UI_MENU_MODS_REPLACES_BUILTIN), m->nsets, m->overrides);
-        else if (m->nsets)
-            snprintf(code, sizeof code, menu_text(UI_MENU_MODS_EXEFS_ON), m->nsets);
+        else if (m->nsets || m->ndata)
+            snprintf(code, sizeof code, menu_text(UI_MENU_MODS_EXEFS_ON), m->nsets + m->ndata);
         switch (m->problem) {
         case MOD_EMPTY: snprintf(problem, sizeof problem, "%s", menu_text(UI_MENU_MODS_EMPTY)); break;
         case MOD_PLUGIN: snprintf(problem, sizeof problem, "%s", menu_text(UI_MENU_MODS_SUBSDK)); break;
@@ -1521,7 +1561,9 @@ extern "C" int menu_overlay_frame(uint64_t frame, const MenuTarget *t, const cha
     uint64_t t0 = plat_now_ns();
     if (!g_ctx_ok) make_context();
     float scale = menu_scale(t->height);
-    if (!vk_up(t) || (g_font_scale != scale && !fonts_up(scale))) {
+    // Check if amiibo page requested font rebuild (e.g., after refreshing directory with new files)
+    bool amiibo_rebuild = amiibo_check_font_rebuild();
+    if (!vk_up(t) || (g_font_scale != scale && !fonts_up(scale)) || (amiibo_rebuild && !fonts_up(scale))) {
         rt_log("menu: Dear ImGui Vulkan back end set-up failed; the menu stays closed");
         g_failed.store(1);
         set_open(0);

runtime/ui_text.c

@@ -71,6 +71,8 @@ static const char *const text_en[UI_TEXT_COUNT] = {
     [UI_MENU_RENDER_SCALE_DESC] = "Size of the 3D scene relative to the game resolution; the game's FSR scales it to "
                                   "that size. Above 100 % smooths jagged edges (supersampling) at a high GPU cost.",
     [UI_MENU_RENDER_SCALE_GAME] = "Game (83 %)",
+    [UI_MENU_DYNAMIC_RES] = "Dynamic resolution",
+    [UI_MENU_DYNAMIC_RES_DESC] = "Lets the game lower the 3D scene to 90 or 80 %. It judges by an estimate made for the Switch that ignores your GPU, so wide scenes drop even on a fast PC. Off keeps the render scale.",
     [UI_MENU_UPSCALER] = "Scaling to the window",
     [UI_MENU_UPSCALER_DESC] = "How the picture is scaled when the window is larger than the game resolution. FSR keeps "
                               "edges and detail sharper than bilinear.",
@@ -181,6 +183,8 @@ static const char *const text_en[UI_TEXT_COUNT] = {
     [UI_MENU_CLOTH_REALTIME_DESC] = "Steps cloth and hair by real time as well.",
     [UI_MENU_HIGHFPS_FIX] = "High frame rate fixes",
     [UI_MENU_HIGHFPS_FIX_DESC] = "Corrects physics that misbehave above 30 frames per second, and objects that stay see-through or invisible above 60.",
+    [UI_MENU_CLIMB_FIX] = "Climbing speed fix",
+    [UI_MENU_CLIMB_FIX_DESC] = "Keeps climbing walls and ladders at full speed above 60 frames per second, so Link reaches the top.",
     [UI_MENU_GAME_LANG] = "Game language", [UI_MENU_GAME_LANG_DESC] = "Language of the game's text and voices.",
     [UI_MENU_GAME_LANG_AUTO] = "Automatic",
     [UI_MENU_OPERATION_MODE] = "Console mode",
@@ -192,6 +196,10 @@ static const char *const text_en[UI_TEXT_COUNT] = {
                                   "window loses focus, as on the Switch when Home is pressed; a load interrupted that "
                                   "way can stop for good.",
 
+    [UI_MENU_PLAY_TIME] = "Play time",
+    [UI_MENU_PLAY_TIME_DESC] = "Time in the game over all runs, counted while the game window has the focus and the game "
+                               "draws frames. Kept in playtime.txt in the profile folder, next to the saves.",
+    [UI_MENU_PLAY_TIME_HM] = "%llu h %02u min", [UI_MENU_PLAY_TIME_SESSION] = "this run: %s",
     [UI_MENU_MODS_BUILTIN] = "Built into the port", [UI_MENU_MODS_INSTALLED] = "Installed mods",
     [UI_MENU_MODS_NONE] = "No mods yet. Unpack a mod into the mods folder, one folder per mod with its romfs folder "
                           "inside, and restart the game. TKMM can export its merged mods into a folder there too.",
@@ -256,7 +264,7 @@ static const char *const text_en[UI_TEXT_COUNT] = {
     [UI_MENU_PROMPTS_MISSING] = "prompt files missing: run the setup again",
     [UI_MENU_PROMPTS_FAILED] = "prompt files damaged: run the setup again",
     [UI_MENU_PROMPTS_RESTART] = "prompts change after restart",
-    [UI_MENU_PROMPTS_OFF_SHAPE] = "off at a game resolution not 16:9",
+    [UI_MENU_PROMPTS_OFF_SHAPE] = "off at this game resolution (no layout made for it)",
     [UI_MENU_PROMPTS_OFF_MODS] = "off: a RomFS mod replaces their files",
     [UI_MENU_PROMPTS_OFF_PATCHES] = "off while TOTK_ROMFS_PATCHES is set",
     [UI_MENU_RESTART] = "Restart now",
@@ -382,6 +390,9 @@ static const char *const text_en[UI_TEXT_COUNT] = {
     [UI_MENU_PREEMPT_OFF] = "Off", [UI_MENU_PREEMPT_ON] = "On", [UI_MENU_PREEMPT_INHERIT] = "Inheritance only",
     [UI_MENU_REPLAY_ASYNC] = "GPU frame prep on its own thread",
     [UI_MENU_REPLAY_ASYNC_DESC] = "Prepares frames for the GPU on a separate thread instead of the game thread. Adds up to one frame of latency.",
+    [UI_MENU_REPLAY_WORKERS] = "GPU frame prep threads",
+    [UI_MENU_REPLAY_WORKERS_DESC] = "Records large frames for the GPU on several threads at once. Helps when the CPU limits the frame rate; needs 8 or more logical processors. Off records on one thread.",
+    [UI_MENU_REPLAY_WORKERS_NOW] = "now: %s",
     [UI_MENU_TIMER_1MS] = "1 ms Windows timer",
     [UI_MENU_TIMER_1MS_DESC] = "Raises the system timer resolution to 1 ms for steadier frame pacing.",
     [UI_MENU_PIPE_WAIT] = "Waiting draw handling",
@@ -433,7 +444,8 @@ static const char *const text_en[UI_TEXT_COUNT] = {
     [UI_MENU_VK_DYN_VERTEX_DESC] = "Forces dynamic vertex input on or off; the default depends on the GPU.",
     [UI_MENU_VK_DYN_A2C] = "Dynamic alpha to coverage",
     [UI_MENU_VK_DYN_A2C_DESC] = "Forces dynamic alpha to coverage on or off. Default: off on AMD's driver for RDNA 2 and "
-                                "older, which draws characters black with it, on elsewhere.",
+                                "older, which draws characters black with it, and on Intel's Windows driver for "
+                                "integrated GPUs, which draws coloured noise on foliage; on elsewhere.",
     [UI_MENU_VK_SUBGROUP_PIN] = "Pin subgroup size", [UI_MENU_VK_SUBGROUP_PIN_DESC] = "Fixes the shader subgroup size.",
     [UI_MENU_TRACE] = "Performance log", [UI_MENU_TRACE_DESC] = "Writes frame timings to a file.",
     [UI_MENU_FLIGHT] = "Flight recorder",
@@ -541,6 +553,8 @@ static const char *const text_ru[UI_TEXT_COUNT] = {
     [UI_MENU_RENDER_SCALE_DESC] = "Размер 3D-сцены относительно разрешения игры; FSR игры доводит её до этого размера. "
                                   "Выше 100 % сглаживает лесенки на краях (суперсэмплинг) ценой большой нагрузки на GPU.",
     [UI_MENU_RENDER_SCALE_GAME] = "Как в игре (83 %)",
+    [UI_MENU_DYNAMIC_RES] = "Динамическое разрешение",
+    [UI_MENU_DYNAMIC_RES_DESC] = "Игра снижает 3D-сцену до 90 или 80 %. Она судит по оценке для Switch, не учитывающей вашу видеокарту, поэтому широкие сцены снижаются даже на быстром ПК. Выключено: остаётся масштаб отрисовки.",
     [UI_MENU_UPSCALER] = "Масштабирование под окно",
     [UI_MENU_UPSCALER_DESC] = "Как картинка растягивается, когда окно больше разрешения игры. FSR сохраняет края и "
                               "детали чётче билинейного.",
@@ -651,6 +665,8 @@ static const char *const text_ru[UI_TEXT_COUNT] = {
     [UI_MENU_CLOTH_REALTIME_DESC] = "Ткань и волосы тоже идут по реальному времени.",
     [UI_MENU_HIGHFPS_FIX] = "Исправления для высокого FPS",
     [UI_MENU_HIGHFPS_FIX_DESC] = "Исправляет физику, которая ломается при частоте выше 30 кадров в секунду, и объекты, которые выше 60 остаются полупрозрачными или невидимыми.",
+    [UI_MENU_CLIMB_FIX] = "Исправление скорости лазания",
+    [UI_MENU_CLIMB_FIX_DESC] = "Сохраняет нормальную скорость лазания по стенам и лестницам выше 60 кадров в секунду, чтобы Линк добирался до верха.",
     [UI_MENU_GAME_LANG] = "Язык игры", [UI_MENU_GAME_LANG_DESC] = "Язык текста и озвучки игры.",
     [UI_MENU_GAME_LANG_AUTO] = "Автоматически",
     [UI_MENU_OPERATION_MODE] = "Режим консоли",
@@ -662,6 +678,10 @@ static const char *const text_ru[UI_TEXT_COUNT] = {
                                   "когда окно теряет фокус, как на Switch при нажатии Home; прерванная так загрузка "
                                   "может остановиться насовсем.",
 
+    [UI_MENU_PLAY_TIME] = "Время в игре",
+    [UI_MENU_PLAY_TIME_DESC] = "Время в игре за все запуски. Считается, пока окно игры в фокусе и игра рисует кадры. "
+                               "Хранится в playtime.txt в папке профиля, рядом с сохранениями.",
+    [UI_MENU_PLAY_TIME_HM] = "%llu ч %02u мин", [UI_MENU_PLAY_TIME_SESSION] = "в этом запуске: %s",
     [UI_MENU_MODS_BUILTIN] = "Встроенные в порт", [UI_MENU_MODS_INSTALLED] = "Установленные моды",
     [UI_MENU_MODS_NONE] = "Модов пока нет. Распакуйте мод в папку mods, каждый мод в свою папку с папкой romfs внутри, и "
                           "перезапустите игру. TKMM тоже может выгрузить туда объединённые моды отдельной папкой.",
@@ -724,7 +744,7 @@ static const char *const text_ru[UI_TEXT_COUNT] = {
     [UI_MENU_PROMPTS_MISSING] = "нет файлов подсказок: запустите установку снова",
     [UI_MENU_PROMPTS_FAILED] = "файлы подсказок повреждены: запустите установку снова",
     [UI_MENU_PROMPTS_RESTART] = "подсказки сменятся после перезапуска",
-    [UI_MENU_PROMPTS_OFF_SHAPE] = "выкл. при разрешении игры не 16:9",
+    [UI_MENU_PROMPTS_OFF_SHAPE] = "выкл. при этом разрешении игры (для него нет раскладки)",
     [UI_MENU_PROMPTS_OFF_MODS] = "выкл.: мод RomFS заменяет их файлы",
     [UI_MENU_PROMPTS_OFF_PATCHES] = "выкл., пока задан TOTK_ROMFS_PATCHES",
     [UI_MENU_RESTART] = "Перезапустить сейчас",
@@ -854,6 +874,9 @@ static const char *const text_ru[UI_TEXT_COUNT] = {
     [UI_MENU_PREEMPT_OFF] = "Выкл.", [UI_MENU_PREEMPT_ON] = "Вкл.", [UI_MENU_PREEMPT_INHERIT] = "Только наследование",
     [UI_MENU_REPLAY_ASYNC] = "Подготовка кадра для GPU в отдельном потоке",
     [UI_MENU_REPLAY_ASYNC_DESC] = "Кадр для GPU готовится в отдельном потоке, а не в потоке игры. Добавляет до одного кадра задержки.",
+    [UI_MENU_REPLAY_WORKERS] = "Потоки подготовки кадра для GPU",
+    [UI_MENU_REPLAY_WORKERS_DESC] = "Большие кадры для GPU записываются сразу в нескольких потоках. Помогает, когда частоту кадров ограничивает процессор; нужно 8 логических процессоров или больше. «Выкл.» — запись в одном потоке.",
+    [UI_MENU_REPLAY_WORKERS_NOW] = "сейчас: %s",
     [UI_MENU_TIMER_1MS] = "Таймер Windows 1 мс",
     [UI_MENU_TIMER_1MS_DESC] = "Повышает точность системного таймера до 1 мс для ровного темпа кадров.",
     [UI_MENU_PIPE_WAIT] = "Обработка ожидающих отрисовок",
@@ -905,8 +928,9 @@ static const char *const text_ru[UI_TEXT_COUNT] = {
     [UI_MENU_VK_DYN_VERTEX_DESC] = "Принудительно включает или выключает динамический ввод вершин; по умолчанию зависит от GPU.",
     [UI_MENU_VK_DYN_A2C] = "Динамический alpha to coverage",
     [UI_MENU_VK_DYN_A2C_DESC] = "Принудительно включает или выключает динамический alpha to coverage. По умолчанию "
-                                "выключен на драйвере AMD для RDNA 2 и старше, где с ним персонажи чёрные, и включён "
-                                "на остальных.",
+                                "выключен на драйвере AMD для RDNA 2 и старше, где с ним персонажи чёрные, и на "
+                                "драйвере Intel для Windows на встроенных GPU, где с ним цветной шум на листве; "
+                                "включён на остальных.",
     [UI_MENU_VK_SUBGROUP_PIN] = "Фиксировать размер подгруппы",
     [UI_MENU_VK_SUBGROUP_PIN_DESC] = "Фиксирует размер подгруппы шейдера.",
     [UI_MENU_TRACE] = "Журнал производительности", [UI_MENU_TRACE_DESC] = "Пишет время кадров в файл.",
@@ -983,6 +1007,10 @@ static const char *const text_pt_br[UI_TEXT_COUNT] = {
     [UI_MENU_TAB_SCREEN] = "Tela", [UI_MENU_TAB_GAME] = "Jogo", [UI_MENU_TAB_SOUND] = "Som",
     [UI_MENU_TAB_CONTROLS] = "Controles", [UI_MENU_TAB_AMIIBO] = "Amiibo", [UI_MENU_TAB_MENU] = "Menu", [UI_MENU_TAB_ADVANCED] = "Avançado",
     [UI_MENU_TAB_MODS] = "Mods",
+    [UI_MENU_PLAY_TIME] = "Tempo de jogo",
+    [UI_MENU_PLAY_TIME_DESC] = "Tempo no jogo somando todas as sessões, contado enquanto a janela do jogo está em foco e o "
+                               "jogo desenha quadros. Fica em playtime.txt na pasta do perfil, junto dos saves.",
+    [UI_MENU_PLAY_TIME_HM] = "%llu h %02u min", [UI_MENU_PLAY_TIME_SESSION] = "nesta sessão: %s",
     [UI_MENU_MODS_BUILTIN] = "Embutidos no port", [UI_MENU_MODS_INSTALLED] = "Mods instalados",
     [UI_MENU_MODS_NONE] = "Nenhum mod ainda. Descompacte um mod na pasta mods, uma pasta por mod com a pasta romfs dentro, "
                           "e reinicie o jogo. O TKMM também pode exportar os mods mesclados para uma pasta ali.",
@@ -1046,6 +1074,8 @@ static const char *const text_pt_br[UI_TEXT_COUNT] = {
                                   "tamanho. Acima de 100 % suaviza bordas serrilhadas (supersampling) com alto custo de "
                                   "GPU.",
     [UI_MENU_RENDER_SCALE_GAME] = "Jogo (83 %)",
+    [UI_MENU_DYNAMIC_RES] = "Resolução dinâmica",
+    [UI_MENU_DYNAMIC_RES_DESC] = "Deixa o jogo reduzir a cena 3D para 90 ou 80 %. Ele decide por uma estimativa feita para o Switch que ignora sua GPU, então cenas largas caem mesmo num PC rápido. Desligada, mantém a escala da renderização.",
     [UI_MENU_UPSCALER] = "Escala para a janela",
     [UI_MENU_UPSCALER_DESC] = "Como a imagem é ampliada quando a janela é maior que a resolução do jogo. O FSR mantém "
                               "bordas e detalhes mais nítidos que o bilinear.",
@@ -1081,6 +1111,8 @@ static const char *const text_pt_br[UI_TEXT_COUNT] = {
     [UI_MENU_CLOTH_REALTIME_DESC] = "Avança tecidos e cabelos pelo tempo real também.",
     [UI_MENU_HIGHFPS_FIX] = "Correções para alta taxa de quadros",
     [UI_MENU_HIGHFPS_FIX_DESC] = "Corrige a física que se comporta mal acima de 30 quadros por segundo e objetos que ficam transparentes ou invisíveis acima de 60.",
+    [UI_MENU_CLIMB_FIX] = "Correção da velocidade de escalada",
+    [UI_MENU_CLIMB_FIX_DESC] = "Mantém a escalada de paredes e escadas na velocidade normal acima de 60 quadros por segundo, para que Link chegue ao topo.",
     [UI_MENU_GAME_LANG] = "Idioma do jogo", [UI_MENU_GAME_LANG_DESC] = "Idioma dos textos e das vozes do jogo.",
     [UI_MENU_GAME_LANG_AUTO] = "Automático",
     [UI_MENU_OPERATION_MODE] = "Modo do console",
@@ -1088,6 +1120,10 @@ static const char *const text_pt_br[UI_TEXT_COUNT] = {
     [UI_MENU_MODE_DOCKED] = "Modo TV", [UI_MENU_MODE_HANDHELD] = "Modo portátil",
 
     [UI_MENU_VOLUME] = "Volume", [UI_MENU_VOLUME_DESC] = "Volume geral do jogo.",
+    [UI_MENU_AUDIO_CHANNELS] = "Alto-falantes",
+    [UI_MENU_AUDIO_CHANNELS_DESC] = "Saída estéreo ou surround 5.1. Automático escolhe 5.1 quando a configuração de "
+                                    "alto-falantes do Windows tem seis canais ou mais. Vale após reiniciar.",
+    [UI_MENU_CHANNELS_AUTO] = "Automático", [UI_MENU_CHANNELS_STEREO] = "Estéreo", [UI_MENU_CHANNELS_51] = "Surround 5.1",
 
     [UI_MENU_MOTION] = "Controles de movimento", [UI_MENU_MOTION_DESC] = "Mirar com o giroscópio do controle.",
     [UI_MENU_RUMBLE_STRENGTH] = "Intensidade da vibração",
@@ -1219,6 +1255,9 @@ static const char *const text_pt_br[UI_TEXT_COUNT] = {
     [UI_MENU_PREEMPT_OFF] = "Desligado", [UI_MENU_PREEMPT_ON] = "Ligado", [UI_MENU_PREEMPT_INHERIT] = "Só herança",
     [UI_MENU_REPLAY_ASYNC] = "Preparo de quadro da GPU em thread própria",
     [UI_MENU_REPLAY_ASYNC_DESC] = "Prepara os quadros para a GPU numa thread separada em vez da thread do jogo. Adiciona até um quadro de latência.",
+    [UI_MENU_REPLAY_WORKERS] = "Threads de preparo de quadro da GPU",
+    [UI_MENU_REPLAY_WORKERS_DESC] = "Grava quadros grandes para a GPU em várias threads ao mesmo tempo. Ajuda quando a CPU limita a taxa de quadros; precisa de 8 ou mais processadores lógicos. Desligado grava numa thread só.",
+    [UI_MENU_REPLAY_WORKERS_NOW] = "agora: %s",
     [UI_MENU_TIMER_1MS] = "Timer do Windows de 1 ms",
     [UI_MENU_TIMER_1MS_DESC] = "Aumenta a resolução do timer do sistema para 1 ms, para um ritmo de quadros mais estável.",
     [UI_MENU_PIPE_WAIT] = "Tratamento de desenhos em espera",
@@ -1272,7 +1311,9 @@ static const char *const text_pt_br[UI_TEXT_COUNT] = {
     [UI_MENU_VK_DYN_VERTEX_DESC] = "Força a entrada dinâmica de vértices ligada ou desligada; o padrão depende da GPU.",
     [UI_MENU_VK_DYN_A2C] = "Alpha to coverage dinâmico",
     [UI_MENU_VK_DYN_A2C_DESC] = "Força o alpha to coverage dinâmico ligado ou desligado. Padrão: desligado no driver da AMD "
-                                "para RDNA 2 e anteriores, que com ele desenha os personagens pretos, ligado nos demais.",
+                                "para RDNA 2 e anteriores, que com ele desenha os personagens pretos, e no driver da "
+                                "Intel para Windows em GPUs integradas, que desenha ruído colorido na folhagem; "
+                                "ligado nos demais.",
     [UI_MENU_VK_SUBGROUP_PIN] = "Fixar tamanho do subgrupo",
     [UI_MENU_VK_SUBGROUP_PIN_DESC] = "Fixa o tamanho do subgrupo dos shaders.",
     [UI_MENU_TRACE] = "Registro de desempenho", [UI_MENU_TRACE_DESC] = "Grava os tempos de quadro num arquivo.",
@@ -1378,7 +1419,7 @@ static const char *const text_pt_br[UI_TEXT_COUNT] = {
     [UI_MENU_PROMPTS_MISSING] = "faltam os arquivos dos ícones: execute a instalação de novo",
     [UI_MENU_PROMPTS_FAILED] = "arquivos dos ícones danificados: execute a instalação de novo",
     [UI_MENU_PROMPTS_RESTART] = "os ícones mudam após reiniciar",
-    [UI_MENU_PROMPTS_OFF_SHAPE] = "desligados numa resolução de jogo que não é 16:9",
+    [UI_MENU_PROMPTS_OFF_SHAPE] = "desligados nesta resolução de jogo (sem layout feito para ela)",
     [UI_MENU_PROMPTS_OFF_MODS] = "desligados: um mod de RomFS substitui os arquivos deles",
     [UI_MENU_PROMPTS_OFF_PATCHES] = "desligados enquanto TOTK_ROMFS_PATCHES está definido",
     [UI_MENU_RESTART] = "Reiniciar agora",
@@ -1492,6 +1533,10 @@ static const char *const text_zh_hans[UI_TEXT_COUNT] = {
     [UI_MENU_TAB_MENU] = "菜单",
     [UI_MENU_TAB_ADVANCED] = "高级",
     [UI_MENU_TAB_MODS] = "模组",
+    [UI_MENU_PLAY_TIME] = "游戏时间",
+    [UI_MENU_PLAY_TIME_DESC] = "所有运行累计的游戏时间，仅在游戏窗口处于焦点且游戏正在绘制画面时计时。"
+                               "保存在配置文件夹的 playtime.txt 中，与存档放在一起。",
+    [UI_MENU_PLAY_TIME_HM] = "%llu 小时 %02u 分钟", [UI_MENU_PLAY_TIME_SESSION] = "本次运行：%s",
     [UI_MENU_MODS_BUILTIN] = "移植版内置", [UI_MENU_MODS_INSTALLED] = "已安装的模组",
     [UI_MENU_MODS_NONE] = "还没有模组。把模组解压到 mods 文件夹，每个模组一个文件夹，里面放 romfs 文件夹，然后重启游戏。"
                           "TKMM 也可以把合并后的模组导出到那里的一个文件夹。",
@@ -1560,6 +1605,8 @@ static const char *const text_zh_hans[UI_TEXT_COUNT] = {
                                   "游戏的 FSR 会将其缩放到该尺寸。"
                                   "高于 100% 会平滑锯齿边缘（超采样），但 GPU 开销很高。",
     [UI_MENU_RENDER_SCALE_GAME] = "游戏（83%）",
+    [UI_MENU_DYNAMIC_RES] = "动态分辨率",
+    [UI_MENU_DYNAMIC_RES_DESC] = "让游戏把 3D 场景降到 90 % 或 80 %。它依据为 Switch 制定的估算，不考虑你的显卡，因此即使在高性能 PC 上宽场景也会降低。关闭时保持渲染比例。",
     [UI_MENU_UPSCALER] = "放大到窗口",
     [UI_MENU_UPSCALER_DESC] = "窗口大于游戏分辨率时画面的放大方式。"
                               "FSR 比双线性更能保留边缘和细节。",
@@ -1617,6 +1664,8 @@ static const char *const text_zh_hans[UI_TEXT_COUNT] = {
     [UI_MENU_HIGHFPS_FIX] = "高帧率修正",
     [UI_MENU_HIGHFPS_FIX_DESC] = "修正帧率高于 30 时出现异常的物理效果，"
                                  "以及帧率高于 60 时一直半透明或不可见的物体。",
+    [UI_MENU_CLIMB_FIX] = "攀爬速度修正",
+    [UI_MENU_CLIMB_FIX_DESC] = "帧率高于 60 时让攀爬墙壁和梯子保持正常速度，林克能爬到顶。",
     [UI_MENU_GAME_LANG] = "游戏语言",
     [UI_MENU_GAME_LANG_DESC] = "游戏文本和语音的语言。",
     [UI_MENU_GAME_LANG_AUTO] = "自动",
@@ -1631,6 +1680,9 @@ static const char *const text_zh_hans[UI_TEXT_COUNT] = {
                                   "这样被打断的读取可能会永久卡住。",
     [UI_MENU_VOLUME] = "音量",
     [UI_MENU_VOLUME_DESC] = "游戏的主音量。",
+    [UI_MENU_AUDIO_CHANNELS] = "扬声器",
+    [UI_MENU_AUDIO_CHANNELS_DESC] = "立体声或 5.1 环绕声输出。选择自动时，若 Windows 扬声器设置有六个或更多声道则使用 5.1。重启后生效。",
+    [UI_MENU_CHANNELS_AUTO] = "自动", [UI_MENU_CHANNELS_STEREO] = "立体声", [UI_MENU_CHANNELS_51] = "5.1 环绕声",
     [UI_MENU_MOTION] = "体感操作",
     [UI_MENU_MOTION_DESC] = "用手柄的陀螺仪瞄准。",
     [UI_MENU_RUMBLE_STRENGTH] = "振动强度",
@@ -1819,6 +1871,10 @@ static const char *const text_zh_hans[UI_TEXT_COUNT] = {
     [UI_MENU_REPLAY_ASYNC] = "GPU 帧准备独立线程",
     [UI_MENU_REPLAY_ASYNC_DESC] = "在单独的线程而不是游戏线程上为 GPU 准备帧。"
                                   "最多增加一帧延迟。",
+    [UI_MENU_REPLAY_WORKERS] = "GPU 帧准备线程数",
+    [UI_MENU_REPLAY_WORKERS_DESC] = "同时在多个线程上为 GPU 录制大帧。CPU 限制帧率时有帮助；需要 8 个或更多逻辑处理器。"
+                                    "关：在单个线程上录制。",
+    [UI_MENU_REPLAY_WORKERS_NOW] = "当前：%s",
     [UI_MENU_TIMER_1MS] = "1 ms Windows 计时器",
     [UI_MENU_TIMER_1MS_DESC] = "将系统计时器精度提高到 1 ms，使帧间隔更平稳。",
     [UI_MENU_PIPE_WAIT] = "等待中绘制的处理",
@@ -1894,7 +1950,8 @@ static const char *const text_zh_hans[UI_TEXT_COUNT] = {
     [UI_MENU_VK_DYN_VERTEX_DESC] = "强制开关动态顶点输入；默认值取决于显卡。",
     [UI_MENU_VK_DYN_A2C] = "动态 Alpha 覆盖",
     [UI_MENU_VK_DYN_A2C_DESC] = "强制开关动态 Alpha 覆盖。默认：AMD RDNA 2 及更早的驱动上为关闭，"
-                                "开启会导致人物变黑；其他显卡上为开启。",
+                                "开启会导致人物变黑；Intel 集成显卡的 Windows 驱动上也为关闭，开启会使植被出现彩色噪点；"
+                                "其他显卡上为开启。",
     [UI_MENU_VK_SUBGROUP_PIN] = "固定子群大小",
     [UI_MENU_VK_SUBGROUP_PIN_DESC] = "固定着色器的子群大小。",
     [UI_MENU_TRACE] = "性能日志",
@@ -1991,7 +2048,7 @@ static const char *const text_zh_hans[UI_TEXT_COUNT] = {
     [UI_MENU_PROMPTS_MISSING] = "缺少按键提示文件：请重新运行安装程序",
     [UI_MENU_PROMPTS_FAILED] = "按键提示文件已损坏：请重新运行安装程序",
     [UI_MENU_PROMPTS_RESTART] = "按键提示重启后生效",
-    [UI_MENU_PROMPTS_OFF_SHAPE] = "游戏分辨率不是 16:9 时关闭",
+    [UI_MENU_PROMPTS_OFF_SHAPE] = "此游戏分辨率下关闭（未为其制作布局）",
     [UI_MENU_PROMPTS_OFF_MODS] = "已关闭：RomFS 模组替换了它们的文件",
     [UI_MENU_PROMPTS_OFF_PATCHES] = "设置了 TOTK_ROMFS_PATCHES 时关闭",
     [UI_MENU_RESTART] = "立即重启",

runtime/ui_text.h

@@ -35,6 +35,7 @@ typedef enum {
     UI_MENU_GRAPHICS,         // group heading over the resolution rows
     UI_MENU_OUTPUT_RES, UI_MENU_OUTPUT_RES_DESC,
     UI_MENU_RENDER_SCALE, UI_MENU_RENDER_SCALE_DESC, UI_MENU_RENDER_SCALE_GAME,
+    UI_MENU_DYNAMIC_RES, UI_MENU_DYNAMIC_RES_DESC,
     UI_MENU_UPSCALER, UI_MENU_UPSCALER_DESC, UI_MENU_UPSCALER_FSR, UI_MENU_UPSCALER_BILINEAR,
     UI_MENU_ASPECT, UI_MENU_ASPECT_DESC, UI_MENU_ASPECT_FIT, UI_MENU_ASPECT_STRETCH,
     UI_MENU_MOVIE_ASPECT, UI_MENU_MOVIE_ASPECT_DESC, UI_MENU_MOVIE_ASPECT_KEEP, UI_MENU_MOVIE_ASPECT_STRETCH,
@@ -67,9 +68,13 @@ typedef enum {
     UI_MENU_REALTIME, UI_MENU_REALTIME_DESC,
     UI_MENU_CLOTH_REALTIME, UI_MENU_CLOTH_REALTIME_DESC,
     UI_MENU_HIGHFPS_FIX, UI_MENU_HIGHFPS_FIX_DESC,
+    UI_MENU_CLIMB_FIX, UI_MENU_CLIMB_FIX_DESC,
     UI_MENU_GAME_LANG, UI_MENU_GAME_LANG_DESC, UI_MENU_GAME_LANG_AUTO,
     UI_MENU_OPERATION_MODE, UI_MENU_OPERATION_MODE_DESC, UI_MENU_MODE_DOCKED, UI_MENU_MODE_HANDHELD,
     UI_MENU_ASSUME_FOCUS, UI_MENU_ASSUME_FOCUS_DESC,
+    UI_MENU_PLAY_TIME, UI_MENU_PLAY_TIME_DESC,
+    UI_MENU_PLAY_TIME_HM,        // hours and minutes: "%llu h %02u min" (unsigned long long hours, unsigned minutes)
+    UI_MENU_PLAY_TIME_SESSION,   // note under the total: "this run: %s" (the hours and minutes above)
 
     // Mods: the port's own patches and the player's mods folder (mods.c)
     UI_MENU_MODS_BUILTIN, UI_MENU_MODS_INSTALLED,
@@ -172,6 +177,7 @@ typedef enum {
     UI_MENU_ASYNC_PREEMPT, UI_MENU_ASYNC_PREEMPT_DESC, UI_MENU_PREEMPT_OFF, UI_MENU_PREEMPT_ON,
     UI_MENU_PREEMPT_INHERIT,
     UI_MENU_REPLAY_ASYNC, UI_MENU_REPLAY_ASYNC_DESC,
+    UI_MENU_REPLAY_WORKERS, UI_MENU_REPLAY_WORKERS_DESC, UI_MENU_REPLAY_WORKERS_NOW,
     UI_MENU_TIMER_1MS, UI_MENU_TIMER_1MS_DESC,
     UI_MENU_PIPE_WAIT, UI_MENU_PIPE_WAIT_DESC, UI_MENU_PIPE_WAIT_DEFAULT, UI_MENU_PIPE_WAIT_DRAW,
     UI_MENU_PIPE_SKIP, UI_MENU_PIPE_SKIP_DESC, UI_MENU_PIPE_SKIP_STRICT, UI_MENU_PIPE_SKIP_ANY,

runtime/vk.h

@@ -44,6 +44,7 @@ typedef struct VkCtx {
     // VK_FEEDBACK_RESTART (the pass ends and restarts behind the full barrier).
     int feedback;                           // VK_FEEDBACK_*
     uint32_t min_imported_host_alignment;   // VK_EXT_external_memory_host
+    VkDeviceSize vma_block;                 // VMA's block size for heaps over 1 GiB (TOTK_VMA_BLOCK_MIB, vk_device.c)
     int device_index;                       // index in vkEnumeratePhysicalDevices
 } VkCtx;
 enum { VK_FEEDBACK_OFF, VK_FEEDBACK_RESTART, VK_FEEDBACK_LAYOUT };
@@ -58,9 +59,10 @@ typedef struct {
     int amd;                           // an AMD GPU (vendor 0x1002), any driver
     int amd_sc;                        // AMD's Windows driver branch for RDNA 1 and 2 ("AMD proprietary shader compiler")
     int mesa;                          // a Mesa driver the game runs on: RADV, ANV or NVK
+    int intel_igpu_win;                // Intel's Windows driver on an integrated GPU (UHD, Iris Xe, Core Ultra graphics)
 } VkCaps;
 extern VkCaps vk_caps;
-void vk_caps_driver(VkPhysicalDevice p);             // first, before vkCreateDevice: vk_caps.amd, amd_sc, mesa
+void vk_caps_driver(VkPhysicalDevice p);             // first, before vkCreateDevice: vk_caps.amd, amd_sc, mesa, intel_igpu_win
 void vk_caps_features(VkPhysicalDeviceFeatures *f);   // before vkCreateDevice (TOTK_VK_NO_FLOAT64 clears shaderFloat64)
 // Before vkCreateDevice, with the driver's graphics pipeline library feature and properties: sets vk_caps.gpl,
 // 1 = enable the extensions (VK_KHR_pipeline_library, VK_EXT_graphics_pipeline_library) and the feature.
@@ -226,6 +228,10 @@ enum { VK_MCLS_NONE, VK_MCLS_HOST_NOACC, VK_MCLS_HOST_UNC, VK_MCLS_HOST_CACHED,
 enum { VK_MUSE_VB, VK_MUSE_IB, VK_MUSE_UBO, VK_MUSE_SSBO, VK_MUSE_IND, VK_MUSE_COPY_SRC, VK_MUSE_GPU_WRITE,
        VK_MUSE_CPU_READ, VK_MUSE_N };
 void vk_mem_count(int cls, int use, uint64_t bytes);
+// A thread recording a chunk of a submission (vk_exec.c) counts into its own acc from vk_mem_count_local(acc) on, until
+// vk_mem_count_local(NULL); vk_mem_count_flush adds acc into the census and clears it.
+void vk_mem_count_local(int64_t (*acc)[VK_MUSE_N]);
+void vk_mem_count_flush(int64_t (*acc)[VK_MUSE_N]);
 // TOTK_TRACE=perf dirty census (vk_memory.c): the replay notes each imported pool it binds as a vertex, index, uniform,
 // storage or indirect buffer; at the start of every submit the pages the CPU wrote in such pools since the previous
 // submit are counted from the shared page generations and logged as `mem: dirty` every 60 frames. Nothing when tracing
@@ -258,10 +264,14 @@ void vk_mem_gpu_wrote_shader(const VkMemSpan *sp, uint64_t size);
 // it records, ubo bindings take the copy (0: not copied, bind memory), GPU writes of the replay drop the copies they touch.
 void vk_usnap_begin(const uint64_t *handles, int n, uint64_t fence);
 void vk_usnap_end(uint64_t fence);
-int vk_usnap_find(uint64_t addr, uint64_t n, uint64_t align, VkDescriptorBufferInfo *out);
+// at: the copies dropped at or before it are not found (vk_usnap_drop_at); UINT32_MAX outside a chunked replay.
+int vk_usnap_find(uint64_t addr, uint64_t n, uint64_t at, uint64_t align, VkDescriptorBufferInfo *out);
 const uint8_t *vk_usnap_map(void);   // a host copy of the current arena, at the offsets vk_usnap_find gives
 const uint8_t *vk_usnap_bytes(uint64_t addr, uint64_t n);   // the copy of [addr, addr+n) as of the submit, NULL: none
 void vk_usnap_drop(uint64_t addr, uint64_t n);   // the replay makes the GPU write [addr, addr+n) of guest memory
+// The state scan of a chunked replay (vk_exec.c): the op that writes [addr, addr+n) is the at-th that may; the chunks
+// pass at with the ops they record (VkExec.drop_n) and find the copies only before it.
+void vk_usnap_drop_at(uint64_t addr, uint64_t n, uint32_t at);
 uint64_t vk_usnap_gen(void);                      // changes whenever a binding may resolve differently
 void vk_usnap_shutdown(void);                     // vk_exec_draw_shutdown: frees the arenas
 // vk_mvprev.c: last frame's copy of the scene's object (c7) and bone (c5) blocks, for per-object motion vectors.
@@ -280,7 +290,13 @@ void vk_mem_mirror_prologue(VkCommandBuffer cb);
 // Shadow storage (vk_memory.c): vk_sync.c calls vk_mem_shadow_sync right before each submit; the replay notes every GPU
 // write into CPU-visible memory at record time (vk_mem_gpu_pending) and reports it at the fence with vk_gpu_written,
 // which copies the bytes back into guest memory before telling the texture cache (vk_guest_written).
-void vk_mem_shadow_sync(void);
+void vk_mem_shadow_sync(uint64_t fence);   // fence: the value of the part about to be submitted (0 untracked)
+// TOTK_SHADOW_DEFER: shadow updates that may not be written by the CPU while the GPU still reads the shadow are queued
+// by vk_mem_shadow_sync as ranges of guest memory. vk_mem_shadow_deferred says whether there are any;
+// vk_mem_shadow_prologue records the copies (staging -> shadow buffers) into the prologue command buffer, before
+// vk_mem_mirror_prologue's copies, which read the shadow.
+int vk_mem_shadow_deferred(void);
+void vk_mem_shadow_prologue(VkCommandBuffer cb, uint64_t fence);
 void vk_mem_gpu_pending(uint64_t addr, uint64_t size);
 void vk_gpu_written(uint64_t addr, uint64_t size);
 int vk_mem_flush_add_probe(void);        // --probe-vkexec: the flush range list; 0 or the number of the failed check
@@ -292,13 +308,24 @@ int vk_mem_class_of(uint64_t addr);               // VK_MCLS_* of the pool at ad
 void vk_mem_census(uint64_t frame);               // at each present: pool census and traffic, logged at frame 1800 and,
                                                   // with TOTK_TRACE=perf, every 1800 frames
 // VRAM in use and its budget, summed over the DEVICE_LOCAL heaps. exact 1: VK_EXT_memory_budget (the whole process),
-// 0: this allocator's own allocations against the heap sizes. 0 when unknown.
+// 0: this allocator's own allocations against the heap sizes. 0 when unknown. Where device memory takes host memory
+// (vk_mem_host_room) the budget is at most the use plus that room.
 int vk_mem_vram(uint64_t *used, uint64_t *budget, int *exact);
+// Bytes the system can still give device memory beyond a reserve (TOTK_MEM_RESERVE_MIB) where device memory takes host
+// memory: a UMA GPU (commit and physical memory), or a discrete GPU whose device-local memory took process commit in
+// vk_mem_host_probe (commit). UINT64_MAX elsewhere.
+uint64_t vk_mem_host_room(void);
+void vk_mem_host_probe(void);                 // vk_device.c, after the device: measures that for a discrete GPU, once
+int vk_mem_uma(void);                         // 1: the device class is uma (vk_mem_class_log), its memory is RAM
 // VRAM headroom (vk_headroom.c): the VMA callbacks that see new device memory, and the thread that allocates the next
 // block of each memory type in use ahead of the render thread.
 const VmaDeviceMemoryCallbacks *vk_headroom_callbacks(void);
 void vk_headroom_start(void);
 void vk_headroom_stop(void);   // before vmaDestroyAllocator
+// The texture pool (vk_memory.c vk_texture_image) gets its next block ahead too; pool NULL frees the anchor in it.
+void vk_headroom_pool(VmaPool pool, uint32_t type, VkDeviceSize block);
+void vk_headroom_release(void);   // after a failed allocation: frees the spare blocks and keeps none for a while;
+                                 // any thread that holds no VMA lock (takes only the headroom lock)
 int vk_mem_resolve(uint64_t addr, uint64_t size, VkMemSpan *out);   // 1 when all of [addr, addr+size) is backed
 // Changes whenever vk_mem_resolve plus VK_MEM_USE may give another span for an address: a pool added, finalized or
 // mapped, a mirror created, refused or forgotten, a submission started.
@@ -329,6 +356,17 @@ void vk_defer_image(VkImage img, VmaAllocation alloc);   // alloc NULL: vkDestro
 void vk_defer_view(VkImageView view);
 void vk_exec_image_gone(VkImage image);   // destroy_tex, texture lock held: the replay's views of image go with it
 void vk_defer_buffer(VkBuffer buf, VmaAllocation alloc);
+// free(p) once the GPU is past the submission being recorded, like the deferrals above: host records that a recording
+// may still read through a pointer taken before another thread dropped them (vk_texture.c destroy_tex).
+void vk_defer_free(void *p);
+// Device-local scratch for the upload batch (storage, transfer src and dst), from a pool of its own (vk_memory.c).
+VkResult vk_scratch_buffer(VkDeviceSize size, VkBuffer *buf, VmaAllocation *alloc);
+// A game texture's image with memory of its own: the texture pool, or a dedicated allocation (vk_memory.c).
+VkResult vk_texture_image(const VkImageCreateInfo *ci, VkImage *img, VmaAllocation *alloc);
+// Staging and scratch bytes the open upload batch holds until it completes. Large texture loads submit the batch early
+// (vk_texture.c UPLOAD_HELD_MAX) so the GPU starts on it and its buffers come back before the pools need new blocks.
+VkDeviceSize vk_upload_held(void);
+void vk_upload_pools_init(void);   // vk_device.c, after the allocator: the staging and scratch pools and their blocks
 // ---- ASTC (vk_astc.c). TOTK_ASTC = auto | gpu | native | cpu | check; see vk_astc.c.
 enum { VK_ASTC_CPU, VK_ASTC_GPU, VK_ASTC_NATIVE, VK_ASTC_CHECK };
 int vk_astc_mode(void);                 // decided once per device (logged); VK_ASTC_CPU before vk_init
@@ -352,8 +390,36 @@ void vk_astc_time_enable(int on);
 int vk_astc_time_get(uint32_t bw, uint32_t bh, uint64_t *texels, uint64_t *calls, double *us);   // totals so far
 void vk_astc_time_summary(void);        // logs the totals
 void vk_astc_release(void);             // vk_device.c, before the device goes: frees the decoder's objects
-void vk_mem_note_submit(uint64_t timeline_value);  // vk_sync.c: after each submit that signals ctx->timeline with this value
+// ---- Block-linear deswizzle on the GPU (vk_deswizzle.c). TOTK_DESWIZZLE = auto | gpu | cpu | check.
+enum { VK_DESWZ_CPU, VK_DESWZ_GPU, VK_DESWZ_CHECK };
+int vk_deswz_mode(void);                // decided once per device (logged); VK_DESWZ_CPU before vk_init
+// One level and layer: its storage starts at byte src of the bound source range and covers segs 16-byte segments
+// (whole blocks, cut at the storage end); its linear copy goes to byte dst of the destination, rows of pitch16 segments,
+// rows x slices of them. th, td, gobs_x, blocks_y: as tex_deswizzle.c nvn_bl_offset.
+typedef struct {
+    VkDeviceSize src, dst;
+    uint32_t segs, pitch16, rows, slices, th, td, gobs_x, blocks_y;
+} VkDeswzRegion;
+// Records into cb a dispatch per region reading [src_off, src_off + src_size) of src (a storage buffer) and writing
+// dst (device-local, dst_size bytes). The caller records the barriers around it. 0 when nothing was recorded: no
+// pipeline, src_off not 16-byte aligned, a range over the device's storage-buffer limit, a region outside the ranges.
+int vk_deswz_record(VkCommandBuffer cb, VkBuffer src, VkDeviceSize src_off, VkDeviceSize src_size, VkBuffer dst,
+                    VkDeviceSize dst_size, const VkDeswzRegion *rg, int n);
+// 1 when vk_deswz_record will take these regions (their dst offsets aside) with destinations of up to max_dst bytes.
+int vk_deswz_fits(VkDeviceSize src_off, VkDeviceSize src_size, const VkDeswzRegion *rg, int n, VkDeviceSize max_dst);
+void vk_deswz_release(void);            // vk_device.c, before the device goes
+// 1 when sp (from vk_mem_resolve) is an imported pool's buffer over guest memory itself, so a GPU read in a submission
+// sees every CPU write made before the submit; 0 for device memory and shadow storage (synced after the upload batch).
+int vk_mem_span_live(const VkMemSpan *sp);
+// vk_sync.c, when it starts recording the submission that will signal ctx->timeline with this value: objects destroyed
+// from then on are freed once the timeline reaches it.
+void vk_mem_note_fence(uint64_t timeline_value);
+void vk_mem_fault_log(void);   // vk_device_fault_report: the deferred-free state, lock-free
 void vk_mem_collect(void);                         // frees what has completed
+// After a failed allocation, on a thread that may hold the texture lock and the upload lock (cmd_lock) but no VMA
+// lock: frees the headroom blocks and the completed upload batches and deferred objects, and requests the release
+// of released textures at the next submit (rt_oom_request).
+void vk_oom_release(void);
 typedef struct { uint64_t imported_pools, physical_pools, virtual_pools, import_failures, upload_batches; } VkMemStats;
 void vk_mem_stats(VkMemStats *s);
 void vk_mem_shutdown(void);                        // before vk_shutdown: waits idle, frees every memory and texture object
@@ -384,12 +450,16 @@ typedef struct {
     uint8_t scale;                 // 1, or TOTK_SHADOW_SCALE for a shadow map made larger than the game's (vk_texture.c
                                    // shadow_scale_of): width and height above are the larger ones, the game's regions
                                    // are multiplied by it
+    uint8_t gone;                  // the entry was dropped (vk_texture.c destroy_tex, under the texture lock): image and
+                                   // record live until the submission being recorded is done, and nothing may be cached
+                                   // by its image handle, which the driver can hand out again after that
 } VkTexImage;
 // The window texture the backend presents (vk_backend.c): *image and *format become the float picture the game copied
 // into it, when the last draw into it was that copy and the swapchain is HDR10.
 void vk_hdr_present_source(const VkTexImage *win, VkImage *image, VkFormat *format);
 void vk_hdr_image_gone(VkImage image);
 const VkTexImage *vk_texture_get(const tex_t *t);   // created and made current; NULL for BUFFER or unmapped formats
+const VkTexImage *vk_texture_revive(const tex_t *t, uint32_t pos);  // the entry vk_texture_get would return if it exists, made live at op pos
 // A texture slot; a slot whose texture was finalized follows the live texture at the same storage (hle_nvn.c).
 const VkTexImage *vk_texture_by_id(int id, view_t *view, int *has_view);
 // The replay looks up many slots per draw: it takes the texture lock once around them (vk_texture_lock) and uses the
@@ -397,6 +467,12 @@ const VkTexImage *vk_texture_by_id(int id, view_t *view, int *has_view);
 void vk_texture_lock(void);
 void vk_texture_unlock(void);
 const VkTexImage *vk_texture_by_id_locked(int id, view_t *view, int *has_view);
+// 1 when slot id's entry had its freshness check in this submission, so vk_texture_by_id_locked would only look it up.
+// No lock: a context of a chunked replay asks before taking the texture lock.
+int vk_texture_slot_checked(int id);
+// 1 when the state scan revived a texture at the slot's storage in this submission: the slot's answer depends on the op
+// a chunk records, so no memo holds it (vk_exec_draw.c); a chunk sees every revival up to its last op.
+int vk_texture_slot_moves(int id);
 VkSampler vk_sampler_by_id_locked(int id);
 // The slot's sampler with nearest filtering and no anisotropy: a depth texture read without a compare lands on one texel
 // on the Switch although the game binds a linear sampler, and MacNativeGames samples it the same way. VK_NULL_HANDLE
@@ -428,6 +504,7 @@ void vk_texture_stats(uint64_t *images, uint64_t *uploads, uint64_t *reuploads);
 // (images not bound into pool memory).
 typedef struct { uint64_t live, released, own_live, own_released, bytes_live, bytes_released; } VkTexCensus;
 void vk_texture_census(VkTexCensus *s);
+void vk_texture_released_log(uint64_t frame);       // mem: released lines of vram_log
 void vk_texture_shutdown(void);                     // vk_mem_shutdown calls it
 void vk_texture_new_submit(void);                   // a queue submit starts: draw bindings check storage again
 uint32_t vk_texture_epoch(void);                    // the write-watch snapshot epoch vk_texture_new_submit started
@@ -458,9 +535,22 @@ int vk_exec_record(VkCommandBuffer cb, const uint64_t *handles, int n, uint64_t
 // batch) without a signal, and returns a new begun one for the rest of the submission. The submission's timeline value
 // is signalled by its last part, whose signal covers every earlier part in submission order.
 VkCommandBuffer vk_sync_split(void);
+// Inside vk_exec_record: submits cb, a part recorded and ended on another thread, after the upload batch and the
+// mirror copies queued so far, without a signal (the submitting thread, in the order of the submission).
+void vk_sync_part(VkCommandBuffer cb);
+// GPU time of a part recorded on another thread (vk_sync.c "GPU time"): the start timestamp after vkBeginCommandBuffer,
+// the pair or -1; the end before vkEndCommandBuffer.
+int vk_sync_ts_open(VkCommandBuffer cb);
+void vk_sync_ts_close(VkCommandBuffer cb, int pair, uint64_t fence);
 // Runs fn(arg) on the completion thread once fence is done on the GPU (event writes, counter reports, sync state).
 // Callbacks run in the order they were added, after those of earlier values; a value already done runs at once there.
 void vk_completion_add(uint64_t fence, void (*fn)(void *), void *arg);
+// Callbacks of a part recorded on another thread: between vk_completion_defer(d) and vk_completion_defer(NULL) the
+// calling thread's vk_completion_add calls collect in d, and vk_completion_flush adds them, in their order, when the
+// part takes its place in the submission (vk_exec.c record_chunks).
+typedef struct { void *items; int n, cap; } VkDeferred;
+void vk_completion_defer(VkDeferred *d);
+void vk_completion_flush(VkDeferred *d);
 // gfx_vulkan_backend (vk_backend.c): presents go to a present thread and run after the submissions made before them.
 void rdoc_init(void);                     // runtime/rdoc.c: TOTK_RDOC debug capture; before vkCreateInstance
 int rdoc_active(void);
@@ -504,6 +594,7 @@ void vk_pipe_state_defaults(VkPipeState *s);   // NVN defaults of every state ob
 enum { VK_PIPE_SUPPORT = GFX_RES_KINDS };      // VkPipeBinding.kind of the support buffer
 #define VK_PIPE_ZERO_BINDING 16                // vertex binding of shader inputs without an attribute: bind a zeroed
                                                // buffer of at least 16 bytes there (stride 0)
+#define VK_PIPE_ALL_BINDINGS 0x1ffffu          // every vertex binding, streams 0-15 and the zero binding
 typedef struct {
     uint16_t binding;                      // in its set
     uint8_t stage;                         // GFX_STAGE_* that declared it (support buffer: the first stage)
@@ -556,8 +647,15 @@ VkPipeline vk_pipeline_compute(const GfxShader *cs, const VkPipeInfo **info);
 // has them dynamic, depth clamp, polygon mode, blend enables, equations, write masks and alpha to coverage. The replay
 // sets the rest: vkCmdSetViewportWithCount, vkCmdSetScissorWithCount, depth bias, blend constants, stencil compare and
 // write masks and references; vertex buffers with vkCmdBindVertexBuffers2 (strides NULL when
-// vk_pipeline_vertex_input_dynamic(), else from VkPipeState.vstream).
-void vk_pipeline_set_dynamic(VkCommandBuffer cb, const VkPipeState *s);
+// vk_pipeline_vertex_input_dynamic(), else from VkPipeState.vstream). Returns the vertex bindings (bit per binding) the
+// state's vertex input reads, whether set here or made into the pipeline.
+uint32_t vk_pipeline_set_dynamic(VkCommandBuffer cb, const VkPipeState *s);
+// The replay's variant (one thread, TOTK_DYN_DELTA): valid 1 says every graphics pipeline bound in cb since the last call
+// was one of vk_pipeline_graphics*'s, which all declare the same dynamic states, and nothing else set them, so cb still
+// holds what that call recorded (Vulkan keeps dynamic state across such binds for the life of the command buffer);
+// then only the commands whose values differ are recorded. valid 0 (a new command buffer, or one whose state another
+// recorder changed) records everything. *cmds: the commands recorded (valid 0: 1). Returns as vk_pipeline_set_dynamic.
+uint32_t vk_pipeline_set_dynamic_delta(VkCommandBuffer cb, const VkPipeState *s, int valid, uint32_t *cmds);
 int vk_pipeline_vertex_input_dynamic(void);    // 1: vertex input is VK_EXT_vertex_input_dynamic_state
 // Translator answers from the bound state: attr_type from the vertex attribute formats (vs), frag_out_type from the
 // colour target formats (fs). Other fields are left as they are; either pointer may be NULL.

runtime/vk_astc.c

@@ -300,17 +300,19 @@ static int create_pipeline(VkCtx *c) {
 // TOTK_ASTC_BC=1: the BC3 encoder, when the device samples BC3 (every desktop GPU does); 0 leaves ASTC as RGBA8.
 // Unset: on when the largest device-local heap is under 5 GiB. There RGBA8 ASTC, four times the size of BC3, fills
 // video memory (a GTX 1650 4 GiB went from 10-23 to 3-12 ms of GPU time a frame, issue #87); BC3 loses some precision
-// (normal maps), so GPUs with more memory keep RGBA8.
+// (normal maps), so GPUs with more memory keep RGBA8. A UMA GPU's heap is the machine's RAM however large it reads
+// (10.6 GiB on an ROG Ally X, 14.9 GiB on a Radeon 880M), so there BC3 is on too: on the RX 9070, BC3 took 737 MiB off
+// VMA's device-local allocations after a minute in the field, at the same GPU time.
 static int create_bc_pipeline(VkCtx *c) {
     const char *e = cfg_get("TOTK_ASTC_BC");
     VkDeviceSize vram = 0;
     for (uint32_t h = 0; h < c->mem_props.memoryHeapCount && h < VK_MAX_MEMORY_HEAPS; h++)
         if (c->mem_props.memoryHeaps[h].flags & VK_MEMORY_HEAP_DEVICE_LOCAL_BIT && c->mem_props.memoryHeaps[h].size > vram)
             vram = c->mem_props.memoryHeaps[h].size;
-    int on = e && *e ? *e != '0' : vram < (5ull << 30);
+    int on = e && *e ? *e != '0' : vram < (5ull << 30) || vk_mem_uma();
     if (!(e && *e))
-        rt_log("vk: TOTK_ASTC_BC unset: BC3 %s (largest device-local heap %llu MiB)", on ? "on" : "off",
-               (unsigned long long)(vram >> 20));
+        rt_log("vk: TOTK_ASTC_BC unset: BC3 %s (largest device-local heap %llu MiB%s)", on ? "on" : "off",
+               (unsigned long long)(vram >> 20), vk_mem_uma() ? ", uma: system memory" : "");
     if (!on) return 0;
     VkFormatProperties p;
     vkGetPhysicalDeviceFormatProperties(c->phys, VK_FORMAT_BC3_SRGB_BLOCK, &p);
@@ -463,13 +465,9 @@ int vk_astc_record(VkCommandBuffer cb, VkBuffer src, VkDeviceSize src_size, cons
     cap_in = (cap_in + al - 1) / al * al;
     cap_out = (cap_out + al - 1) / al * al;
     if (!bc) cap_bc = 0;
-    VkBufferCreateInfo bi = {.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO, .size = cap_in + cap_out + cap_bc,
-                             .usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT |
-                                      VK_BUFFER_USAGE_TRANSFER_DST_BIT};
-    VmaAllocationCreateInfo aci = {.usage = VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE};
     VkBuffer sb;
     VmaAllocation sa;
-    if (vmaCreateBuffer(c->vma, &bi, &aci, &sb, &sa, NULL) != VK_SUCCESS) {
+    if (vk_scratch_buffer(cap_in + cap_out + cap_bc, &sb, &sa) != VK_SUCCESS) {
         if (plat_inc32(&n_fail) <= 16)
             rt_log("vk: ASTC scratch buffer of %llu bytes failed: texture left undecoded",
                    (unsigned long long)(cap_in + cap_out));

runtime/vk_backend.c

@@ -49,6 +49,7 @@ static void wait_until(uint64_t at) {
 
 static void present_main(void *arg) {
     (void)arg;
+    perf_core_thread(PCORE_PRESENT);
     uint64_t last = 0, last_queued = 0;
     plat_mutex_lock(&pl);
     for (;;) {
@@ -159,6 +160,7 @@ static PlatCond rcv = PLAT_COND_INIT;   // item queued, item taken, item done
 static ReplayItem rq[RQ_CAP];
 static int rq_head, rq_n, r_busy, r_started;
 static uint64_t r_presents_queued, r_presents_done, r_submit_done, game_frames;
+static uint64_t r_value_done;   // under rl: the newest timeline value the replay thread submitted
 // Under rl: calls, calls that waited and the time waited per cause, since the last replay_log line and over the run.
 static uint64_t dr_calls[DR_N], dr_waits[DR_N], dr_ns[DR_N], dr_tcalls[DR_N], dr_twaits[DR_N], dr_tns[DR_N];
 // TOTK_REPLAY_CHECK: ReadTexels compared with a read after a full drain (under rl): checked, differing, differing
@@ -191,6 +193,15 @@ uint64_t gfx_replayed_submits(void) {
     return v;
 }
 
+// TOTK_SYNC_AHEAD (nvn_api_core.c nvnSyncWait): waits until the replay thread submitted value v, so every shadow copy
+// of v was made before the game writes memory v reads. The caller has given its core up.
+void gfx_replay_wait_value(uint64_t v) {
+    if (!gfx_replay_async()) return;
+    plat_mutex_lock(&rl);
+    while (r_value_done < v) plat_cond_wait(&rcv, &rl);
+    plat_mutex_unlock(&rl);
+}
+
 static uint64_t submit_sync(const uint64_t *handles, int n) {
     vk_sync_start();
     vk_present_drain();
@@ -201,6 +212,7 @@ static uint64_t submit_sync(const uint64_t *handles, int n) {
 static void replay_main(void *arg) {
     (void)arg;
     perf_replay_thread();
+    perf_core_thread(PCORE_REPLAY);
     plat_mutex_lock(&rl);
     for (;;) {
         while (!rq_n) plat_cond_wait(&rcv, &rl);
@@ -225,6 +237,7 @@ static void replay_main(void *arg) {
             present_image(it.tex, it.has_crop ? it.crop : NULL, it.at, it.queued, frame, it.value);
         }
         plat_mutex_lock(&rl);
+        if (it.kind != RI_RELEASE) r_value_done = it.value;
         if (it.kind == RI_SUBMIT) r_submit_done = it.submit_no;
         else r_presents_done++;
         r_busy = 0;
@@ -326,6 +339,8 @@ void gfx_exit_log(void) {
 }
 
 static void vk_be_present(const tex_t *t, const int crop[4], uint64_t at_ns) {
+    static _Thread_local int pcore;   // the game's render thread: the one that presents
+    if (!pcore) pcore = 1, perf_core_thread(PCORE_RENDER);
     vk_sync_start();
     if (!gfx_replay_async()) {
         uint64_t frame = __atomic_add_fetch(&gfx_counters.frames, 1, __ATOMIC_RELAXED);

runtime/vk_caps.c

@@ -45,6 +45,8 @@ void vk_caps_driver(VkPhysicalDevice p) {
     vk_caps.amd_sc = drv.driverID == VK_DRIVER_ID_AMD_PROPRIETARY && strstr(drv.driverInfo, "shader compiler");
     vk_caps.mesa = drv.driverID == VK_DRIVER_ID_MESA_RADV || drv.driverID == VK_DRIVER_ID_INTEL_OPEN_SOURCE_MESA ||
                    drv.driverID == VK_DRIVER_ID_MESA_NVK;
+    vk_caps.intel_igpu_win = drv.driverID == VK_DRIVER_ID_INTEL_PROPRIETARY_WINDOWS &&
+                             p2.properties.deviceType == VK_PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU;
 }
 
 // TOTK_GPL=0 turns graphics pipeline libraries off; they are on wherever the driver has fast linking. AMD's RDNA 1/2

runtime/vk_contact.c

@@ -7,9 +7,13 @@
 // passes behind nearby geometry: the small shadows at feet, under stones and in grass that the shadow maps are too
 // coarse for. Transparent surfaces, the sky and post-processing come after, untouched.
 //
-// Inputs from the game, read from the fragment stage's uniform block c4 bound in the composite pass (guest memory,
-// so it may be one frame newer than the replay): the projection scales c4[7].x and c4[8].y, the far plane c4[14].y,
-// and the first sun cascade's depth row c4[44].xyz, whose negated direction points from a view-space position toward
+// vk_contact_plan decides that in replay order and vk_contact_exec records it: a chunked replay's scan
+// plans the march (vk_exec.c scan_contact) and the chunk that holds its op records it.
+//
+// Inputs from the game, read where the march is planned from the fragment stage's uniform block c4 bound in the
+// composite pass (guest memory, so it may be one frame newer than the replay): the projection scales c4[7].x and
+// c4[8].y, the far plane c4[14].y, and the first sun cascade's depth row c4[44].xyz, whose negated direction points from
+// a view-space position toward
 // the sun (the G-buffer shaders compare dot(c4[44].xyz, pos) + c4[44].w with the cascade map). The scene colour images
 // get STORAGE usage when the setting is on (vk_texture.c create_tex).
 #include "vk.h"
@@ -23,22 +27,22 @@
 #include "vk_exec.h"
 
 static struct {
-    int tried, ok, strength, debug, logged;
+    int read, tried, ok, strength, debug, logged;
     PFN_vkCmdPushDescriptorSetKHR push;
     VkDescriptorSetLayout dsl;
     VkPipelineLayout layout;
     VkPipeline pipe;
     VkSampler sampler;
-    const VkTexImage *scene, *depth;   // the last G-buffer's scene colour and linear depth
-    uint32_t w, h;
+    VkExecTarget scene, depth;         // the last G-buffer's scene colour and linear depth
     int pending;                       // the composite into scene ran; the march waits for the next pass
     uint64_t c4;
 } C;
 
 int vk_contact_strength(void) {
-    if (!C.tried) {
+    if (!C.read) {   // once: the scan asks at every draw
         int s = cfg_int("TOTK_CONTACT_SHADOWS");
         C.strength = s < 0 ? 0 : s > 100 ? 100 : s;
+        C.read = 1;
     }
     return C.strength;
 }
@@ -90,16 +94,16 @@ static int init(VkCtx *c) {
 }
 
 // A G-buffer pass: a B10G11R11 scene colour, an R32F linear depth and a D32F depth, all one size, at least 640 wide.
-static int is_gbuffer(const VkExec *ex, const VkTexImage **scene, const VkTexImage **depth) {
+static int is_gbuffer(const VkExec *ex, const VkExecTarget **scene, const VkExecTarget **depth) {
     *scene = *depth = NULL;
     if (ex->pass_noatt || ex->pass_w < 640 || ex->depth.format != VK_FORMAT_D32_SFLOAT || ex->depth.level) return 0;
     for (uint32_t i = 0; i < ex->pass_ncolor && i < 8; i++) {
         const VkExecTarget *t = &ex->color[i];
         if (!t->image || t->level || t->width != ex->pass_w || t->height != ex->pass_h) continue;
-        if (t->format == VK_FORMAT_B10G11R11_UFLOAT_PACK32 && !*scene) *scene = t->img;
-        if (t->format == VK_FORMAT_R32_SFLOAT && !*depth) *depth = t->img;
+        if (t->format == VK_FORMAT_B10G11R11_UFLOAT_PACK32 && !*scene) *scene = t;
+        if (t->format == VK_FORMAT_R32_SFLOAT && !*depth) *depth = t;
     }
-    return *scene && *depth && ((*scene)->usage & VK_IMAGE_USAGE_STORAGE_BIT);
+    return *scene && *depth && (*scene)->img && ((*scene)->img->usage & VK_IMAGE_USAGE_STORAGE_BIT);
 }
 
 static void mem_barrier(VkCommandBuffer cb) {
@@ -112,30 +116,66 @@ static void mem_barrier(VkCommandBuffer cb) {
     vkCmdPipelineBarrier2(cb, &d);
 }
 
-void vk_contact_flush(VkExec *ex) {
-    if (!C.pending) return;
+// The lighting composite: the remembered scene colour as the only target, with a D16 depth of its size.
+static int is_composite(const VkExec *ex) {
+    return C.scene.image && ex->pass_ncolor >= 1 && ex->color[0].image == C.scene.image && !ex->color[0].level &&
+           ex->depth.format == VK_FORMAT_D16_UNORM && ex->pass_w == C.scene.width && ex->pass_h == C.scene.height;
+}
+
+int vk_contact_plan(const VkExec *ex, int pass, VkExecJob *j) {
+    if (!vk_contact_strength()) return 0;
+    if (pass) {
+        const VkExecTarget *scene, *depth;
+        // The camera block: c4 of the fragment stage, 0x940 bytes. The composite binds it only after its pass begins, so
+        // the newest one seen at any pass start serves.
+        const p_bind_buf *b = &ex->ubo[1][1];
+        if (b->addr && b->size == 0x940) C.c4 = b->addr;
+        if (is_gbuffer(ex, &scene, &depth)) {
+            C.pending = 0;
+            C.scene = *scene, C.depth = *depth;
+            return 0;
+        }
+        if (is_composite(ex)) {
+            C.pending = C.c4 != 0;
+            return 0;
+        }
+    }
+    if (!C.pending) return 0;
     C.pending = 0;
-    if (ex->in_pass) return;
     const float *u = (const float *)(uintptr_t)C.c4;
     float lx = -u[44 * 4 + 0], ly = -u[44 * 4 + 1], lz = -u[44 * 4 + 2], ll = sqrtf(lx * lx + ly * ly + lz * lz);
     float px = u[7 * 4 + 0], py = u[8 * 4 + 1], zfar = u[14 * 4 + 1];
-    if (!(ll > 1e-6f) || !(px > 0) || !(py > 0) || !(zfar > 1)) return;
-    VkImageView in = vk_exec_target_view(C.depth, VK_FORMAT_R32_SFLOAT, VK_IMAGE_VIEW_TYPE_2D, VK_IMAGE_ASPECT_COLOR_BIT,
+    if (!(ll > 1e-6f) || !(px > 0) || !(py > 0) || !(zfar > 1)) return 0;
+    float w = (float)C.scene.width, h = (float)C.scene.height;
+    *j = (VkExecJob){.scene = C.scene, .depth = C.depth,
+                     .pc = {px, py, zfar, (float)C.strength / 100.0f,
+                            lx / ll, ly / ll, lz / ll, 0.6f,
+                            w, h, 1.0f / w, 1.0f / h,
+                            0, 0, 0, 0}};
+    return 1;
+}
+
+void vk_contact_exec(VkExec *ex, const VkExecJob *j) {
+    if (!C.tried) init(vk_ctx());
+    if (!C.ok) return;
+    VkImageView in = vk_exec_target_view(j->depth.img, VK_FORMAT_R32_SFLOAT, VK_IMAGE_VIEW_TYPE_2D, VK_IMAGE_ASPECT_COLOR_BIT,
                                          0, 0, 1, 0);
-    VkImageView out = vk_exec_target_view(C.scene, VK_FORMAT_B10G11R11_UFLOAT_PACK32, VK_IMAGE_VIEW_TYPE_2D,
+    VkImageView out = vk_exec_target_view(j->scene.img, VK_FORMAT_B10G11R11_UFLOAT_PACK32, VK_IMAGE_VIEW_TYPE_2D,
                                           VK_IMAGE_ASPECT_COLOR_BIT, 0, 0, 1, 0);
     if (!in || !out) return;
+    // A full barrier that also takes the images in the feedback layout back to GENERAL (vk_exec.c barrier_to): the
+    // G-buffer's linear depth and scene colour are such images once a draw sampled them in their own pass, and the march
+    // binds them in GENERAL. A memory barrier alone left them in ATTACHMENT_FEEDBACK_LOOP_OPTIMAL_EXT.
+    vk_exec_barrier(ex);
     VkCommandBuffer cb = ex->cb;
-    float pcv[16] = {px, py, zfar, (float)C.strength / 100.0f,
-                     lx / ll, ly / ll, lz / ll, 0.6f,
-                     (float)C.w, (float)C.h, 1.0f / (float)C.w, 1.0f / (float)C.h,
-                     (float)C.debug, 0, 0, 0};
+    float pcv[16];
+    memcpy(pcv, j->pc, sizeof pcv);
+    pcv[12] = (float)C.debug;
     if (!C.logged) {
         C.logged = 1;
-        rt_log("vk: contact shadows on a %ux%u scene, sun toward %.3f %.3f %.3f (view space), far %g", C.w, C.h,
-               pcv[4], pcv[5], pcv[6], zfar);
+        rt_log("vk: contact shadows on a %ux%u scene, sun toward %.3f %.3f %.3f (view space), far %g", j->scene.width, j->scene.height,
+               pcv[4], pcv[5], pcv[6], pcv[2]);
     }
-    mem_barrier(cb);   // the compose passes' writes, before the march reads and writes them
     VkDescriptorImageInfo ii = {C.sampler, in, VK_IMAGE_LAYOUT_GENERAL}, oi = {VK_NULL_HANDLE, out, VK_IMAGE_LAYOUT_GENERAL};
     VkWriteDescriptorSet w[2] = {
         {.sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET, .dstBinding = 0, .descriptorCount = 1,
@@ -145,36 +185,21 @@ void vk_contact_flush(VkExec *ex) {
     vkCmdBindPipeline(cb, VK_PIPELINE_BIND_POINT_COMPUTE, C.pipe);
     rec_push(C.push, cb, VK_PIPELINE_BIND_POINT_COMPUTE, C.layout, 0, 2, w);
     vkCmdPushConstants(cb, C.layout, VK_SHADER_STAGE_COMPUTE_BIT, 0, sizeof pcv, pcv);
-    vkCmdDispatch(cb, (C.w + 7) / 8, (C.h + 7) / 8, 1);
+    vkCmdDispatch(cb, (j->scene.width + 7) / 8, (j->scene.height + 7) / 8, 1);
     mem_barrier(cb);   // and what follows sees the darkened colour
     ex->recorded += 6;
-    vk_exec_draw_new_cb();   // the game's bound compute pipeline and descriptors were replaced: bind them again
-}
-
-// The lighting composite: the remembered scene colour as the only target, with a D16 depth of its size.
-static int is_composite(const VkExec *ex) {
-    return C.scene && ex->pass_ncolor >= 1 && ex->color[0].img == C.scene && !ex->color[0].level &&
-           ex->depth.format == VK_FORMAT_D16_UNORM && ex->pass_w == C.w && ex->pass_h == C.h;
+    vk_exec_draw_new_cb(ex);   // the game's bound compute pipeline and descriptors were replaced: bind them again
 }
 
+// The sequential replay's hooks; a chunk leaves the marches to the scan.
 void vk_contact_pass_begin(VkExec *ex) {
-    if (!C.tried && !init(vk_ctx())) return;
-    if (!C.ok) return;
-    const VkTexImage *scene, *depth;
-    // The camera block: c4 of the fragment stage, 0x940 bytes. The composite binds it only after its pass begins, so
-    // the newest one seen at any pass start serves.
-    const p_bind_buf *b = &ex->ubo[1][1];
-    if (b->addr && b->size == 0x940) C.c4 = b->addr;
-    if (is_gbuffer(ex, &scene, &depth)) {
-        C.pending = 0;
-        C.scene = scene, C.depth = depth, C.w = ex->pass_w, C.h = ex->pass_h;
-        return;
-    }
-    if (is_composite(ex)) {
-        C.pending = C.c4 != 0;
-        return;
-    }
-    vk_contact_flush(ex);
+    VkExecJob j;
+    if (!ex->chunked && vk_contact_plan(ex, 1, &j)) vk_contact_exec(ex, &j);
+}
+
+void vk_contact_flush(VkExec *ex) {
+    VkExecJob j;
+    if (!ex->chunked && C.pending && vk_contact_plan(ex, 0, &j)) vk_contact_exec(ex, &j);
 }
 
 void vk_contact_release(void) {

runtime/vk_crumbs.c

@@ -91,7 +91,7 @@ void vk_crumbs_report(void) {
 // records changed pipeline state also notes the shaders of a pass's first draw. A full barrier already precedes every
 // pass and dispatch (vk_exec.c vk_exec_pass_begin, vk_exec_draw.c op_dispatch), so the end-of-pipe write adds no wait.
 static VkPassMark ring[VK_PASS_MARKS];
-static uint32_t seq;                        // marks made; only the replay thread writes it
+static uint32_t seq;                        // marks made (atomic: the recording threads of a chunked replay)
 
 // TOTK_PASS_CRUMBS: unset, buffer marks where VK_AMD_buffer_marker exists; 1, also NVIDIA checkpoints; 0, neither.
 int vk_pass_marks_want_nv(void) {
@@ -124,8 +124,8 @@ void vk_pass_mark_cmd(VkCommandBuffer cb, uint32_t id, int end) {
 
 VkPassMark *vk_pass_mark_begin(VkCommandBuffer cb, int kind, uint64_t fence) {
     if (!vk_pass_marks_on || !cb) return NULL;
-    uint32_t id = ++seq;
-    if (!id) id = ++seq;   // 0 means no mark
+    uint32_t id = __atomic_add_fetch(&seq, 1, __ATOMIC_RELAXED);   // the recording threads of a chunked replay
+    if (!id) id = __atomic_add_fetch(&seq, 1, __ATOMIC_RELAXED);   // 0 means no mark
     VkPassMark *m = &ring[id & (VK_PASS_MARKS - 1)];
     m->id = id, m->kind = (uint8_t)kind, m->fence = fence, m->draws = UINT32_MAX;
     rec_pass_mark(cb, id, 0);

runtime/vk_deswizzle.c

@@ -0,0 +1,176 @@
+// Block-linear textures deswizzled by the GPU (runtime/shaders/bl_deswizzle.comp): vk_texture.c upload binds the
+// texture's storage in its pool's imported buffer and lets a compute pass write the linear copy that a buffer-to-image
+// copy (or the ASTC decoder) then takes, instead of tex_deswizzle.c walking every 16 bytes on the replay thread
+// (50-236 ms frames on entering new areas, 2026-11-07 play sessions). The same design as MacNativeGames' TotK port
+// (d2r_gfx nvn_mtl.m: deswizzle from the pool at GPU execution time), written for Vulkan from our CPU layout code.
+//
+// TOTK_DESWIZZLE = auto (default) / gpu: the compute pass where the storage is an imported pool (vk_mem_span_live),
+// the CPU otherwise; cpu: always the CPU; check: gpu, and every upload is also deswizzled on the CPU and compared
+// (waits for the GPU; for testing only).
+#include <stdlib.h>
+#include <string.h>
+
+#include "bl_deswizzle_spv.h"
+#include "cfg.h"
+#include "host.h"
+#include "vk.h"
+#include "vk_rec.h"
+#include "platform/platform.h"
+
+typedef struct {
+    uint32_t src_base, dst_base, segs, pitch16, rows, slices, th, td, gobs_x, blocks_y, nx;
+} Push;
+
+static PlatMutex g_lock = PLAT_MUTEX_INIT;
+static struct {
+    VkDevice dev;
+    int mode;   // VK_DESWZ_*; -1 before the first vk_deswz_mode with a device
+    VkDescriptorSetLayout dsl;
+    VkPipelineLayout layout;
+    VkPipeline pipe;
+    PFN_vkCmdPushDescriptorSetKHR push;
+} G = {.mode = -1};
+static volatile int64_t n_regions, n_bytes;
+
+static int create_pipeline(VkCtx *c) {
+    G.push = (PFN_vkCmdPushDescriptorSetKHR)vkGetDeviceProcAddr(c->device, "vkCmdPushDescriptorSetKHR");
+    if (!G.push) return 0;
+    VkDescriptorSetLayoutBinding b[2];
+    for (int i = 0; i < 2; i++)
+        b[i] = (VkDescriptorSetLayoutBinding){.binding = (uint32_t)i, .descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
+                                              .descriptorCount = 1, .stageFlags = VK_SHADER_STAGE_COMPUTE_BIT};
+    VkDescriptorSetLayoutCreateInfo dci = {.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO,
+                                           .flags = VK_DESCRIPTOR_SET_LAYOUT_CREATE_PUSH_DESCRIPTOR_BIT_KHR,
+                                           .bindingCount = 2, .pBindings = b};
+    if (vkCreateDescriptorSetLayout(c->device, &dci, NULL, &G.dsl) != VK_SUCCESS) return 0;
+    VkPushConstantRange pr = {VK_SHADER_STAGE_COMPUTE_BIT, 0, sizeof(Push)};
+    VkPipelineLayoutCreateInfo li = {.sType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO, .setLayoutCount = 1,
+                                     .pSetLayouts = &G.dsl, .pushConstantRangeCount = 1, .pPushConstantRanges = &pr};
+    if (vkCreatePipelineLayout(c->device, &li, NULL, &G.layout) != VK_SUCCESS) return 0;
+    VkShaderModuleCreateInfo mi = {.sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO, .codeSize = sizeof bl_deswizzle_spv,
+                                   .pCode = bl_deswizzle_spv};
+    VkShaderModule m = VK_NULL_HANDLE;
+    if (vkCreateShaderModule(c->device, &mi, NULL, &m) != VK_SUCCESS) return 0;
+    VkComputePipelineCreateInfo ci = {.sType = VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO,
+                                      .stage = {.sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO,
+                                                .stage = VK_SHADER_STAGE_COMPUTE_BIT, .module = m, .pName = "main"},
+                                      .layout = G.layout};
+    VkResult r = vkCreateComputePipelines(c->device, VK_NULL_HANDLE, 1, &ci, NULL, &G.pipe);
+    vkDestroyShaderModule(c->device, m, NULL);
+    if (r != VK_SUCCESS) G.pipe = VK_NULL_HANDLE;
+    return r == VK_SUCCESS;
+}
+
+int vk_deswz_mode(void) {
+    VkCtx *c = vk_ctx();
+    if (!c) return VK_DESWZ_CPU;
+    plat_mutex_lock_shared(&g_lock);
+    int m = G.dev == c->device ? G.mode : -1;
+    plat_mutex_unlock_shared(&g_lock);
+    if (m >= 0) return m;
+    plat_mutex_lock(&g_lock);
+    if (G.dev != c->device) {   // the old device's objects went with it (probes recreate the device)
+        memset(&G, 0, sizeof G);
+        G.dev = c->device;
+        G.mode = -1;
+    }
+    if (G.mode < 0) {
+        const char *e = cfg_get("TOTK_DESWIZZLE");
+        if (!e || !*e) e = "auto";
+        int want = !strcmp(e, "cpu") || !strcmp(e, "0") ? VK_DESWZ_CPU : !strcmp(e, "check") ? VK_DESWZ_CHECK : VK_DESWZ_GPU;
+        if (strcmp(e, "auto") && strcmp(e, "gpu") && strcmp(e, "1") && strcmp(e, "cpu") && strcmp(e, "0") &&
+            strcmp(e, "check"))
+            rt_log("vk: TOTK_DESWIZZLE=%s is not auto, gpu, cpu or check: auto", e);
+        if (want != VK_DESWZ_CPU && !create_pipeline(c)) {
+            rt_log("vk: the deswizzle compute pass is unavailable on %s: textures deswizzle on the CPU", c->props.deviceName);
+            want = VK_DESWZ_CPU;
+        }
+        G.mode = want;
+        rt_log("vk: block-linear textures deswizzle on the %s%s", want == VK_DESWZ_CPU ? "CPU" : "GPU where the storage is "
+               "an imported pool", want == VK_DESWZ_CHECK ? ", checked against the CPU (TOTK_DESWIZZLE=check)" : "");
+    }
+    m = G.mode;
+    plat_mutex_unlock(&g_lock);
+    return m;
+}
+
+static uint32_t max_groups_x(const VkCtx *c) {
+    uint32_t m = c->props.limits.maxComputeWorkGroupCount[0];
+    return m > 65535 ? 65535 : m;
+}
+
+// The bound source range starts at the alignment below src_off.
+static VkDeviceSize src_lo(const VkCtx *c, VkDeviceSize src_off) {
+    VkDeviceSize al = c->props.limits.minStorageBufferOffsetAlignment;
+    if (al < 16) al = 16;
+    return src_off / al * al;
+}
+
+int vk_deswz_fits(VkDeviceSize src_off, VkDeviceSize src_size, const VkDeswzRegion *rg, int n, VkDeviceSize max_dst) {
+    VkCtx *c = vk_ctx();
+    if (!c || !n || (src_off & 15) || vk_deswz_mode() == VK_DESWZ_CPU) return 0;
+    VkDeviceSize range = src_off + src_size - src_lo(c, src_off), lim = c->props.limits.maxStorageBufferRange;
+    if (range > lim || max_dst > lim) return 0;
+    uint32_t maxg = max_groups_x(c);
+    for (int i = 0; i < n; i++) {   // the 32-bit segment indices of the shader stay inside the bindings
+        uint64_t groups = ((uint64_t)rg[i].segs + 63) / 64, out = (uint64_t)rg[i].pitch16 * rg[i].rows * rg[i].slices;
+        if ((rg[i].src & 15) || rg[i].src + (uint64_t)rg[i].segs * 16 > src_size || !rg[i].segs ||
+            (src_off - src_lo(c, src_off) + rg[i].src) / 16 + rg[i].segs > UINT32_MAX || out > UINT32_MAX ||
+            out * 16 > max_dst || (groups + maxg - 1) / maxg > c->props.limits.maxComputeWorkGroupCount[1])
+            return 0;
+    }
+    return 1;
+}
+
+int vk_deswz_record(VkCommandBuffer cb, VkBuffer src, VkDeviceSize src_off, VkDeviceSize src_size, VkBuffer dst,
+                    VkDeviceSize dst_size, const VkDeswzRegion *rg, int n) {
+    VkCtx *c = vk_ctx();
+    if (!vk_deswz_fits(src_off, src_size, rg, n, dst_size)) return 0;
+    for (int i = 0; i < n; i++)
+        if ((rg[i].dst & 15) || rg[i].dst + (uint64_t)rg[i].pitch16 * 16 * rg[i].rows * rg[i].slices > dst_size) return 0;
+    plat_mutex_lock_shared(&g_lock);
+    VkPipeline pipe = G.dev == c->device ? G.pipe : VK_NULL_HANDLE;
+    plat_mutex_unlock_shared(&g_lock);
+    if (!pipe) return 0;
+    VkDeviceSize lo = src_lo(c, src_off), range = src_off + src_size - lo;
+    uint32_t maxg = max_groups_x(c);
+    VkDescriptorBufferInfo dbi[2] = {{src, lo, range}, {dst, 0, dst_size}};
+    VkWriteDescriptorSet w[2];
+    for (int i = 0; i < 2; i++)
+        w[i] = (VkWriteDescriptorSet){.sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET, .dstBinding = (uint32_t)i,
+                                      .descriptorCount = 1, .descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
+                                      .pBufferInfo = &dbi[i]};
+    vkCmdBindPipeline(cb, VK_PIPELINE_BIND_POINT_COMPUTE, pipe);
+    rec_push(G.push, cb, VK_PIPELINE_BIND_POINT_COMPUTE, G.layout, 0, 2, w);
+    uint64_t bytes = 0;
+    for (int i = 0; i < n; i++) {
+        const VkDeswzRegion *r = &rg[i];
+        uint64_t groups = ((uint64_t)r->segs + 63) / 64;
+        uint32_t nx = groups < maxg ? (uint32_t)groups : maxg, ny = (uint32_t)((groups + nx - 1) / nx);
+        Push pc = {.src_base = (uint32_t)((src_off - lo + r->src) / 16), .dst_base = (uint32_t)(r->dst / 16),
+                   .segs = r->segs, .pitch16 = r->pitch16, .rows = r->rows, .slices = r->slices, .th = r->th, .td = r->td,
+                   .gobs_x = r->gobs_x, .blocks_y = r->blocks_y, .nx = nx};
+        vkCmdPushConstants(cb, G.layout, VK_SHADER_STAGE_COMPUTE_BIT, 0, sizeof pc, &pc);
+        vkCmdDispatch(cb, nx, ny, 1);
+        bytes += (uint64_t)r->segs * 16;
+    }
+    int64_t before = plat_add64(&n_regions, n) - n;
+    plat_add64(&n_bytes, (int64_t)bytes);
+    if (!before || (before + n) / 4096 != before / 4096)
+        rt_log("vk: deswizzle: %lld regions, %.1f MiB of block-linear storage deswizzled on the GPU so far",
+               (long long)(before + n), (double)n_bytes / 1048576.0);
+    return 1;
+}
+
+void vk_deswz_release(void) {
+    VkCtx *c = vk_ctx();
+    plat_mutex_lock(&g_lock);
+    if (c && G.dev == c->device) {
+        if (G.pipe) vkDestroyPipeline(c->device, G.pipe, NULL);
+        if (G.layout) vkDestroyPipelineLayout(c->device, G.layout, NULL);
+        if (G.dsl) vkDestroyDescriptorSetLayout(c->device, G.dsl, NULL);
+    }
+    memset(&G, 0, sizeof G);
+    G.mode = -1;
+    plat_mutex_unlock(&g_lock);
+}

runtime/vk_device.c

@@ -16,9 +16,69 @@ static VkCtx g_vk;
 static int g_ready;
 static PlatMutex g_lock = PLAT_MUTEX_INIT;
 
+// ---- GPU address map for device faults
+// VK_EXT_device_address_binding_report: the driver reports each bind and unbind of an object's GPU virtual range through
+// the debug messenger. The newest AB_N events stay in a ring written without a lock (one atomic add each), so a device
+// fault names the object behind its address and whether it was unbound before the fault, which is memory freed while
+// a submission still used it. One event per image, buffer or memory block made or freed, none per draw.
+// TOTK_ADDR_REPORT=0 turns it off.
+#define AB_N (1u << 15)
+typedef struct { uint64_t seq, base, size, handle, issued, done; int32_t type; uint32_t unbind; } AbEvent;
+static AbEvent *ab_ring;
+static uint64_t ab_head;
+static int ab_want;   // the instance has the messenger for it (create_instance)
+
+static void ab_record(const VkDebugUtilsMessengerCallbackDataEXT *d) {
+    const VkDeviceAddressBindingCallbackDataEXT *b = NULL;
+    for (const VkBaseInStructure *p = d->pNext; p; p = p->pNext)
+        if (p->sType == VK_STRUCTURE_TYPE_DEVICE_ADDRESS_BINDING_CALLBACK_DATA_EXT) b = (const void *)p;
+    if (!b || !ab_ring) return;
+    uint64_t i = __atomic_fetch_add(&ab_head, 1, __ATOMIC_RELAXED);
+    AbEvent *e = &ab_ring[i & (AB_N - 1)];
+    __atomic_store_n(&e->seq, 0, __ATOMIC_RELAXED);
+    e->base = b->baseAddress, e->size = b->size;
+    e->handle = d->objectCount ? d->pObjects[0].objectHandle : 0;
+    e->type = d->objectCount ? (int32_t)d->pObjects[0].objectType : 0;
+    e->unbind = b->bindingType == VK_DEVICE_ADDRESS_BINDING_TYPE_UNBIND_EXT;
+    e->issued = vk_sync_issued(), e->done = vk_sync_fence_done();
+    __atomic_store_n(&e->seq, i + 1, __ATOMIC_RELEASE);
+}
+
+static const char *ab_type_name(int32_t t, char *buf, size_t n) {
+    switch (t) {
+    case VK_OBJECT_TYPE_IMAGE: return "image";
+    case VK_OBJECT_TYPE_BUFFER: return "buffer";
+    case VK_OBJECT_TYPE_DEVICE_MEMORY: return "memory";
+    default: snprintf(buf, n, "object type %d", (int)t); return buf;
+    }
+}
+
+// The newest events whose range meets the precision-aligned range around a fault address, newest first.
+static void ab_report(uint64_t addr, uint64_t precision) {
+    if (!ab_ring) return;
+    uint64_t lo = precision > 1 ? addr & ~(precision - 1) : addr, hi = lo + (precision > 1 ? precision : 1);
+    uint64_t head = __atomic_load_n(&ab_head, __ATOMIC_ACQUIRE), n = head < AB_N ? head : AB_N;
+    int found = 0;
+    for (uint64_t k = 1; k <= n && found < 6; k++) {
+        const AbEvent *e = &ab_ring[(head - k) & (AB_N - 1)];
+        if (__atomic_load_n(&e->seq, __ATOMIC_ACQUIRE) != head - k + 1) continue;
+        if (e->base >= hi || e->base + e->size <= lo) continue;
+        found++;
+        char tn[32];
+        rt_log("vk: fault address %#llx: %s %#llx [%#llx, +%#llx] %s %llu address events before the fault, "
+               "submitted %llu, done %llu", (unsigned long long)addr, ab_type_name(e->type, tn, sizeof tn),(unsigned long long)e->handle,
+               (unsigned long long)e->base, (unsigned long long)e->size, e->unbind ? "unbound" : "bound",
+               (unsigned long long)k, (unsigned long long)e->issued, (unsigned long long)e->done);
+    }
+    if (!found) rt_log("vk: fault address %#llx: no binding in the last %llu address events", (unsigned long long)addr,
+                       (unsigned long long)n);
+}
+
 static VKAPI_ATTR VkBool32 VKAPI_CALL debug_cb(VkDebugUtilsMessageSeverityFlagBitsEXT sev, VkDebugUtilsMessageTypeFlagsEXT type,
                                                const VkDebugUtilsMessengerCallbackDataEXT *d, void *user) {
-    (void)type, (void)user;
+    (void)user;
+    if (type & VK_DEBUG_UTILS_MESSAGE_TYPE_DEVICE_ADDRESS_BINDING_BIT_EXT) { ab_record(d); return VK_FALSE; }
+    if (sev < VK_DEBUG_UTILS_MESSAGE_SEVERITY_WARNING_BIT_EXT) return VK_FALSE;   // info, asked for the address events
     int err = (sev & VK_DEBUG_UTILS_MESSAGE_SEVERITY_ERROR_BIT_EXT) != 0;
     if (err) plat_inc32(&g_vk.validation_errors);
     static volatile int32_t logged;
@@ -76,7 +136,10 @@ static int create_instance(void) {
         rt_log("vk: TOTK_VK_VALIDATION set but %s is not installed; continuing without", layers[0]);
         validate = 0;
     }
-    if (validate) ext[n_ext++] = VK_EXT_DEBUG_UTILS_EXTENSION_NAME;
+    const char *ar = cfg_get("TOTK_ADDR_REPORT");
+    ab_want = !(ar && !strcmp(ar, "0")) && instance_ext_present(VK_EXT_DEBUG_UTILS_EXTENSION_NAME);
+    if (ab_want && !ab_ring && !(ab_ring = calloc(AB_N, sizeof *ab_ring))) ab_want = 0;
+    if (validate || ab_want) ext[n_ext++] = VK_EXT_DEBUG_UTILS_EXTENSION_NAME;
     if (vk_hdr_wanted() && instance_ext_present(VK_EXT_SWAPCHAIN_COLOR_SPACE_EXTENSION_NAME))   // HDR10 surface formats
         ext[n_ext++] = VK_EXT_SWAPCHAIN_COLOR_SPACE_EXTENSION_NAME;
     vk_dlaa_instance_exts(ext, &n_ext, 12);
@@ -89,16 +152,18 @@ static int create_instance(void) {
     VkResult r = vkCreateInstance(&ci, NULL, &g_vk.instance);
     if (r != VK_SUCCESS) { rt_log("vk: vkCreateInstance failed (%d); is a Vulkan 1.3 driver installed?", (int)r); return 0; }
     g_vk.validation = validate;
-    if (validate) {
+    if (validate || ab_want) {
         PFN_vkCreateDebugUtilsMessengerEXT f =
             (PFN_vkCreateDebugUtilsMessengerEXT)vkGetInstanceProcAddr(g_vk.instance, "vkCreateDebugUtilsMessengerEXT");
         VkDebugUtilsMessengerCreateInfoEXT mi = {
             .sType = VK_STRUCTURE_TYPE_DEBUG_UTILS_MESSENGER_CREATE_INFO_EXT,
-            .messageSeverity = VK_DEBUG_UTILS_MESSAGE_SEVERITY_WARNING_BIT_EXT | VK_DEBUG_UTILS_MESSAGE_SEVERITY_ERROR_BIT_EXT,
-            .messageType = VK_DEBUG_UTILS_MESSAGE_TYPE_GENERAL_BIT_EXT | VK_DEBUG_UTILS_MESSAGE_TYPE_VALIDATION_BIT_EXT |
-                           VK_DEBUG_UTILS_MESSAGE_TYPE_PERFORMANCE_BIT_EXT,
+            .messageSeverity = VK_DEBUG_UTILS_MESSAGE_SEVERITY_WARNING_BIT_EXT | VK_DEBUG_UTILS_MESSAGE_SEVERITY_ERROR_BIT_EXT |
+                               (ab_want ? VK_DEBUG_UTILS_MESSAGE_SEVERITY_INFO_BIT_EXT : 0),
+            .messageType = (validate ? VK_DEBUG_UTILS_MESSAGE_TYPE_GENERAL_BIT_EXT | VK_DEBUG_UTILS_MESSAGE_TYPE_VALIDATION_BIT_EXT |
+                                       VK_DEBUG_UTILS_MESSAGE_TYPE_PERFORMANCE_BIT_EXT : 0) |
+                           (ab_want ? VK_DEBUG_UTILS_MESSAGE_TYPE_DEVICE_ADDRESS_BINDING_BIT_EXT : 0),
             .pfnUserCallback = debug_cb};
-        if (f) f(g_vk.instance, &mi, NULL, &g_vk.messenger);
+        if (!f || f(g_vk.instance, &mi, NULL, &g_vk.messenger) != VK_SUCCESS) ab_want = 0;
     }
     return 1;
 }
@@ -217,6 +282,7 @@ typedef struct {
     VkPhysicalDeviceGraphicsPipelineLibraryFeaturesEXT gpl;
     VkPhysicalDeviceAttachmentFeedbackLoopLayoutFeaturesEXT fbl;
     VkPhysicalDeviceDynamicRenderingLocalReadFeaturesKHR lr;
+    VkPhysicalDeviceAddressBindingReportFeaturesEXT abr;
 } FeatureChain;
 
 static int create_device(void) {
@@ -251,6 +317,13 @@ static int create_device(void) {
     // Ryujinx's SPIR-V declares fragment shader interlock whenever the host reports it (vk_pipeline.c spirv_fixup).
     OPTIONAL_EXT(has_fragment_shader_interlock, VK_EXT_FRAGMENT_SHADER_INTERLOCK_EXTENSION_NAME, fsi,
                  VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FRAGMENT_SHADER_INTERLOCK_FEATURES_EXT);
+    // The GPU address map for device faults (ab_record), with the messenger made for it
+    int has_abr = 0;
+    if (ab_want && has_ext(ep, ne, VK_EXT_DEVICE_ADDRESS_BINDING_REPORT_EXTENSION_NAME)) {
+        has_abr = 1;
+        ext[n_ext++] = VK_EXT_DEVICE_ADDRESS_BINDING_REPORT_EXTENSION_NAME;
+        CHAIN(abr, VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_ADDRESS_BINDING_REPORT_FEATURES_EXT);
+    }
     // Feedback loops (vk.h VkCtx.feedback): the layout, and barriers inside dynamic rendering
     int has_fbl = 0, has_lr = 0;
     if (has_ext(ep, ne, VK_EXT_ATTACHMENT_FEEDBACK_LOOP_LAYOUT_EXTENSION_NAME) &&
@@ -314,6 +387,10 @@ static int create_device(void) {
     else if (!h->textureCompressionBC) miss = "textureCompressionBC";
     if (miss) { rt_log("vk: required feature missing on %s: %s", g_vk.props.deviceName, miss); free(c); return 0; }
     if (!c->fault.deviceFault) g_vk.has_device_fault = 0;   // vkGetDeviceFaultInfoEXT needs the feature, not just the extension
+    rt_log("vk: GPU address map for device faults: %s",
+           has_abr && c->abr.reportAddressBinding ? "on (VK_EXT_device_address_binding_report; TOTK_ADDR_REPORT=0 turns it off)"
+           : !ab_want ? "off (TOTK_ADDR_REPORT=0 or no VK_EXT_debug_utils)"
+                      : "unavailable (no VK_EXT_device_address_binding_report)");
     g_vk.has_buffer_device_address = c->f12.bufferDeviceAddress;
     g_vk.has_astc_ldr = h->textureCompressionASTC_LDR;
     {
@@ -352,8 +429,14 @@ static int create_device(void) {
     g_vk.min_imported_host_alignment = (uint32_t)emh.minImportedHostPointerAlignment;
     vk_caps_init(&g_vk);
 
+    // Blocks of 64 MiB instead of VMA's 256 (TOTK_VMA_BLOCK_MIB, 0 = VMA's). Since the textures moved to their own pool
+    // (vk_memory.c vk_texture_image), the default device-local blocks hold mirrors and other buffers: on the RX 9070
+    // after 10 minutes in the field one 256 MiB block held 53 MiB of them, and the headroom thread kept half a block of
+    // each device-local type ahead (128 MiB each). Larger buffers than half a block get memory of their own.
+    g_vk.vma_block = (VkDeviceSize)cfg_int("TOTK_VMA_BLOCK_MIB") << 20;
+    if (!g_vk.vma_block) g_vk.vma_block = 256ull << 20;   // VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE
     VmaAllocatorCreateInfo ai = {.physicalDevice = g_vk.phys, .device = g_vk.device, .instance = g_vk.instance,
-                                 .vulkanApiVersion = VK_API_VERSION_1_3,
+                                 .vulkanApiVersion = VK_API_VERSION_1_3, .preferredLargeHeapBlockSize = g_vk.vma_block,
                                  .flags = (g_vk.has_buffer_device_address ? VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT : 0) |
                                           (g_vk.has_memory_budget ? VMA_ALLOCATOR_CREATE_EXT_MEMORY_BUDGET_BIT : 0),
                                  .pDeviceMemoryCallbacks = vk_headroom_callbacks()};
@@ -376,6 +459,7 @@ static void destroy_all(void) {
     gfx_window_release_vk();
     if (g_vk.device) vkDeviceWaitIdle(g_vk.device);
     if (g_vk.device) vk_astc_release();
+    if (g_vk.device) vk_deswz_release();
     if (g_vk.device) vk_crumbs_release();
     if (g_vk.timeline) vkDestroySemaphore(g_vk.device, g_vk.timeline, NULL);
     vk_headroom_stop();
@@ -397,7 +481,8 @@ int vk_init(void) {
     plat_mutex_lock(&g_lock);
     if (!g_ready) {
         memset(&g_vk, 0, sizeof g_vk);
-        if (create_instance() && pick_device() && create_device()) g_ready = 1, vk_headroom_start(), vk_dlaa_init();
+        if (create_instance() && pick_device() && create_device())
+            g_ready = 1, vk_mem_host_probe(), vk_upload_pools_init(), vk_headroom_start(), vk_dlaa_init();
         else destroy_all();
     }
     int ok = g_ready;
@@ -437,6 +522,7 @@ void vk_device_fault_report(VkResult r) {
     done = 1;
     vk_crumbs_report();
     vk_pass_marks_report();
+    vk_mem_fault_log();
     VkCtx *c = vk_ctx();
     PFN_vkGetDeviceFaultInfoEXT get =
         c && c->has_device_fault ? (PFN_vkGetDeviceFaultInfoEXT)vkGetDeviceProcAddr(c->device, "vkGetDeviceFaultInfoEXT")
@@ -461,9 +547,11 @@ void vk_device_fault_report(VkResult r) {
         rt_log("vk: device fault: \"%s\" (%u address infos, %u vendor infos, vendor binary %llu bytes)", info.description,
                n.addressInfoCount, n.vendorInfoCount, (unsigned long long)binary);
         // The faulting access lies in the precision-aligned range around the reported address.
-        for (uint32_t i = 0; i < n.addressInfoCount; i++)
+        for (uint32_t i = 0; i < n.addressInfoCount; i++) {
             rt_log("vk: device fault address %u: %s at %#llx, precision %#llx", i, fault_address_type(ai[i].addressType),
                    (unsigned long long)ai[i].reportedAddress, (unsigned long long)ai[i].addressPrecision);
+            ab_report(ai[i].reportedAddress, ai[i].addressPrecision);
+        }
         for (uint32_t i = 0; i < n.vendorInfoCount; i++)
             rt_log("vk: device fault vendor info %u: \"%s\" code %#llx data %#llx", i, vi[i].description,
                    (unsigned long long)vi[i].vendorFaultCode, (unsigned long long)vi[i].vendorFaultData);

runtime/vk_dlaa.c

@@ -11,9 +11,6 @@
 // layout with a set-then-get round trip before anything relies on it.
 //
 // Where DLAA runs in the frame, the jitter and the motion vectors: see the sections below.
-#ifdef _WIN32
-#include <windows.h>
-#endif
 #include <math.h>
 #include <stdio.h>
 #include <stdlib.h>
@@ -115,31 +112,19 @@ int vk_dlaa_wanted(void) {
 static int core_load(void) {
     if (N.tried) return N.loaded;
     N.tried = 1;
-#ifdef _WIN32
-    char dir[MAX_PATH] = {0}, path[MAX_PATH + 32];
-    DWORD n = sizeof dir;
-    if (RegGetValueA(HKEY_LOCAL_MACHINE, "SOFTWARE\\NVIDIA Corporation\\Global\\NGXCore", "FullPath", RRF_RT_REG_SZ,
-                     NULL, dir, &n) != ERROR_SUCCESS) {
+    char path[PLAT_PATH_MAX];
+    if (!plat_ngx_core_path(path, sizeof path)) {
         rt_log("dlaa: the driver has no NGX core (registry NGXCore missing): DLAA off");
         return 0;
     }
-    snprintf(path, sizeof path, "%s\\_nvngx.dll", dir);
-    N.core = LoadLibraryA(path);
-    if (!N.core) {
-        rt_log("dlaa: cannot load %s (error %lu): DLAA off", path, GetLastError());
-        return 0;
-    }
-#define FN(f, name) N.f = (void *)GetProcAddress((HMODULE)N.core, name)
-#else
-    // The driver's NGX core; the dynamic loader finds it where the driver put it (/usr/lib, /usr/lib/x86_64-linux-gnu).
-    const char *path = "libnvidia-ngx.so.1";
     N.core = plat_lib_open(path);
     if (!N.core) {
-        rt_log("dlaa: cannot load %s (not an NVIDIA driver, or one without NGX): DLAA off", path);
+        char why[256];
+        plat_lib_error(why, sizeof why);
+        rt_log("dlaa: cannot load %s (%s; not an NVIDIA driver, or one without NGX): DLAA off", path, why);
         return 0;
     }
 #define FN(f, name) N.f = plat_lib_sym(N.core, name)
-#endif
     FN(init, "NVSDK_NGX_VULKAN_Init_Ext2");
     FN(req_ext, "NVSDK_NGX_VULKAN_RequiredExtensions");
     FN(caps, "NVSDK_NGX_VULKAN_GetCapabilityParameters");
@@ -203,46 +188,14 @@ static int vtable_ok(void) {
     return ok;
 }
 
-#ifndef _WIN32
-// UTF-8 to UTF-32 wchar_t, without the C library's locale (the runtime never sets one, so mbstowcs would stop at the
-// first non-ASCII byte). Returns 0 on malformed input or when out has no room.
-static int utf8_to_wide(const char *in, wchar_t *out, size_t cap) {
-    const unsigned char *u = (const unsigned char *)in;
-    size_t n = 0;
-    while (*u) {
-        unsigned c = *u++, more = c < 0x80 ? 0 : c >= 0xF0 ? 3 : c >= 0xE0 ? 2 : c >= 0xC0 ? 1 : 4;
-        if (more == 4) return 0;
-        if (more) c &= 0x3F >> more;
-        for (unsigned k = 0; k < more; k++, u++) {
-            if ((*u & 0xC0) != 0x80) return 0;
-            c = c << 6 | (*u & 0x3F);
-        }
-        if (n + 1 >= cap) return 0;
-        out[n++] = (wchar_t)c;
-    }
-    out[n] = 0;
-    return 1;
-}
-#endif
-
 void vk_dlaa_init(void) {
     if (!vk_dlaa_wanted() || !core_load()) return;
-#ifdef _WIN32
-    wchar_t wdir[MAX_PATH];
-    DWORD len = GetModuleFileNameW(NULL, wdir, MAX_PATH);
-    while (len && wdir[len - 1] != L'\\') len--;
-    wdir[len] = 0;
-#else
-    // The executable's folder, where NGX looks for libnvidia-ngx-dlss.so and writes its logs, as a wchar_t string:
-    // 32-bit code points here, as the Linux core expects.
-    char dir[PLAT_PATH_MAX];
+    // The executable's folder, where NGX looks for the DLSS model library and writes its logs.
     wchar_t wdir[PLAT_PATH_MAX];
-    char *cut = plat_exe_path(dir, sizeof dir) ? plat_last_sep(dir) : NULL;
-    if (!cut || !utf8_to_wide((cut[1] = 0, dir), wdir, PLAT_PATH_MAX)) {
+    if (!plat_exe_dir_wide(wdir, PLAT_PATH_MAX)) {
         rt_log("dlaa: cannot name the executable's folder for NGX: DLAA off");
         return;
     }
-#endif
     // 231313132: the application id NVIDIA's DLSS samples use for builds without an NVIDIA-issued id
     const VkCtx *c = vk_ctx();
     NgxResult r = N.init(231313132ull, wdir, c->instance, c->phys, c->device, vkGetInstanceProcAddr,
@@ -289,7 +242,11 @@ int vk_dlaa_available(void) { return vk_dlaa_wanted() && N.available; }
 static struct {
     VkExecTarget color, depth;   // the armed pair, valid in the frame it was armed
     int armed, done, gbuf;       // this frame: armed on the light target, DLAA ran, the G-buffer was drawn
+    uint32_t gbuf_w;             // the widest G-buffer's width this frame
+    VkImage nodepth;             // this frame: a G-buffer-size target drawn without depth while nothing was armed
     int ended, up;               // this frame: the scene is drawn (jitter over), DLSS wrote the scaled output
+    uint64_t end_fence;          // a chunked replay: the submission and op the scan ended the scene at (ended_at)
+    uint32_t end_pos;
     struct Size {                // a DLSS feature and its work images per scene size (dynamic resolution flips it)
         uint32_t w, h, ow, oh;   // render size, output size
         VkImage out, mv;
@@ -604,7 +561,7 @@ static int mv_pass(VkExec *ex, struct Size *e, VkImageView zv, float jx, float j
     VkCommandBuffer cb = ex->cb;
     uint32_t w = e->w, h = e->h;
     VkDescriptorBufferInfo cam;
-    if (!mv_init() || !D.cam_addr || !vk_usnap_find(D.cam_addr, CAM_BLOCK, 4, &cam)) {
+    if (!mv_init() || !D.cam_addr || !vk_usnap_find(D.cam_addr, CAM_BLOCK, UINT32_MAX, 4, &cam)) {
         D.mv_none++;
         D.cam_prev_ok = 0;
         e->chars_ok = 0;
@@ -824,10 +781,9 @@ static struct Snap *snap_for(VkCommandBuffer cb, uint32_t w, uint32_t h, VkForma
 }
 
 // At arming: the light target as it is before the transparents, outside any pass.
-static void mask_snapshot(VkExec *ex) {
+static void mask_snapshot(VkExec *ex, const VkExecTarget *t) {
     R.cur = NULL;
     if (mask_init() <= 0) return;
-    const VkExecTarget *t = &D.color;
     struct Snap *s = snap_for(ex->cb, t->width, t->height, t->format);
     if (!s) return;
     vk_exec_barrier(ex);
@@ -876,14 +832,33 @@ static float halton(uint32_t i, uint32_t b) {
     return r;
 }
 
+// The scene is drawn at the op ex records. A chunked replay's scan ends it ahead of the chunks that still record the
+// scene's draws, so a chunk compares its op with the scan's (the chunks after it are taken once the scan is past their
+// ops); a later submission of the frame is past it.
+static int ended_at(const VkExec *ex) {
+    if (!__atomic_load_n(&D.ended, __ATOMIC_ACQUIRE)) return 0;
+    return !ex->chunked || ex->fence != D.end_fence || ex->pos >= D.end_pos;
+}
+
 void vk_dlaa_viewport(VkExec *ex, VkViewport *v) {
-    if (!D.jit_ok || D.ended || !ex->depth.image || ex->depth.image != D.jdepth) return;
+    if (!D.jit_ok || !ex->depth.image || ex->depth.image != D.jdepth || ended_at(ex)) return;
     v->x += D.jx, v->y += D.jy;
-    D.ux = D.jx, D.uy = D.jy, D.jit_used = 1;
+    D.ux = D.jx, D.uy = D.jy;
+    __atomic_store_n(&D.jit_used, 1, __ATOMIC_RELAXED);
+}
+
+// The scene camera: c4 of the last scene draw before the scene ends. A chunked replay's scan notes it in replay order
+// (vk_dlaa_scan_draw): chunks recording side by side would leave the block of whichever drew last.
+static void note_camera(const VkExec *ex) {
+    const p_bind_buf *b = &ex->ubo[0][1];   // constant bank c4: the replay keeps user banks from c3 on
+    const VkExecTarget *z = &ex->depth;
+    if (b->size == 0x940 && b->addr && z->image && z->width >= DLAA_SCENE_MIN_W && z->format == VK_FORMAT_D32_SFLOAT &&
+        (!D.jdepth || z->image == D.jdepth))
+        D.cam_addr = b->addr;
 }
 
 void vk_dlaa_draw(VkExec *ex) {
-    if (!dl_active() || D.ended) return;
+    if (!dl_active() || ended_at(ex)) return;
     static int survey = -1;   // TOTK_DLAA_SURVEY=1 (test): what is drawn into the armed light target
     if (survey < 0) survey = rt_knob("DLAA_SURVEY", "0");
     static uint64_t dfn[16];   // scene draws writing depth, by depth test (reversed depth: GREATER/GEQUAL)
@@ -914,11 +889,11 @@ void vk_dlaa_draw(VkExec *ex) {
             }
         }
     }
-    const p_bind_buf *b = &ex->ubo[0][1];   // constant bank c4: the replay keeps user banks from c3 on
-    const VkExecTarget *z = &ex->depth;
-    if (b->size == 0x940 && b->addr && z->image && z->width >= DLAA_SCENE_MIN_W && z->format == VK_FORMAT_D32_SFLOAT &&
-        (!D.jdepth || z->image == D.jdepth))
-        D.cam_addr = b->addr;
+    if (!ex->chunked) note_camera(ex);
+}
+
+void vk_dlaa_scan_draw(const VkExec *ex) {
+    if (dl_active() && !D.ended) note_camera(ex);
 }
 
 // The jitter moves on right after DLAA ran, not at the present: the replay can record the next frame's scene before
@@ -1092,7 +1067,7 @@ static void evaluate(VkExec *ex, const VkExecTarget *dst) {
     full_barrier(cb);
     if (dst != t) vk_dlaa_swap_img = t->img, vk_dlaa_swap_view = e->out_view;   // the scaling draw reads DLSS's picture
     ex->recorded += 4;
-    vk_exec_draw_new_cb();   // NGX bound pipelines and descriptor sets of its own: the next draw binds everything again
+    vk_exec_draw_new_cb(ex);   // NGX bound pipelines and descriptor sets of its own: the next draw binds everything again
     D.jdepth = z->image;
     next_jitter(w, ow);
     if (++D.evals == 1 || D.evals == 300 || D.evals % 3600 == 0)   // 300: one early line with the frames counted
@@ -1109,51 +1084,110 @@ static int scene_color(VkFormat f) {
     return f == VK_FORMAT_B10G11R11_UFLOAT_PACK32 || f == VK_FORMAT_R16G16B16A16_SFLOAT;
 }
 
-void vk_dlaa_pass(VkExec *ex) {
+int vk_dlaa_plan(const VkExec *ex, VkExecJob *j) {
     const VkExecTarget *c = &ex->color[0], *z = &ex->depth;
-    if (!dl_active() || D.done) return;
+    if (!dl_active() || D.done) return 0;
     // The G-buffer: several colour targets with a D32F depth of at least DLAA_SCENE_MIN_W wide. Only the light target drawn after it
     // arms the frame; at night the game draws into the light target alone with the scene depth before the G-buffer too,
     // and DLAA then ran at a small pass ahead of the whole scene, which reached the screen jittered.
     if (ex->pass_ncolor >= 3 && z->image && z->format == VK_FORMAT_D32_SFLOAT && z->width >= DLAA_SCENE_MIN_W) {
+        D.gbuf_w = D.gbuf && D.gbuf_w > z->width ? D.gbuf_w : z->width;
         D.gbuf = 1, D.armed = 0;
         // target 0 is the object-class buffer (RGBA8, the scene size): the characters' bias mask reads its green. A
         // smaller pass of the same shape (640x360, seen while loading) does not replace the scene's in the same frame.
         if (c->image && c->format == VK_FORMAT_R8G8B8A8_UNORM && c->width == z->width && c->height == z->height &&
             c->samples <= 1 && !c->is3d && (!D.ids_ok || c->width >= D.ids.width))
             D.ids = *c, D.ids_ok = 1;
-        return;
+        return 0;
+    }
+    // A pass into one scene colour target of the G-buffer's size or larger without depth while nothing is armed ends
+    // the scene once the bloom chain follows with still nothing armed: in the inventory (1.4.3) the preview's light
+    // target is drawn that way before the bloom, and with the scene depth only after it, while the menu is composed;
+    // DLAA ran on the finished menu picture with the previous frame's history, which left stray dark pixels around Link.
+    // Arming cancels the end, on that target too: at some camera angles the field's effects composite (a full-screen
+    // draw without depth into the light target) comes before the first pass with the scene depth, and with that target
+    // barred from arming DLSS stopped for as long as the view held (in the inventory the bloom chain always ended the
+    // scene first, in both modes, menu opening included). While the scope zooms in, a second G-buffer pass disarms the
+    // frame, the scene is copied into an output-sized target without depth, and only then is the light target drawn
+    // again with the scene depth.
+    // The G-buffer's width is the widest of the frame: the game also draws a half-size G-buffer, after which a
+    // half-size pass without depth ended the scene at some camera angles.
+    if (D.gbuf && !D.armed && !D.ended && !z->image && ex->pass_ncolor == 1 && c->image && scene_color(c->format) &&
+        c->width >= D.gbuf_w && !c->level)
+        D.nodepth = c->image;
+    if (D.nodepth && !D.armed && !D.ended && ex->pass_w && ex->pass_w * 4 <= D.gbuf_w && c->image &&
+        scene_color(c->format)) {
+        D.end_fence = ex->fence, D.end_pos = ex->pos;
+        __atomic_store_n(&D.ended, 1, __ATOMIC_RELEASE);
     }
     // the scene depth is D32F; a D16 target of the scene size also serves one composite pass before the transparents.
     // A smaller pair never takes over an armed one: the camera rune draws a half-size RGBA16F target with a half-size
     // D32F depth after the lighting (852x480 under a 1704x960 scene), and DLSS ran on that and wrote its picture into
-    // the scene's own light target, which left every object black.
-    if (D.gbuf && ex->pass_ncolor == 1 && c->image && scene_color(c->format) && z->image && z->format == VK_FORMAT_D32_SFLOAT &&
-        c->width == z->width && c->height == z->height && c->width >= DLAA_SCENE_MIN_W &&
+    // the scene's own light target, which left every object black. Nor does a pair narrower than the frame's widest
+    // G-buffer arm at all: the game draws its particles into a half-size RGBA16F target with a half-size D32F depth, and
+    // with the scope open, where the light target is drawn with a D16 depth and never arms, that effects pass was the
+    // first pair to arm; DLSS then scaled the effects into the light target, which filled the lens with garbage in the
+    // DLAA mode.
+    if (D.gbuf && !D.ended && ex->pass_ncolor == 1 && c->image && scene_color(c->format) && z->image && z->format == VK_FORMAT_D32_SFLOAT &&
+        c->width == z->width && c->height == z->height && c->width >= DLAA_SCENE_MIN_W && c->width >= D.gbuf_w &&
         (!D.armed || (c->width >= D.color.width && c->height >= D.color.height)) &&
         c->samples <= 1 && !c->is3d) {
         int first = !D.armed;
-        D.color = *c, D.depth = *z, D.armed = 1;
-        if (first) mask_snapshot(ex);   // the lit scene, before the transparents (reactive mask)
-        return;
+        D.color = *c, D.depth = *z, D.armed = 1, D.nodepth = VK_NULL_HANDLE;
+        if (!first) return 0;
+        *j = (VkExecJob){.kind = VK_JOB_DLSS_MASK, .scene = *c};   // the lit scene, before the transparents (reactive mask)
+        return 1;
     }
-    if (!D.armed) return;
+    if (!D.armed) return 0;
     if (!D.ended) {
-        if (!ex->pass_w || ex->pass_w * 4 > D.color.width) return;
-        D.ended = 1;
+        // The bloom chain's first pass ends the scene: a pass at most a quarter of the scene's width into a scene
+        // colour target. With the camera rune or the scope open the game draws a 160-wide depth-only pass and 160x90
+        // RG16F and RGBA8 passes right after the lighting, then more geometry and the transparents; DLSS ended the
+        // scene at those, the next G-buffer pass disarmed it and the scaling pass found nothing to evaluate.
+        if (!ex->pass_w || ex->pass_w * 4 > D.color.width || !c->image || !scene_color(c->format)) return 0;
+        D.end_fence = ex->fence, D.end_pos = ex->pos;
+        __atomic_store_n(&D.ended, 1, __ATOMIC_RELEASE);
         unsigned o[2], s[2];
         res_sizes(o, s);
-        if (D.color.width >= o[0]) D.done = 1, evaluate(ex, NULL);   // DLAA in place; else wait for the scaling pass
-        return;
+        if (D.color.width < o[0]) return 0;   // wait for the scaling pass
+        D.done = 1;
+        *j = (VkExecJob){.kind = VK_JOB_DLSS};   // DLAA in place
+        return 1;
     }
     // the game's scaling pass: one colour target of the output size, no depth
     unsigned o[2], s[2];
     res_sizes(o, s);
     if (ex->pass_ncolor == 1 && c->image && !z->image && scene_color(c->format) && c->width > D.color.width &&
         c->width == o[0] && c->height == o[1] && c->samples <= 1 && !c->is3d) {
-        D.done = D.up = 1;
-        evaluate(ex, c);
+        D.done = 1;
+        *j = (VkExecJob){.kind = VK_JOB_DLSS, .scene = *c};
+        return 1;
     }
+    return 0;
+}
+
+// The scan resolved its targets without making entries (vk_texture_revive); a record made on a chunk takes their images
+// now, as the scan's pointers may be stale by then.
+static void target_now(VkExecTarget *t) {
+    const VkTexImage *i = t->tex ? vk_texture_get(t->tex) : NULL;
+    if (i) t->img = i, t->image = i->image;
+}
+
+void vk_dlaa_exec(VkExec *ex, const VkExecJob *j) {
+    VkExecTarget t = j->scene;
+    if (ex->chunked) {
+        target_now(&t), target_now(&D.color), target_now(&D.depth);
+        if (D.ids_ok) target_now(&D.ids);
+    }
+    if (j->kind == VK_JOB_DLSS_MASK) return mask_snapshot(ex, &t);
+    if (!t.tex) return evaluate(ex, NULL);
+    D.up = 1;
+    evaluate(ex, &t);
+}
+
+void vk_dlaa_pass(VkExec *ex) {
+    VkExecJob j;
+    if (!ex->chunked && vk_dlaa_plan(ex, &j)) vk_dlaa_exec(ex, &j);
 }
 
 // TOTK_DLSS, read at each present (the menu switches it live): the scene size of the mode (resolution.c) and the
@@ -1196,6 +1230,7 @@ void vk_dlaa_frame(void) {
         D.on = on;
         // either way the next DLAA frame starts a new history from an unjittered frame
         D.reset = 1, D.cam_prev_ok = 0, D.jit_ok = D.jit_used = D.armed = D.done = D.gbuf = D.ended = D.up = 0;
+        D.nodepth = VK_NULL_HANDLE;
         D.hold = 0;
         return;
     }
@@ -1204,7 +1239,7 @@ void vk_dlaa_frame(void) {
     if (D.jit_used && !D.done) D.hold = 60, D.holds++;   // jittered but not resolved: that frame shook
     else if (D.hold) D.hold--;
     if (!D.done) D.jit_ok = 0;
-    D.armed = D.done = D.jit_used = D.gbuf = D.ended = D.up = D.ids_ok = 0;
+    D.armed = D.done = D.jit_used = D.gbuf = D.ended = D.up = D.ids_ok = 0, D.nodepth = VK_NULL_HANDLE;
     D.cam_addr = 0;
 }
 
@@ -1225,18 +1260,29 @@ void vk_dlaa_frame(void) {
 static const uint64_t FXAA_FS[] = {0xf450003ebeb3dc52ull, 0x0a12f4bc2a9a4fbeull, 0x5c43ae18948d52cbull,
                                    0xaad42238801d31d8ull};
 
+// Chunks of one submission look shaders up side by side: an entry's hash is stored before its shader, and entries are
+// added under the lock.
 static uint64_t spirv_hash(const GfxShader *s) {
     static struct { const GfxShader *s; uint64_t h; } tab[8192];
+    static PlatMutex lk = PLAT_MUTEX_INIT;
     uintptr_t k = ((uintptr_t)s >> 4) * 0x9E3779B97F4A7C15ull;
     for (uint32_t i = 0, j = (uint32_t)(k >> 51); i < 8192; i++, j = (j + 1) & 8191) {
-        if (tab[j].s == s) return tab[j].h;
-        if (!tab[j].s) {
+        const GfxShader *e = __atomic_load_n(&tab[j].s, __ATOMIC_ACQUIRE);
+        if (e == s) return tab[j].h;
+        if (e) continue;
+        plat_mutex_lock(&lk);
+        e = tab[j].s;
+        if (!e) {
             uint64_t words = 0, h = 0xcbf29ce484222325ull;
             const uint8_t *b = (const uint8_t *)gfx_shader_spirv(s, &words);
             for (uint64_t n = 0; b && n < words * 4; n++) h = (h ^ b[n]) * 0x100000001b3ull;
-            tab[j].s = s, tab[j].h = h;
+            tab[j].h = h;
+            __atomic_store_n(&tab[j].s, s, __ATOMIC_RELEASE);
+            plat_mutex_unlock(&lk);
             return h;
         }
+        plat_mutex_unlock(&lk);
+        if (e == s) return tab[j].h;
     }
     return 0;   // table full: no skip
 }

runtime/vk_dlaa.h

@@ -18,11 +18,17 @@ int vk_dlaa_available(void);
 // scene's light target or, at the first quarter-width pass after it, runs DLAA on that target. Each present ends the
 // frame (vk_dlaa_frame).
 typedef struct VkExec VkExec;
+typedef struct VkExecJob VkExecJob;
 void vk_dlaa_pass(VkExec *ex);
+// The same decision for a chunked replay's scan, in replay order: 1 with the record due before this pass (j), which
+// vk_dlaa_exec makes on the chunk that holds the op (vk_exec.c run_job).
+int vk_dlaa_plan(const VkExec *ex, VkExecJob *j);
+void vk_dlaa_exec(VkExec *ex, const VkExecJob *j);
 void vk_dlaa_frame(void);
 // vk_exec_draw.c, each draw before its fixed state is recorded: notes the scene camera block (vk_dlaa_draw), and moves
 // the viewport of the scene's draws by the frame's jitter (vk_dlaa_viewport, when the viewport is recorded).
 void vk_dlaa_draw(VkExec *ex);
+void vk_dlaa_scan_draw(const VkExec *ex);   // a chunked replay's scan, at each draw: the camera note of vk_dlaa_draw
 void vk_dlaa_viewport(VkExec *ex, VkViewport *v);
 // vk_mvsurvey.c (TOTK_MV_SURVEY): notes the uniform blocks of each scene draw, beside vk_dlaa_draw; each present
 // compares them with the previous frames' and logs a summary now and then.

runtime/vk_exec.h

@@ -2,10 +2,12 @@
 // handles of a submission, keeps render targets, passes, clears and fixed-function state; draws, binds and dispatches
 // (vk_exec_draw.c) and copies, counters, events and conditional rendering (vk_exec_xfer.c) plug in through the two
 // hooks at the bottom.
-// Runs on the submitting thread under vk_sync_submit's lock: no locking inside.
+// Runs on the submitting thread under vk_sync_submit's lock; a chunked replay (TOTK_REPLAY_WORKERS, vk_exec.c) records
+// on several threads, each with a VkExec of its own, and shares only the locked tables named where they are declared.
 // The op loop follows MacNativeGames nvn_mtl.m (MIT, Copyright (c) 2026 jhaizhou) and the pass handling LAS Recomp
 // (MIT, Copyright (c) 2026 Peiikai); vk_exec.c names the sources.
 #pragma once
+#include <stddef.h>
 #include <stdint.h>
 
 #include "nvn_api.h"
@@ -38,8 +40,36 @@ typedef struct {
     uint8_t scale;                       // VkTexImage.scale
 } VkExecTarget;
 
-// The replay state. One instance lives for the run: NVN state is queue state and carries over from one submission to
-// the next, as in MacNativeGames' and LAS Recomp's replays. The "per command buffer" fields are reset by vk_exec_record.
+// TOTK_TRACE=gfx: replay counters of the current frame, logged and reset at each present (vk_exec_frame_log).
+typedef struct {
+    uint64_t passes, draws_seen, draws, clears, copies, dispatches, submits;
+    uint64_t fb_draws, fb_barriers, fb_restarts, fb_layouts;   // feedback loops: draws sampling a written target of
+                                                               // their pass, in-pass barriers, pass restarts, layout
+                                                               // transitions
+} VkExecFrame;
+
+// A record the scan of a chunked replay planned before an op: a contact shadow march (vk_contact.c: vk_contact_plan
+// decides it in replay order, vk_contact_exec records it), or DLSS's copy of the lit scene or its evaluation
+// (vk_dlaa.c: vk_dlaa_plan, vk_dlaa_exec). The chunk that holds the op records it.
+enum { VK_JOB_CONTACT, VK_JOB_DLSS_MASK, VK_JOB_DLSS };
+typedef struct VkExecJob {
+    const void *op;                      // vk_exec.c: the op it precedes in a scanned submission
+    int kind;                            // VK_JOB_*
+    int chunk;                           // vk_exec.c: the chunk that holds the op
+    VkExecTarget scene, depth;           // contact: the scene colour and the linear depth; DLSS: the light target to copy
+                                         // (mask) or the scaling pass's target (evaluation; no tex: in place)
+    float pc[16];                        // the march's push constants
+} VkExecJob;
+
+// An OP_UPDATE_UB of the submission: [lo, hi) of guest memory takes data (vk_exec_draw.c ubo_ptr reads them).
+typedef struct { uint64_t lo, hi; const uint8_t *data; } VkExecUpd;
+
+struct VkExecDraw;   // vk_exec_draw.c: the memos and command buffer state of one recording context
+
+// The replay state and one recording context. NVN state is queue state and carries over from one submission to the
+// next, as in MacNativeGames' and LAS Recomp's replays. The "per command buffer" fields are reset by vk_exec_record.
+// The fields from rt up to views are that queue state (VK_EXEC_STATE_BEGIN/END); the rest belongs to the context that
+// records with it.
 typedef struct VkExec {
     // ---- per command buffer
     VkCommandBuffer cb;                  // being recorded
@@ -94,8 +124,14 @@ typedef struct VkExec {
     int render_disabled;                 // OP_RENDER_ENABLE false or a failed OP_RENDER_COND: clears and draws skip
     p_render_cond render_cond;           // last OP_RENDER_COND
     int has_render_cond;
+    // samples-passed counting (vk_exec_xfer.c): a reset since the last report, a report since the last reset, the game
+    // reported twice without a reset (every draw counts from then on)
+    int oq_counting, oq_reported, oq_keep;
 
     // ---- internal to vk_exec.c
+    VkExecFrame fr;                      // counters of this context, added to vk_exec_frame at the end of a submission
+    int split_draws, part_start;         // draws and dispatches since the last part; recorded at its start
+    uint64_t part_t0;                    // replay time at its start
     void *views;                         // image views created for this command buffer (freed at its fence)
     int nviews, cap_views;
     int barrier_at;                      // recorded right after the last full barrier (-1: none yet)
@@ -112,8 +148,30 @@ typedef struct VkExec {
     const void *fb_fs;                   // feedback_scan's memo: fragment shader, texture bind serial, pass serial
     uint32_t fb_serial, fb_pass, fb_sampled;
     uint32_t tex_serial, pass_serial;    // bumped by every texture or image bind and pass begin
+
+    // ---- vk_exec_xfer.c: the occlusion query of the open pass and the queries closed since the last report or reset
+    int oq_open;                         // slot of the active query, -1 none
+    uint32_t *oq_seg, oq_nseg, oq_capseg;
+    int oq_lost;                         // the segment missed a draw (ring full)
+
+    struct VkExecDraw *draw;             // vk_exec_draw.c, made on the context's first draw op
+
+    // ---- a chunk of a scanned submission (vk_exec.c): the scan's answers, taken in op order
+    int chunked;                         // records a chunk: what depends on earlier chunks comes from the scan
+    const VkExecUpd *upd;                // the submission's inline uniform updates
+    const uint8_t *cond;                 // its RENDER_COND results (1 renders)
+    const VkExecJob *jobs;               // its planned records (contact shadows, DLSS)
+    uint32_t njobs;
+    uint32_t upd_n, cond_i, job_i;       // updates, results and records before the op being recorded
+    uint32_t drop_n;                     // ops before it that may write uniform ranges (vk_usnap_drop_at)
+    const uint32_t *wpos;                // the positions of the draws and dispatches among those ops (vk_exec_draw_writes)
+    uint32_t wpos_i, wpos_end;           // the next of them, and the end of the chunk's
+    uint32_t pos;                        // ops of the submission up to and with it (the scan's too)
 } VkExec;
 
+#define VK_EXEC_STATE_BEGIN offsetof(VkExec, rt)
+#define VK_EXEC_STATE_END offsetof(VkExec, fr)
+
 // Begins dynamic rendering with the current targets unless a pass is open (a full memory barrier precedes it). Call
 // before a draw or an attachment clear.
 void vk_exec_pass_begin(VkExec *ex);
@@ -122,6 +180,12 @@ void vk_exec_pass_end(VkExec *ex);
 // Full memory barrier (all commands, all memory); ends the open pass first. Copies and dispatches order themselves with
 // it. Skipped when nothing was recorded since the last one.
 void vk_exec_barrier(VkExec *ex);
+// UpdateUniformBuffer overlays (vk_exec_draw.c): the copies into the real buffers that updates inside a pass left for
+// the next barrier (barrier_to records them first), and a GPU write of another kind into [addr, addr+n) of this
+// submission (a copy, a fill, a storage write), which retires the overlays it touches.
+void vk_exec_ub_flush(VkExec *ex);
+void vk_exec_ub_foreign(VkExec *ex, uint64_t addr, uint64_t n);
+void vk_exec_ub_perf(void);
 // vk_texture.c (TOTK_TEX_ALIAS): commands that must run at the replay's current position. vk_exec_inline_begin returns
 // the command buffer being recorded after a full barrier (an open pass is ended; *reopen says so), or VK_NULL_HANDLE
 // when this thread is not recording a replay. vk_exec_inline_end begins the pass again when it was open.
@@ -130,9 +194,16 @@ VkCommandBuffer vk_exec_inline_begin(int *reopen);
 void vk_exec_inline_end(int reopen);
 // vk_exec_draw.c: forget what the command buffer being recorded has bound (pipeline, sets, dynamic state, vertex
 // buffers): the replay continues in a new one (vk_sync_split).
-void vk_exec_draw_new_cb(void);
+void vk_exec_draw_new_cb(VkExec *ex);
+// The recording threads of the chunked replay as this run read TOTK_REPLAY_WORKERS (0 off), -1 before the first
+// submission read it (the settings menu shows it).
+int vk_exec_replay_workers(void);
+// vk_exec_draw.c, for the state scan of a chunked replay: the storage ranges the shaders of a draw (dispatch 0) or a
+// dispatch at state s write, as {address, size} into out; their count, or -1 when there are more than max or a stage's
+// shader is not known yet (no context has translated it). Reads uniform blocks through s->upd like a chunk.
+int vk_exec_draw_writes(const VkExec *s, int dispatch, uint64_t (*out)[2], int max);
 // vk_exec_draw.c: the vertex and fragment shader of the last draw recorded into this command buffer (NULL: none yet).
-void vk_exec_draw_shaders(const GfxShader **vs, const GfxShader **fs);
+void vk_exec_draw_shaders(const VkExec *ex, const GfxShader **vs, const GfxShader **fs);
 VkBuffer vk_exec_inline_buffer(VkDeviceSize size);
 // vk_exec.c: the stand-in a 3D BC render target is drawn through goes with its image (vk_exec_image_gone).
 void vk_exec_bc_gone(VkImage image);
@@ -149,16 +220,15 @@ int vk_exec_targets_recent(VkExec *ex, int any);
 // The current pass's targets with their formats, extents and the frames they were last rendered in (diagnostics).
 void vk_exec_targets_describe(VkExec *ex, char *buf, size_t cap);
 
-// TOTK_TRACE=gfx: replay counters of the current frame, logged and reset at each present (vk_exec_frame_log).
-typedef struct {
-    uint64_t passes, draws_seen, draws, clears, copies, dispatches, submits;
-    uint64_t fb_draws, fb_barriers, fb_restarts, fb_layouts;   // feedback loops: draws sampling a written target of
-                                                               // their pass, in-pass barriers, pass restarts, layout
-                                                               // transitions
-} VkExecFrame;
 extern VkExecFrame vk_exec_frame;
 void vk_exec_frame_written(const tex_t *t);                    // vk_texture_gpu_written: t rendered or copied into
 uint64_t vk_exec_recording_fence(void);                        // fence of the submission this thread records, else 0
+uint32_t vk_exec_replay_pos(void);                             // the op this thread records in a chunk, else UINT32_MAX
+// A chunk that is about to make a texture entry whose content depends on the textures at its storage (vk_texture.c
+// entry_of): whether a chunk before it is still being recorded, and the wait until none is. The caller holds no lock
+// but the texture lock, which it releases for the wait.
+int vk_exec_chunk_behind(void);
+void vk_exec_chunk_settle(void);
 void vk_exec_frame_log(const tex_t *presented, uint64_t frame);
 
 // TOTK_DUMP=<frame> (vk_dump.c): between presents frame-1 and frame, every pass's targets and every dispatch's storage
@@ -182,6 +252,7 @@ void vk_c0_fill(uint8_t bank[VK_C0_BYTES], int stage, const p_bind_buf ssbo[16],
 // vk_exec_draw.c: OP_DRAW, OP_DRAW_IDX, OP_DISPATCH, OP_BIND_* (program, state, vattr, vstream, vbuf, ubo, ssbo, tex,
 // samp, img), OP_UPDATE_UB, OP_DRAW_IDX_IND, OP_DISPATCH_IND. payload: nvn_op_payload of the record.
 void vk_exec_op_draw(VkExec *ex, uint32_t op, const void *payload);
+int vk_exec_bind(VkExec *ex, uint32_t op, const void *payload);   // a bind op's writes into ex; 1: VkExecDraw.bind_gen
 // The aspect a sampled view of img reads (vk_exec_draw.c get_view): stencil for a view in STENCIL depth-stencil mode.
 VkImageAspectFlags vk_exec_view_aspect(const VkTexImage *img, const view_t *v, int hv);
 VkImageView vk_exec_target_view(const VkTexImage *img, VkFormat format, VkImageViewType type, uint32_t aspect,
@@ -193,10 +264,26 @@ void vk_exec_op_xfer(VkExec *ex, uint32_t op, const void *payload);
 // sample DLSS's output vk_dlaa_swap_view in its place, so the game's scaling draw scales DLSS's picture.
 extern const VkTexImage *vk_dlaa_swap_img;
 extern VkImageView vk_dlaa_swap_view;
-// vk_contact.c: a pass about to begin (its targets resolved, no pass open), and a dispatch or copy about to be recorded
+// The state scan's part of the transfer ops: render enable, conditional rendering (returns the condition: 1 renders),
+// samples-passed counting state and the pending reports conditions read. Returns 1 for every other op.
+// The state scan of a chunked replay (vk_exec.c): the textures op uses go live in replay order (vk_texture_revive); pos:
+// the op's place in the submission. 1 when one of them has no entry yet.
+int vk_exec_xfer_revive(uint32_t op, const void *payload, uint32_t pos);
+int vk_exec_xfer_scan(VkExec *ex, uint32_t op, const void *payload);
+// The guest memory a COPY_B2B, COPY_T2B or CLEAR_BUF op makes the GPU write, as far as the scan can tell without the
+// texture's image: [*addr, *addr + return value).
+uint64_t vk_exec_xfer_written(const VkExec *ex, uint32_t op, const void *payload, uint64_t *addr);
+// vk_contact.c: a pass about to begin (its targets resolved, no pass open), and a dispatch or copy about to be recorded.
+// vk_contact_plan is what they do in replay order (pass: the first; else the second), for the scan; 1: a march is due (j).
 void vk_contact_pass_begin(VkExec *ex);
 void vk_contact_flush(VkExec *ex);
+int vk_contact_plan(const VkExec *ex, int pass, VkExecJob *j);
+void vk_contact_exec(VkExec *ex, const VkExecJob *j);
 // vk_exec_xfer.c, samples-passed counting: a draw about to be recorded in the open pass (begins an occlusion query
 // when counting and none is active), and the open pass about to end (ends the active query).
 void vk_exec_oq_draw(VkExec *ex);
 void vk_exec_oq_pass_end(VkExec *ex);
+void vk_exec_oq_forget(VkExec *ex);
+void vk_exec_oq_flush(VkExec *ex);    // the end of a chunk: its closed queries counted as a segment without a report   // vk_exec_xfer_shutdown: the context's queries belonged to the old pool
+// vk_exec.c: fn on every recording context (teardown, the device idle).
+void vk_exec_contexts(void (*fn)(VkExec *ex));

runtime/vk_exec_probe.c

@@ -968,6 +968,9 @@ int probe_vkexec(void) {
     const GfxBackend *b = &gfx_vulkan_backend;
     uint64_t heap = 0;
     CHECK(mem_set_heap_size(16ull << 20, &heap) == 0 && heap, "SetHeapSize 16 MiB");
+    // Out-of-memory waits (oom.c): the first two staging buffers, the first texture image and the first draw-data chunk
+    // fail; every part below passes through the wait or the retry that follows.
+    rt_oom_force(RT_OOM_STAGING, 2), rt_oom_force(RT_OOM_IMAGE, 1), rt_oom_force(RT_OOM_CHUNK, 1);
     uint8_t *guest = (uint8_t *)(uintptr_t)heap;
     uint64_t first = 0, last = 0;
     int w = 0, h = 0, inside = 0, xfer_bytes = 0;
@@ -989,6 +992,12 @@ int probe_vkexec(void) {
     vk_exec_xfer_shutdown();
     vk_backend_shutdown();
     long errors = vk_validation_errors_total();
+    int oom_left[3] = {rt_oom_forced_left(RT_OOM_STAGING), rt_oom_forced_left(RT_OOM_IMAGE), rt_oom_forced_left(RT_OOM_CHUNK)};
+    if (!bad && !errors && (oom_left[0] || oom_left[1] || oom_left[2])) {
+        printf("probe-vkexec: FAILED (forced allocation failures never reached: staging %d, image %d, chunk %d)\n",
+               oom_left[0], oom_left[1], oom_left[2]);
+        return 1;
+    }
     if (bad || errors) {
         printf("probe-vkexec: FAILED (%s, %ld validation errors)\n",
                sync_bad ? "a sync check failed" : draw_bad ? "a draw check failed" : bad ? "a clear or xfer check failed" : "checks passed", errors);
@@ -1034,5 +1043,7 @@ int probe_vkexec(void) {
                "finished mark is the last pass, ring records exact); the report reads:\n%s\n", marks_text);
     printf("probe-vkexec: empty pool part passed (a zero-byte pool at the start of another: no Vulkan objects, lookups "
            "there find the other pool before and after its Finalize)\n");
+    printf("probe-vkexec: out-of-memory part passed (2 failed staging buffers waited for, 1 failed texture image retried, "
+           "1 failed draw-data chunk waited for)\n");
     return 0;
 }

runtime/vk_exec_xfer.c

@@ -72,7 +72,8 @@ static void guest_written_cb(void *arg) {
 
 // The GPU wrote [addr, addr+size) of CPU-visible memory: tell the texture cache once the fence is reached.
 static void note_gpu_write(VkExec *ex, uint64_t addr, uint64_t size) {
-    vk_usnap_drop(addr, size);
+    if (!ex->chunked) vk_usnap_drop(addr, size);   // a chunk: the scan dropped it in its place
+    vk_exec_ub_foreign(ex, addr, size);
     if (nvn_gpu_only(addr)) return;
     Range *r = malloc(sizeof *r);
     if (!r) rt_fatal("vk: out of memory");
@@ -128,13 +129,6 @@ static int oq_state;                     // 0 not set up, 1 counting available,
 static VkQueryControlFlags oq_flags;
 static volatile uint8_t oq_busy[OQ_N];   // from begin until the completion thread read and reset the slot
 static uint32_t oq_next;                 // replay: next ring slot
-static int oq_counting;                  // replay: draws are counted (a reset since the last report, or oq_keep)
-static int oq_reported;                  // replay: a samples-passed report since the last reset
-static int oq_keep;                      // replay: the game reported twice without a reset: count every draw
-static int oq_open = -1;                 // replay: the slot whose query is active in the open pass
-static uint32_t *oq_seg;                 // replay: slots closed since the last report or reset
-static uint32_t oq_nseg, oq_capseg;
-static int oq_lost;                      // replay: the segment missed a draw (ring full)
 static uint64_t oq_count;                // completion thread: samples since the last reset
 
 typedef struct { uint64_t addr; int reset, lost; uint32_t n; uint32_t q[]; } OqSegment;
@@ -165,39 +159,49 @@ static int oq_ready(void) {
 }
 
 static void oq_close(VkExec *ex) {
-    if (oq_open < 0) return;
-    vkCmdEndQuery(ex->cb, oq_pool, (uint32_t)oq_open);
+    if (ex->oq_open < 0) return;
+    vkCmdEndQuery(ex->cb, oq_pool, (uint32_t)ex->oq_open);
     ex->recorded++;
-    if (oq_nseg == oq_capseg) {
-        oq_capseg = oq_capseg ? oq_capseg * 2 : 64;
-        oq_seg = realloc(oq_seg, sizeof *oq_seg * oq_capseg);
-        if (!oq_seg) rt_fatal("vk: out of memory");
+    if (ex->oq_nseg == ex->oq_capseg) {
+        ex->oq_capseg = ex->oq_capseg ? ex->oq_capseg * 2 : 64;
+        ex->oq_seg = realloc(ex->oq_seg, sizeof *ex->oq_seg * ex->oq_capseg);
+        if (!ex->oq_seg) rt_fatal("vk: out of memory");
     }
-    oq_seg[oq_nseg++] = (uint32_t)oq_open;
-    oq_open = -1;
+    ex->oq_seg[ex->oq_nseg++] = (uint32_t)ex->oq_open;
+    ex->oq_open = -1;
 }
 
 void vk_exec_oq_draw(VkExec *ex) {
-    if (!oq_counting || oq_open >= 0) return;
-    uint32_t q = oq_next % OQ_N;
-    if (__atomic_load_n(&oq_busy[q], __ATOMIC_ACQUIRE)) {
+    if (!ex->oq_counting || ex->oq_open >= 0) return;
+    uint32_t q = __atomic_fetch_add(&oq_next, 1, __ATOMIC_RELAXED) % OQ_N;   // threads recording chunks take slots too
+    if (__atomic_exchange_n(&oq_busy[q], 1, __ATOMIC_ACQUIRE)) {
         warn_once(W_QUERY, "occlusion query ring full: draws left uncounted");
-        oq_lost = 1;
+        ex->oq_lost = 1;
         return;
     }
-    oq_busy[q] = 1;
-    oq_next++;
     vkCmdBeginQuery(ex->cb, oq_pool, q, oq_flags);
     ex->recorded++;
-    oq_open = (int)q;
+    ex->oq_open = (int)q;
 }
 
 void vk_exec_oq_pass_end(VkExec *ex) { oq_close(ex); }
 
+static void oq_segment(VkExec *ex, uint64_t addr, int reset);
+
+// The queries a chunk closed: the completion thread adds them to the count before the next chunk's (vk_exec.c).
+void vk_exec_oq_flush(VkExec *ex) {
+    if (ex->oq_nseg || ex->oq_lost) oq_segment(ex, 0, 0);
+}
+
+void vk_exec_oq_forget(VkExec *ex) {
+    ex->oq_counting = ex->oq_reported = ex->oq_keep = 0, ex->oq_open = -1;
+    ex->oq_nseg = 0, ex->oq_lost = 0;
+}
+
 void vk_exec_xfer_shutdown(void) {
     if (oq_state > 0) vkDestroyQueryPool(vk_ctx()->device, oq_pool, NULL);
-    oq_pool = VK_NULL_HANDLE, oq_state = 0, oq_next = 0, oq_counting = oq_reported = oq_keep = 0, oq_open = -1;
-    oq_nseg = 0, oq_lost = 0, oq_count = 0;
+    oq_pool = VK_NULL_HANDLE, oq_state = 0, oq_next = 0, oq_count = 0;
+    vk_exec_contexts(vk_exec_oq_forget);
     memset((void *)oq_busy, 0, sizeof oq_busy);
 }
 
@@ -239,30 +243,34 @@ static void oq_cb(void *arg) {
 // Hands the segment to the completion thread with a report (addr; 0: no CPU-visible memory) or a reset.
 static void oq_segment(VkExec *ex, uint64_t addr, int reset) {
     oq_close(ex);
-    OqSegment *s = malloc(sizeof *s + sizeof *s->q * oq_nseg);
+    OqSegment *s = malloc(sizeof *s + sizeof *s->q * ex->oq_nseg);
     if (!s) rt_fatal("vk: out of memory");
-    s->addr = addr, s->reset = reset, s->lost = oq_lost, s->n = oq_nseg;
-    memcpy(s->q, oq_seg, sizeof *s->q * oq_nseg);
-    oq_nseg = 0, oq_lost = 0;
+    s->addr = addr, s->reset = reset, s->lost = ex->oq_lost, s->n = ex->oq_nseg;
+    memcpy(s->q, ex->oq_seg, sizeof *s->q * ex->oq_nseg);
+    ex->oq_nseg = 0, ex->oq_lost = 0;
     vk_completion_add(ex->fence, oq_cb, s);
 }
 
 static void reset_counter(VkExec *ex, const p_reset_counter *p) {
     if (p->type != NVN_COUNTER_TYPE_SAMPLES_PASSED || !oq_ready()) return;
     oq_segment(ex, 0, 1);
-    oq_counting = 1, oq_reported = 0;
+    ex->oq_counting = 1, ex->oq_reported = 0;
 }
 
 // A samples-passed report: the segment goes with it (addr 0: no CPU-visible memory) and counting stops until the next
 // reset, unless reports came without resets before.
+static void oq_report_state(VkExec *ex) {
+    if (ex->oq_reported && !ex->oq_keep) {
+        ex->oq_keep = 1;
+        LOG_ONCE("vk: samples passed reported twice without a reset: counting every draw from now on");
+    }
+    ex->oq_reported = 1;
+    if (!ex->oq_keep) ex->oq_counting = 0;
+}
+
 static void oq_report(VkExec *ex, uint64_t addr) {
     oq_segment(ex, addr, 0);
-    if (oq_reported && !oq_keep) {
-        oq_keep = 1;
-        rt_log("vk: samples passed reported twice without a reset: counting every draw from now on");
-    }
-    oq_reported = 1;
-    if (!oq_keep) oq_counting = 0;
+    oq_report_state(ex);
 }
 
 static void signal_event(VkExec *ex, const p_signal_event *p) {
@@ -289,7 +297,7 @@ static void report_counter(VkExec *ex, const p_report_counter *p) {
         *a = p->addr;
         vk_completion_add(ex->fence, counter_cb, a);
     }
-    pending[pending_head++ % PENDING] = (typeof(pending[0])){p->addr, ex->fence, samples};
+    if (!ex->chunked) pending[pending_head++ % PENDING] = (typeof(pending[0])){p->addr, ex->fence, samples};   // else the scan's
 }
 
 // ---- conditional rendering
@@ -611,8 +619,77 @@ void vk_exec_op_xfer(VkExec *ex, uint32_t op, const void *payload) {
     case OP_RENDER_COND:
         ex->render_cond = *(const p_render_cond *)payload;
         ex->has_render_cond = 1;
-        ex->render_disabled = !cond_enabled(payload);
+        ex->render_disabled = !(ex->chunked ? ex->cond[ex->cond_i++] : cond_enabled(payload));
         break;
     default: rt_fatal("vk: op %s is not a transfer", nvn_op_name(op));
     }
 }
+
+static int revive_new(const tex_t *t, uint32_t pos) { return t && t->mem && !vk_texture_revive(t, pos); }
+
+int vk_exec_xfer_revive(uint32_t op, const void *payload, uint32_t pos) {
+    if (op == OP_COPY_T2T) {
+        int n = revive_new(((const p_copy_t2t *)payload)->src, pos);
+        return revive_new(((const p_copy_t2t *)payload)->dst, pos) | n;
+    }
+    if (op == OP_COPY_B2T || op == OP_COPY_T2B) return revive_new(((const p_copy_b2t *)payload)->tex, pos);
+    if (op == OP_CLEAR_TEX) return revive_new(((const p_clear_tex *)payload)->tex, pos);
+    return 0;
+}
+
+uint64_t vk_exec_xfer_written(const VkExec *ex, uint32_t op, const void *payload, uint64_t *addr) {
+    VkMemSpan m;
+    if (op == OP_COPY_B2B) {
+        const p_copy_b2b *p = payload;
+        *addr = p->dst;
+        return vk_mem_resolve(p->src, 1, &m) && vk_mem_resolve(p->dst, 1, &m) ? p->size : 0;
+    }
+    if (op == OP_CLEAR_BUF) {
+        const p_clear_buf *p = payload;
+        *addr = p->addr;
+        return (p->addr | p->size) & 3 || !vk_mem_resolve(p->addr, 1, &m) ? 0 : p->size;
+    }
+    // COPY_T2B: the buffer side of copy_buffer_image from the texture descriptor (sub_resolve, buffer_side)
+    const p_copy_b2t *p = payload;
+    const tex_t *t = p->tex;
+    int bw, bh, bpb = t ? nvn_fmt_bpb(t->d.format, &bw, &bh) : 0;
+    *addr = p->src;
+    if (!bpb) return 0;
+    uint32_t level = p->has_view && p->view.set_levels ? p->view.base_level : 0;
+    uint32_t lw = max1((uint32_t)t->d.width >> level), lh = max1((uint32_t)t->d.height >> level);
+    const NVNcopyRegion *r = &p->r;
+    uint32_t w = r->width > 0 ? (uint32_t)r->width : r->xoffset < (int32_t)lw ? lw - (uint32_t)r->xoffset : 0;
+    uint32_t h = r->height > 0 ? (uint32_t)r->height : r->yoffset < (int32_t)lh ? lh - (uint32_t)r->yoffset : 0;
+    uint32_t nz = r->depth > 0 ? (uint32_t)r->depth : 1;
+    uint32_t wb = (w + (uint32_t)bw - 1) / (uint32_t)bw, hb = (h + (uint32_t)bh - 1) / (uint32_t)bh;
+    if (!wb || !hb) return 0;
+    uint64_t row = ex->copy_stride > 0 ? (uint64_t)ex->copy_stride : (uint64_t)wb * (uint64_t)bpb;
+    return row * hb * (nz - 1) + row * (hb - 1) + (uint64_t)wb * (uint64_t)bpb;
+}
+
+int vk_exec_xfer_scan(VkExec *ex, uint32_t op, const void *payload) {
+    switch (op) {
+    case OP_RENDER_ENABLE: ex->render_disabled = !((const p_flag *)payload)->value; break;
+    case OP_RENDER_COND: {
+        int on = cond_enabled(payload);
+        ex->render_cond = *(const p_render_cond *)payload;
+        ex->has_render_cond = 1;
+        ex->render_disabled = !on;
+        return on;
+    }
+    case OP_RESET_COUNTER:
+        if (((const p_reset_counter *)payload)->type == NVN_COUNTER_TYPE_SAMPLES_PASSED && oq_ready())
+            ex->oq_counting = 1, ex->oq_reported = 0;
+        break;
+    case OP_REPORT_COUNTER: {
+        const p_report_counter *p = payload;
+        int samples = p->type == NVN_COUNTER_TYPE_SAMPLES_PASSED && oq_ready();
+        VkMemSpan m;
+        if (!nvn_gpu_only(p->addr) && !vk_mem_resolve(p->addr, sizeof(NVNcounterData), &m)) break;   // the recording stops
+        if (samples) oq_report_state(ex);
+        if (!nvn_gpu_only(p->addr)) pending[pending_head++ % PENDING] = (typeof(pending[0])){p->addr, ex->fence, samples};
+        break;
+    }
+    }
+    return 1;
+}

runtime/vk_hdr.c

@@ -243,7 +243,9 @@ void vk_hdr_shader_swap(uint32_t **spirv, uint64_t *words) {
 int vk_hdr_track;                 // vk_exec_draw.c calls vk_hdr_draw: set at the first present with TOTK_HDR=1
 static volatile int32_t g_active;   // the swapchain is HDR10 (gfx_present.c vk_hdr_set_active)
 static PlatMutex rec_lock = PLAT_MUTEX_INIT;
-static struct { VkImage win, src; VkFormat fmt; } recs[4];   // per presented window image: what the last draw copied
+// Per presented window image: what the last draw in replay order copied, and where that draw is. Chunks of a replay
+// record at once, so a draw of an earlier chunk can get here after one of a later chunk.
+static struct { VkImage win, src; VkFormat fmt; uint64_t fence; uint32_t pos; } recs[4];
 static int nrecs;
 
 void vk_hdr_set_active(int on) { plat_xchg32(&g_active, on); }
@@ -272,12 +274,13 @@ void vk_hdr_draw(VkExec *ex, const VkPipeInfo *info) {
     }
     if (ntex == 1 && slot >= 0 && slot < 128 && ex->tex[1][slot]) {
         const VkTexImage *s = vk_texture_by_id(NVN_HANDLE_TEX(ex->tex[1][slot]), NULL, NULL);
-        if (s && is_float(s->format) && s->width == t->width && s->height == t->height && s->samples == 1)
+        if (s && !s->gone && is_float(s->format) && s->width == t->width && s->height == t->height && s->samples == 1)
             src = s->image, fmt = s->format;
     }
     plat_mutex_lock(&rec_lock);
     for (int i = 0; i < nrecs; i++)
-        if (recs[i].win == t->image) recs[i].src = src, recs[i].fmt = fmt;
+        if (recs[i].win == t->image && (ex->fence > recs[i].fence || (ex->fence == recs[i].fence && ex->pos >= recs[i].pos)))
+            recs[i].src = src, recs[i].fmt = fmt, recs[i].fence = ex->fence, recs[i].pos = ex->pos;
     plat_mutex_unlock(&rec_lock);
 }
 

runtime/vk_headroom.c

@@ -12,7 +12,9 @@
 // a block per memory type is held back.
 // Under VRAM pressure no headroom is kept: the spare block counts against the budget that decides whether a pool gets
 // its VRAM mirror (vk_memory.c), so while use plus two blocks passes 85% of the budget the anchor goes and nothing new
-// is allocated.
+// is allocated. The same holds when the system runs short of memory (room), and for HR_HOLD_MS after an allocation
+// failed anywhere (vk_headroom_release).
+// The texture pool (vk_memory.c vk_texture_image) is kept ahead the same way, with its own anchor of half its block.
 // TOTK_VRAM_HEADROOM=0 turns it off.
 
 #include "cfg.h"
@@ -22,6 +24,7 @@
 
 #define HR_PERIOD_MS 500
 #define HR_SMALL_HEAP (1ull << 30)   // VMA_SMALL_HEAP_MAX_SIZE: smaller heaps use blocks of a heap eighth
+#define HR_HOLD_MS 30000u
 
 static struct {
     int on, stop;
@@ -33,6 +36,11 @@ static struct {
     volatile int64_t made;     // blocks this thread brought in
     volatile int64_t other;    // device memory of 32 MiB or more other threads allocated (VMA blocks, large dedicated)
     VmaAllocation anchor[64]; // per memory type: the half block that keeps the newest spare block from being deleted
+    VmaPool pool;             // the texture pool, its memory type and block size, and its anchor
+    uint32_t pool_type;
+    VkDeviceSize pool_block;
+    VmaAllocation pool_anchor;
+    volatile uint64_t hold_until;   // plat_now_ms before which no spare block is kept (vk_headroom_release)
 } H;
 
 static void VKAPI_PTR on_allocate(VmaAllocator a, uint32_t type, VkDeviceMemory m, VkDeviceSize size, void *user) {
@@ -55,29 +63,36 @@ const VmaDeviceMemoryCallbacks *vk_headroom_callbacks(void) {
 
 static VkDeviceSize block_size(const VkCtx *c, uint32_t type) {
     VkDeviceSize heap = c->mem_props.memoryHeaps[c->mem_props.memoryTypes[type].heapIndex].size;
-    return heap <= HR_SMALL_HEAP ? heap / 8 : 256ull << 20;   // VMA's default preferredLargeHeapBlockSize
+    return heap <= HR_SMALL_HEAP ? heap / 8 : c->vma_block;   // VMA's preferredLargeHeapBlockSize
 }
 
+// The spare block is memory the game does not use yet; where device memory takes host memory it is kept only while the
+// system has four blocks to spare beyond the reserve (vk_mem_host_room). On the RX 9070 the spare blocks held 1.4 GB
+// of commit.
 static int room(const VkCtx *c, uint32_t type, VkDeviceSize block) {
     VmaBudget b[VK_MAX_MEMORY_HEAPS];
     vmaGetHeapBudgets(c->vma, b);
     const VmaBudget *h = &b[c->mem_props.memoryTypes[type].heapIndex];
-    return h->budget && h->usage + 2 * block <= h->budget / 100 * 85;
+    return h->budget && h->usage + 2 * block <= h->budget / 100 * 85 && vk_mem_host_room() >= 4 * block;
 }
 
-static void drop_anchor(const VkCtx *c, uint32_t type) {
-    if (H.anchor[type]) vmaFreeMemory(c->vma, H.anchor[type]);
-    H.anchor[type] = NULL;
+static void drop_anchor(const VkCtx *c, VmaAllocation *anchor) {
+    if (*anchor) vmaFreeMemory(c->vma, *anchor);
+    *anchor = NULL;
 }
 
-static void top_up(const VkCtx *c, uint32_t type) {
-    VkDeviceSize block = block_size(c, type);
-    if (!room(c, type, block)) {
-        drop_anchor(c, type);
+// pool NULL: the default pool of the type
+static void top_up(const VkCtx *c, uint32_t type, VmaPool pool, VkDeviceSize block, VmaAllocation *anchor) {
+    // Device-local types only: textures and mirrors grow there. Host memory types are woken by the staging pool's own
+    // blocks (vk_memory.c vk_staging), while their default pool held 4 MiB: on the RX 9070 the spare blocks of types 1
+    // and 3 held 512 MiB of commit for nothing.
+    if (!(c->mem_props.memoryTypes[type].propertyFlags & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT) ||
+        plat_now_ms() < __atomic_load_n(&H.hold_until, __ATOMIC_RELAXED) || !room(c, type, block)) {
+        drop_anchor(c, anchor);
         return;
     }
     VkMemoryRequirements req = {.size = block / 2, .alignment = 256, .memoryTypeBits = 1u << type};
-    VmaAllocationCreateInfo ai = {.usage = VMA_MEMORY_USAGE_UNKNOWN};
+    VmaAllocationCreateInfo ai = {.usage = VMA_MEMORY_USAGE_UNKNOWN, .pool = pool};
     VmaAllocation a;
     int64_t before = H.made;
     if (vmaAllocateMemory(c->vma, &req, &ai, &a, NULL) != VK_SUCCESS) return;
@@ -85,12 +100,13 @@ static void top_up(const VkCtx *c, uint32_t type) {
         vmaFreeMemory(c->vma, a);
         return;
     }
-    drop_anchor(c, type);
-    H.anchor[type] = a;
+    drop_anchor(c, anchor);
+    *anchor = a;
     static volatile int32_t logged;
     if (plat_inc32(&logged) <= 32)
-        rt_log("vk: headroom: a %.0f MiB block of memory type %u allocated ahead (%lld so far; %lld allocations of 32 MiB "
-               "or more on other threads)", (double)block / 1048576.0, type, (long long)H.made, (long long)H.other);
+        rt_log("vk: headroom: a %.0f MiB block of memory type %u%s allocated ahead (%lld so far; %lld allocations of "
+               "32 MiB or more on other threads)", (double)block / 1048576.0, type, pool ? " (texture pool)" : "",
+               (long long)H.made, (long long)H.other);
 }
 
 static void headroom_main(void *arg) {
@@ -105,10 +121,15 @@ static void headroom_main(void *arg) {
         plat_mutex_lock(&H.lock);
         const VkCtx *c = H.stop ? NULL : vk_ctx();
         for (uint32_t t = 0; c && t < c->mem_props.memoryTypeCount && t < 64; t++)
-            if (watch >> t & 1) top_up(c, t);
+            if (watch >> t & 1) {
+                top_up(c, t, NULL, block_size(c, t), &H.anchor[t]);
+                if (H.pool && t == H.pool_type) top_up(c, t, H.pool, H.pool_block, &H.pool_anchor);
+            }
         int stop = H.stop;
-        if (stop && (c = vk_ctx()))
-            for (uint32_t t = 0; t < 64; t++) drop_anchor(c, t);
+        if (stop && (c = vk_ctx())) {
+            for (uint32_t t = 0; t < 64; t++) drop_anchor(c, &H.anchor[t]);
+            drop_anchor(c, &H.pool_anchor);
+        }
         plat_mutex_unlock(&H.lock);
         if (stop) return;
     }
@@ -120,11 +141,38 @@ void vk_headroom_stop(void) {
     plat_mutex_lock(&H.lock);
     H.stop = 1;
     const VkCtx *c = vk_ctx();
-    for (uint32_t t = 0; c && t < 64; t++) drop_anchor(c, t);
+    for (uint32_t t = 0; c && t < 64; t++) drop_anchor(c, &H.anchor[t]);
+    if (c) drop_anchor(c, &H.pool_anchor);
     plat_mutex_unlock(&H.lock);
     plat_event_set(&H.ev);
 }
 
+void vk_headroom_pool(VmaPool pool, uint32_t type, VkDeviceSize block) {
+    if (!H.started) return;
+    plat_mutex_lock(&H.lock);
+    const VkCtx *c = vk_ctx();
+    if (c) drop_anchor(c, &H.pool_anchor);
+    H.pool = pool, H.pool_type = type, H.pool_block = block;
+    plat_mutex_unlock(&H.lock);
+}
+
+// After a failed allocation: the spare blocks go at once (an empty VMA block takes the next allocation of its type or
+// is given back), and none is kept for HR_HOLD_MS. H.lock is held only around VMA calls, so the caller may hold any lock
+// of ours, just no VMA lock.
+void vk_headroom_release(void) {
+    __atomic_store_n(&H.hold_until, plat_now_ms() + HR_HOLD_MS, __ATOMIC_RELAXED);
+    if (!H.started) return;
+    int n = 0;
+    plat_mutex_lock(&H.lock);
+    const VkCtx *c = H.stop ? NULL : vk_ctx();
+    for (uint32_t t = 0; c && t < 64; t++)
+        if (H.anchor[t]) drop_anchor(c, &H.anchor[t]), n++;
+    if (c && H.pool_anchor) drop_anchor(c, &H.pool_anchor), n++;
+    plat_mutex_unlock(&H.lock);
+    static volatile int32_t logged;
+    if (n && plat_inc32(&logged) <= 8) rt_log("vk: headroom: %d spare blocks freed after a failed allocation", n);
+}
+
 void vk_headroom_start(void) {
     if (!H.on || H.started) return;
     plat_mutex_init(&H.lock);

runtime/vk_mvobj_fs.h

@@ -1,7 +1,7 @@
 // Generated by tools/make_mvobj_fs.py: the per-object motion fragment shader (vk_mvobj.c). Do not edit.
 #define MVOBJ_FS_LOCATION_WORD 24   // the input Location literal, patched to the variant's
 static const uint32_t mvobj_fs_spv[] = {
-    0x07230203, 0x00010000, 0x00000000, 0x00000034, 0x00000000, 0x00020011, 0x00000001, 0x0003000e,
+    0x07230203, 0x00010000, 0x00000000, 0x00000036, 0x00000000, 0x00020011, 0x00000001, 0x0003000e,
     0x00000000, 0x00000001, 0x0008000f, 0x00000004, 0x00000001, 0x6e69616d, 0x00000000, 0x0000000e,
     0x0000000f, 0x00000010, 0x00030010, 0x00000001, 0x00000007, 0x00040047, 0x0000000e, 0x0000001e,
     0x00000000, 0x00040047, 0x0000000f, 0x0000000b, 0x0000000f, 0x00040047, 0x00000010, 0x0000001e,
@@ -9,31 +9,33 @@ static const uint32_t mvobj_fs_spv[] = {
     0x00000000, 0x00040047, 0x00000011, 0x00000022, 0x00000000, 0x00040047, 0x00000011, 0x00000021,
     0x00000001, 0x00020013, 0x00000002, 0x00030021, 0x00000003, 0x00000002, 0x00030016, 0x00000004,
     0x00000020, 0x00040017, 0x00000005, 0x00000004, 0x00000002, 0x00040017, 0x00000006, 0x00000004,
-    0x00000004, 0x00040015, 0x00000007, 0x00000020, 0x00000000, 0x00020014, 0x00000008, 0x00040020,
-    0x00000009, 0x00000001, 0x00000006, 0x00040020, 0x0000000a, 0x00000003, 0x00000006, 0x0003001e,
-    0x0000000b, 0x00000006, 0x00040020, 0x0000000c, 0x00000002, 0x0000000b, 0x00040020, 0x00000017,
-    0x00000002, 0x00000006, 0x0004002b, 0x00000004, 0x00000012, 0x3f000000, 0x0004002b, 0x00000004,
-    0x00000013, 0x00000000, 0x0004002b, 0x00000004, 0x00000014, 0x358637bd, 0x0004002b, 0x00000007,
-    0x00000015, 0x00000000, 0x0007002c, 0x00000006, 0x00000016, 0x00000013, 0x00000013, 0x00000013,
-    0x00000013, 0x0004003b, 0x00000009, 0x0000000e, 0x00000001, 0x0004003b, 0x00000009, 0x0000000f,
-    0x00000001, 0x0004003b, 0x0000000a, 0x00000010, 0x00000003, 0x0004003b, 0x0000000c, 0x00000011,
-    0x00000002, 0x00050036, 0x00000002, 0x00000001, 0x00000000, 0x00000003, 0x000200f8, 0x00000018,
-    0x00050041, 0x00000017, 0x00000019, 0x00000011, 0x00000015, 0x0004003d, 0x00000006, 0x0000001a,
-    0x00000019, 0x0004003d, 0x00000006, 0x0000001b, 0x0000000e, 0x0004003d, 0x00000006, 0x0000001c,
-    0x0000000f, 0x0007004f, 0x00000005, 0x0000001d, 0x0000001c, 0x0000001c, 0x00000000, 0x00000001,
-    0x0007004f, 0x00000005, 0x0000001e, 0x0000001a, 0x0000001a, 0x00000002, 0x00000003, 0x00050083,
-    0x00000005, 0x0000001f, 0x0000001d, 0x0000001e, 0x00050051, 0x00000004, 0x00000020, 0x0000001b,
-    0x00000000, 0x00050051, 0x00000004, 0x00000021, 0x0000001b, 0x00000001, 0x00050051, 0x00000004,
-    0x00000022, 0x0000001b, 0x00000003, 0x00050088, 0x00000004, 0x00000023, 0x00000020, 0x00000022,
-    0x00050088, 0x00000004, 0x00000024, 0x00000021, 0x00000022, 0x00050051, 0x00000004, 0x00000025,
-    0x0000001a, 0x00000000, 0x00050051, 0x00000004, 0x00000026, 0x0000001a, 0x00000001, 0x00050085,
-    0x00000004, 0x00000027, 0x00000023, 0x00000012, 0x00050081, 0x00000004, 0x00000028, 0x00000027,
-    0x00000012, 0x00050085, 0x00000004, 0x00000029, 0x00000028, 0x00000025, 0x00050085, 0x00000004,
-    0x0000002a, 0x00000024, 0x00000012, 0x00050083, 0x00000004, 0x0000002b, 0x00000012, 0x0000002a,
-    0x00050085, 0x00000004, 0x0000002c, 0x0000002b, 0x00000026, 0x00050051, 0x00000004, 0x0000002d,
-    0x0000001f, 0x00000000, 0x00050051, 0x00000004, 0x0000002e, 0x0000001f, 0x00000001, 0x00050083,
-    0x00000004, 0x0000002f, 0x00000029, 0x0000002d, 0x00050083, 0x00000004, 0x00000030, 0x0000002c,
-    0x0000002e, 0x00070050, 0x00000006, 0x00000031, 0x0000002f, 0x00000030, 0x00000013, 0x00000013,
-    0x000500ba, 0x00000008, 0x00000032, 0x00000022, 0x00000014, 0x000600a9, 0x00000006, 0x00000033,
-    0x00000032, 0x00000031, 0x00000016, 0x0003003e, 0x00000010, 0x00000033, 0x000100fd, 0x00010038
+    0x00000004, 0x00040015, 0x00000007, 0x00000020, 0x00000000, 0x00020014, 0x00000008, 0x00040017,
+    0x00000018, 0x00000008, 0x00000004, 0x00040020, 0x00000009, 0x00000001, 0x00000006, 0x00040020,
+    0x0000000a, 0x00000003, 0x00000006, 0x0003001e, 0x0000000b, 0x00000006, 0x00040020, 0x0000000c,
+    0x00000002, 0x0000000b, 0x00040020, 0x00000017, 0x00000002, 0x00000006, 0x0004002b, 0x00000004,
+    0x00000012, 0x3f000000, 0x0004002b, 0x00000004, 0x00000013, 0x00000000, 0x0004002b, 0x00000004,
+    0x00000014, 0x358637bd, 0x0004002b, 0x00000007, 0x00000015, 0x00000000, 0x0007002c, 0x00000006,
+    0x00000016, 0x00000013, 0x00000013, 0x00000013, 0x00000013, 0x0004003b, 0x00000009, 0x0000000e,
+    0x00000001, 0x0004003b, 0x00000009, 0x0000000f, 0x00000001, 0x0004003b, 0x0000000a, 0x00000010,
+    0x00000003, 0x0004003b, 0x0000000c, 0x00000011, 0x00000002, 0x00050036, 0x00000002, 0x00000001,
+    0x00000000, 0x00000003, 0x000200f8, 0x00000019, 0x00050041, 0x00000017, 0x0000001a, 0x00000011,
+    0x00000015, 0x0004003d, 0x00000006, 0x0000001b, 0x0000001a, 0x0004003d, 0x00000006, 0x0000001c,
+    0x0000000e, 0x0004003d, 0x00000006, 0x0000001d, 0x0000000f, 0x0007004f, 0x00000005, 0x0000001e,
+    0x0000001d, 0x0000001d, 0x00000000, 0x00000001, 0x0007004f, 0x00000005, 0x0000001f, 0x0000001b,
+    0x0000001b, 0x00000002, 0x00000003, 0x00050083, 0x00000005, 0x00000020, 0x0000001e, 0x0000001f,
+    0x00050051, 0x00000004, 0x00000021, 0x0000001c, 0x00000000, 0x00050051, 0x00000004, 0x00000022,
+    0x0000001c, 0x00000001, 0x00050051, 0x00000004, 0x00000023, 0x0000001c, 0x00000003, 0x00050088,
+    0x00000004, 0x00000024, 0x00000021, 0x00000023, 0x00050088, 0x00000004, 0x00000025, 0x00000022,
+    0x00000023, 0x00050051, 0x00000004, 0x00000026, 0x0000001b, 0x00000000, 0x00050051, 0x00000004,
+    0x00000027, 0x0000001b, 0x00000001, 0x00050085, 0x00000004, 0x00000028, 0x00000024, 0x00000012,
+    0x00050081, 0x00000004, 0x00000029, 0x00000028, 0x00000012, 0x00050085, 0x00000004, 0x0000002a,
+    0x00000029, 0x00000026, 0x00050085, 0x00000004, 0x0000002b, 0x00000025, 0x00000012, 0x00050083,
+    0x00000004, 0x0000002c, 0x00000012, 0x0000002b, 0x00050085, 0x00000004, 0x0000002d, 0x0000002c,
+    0x00000027, 0x00050051, 0x00000004, 0x0000002e, 0x00000020, 0x00000000, 0x00050051, 0x00000004,
+    0x0000002f, 0x00000020, 0x00000001, 0x00050083, 0x00000004, 0x00000030, 0x0000002a, 0x0000002e,
+    0x00050083, 0x00000004, 0x00000031, 0x0000002d, 0x0000002f, 0x00070050, 0x00000006, 0x00000032,
+    0x00000030, 0x00000031, 0x00000013, 0x00000013, 0x000500ba, 0x00000008, 0x00000033, 0x00000023,
+    0x00000014, 0x00070050, 0x00000018, 0x00000034, 0x00000033, 0x00000033, 0x00000033, 0x00000033,
+    0x000600a9, 0x00000006, 0x00000035, 0x00000034, 0x00000032, 0x00000016, 0x0003003e, 0x00000010,
+    0x00000035, 0x000100fd, 0x00010038
 };

runtime/vk_mvsurvey.c

@@ -157,10 +157,10 @@ void vk_mvsurvey_draw(VkExec *ex) {
         k->size = (uint32_t)u->size;
         uint32_t want = u->size < SNAP ? (uint32_t)u->size : SNAP;
         VkDescriptorBufferInfo bi;
-        const uint8_t *src = vk_usnap_find(u->addr, want, 1, &bi) ? vk_usnap_map() + bi.offset : NULL;
+        const uint8_t *src = vk_usnap_find(u->addr, want, UINT32_MAX, 1, &bi) ? vk_usnap_map() + bi.offset : NULL;
         k->have = src ? want : 0;
         if (src) memcpy(k->bytes, src, want);
-        const uint8_t *all = vk_usnap_find(u->addr, u->size, 1, &bi) ? vk_usnap_map() + bi.offset : NULL;
+        const uint8_t *all = vk_usnap_find(u->addr, u->size, UINT32_MAX, 1, &bi) ? vk_usnap_map() + bi.offset : NULL;
         k->mats = all ? lead_mats((const float *)all, (uint32_t)(u->size / 4)) : 0;
         if (all) {
             Stat *s = &S.st[b];

runtime/vk_pipeline.c

@@ -101,6 +101,12 @@ static struct {
     VkPipeStats stats;
 } P = {.lock = PLAT_MUTEX_INIT};
 
+// Builds into P.cache take it shared, vk_pipeline_save takes it to copy the cache out. Vulkan lets
+// vkGetPipelineCacheData run alongside builds into a cache created without the EXTERNALLY_SYNCHRONIZED bit, but
+// Intel's Windows driver (101.7085, Iris Xe) faults in it (igvk64.dll, reading 0x170) when the first autosave meets the
+// boot's pipeline builds (#681). Nothing is taken while it is held.
+static PlatMutex cache_use = PLAT_MUTEX_INIT;
+
 typedef struct Layout {
     struct Layout *next;
     uint64_t hash;
@@ -260,13 +266,19 @@ static int ensure_init(void) {
     }
     // TOTK_VK_DYN_VERTEX=0/1 forces dynamic vertex input off or on. Default off on Intel's frozen Windows driver for its
     // older integrated GPUs (UHD 630, Ice Lake: Vulkan 1.3.215 and older), which draws nothing with it; LAS Recomp
-    // turns it off there for the same reason.
+    // turns it off there for the same reason. Also off on Intel's current Windows driver for its integrated GPUs
+    // (vk_caps.intel_igpu_win): on UHD Graphics (i3-1215U, 101.6129; i3-1305U, 101.7088) and Iris Xe (101.7092) it
+    // drew black rectangles over walls, trees and floating islands, and vk_dyn_vertex = 0 removed them (issues #970,
+    // #992, #81). Mesa's ANV on the same GPUs keeps it: a report there (#989) had both switches off and its glitch stayed.
     const char *dv = cfg_get("TOTK_VK_DYN_VERTEX");   // no fixed default: it depends on the GPU, see above
     int intel_legacy = c->props.vendorID == 0x8086 && c->props.apiVersion <= VK_MAKE_API_VERSION(0, 1, 3, 215);
-    int dv_on = dv && *dv ? strcmp(dv, "0") != 0 : !intel_legacy;
+    int dv_on = dv && *dv ? strcmp(dv, "0") != 0 : !intel_legacy && !vk_caps.intel_igpu_win;
     P.dyn_vi = dv_on && c->has_vertex_input_dynamic_state && PROC(set_vi, "vkCmdSetVertexInputEXT");
     if (!dv_on && c->has_vertex_input_dynamic_state)
-        rt_log("vk_pipeline: dynamic vertex input off (%s)", dv && *dv ? "TOTK_VK_DYN_VERTEX=0" : "Intel's older Windows driver draws nothing with it");
+        rt_log("vk_pipeline: dynamic vertex input off (%s)",
+               dv && *dv      ? "TOTK_VK_DYN_VERTEX=0"
+               : intel_legacy ? "Intel's older Windows driver draws nothing with it"
+                              : "Intel's Windows driver for integrated GPUs draws black patches with it");
     if (c->has_eds3) {
         P.dyn_clamp = e3.extendedDynamicState3DepthClampEnable && P.f_depth_clamp && PROC(set_clamp, "vkCmdSetDepthClampEnableEXT");
         P.dyn_poly = e3.extendedDynamicState3PolygonMode && PROC(set_poly, "vkCmdSetPolygonModeEXT");
@@ -279,12 +291,17 @@ static int ensure_init(void) {
     // RDNA 2 and older (vk_caps.amd_sc), which applies alpha to coverage while the dynamic state says off: on an RX 590
     // (24.9.1) TotK, which never turns it on, drew Link and the NPCs with depth but no colour (black silhouettes) and
     // dithered other surfaces into checkerboards (issue #67, RX 470/570/580). Taken out of the dynamic state, it goes
-    // into the pipeline key and the characters draw.
+    // into the pipeline key and the characters draw. Also off on Intel's Windows driver for its integrated GPUs
+    // (vk_caps.intel_igpu_win): UHD Graphics and Iris Xe drew coloured noise on branches, foliage and wood with it, and
+    // vk_dyn_a2c = 0 removed it (issues #970, #992, #419).
     const char *da = cfg_get("TOTK_VK_DYN_A2C");   // no fixed default: it depends on the driver, see above
-    int da_on = da && *da ? strcmp(da, "0") != 0 : !vk_caps.amd_sc;
+    int da_on = da && *da ? strcmp(da, "0") != 0 : !vk_caps.amd_sc && !vk_caps.intel_igpu_win;
     if (!da_on && P.dyn_a2c) {
         P.dyn_a2c = 0;
-        rt_log("vk_pipeline: dynamic alpha to coverage off (%s)", da && *da ? "TOTK_VK_DYN_A2C=0" : "AMD's driver for RDNA 2 and older");
+        rt_log("vk_pipeline: dynamic alpha to coverage off (%s)",
+               da && *da        ? "TOTK_VK_DYN_A2C=0"
+               : vk_caps.amd_sc ? "AMD's driver for RDNA 2 and older"
+                                : "Intel's Windows driver for integrated GPUs draws coloured noise with it");
     }
 #undef PROC
     static const VkDynamicState core[] = {
@@ -1440,7 +1457,9 @@ static VkPipeline create_graphics(const VkPipeState *s, const GKey *k, const Lay
     if (k->fbl & 1) pi.flags |= VK_PIPELINE_CREATE_COLOR_ATTACHMENT_FEEDBACK_LOOP_BIT_EXT;
     if (k->fbl & 2) pi.flags |= VK_PIPELINE_CREATE_DEPTH_STENCIL_ATTACHMENT_FEEDBACK_LOOP_BIT_EXT;
     uint64_t p0 = perf_begin();
+    plat_mutex_lock_shared(&cache_use);
     VkResult r = vkCreateGraphicsPipelines(P.device, P.cache, 1, &pi, NULL, &pipe);
+    plat_mutex_unlock_shared(&cache_use);
     perf_end(PERF_PIPE, p0);   // counts on the render thread only, so background builds stay out of it
     if (r != VK_SUCCESS) {
         rt_log("vk_pipeline: vkCreateGraphicsPipelines failed (%d)", (int)r);
@@ -1591,7 +1610,9 @@ static VkPipeline lib_create(int kind, const VkPipeState *s, const GKey *k, VkPi
         ri.depthAttachmentFormat = (VkFormat)k->depth, ri.stencilAttachmentFormat = (VkFormat)k->stencil;
         pi.pColorBlendState = &cb, pi.pMultisampleState = &ms;
     }
+    plat_mutex_lock_shared(&cache_use);
     if (vkCreateGraphicsPipelines(P.device, P.cache, 1, &pi, NULL, &pipe) != VK_SUCCESS) pipe = VK_NULL_HANDLE;
+    plat_mutex_unlock_shared(&cache_use);
 out:
     if (m) vkDestroyShaderModule(P.device, m, NULL);
     free(patched);
@@ -1803,7 +1824,8 @@ static VkPipeline gpl_link(const VkPipeState *s, const GKey *k, const Layout *lc
 // shaders (gfx_shader_get results) live for the process (gfx_shader_drop_memory is the shader probe's, which builds no
 // pipelines), layouts until vk_pipeline_shutdown, which drains the queue first. Workers call vkCreateGraphicsPipelines
 // without P.lock: P.cache is created without VK_PIPELINE_CACHE_CREATE_EXTERNALLY_SYNCHRONIZED_BIT, so the driver
-// synchronises it, vkGetPipelineCacheData in vk_pipeline_save included. Compute pipelines stay synchronous: a skipped
+// synchronises the builds among themselves. vk_pipeline_save still keeps them apart from vkGetPipelineCacheData (see
+// cache_use). Compute pipelines stay synchronous: a skipped
 // dispatch can change what the game computes.
 //
 // Two queues: draws' misses (A.head) and the boot prebuild (A.low, see "pipeline keys" below). Workers take the low
@@ -2512,7 +2534,7 @@ static void prebuild_start(void) {
     plat_mutex_unlock(&P.lock);
 }
 
-// A compute pipeline for cs in layout l; VK_NULL_HANDLE on failure. No lock needed (P.cache is driver-synchronised).
+// A compute pipeline for cs in layout l; VK_NULL_HANDLE on failure. Takes cache_use shared, like the graphics builds.
 static VkPipeline compute_create(const GfxShader *cs, const Layout *l) {
     uint64_t words;
     const uint32_t *code = gfx_shader_spirv(cs, &words);
@@ -2525,7 +2547,9 @@ static VkPipeline compute_create(const GfxShader *cs, const Layout *l) {
                                       .layout = l->layout};
     pin_subgroup32(&ci.stage, &rqc, code, words);
     VkPipeline pipe = VK_NULL_HANDLE;
+    plat_mutex_lock_shared(&cache_use);
     if (vkCreateComputePipelines(P.device, P.cache, 1, &ci, NULL, &pipe) != VK_SUCCESS) pipe = VK_NULL_HANDLE;
+    plat_mutex_unlock_shared(&cache_use);
     vkDestroyShaderModule(P.device, m, NULL);
     return pipe;
 }
@@ -3113,8 +3137,8 @@ int vk_pipeline_warm_shaders(void) {
 
 // ---- dynamic state
 
-void vk_pipeline_set_dynamic(VkCommandBuffer cb, const VkPipeState *s) {
-    if (!ensure_init()) return;
+uint32_t vk_pipeline_set_dynamic(VkCommandBuffer cb, const VkPipeState *s) {
+    if (!ensure_init()) return VK_PIPE_ALL_BINDINGS;
     const poly_st *pl = &s->poly;
     const ds_st *d = &s->ds;
     // The front face maps unchanged: TOTK sets the upper-left window origin, and the y-flipped viewport (y_flip) puts
@@ -3137,16 +3161,21 @@ void vk_pipeline_set_dynamic(VkCommandBuffer cb, const VkPipeState *s) {
     vkCmdSetDepthBiasEnable(cb, (pl->offset_en >> mode & 1) != 0);
     vkCmdSetRasterizerDiscardEnable(cb, s->raster_discard != 0);
     vkCmdSetPrimitiveRestartEnable(cb, s->prim_restart != 0);
-    if (P.dyn_vi && s->vs) {
+    // The vertex bindings the draw reads: those of its vertex input, set here or made into the pipeline from the same
+    // state by the same vertex_input.
+    uint32_t streams = VK_PIPE_ALL_BINDINGS;
+    if (s->vs) {
         VtxInput v;
         vertex_input(s, &v);
-        rec_vertex_input(P.set_vi, cb, v.n_bind, v.b, v.n_attr, v.a);
+        if (P.dyn_vi) rec_vertex_input(P.set_vi, cb, v.n_bind, v.b, v.n_attr, v.a);
+        streams = 0;
+        for (uint32_t i = 0; i < v.n_bind; i++) streams |= 1u << v.b[i].binding;
     }
     if (P.dyn_clamp) rec_bool(P.set_clamp, cb, s->depth_clamp != 0);
     if (P.dyn_poly) rec_polygon_mode(P.set_poly, cb, polygon_mode(pl->mode));
     if (P.dyn_a2c) rec_bool((PFN_vkCmdSetDepthClampEnableEXT)P.set_a2c, cb, s->ms.a2c != 0);   // the same signature
     int n = target_count(s);
-    if (!n) return;
+    if (!n) return streams;
     VkBool32 en[8];
     VkColorBlendEquationEXT eq[8];
     VkColorComponentFlags wm[8];
@@ -3160,6 +3189,120 @@ void vk_pipeline_set_dynamic(VkCommandBuffer cb, const VkPipeState *s) {
     if (P.dyn_blend_en) rec_blend_enable(P.set_blend_en, cb, 0, (uint32_t)n, en);
     if (P.dyn_blend_eq) rec_blend_equation(P.set_blend_eq, cb, 0, (uint32_t)n, eq);
     if (P.dyn_wmask) rec_write_mask(P.set_wmask, cb, 0, (uint32_t)n, wm);
+    return streams;
+}
+
+// The values vk_pipeline_set_dynamic records for a state, as the commands take them.
+typedef struct {
+    VkCullModeFlags cull;
+    VkFrontFace front;
+    VkPrimitiveTopology topo;
+    VkBool32 dt, dw, st, bias, discard, restart, clamp, a2c;
+    VkCompareOp df, scmp[2];
+    VkStencilOp sop[2][3];               // fail, pass, depth fail; front, back
+    VkPolygonMode poly;
+    int n;                               // colour attachments en, eq and wm hold
+    VkBool32 en[8];
+    VkColorBlendEquationEXT eq[8];
+    VkColorComponentFlags wm[8];
+} DynVals;
+
+static void dyn_vals(const VkPipeState *s, DynVals *o) {
+    const poly_st *pl = &s->poly;
+    const ds_st *d = &s->ds;
+    memset(o, 0, sizeof *o);
+    o->cull = (VkCullModeFlags)(pl->cull & 3);
+    o->front = pl->front == NVN_FRONT_FACE_CW ? VK_FRONT_FACE_CLOCKWISE : VK_FRONT_FACE_COUNTER_CLOCKWISE;
+    o->topo = vk_pipeline_topology(s->prim);
+    o->dt = d->dt, o->dw = d->dw, o->st = d->st;
+    o->df = compare_op(d->df);
+    o->sop[0][0] = stencil_op(d->fail_f), o->sop[0][1] = stencil_op(d->zpass_f), o->sop[0][2] = stencil_op(d->zfail_f);
+    o->sop[1][0] = stencil_op(d->fail_b), o->sop[1][1] = stencil_op(d->zpass_b), o->sop[1][2] = stencil_op(d->zfail_b);
+    o->scmp[0] = compare_op(d->sf_f), o->scmp[1] = compare_op(d->sf_b);
+    unsigned mode = pl->mode <= NVN_POLYGON_MODE_FILL ? pl->mode : NVN_POLYGON_MODE_FILL;
+    o->bias = (pl->offset_en >> mode & 1) != 0;
+    o->discard = s->raster_discard != 0, o->restart = s->prim_restart != 0;
+    o->clamp = s->depth_clamp != 0, o->a2c = s->ms.a2c != 0;
+    o->poly = polygon_mode(pl->mode);
+    o->n = target_count(s);
+    for (int i = 0; i < o->n; i++) {
+        const blend_st *b = &s->blend[i];
+        o->en[i] = (s->color.blend_en >> i & 1) && blendable(target_format(s->color_fmt[i], NULL, NULL));
+        o->eq[i] = (VkColorBlendEquationEXT){blend_factor(b->src_rgb), blend_factor(b->dst_rgb), blend_op(b->eq_rgb),
+                                             blend_factor(b->src_a), blend_factor(b->dst_a), blend_op(b->eq_a)};
+        o->wm[i] = target_wmask(s, i);
+    }
+}
+
+// What the replay's command buffer holds, as vk_pipeline_set_dynamic_delta last recorded it: the values, the inputs of
+// the vertex input and the vertex bindings it reads. Invariant while the caller passes valid 1: cb holds DC.v for every
+// dynamic state of P.dyn, colour attachments [0, DC.v.n) included. Per thread: the threads of a chunked replay
+// (vk_exec.c) each record their own command buffers, and a new one starts with valid 0.
+static _Thread_local struct {
+    DynVals v;
+    const GfxShader *vs;
+    uint32_t vattr[16];
+    vs_st vstream[16];
+    uint32_t streams;
+} DC;
+
+uint32_t vk_pipeline_set_dynamic_delta(VkCommandBuffer cb, const VkPipeState *s, int valid, uint32_t *cmds) {
+    *cmds = 0;
+    if (!ensure_init()) return VK_PIPE_ALL_BINDINGS;
+    if (!valid) {   // a new command buffer, or one whose dynamic state someone else set: everything, as before
+        uint32_t streams = vk_pipeline_set_dynamic(cb, s);
+        dyn_vals(s, &DC.v);
+        DC.vs = s->vs, DC.streams = streams;
+        memcpy(DC.vattr, s->vattr, sizeof DC.vattr), memcpy(DC.vstream, s->vstream, sizeof DC.vstream);
+        *cmds = 1;
+        return streams;
+    }
+    DynVals v;
+    dyn_vals(s, &v);
+    const DynVals *o = &DC.v;
+    uint32_t k = 0;
+    if (v.cull != o->cull) vkCmdSetCullMode(cb, v.cull), k++;
+    if (v.front != o->front) vkCmdSetFrontFace(cb, v.front), k++;
+    if (v.topo != o->topo) vkCmdSetPrimitiveTopology(cb, v.topo), k++;
+    if (v.dt != o->dt) vkCmdSetDepthTestEnable(cb, v.dt), k++;
+    if (v.dw != o->dw) vkCmdSetDepthWriteEnable(cb, v.dw), k++;
+    if (v.df != o->df) vkCmdSetDepthCompareOp(cb, v.df), k++;
+    if (v.st != o->st) vkCmdSetStencilTestEnable(cb, v.st), k++;
+    for (int f = 0; f < 2; f++)
+        if (memcmp(v.sop[f], o->sop[f], sizeof v.sop[f]) || v.scmp[f] != o->scmp[f])
+            vkCmdSetStencilOp(cb, f ? VK_STENCIL_FACE_BACK_BIT : VK_STENCIL_FACE_FRONT_BIT, v.sop[f][0], v.sop[f][1],
+                              v.sop[f][2], v.scmp[f]),
+                k++;
+    if (v.bias != o->bias) vkCmdSetDepthBiasEnable(cb, v.bias), k++;
+    if (v.discard != o->discard) vkCmdSetRasterizerDiscardEnable(cb, v.discard), k++;
+    if (v.restart != o->restart) vkCmdSetPrimitiveRestartEnable(cb, v.restart), k++;
+    // The vertex input is made from the vertex shader's inputs and the attribute and stream states alone
+    // (vertex_input): a draw that keeps all three keeps it, and the bindings it reads.
+    if (s->vs != DC.vs || memcmp(s->vattr, DC.vattr, sizeof DC.vattr) || memcmp(s->vstream, DC.vstream, sizeof DC.vstream)) {
+        uint32_t streams = VK_PIPE_ALL_BINDINGS;
+        if (s->vs) {
+            VtxInput vi;
+            vertex_input(s, &vi);
+            if (P.dyn_vi) rec_vertex_input(P.set_vi, cb, vi.n_bind, vi.b, vi.n_attr, vi.a), k++;
+            streams = 0;
+            for (uint32_t i = 0; i < vi.n_bind; i++) streams |= 1u << vi.b[i].binding;
+        }
+        DC.vs = s->vs, DC.streams = streams;
+        memcpy(DC.vattr, s->vattr, sizeof DC.vattr), memcpy(DC.vstream, s->vstream, sizeof DC.vstream);
+    }
+    if (P.dyn_clamp && v.clamp != o->clamp) rec_bool(P.set_clamp, cb, v.clamp), k++;
+    if (P.dyn_poly && v.poly != o->poly) rec_polygon_mode(P.set_poly, cb, v.poly), k++;
+    if (P.dyn_a2c && v.a2c != o->a2c) rec_bool((PFN_vkCmdSetDepthClampEnableEXT)P.set_a2c, cb, v.a2c), k++;
+    // Attachments past o->n may hold nothing yet in this command buffer: a state with more targets sets all of them.
+    // One with fewer leaves the ones past its count as they were, which no draw of it reads.
+    uint32_t n = (uint32_t)v.n;
+    int all = v.n > o->n;
+    if (n && P.dyn_blend_en && (all || memcmp(v.en, o->en, n * sizeof *v.en))) rec_blend_enable(P.set_blend_en, cb, 0, n, v.en), k++;
+    if (n && P.dyn_blend_eq && (all || memcmp(v.eq, o->eq, n * sizeof *v.eq))) rec_blend_equation(P.set_blend_eq, cb, 0, n, v.eq), k++;
+    if (n && P.dyn_wmask && (all || memcmp(v.wm, o->wm, n * sizeof *v.wm))) rec_write_mask(P.set_wmask, cb, 0, n, v.wm), k++;
+    DC.v = v;   // the invariant now covers [0, v.n) only
+    *cmds = k;
+    return DC.streams;
 }
 
 // ---- statistics, persistence, teardown
@@ -3172,7 +3315,8 @@ void vk_pipeline_stats(VkPipeStats *s) {
 
 static PlatMutex save_lock = PLAT_MUTEX_INIT;   // one cache save at a time; shutdown takes it to destroy the cache
 
-void vk_pipeline_save(void) {
+// One cache save; wait: queue for cache_use (the exit saves) rather than try for it (the autosave).
+static void cache_save(int wait) {
     if (P.ready) keys_flush();
     if (P.ready && P.gpl) {
         rt_log("vk_pipeline: gpl: %llu fast links (%llu by workers; avg %.2f ms, max %.2f ms, %llu failed), %llu draws "
@@ -3199,23 +3343,46 @@ void vk_pipeline_save(void) {
     }
     gfx_shader_record_flush(1);
     if (!P.ready || !P.cache || !P.cache_path[0]) return;
-    // Not under P.lock: the driver synchronises P.cache, and holding P.lock through the copy and the file write (about
-    // 60 ms on the 1.4.3 fight) stopped every pipeline lookup of the replay, a freeze at each 20 s autosave.
+    // Not under P.lock: holding it through the copy and the file write (about 60 ms on the 1.4.3 fight) stopped every
+    // pipeline lookup of the replay, a freeze at each 20 s autosave. Only the copy takes cache_use, so no pipeline is
+    // being built into P.cache while the driver reads it.
     plat_mutex_lock(&save_lock);
     if (!P.cache) {   // vk_pipeline_shutdown destroyed it
         plat_mutex_unlock(&save_lock);
         return;
     }
+    uint64_t t0 = qpc();
+    if (wait) {
+        plat_mutex_lock(&cache_use);
+    } else {
+        // The autosave never queues for cache_use: a queued exclusive request on an SRW lock holds back new shared
+        // ones, so the drawing thread's next build would wait for a worker's build to end. It tries for about 2 s
+        // and otherwise saves on its next round.
+        int got = 0;
+        for (int i = 0; i < 1000 && !(got = plat_mutex_trylock(&cache_use)); i++) plat_sleep_ms(2);
+        if (!got) {
+            rt_log("vk_pipeline: cache save put off, pipelines were being built for %.0f ms", ms_since(t0));
+            plat_mutex_unlock(&save_lock);
+            return;
+        }
+    }
+    uint64_t t1 = qpc();
     size_t size = 0;
     void *data = NULL;
-    if (vkGetPipelineCacheData(P.device, P.cache, &size, NULL) == VK_SUCCESS && size && (data = malloc(size)) &&
-        vkGetPipelineCacheData(P.device, P.cache, &size, data) == VK_SUCCESS) {
-        if (!plat_file_write_atomic(P.cache_path, data, size)) rt_log("vk_pipeline: cannot write %s", P.cache_path);
-    }
+    int ok = vkGetPipelineCacheData(P.device, P.cache, &size, NULL) == VK_SUCCESS && size && (data = malloc(size)) &&
+             vkGetPipelineCacheData(P.device, P.cache, &size, data) == VK_SUCCESS;
+    plat_mutex_unlock(&cache_use);
+    // The copy time is how long a pipeline build on the drawing thread (a compute miss, under P.lock) can wait for
+    // this save; the wait is how long the save waited for builds in flight.
+    rt_log("vk_pipeline: cache %s: %.1f MB copied in %.1f ms, after %.1f ms waiting for builds", wait ? "save" : "autosave",
+           (double)size / 1e6, ms_since(t1), (double)(t1 - t0) / 1e6);
+    if (ok && !plat_file_write_atomic(P.cache_path, data, size)) rt_log("vk_pipeline: cannot write %s", P.cache_path);
     free(data);
     plat_mutex_unlock(&save_lock);
 }
 
+void vk_pipeline_save(void) { cache_save(1); }
+
 // The game never shuts the device down (boot.c and the window end the process), so the cache is saved while it
 // runs: at most every 20 s, and only after new pipelines were built. New pipeline keys are appended every 2 s.
 static void autosave_once(void) {
@@ -3229,7 +3396,7 @@ static void autosave_once(void) {
     vk_pipeline_stats(&s);
     if (s.graphics + s.compute == last_n) return;
     last_n = s.graphics + s.compute, last_ms = now;
-    vk_pipeline_save();
+    cache_save(0);
 }
 
 // On its own thread: the save (vkGetPipelineCacheData and the file write, about 60 ms) and the key appends (a flush to

runtime/vk_rec.c

@@ -3,7 +3,8 @@
 // call's arguments; its pointers point into the packet itself, so the recorder passes the payload to the driver as
 // it is. A packet never wraps: one that does not fit before the end is preceded by a pad. The producer publishes head
 // after each packet; the recorder makes the calls and advances tail, which the producer waits on when the ring is
-// full and in rec_drain. While a part is recorded the recorder spins on head; idle, it sleeps on an event.
+// full and in rec_drain. While a part is recorded the recorder spins on head; idle, it sleeps on an event. Packets also
+// begin, end and submit the parts (rec_begin, rec_end_cb, rec_submit), so each part stays on the recorder throughout.
 // Design: Savage Wind Recomp #189 and LAS Recomp runtime/nvn/vulkan/record.c (https://github.com/peiikai/LASRecomp
 // @ a4b53a3, MIT, Copyright (c) 2026 Peiikai); the code is ours.
 #define VK_REC_IMPL
@@ -18,21 +19,44 @@
 
 #define RING_WORDS (1u << 21)   // 16 MiB: a part of a submission is ~1-2 MiB of packets in the town
 static uint64_t ring[RING_WORDS];
-static volatile uint64_t head, tail;   // words published by the producer / consumed by the recorder
-static volatile int32_t sleeping;         // the recorder waits for wake_ev
+// The queue state, a cache line per writer. The recorder spins on head while it waits for packets, so a line the
+// producer writes on every packet and the recorder reads or writes goes back and forth between their cores each time:
+// with tail, Q.put_h, head and the producer's statistics in one line (the linker's order of these statics), handing
+// packets over took 6% of the replay thread in the fort, more than building the descriptors.
+static struct {
+    _Alignas(64) volatile uint64_t head;      // words published by the producer; the recorder reads it
+    _Alignas(64) volatile uint64_t tail;      // words consumed by the recorder; the producer reads it when it seems full
+    _Alignas(64) volatile int32_t sleeping;   // the recorder waits for wake_ev; written only around a sleep
+    _Alignas(64) uint64_t put_h;              // producer only: words written
+    uint64_t pub_h;                           // the last head published
+    uint64_t tail_seen;                       // tail as last read, at most the real one: the room it leaves is there
+    uint64_t pub_words;                       // TOTK_REC_PUB: publish head once this many words wait (pub)
+} Q;
 static PlatEvent wake_ev;              // auto-reset
-static uint64_t put_h;                 // producer: words written (published up to head)
-static VkCommandBuffer rec_cb;         // the part being recorded (set while the recorder is idle)
+static VkCommandBuffer rec_cb;         // producer: the part being recorded, whose vkCmd* become packets
+static VkCommandBuffer r_cb;           // recorder: the command buffer its packets go into (set by a BeginCb packet)
 static int rec_live;                   // producer: rec_begin started the recorder for rec_cb
+static int end_mode = -1;              // TOTK_REC_END (rec_own)
+static int sub_pending;                // producer: the recorder ended a part that rec_submit has not handed over yet
+static volatile VkResult r_sub_res;    // recorder: the result of its last vkQueueSubmit2, read after the drain
+static volatile VkResult r_sub_err;    // recorder: the first vkQueueSubmit2 that failed (rec_submit reports it)
+static volatile uint64_t q_subs;       // producer: submissions queued for the recorder (rec_submit)
+static volatile uint64_t r_subs;       // recorder: submissions it handed to the driver (submit_run)
+static _Thread_local int on_rec;       // the calling thread is the recorder
+static int rec_async = -1;             // TOTK_REC_ASYNC (rec_submit)
+static struct { uint64_t n, ns; } RQ_; // waits in rec_queue_wait, logged by rec_stats_log
 static PlatThread *rec_thread;
 static int rec_prio;                   // the submitting thread's priority, which the recorder takes
 static int rec_chk;                    // TOTK_REC_CHECK (rec_check_handle)
 static uint64_t rec_delay_ns;          // TOTK_REC_DELAY_US (rec_main)
-static volatile uint64_t part_h0;      // put_h at rec_begin: where the packets of the part being recorded start
+static uint64_t rec_spin_ns;           // TOTK_REC_SPIN_US (rec_main); 0: 4096 pauses
+static int rec_spin_ab;                // TOTK_REC_SPIN_US=ab: the timed poll in the B blocks of TOTK_AB only
+static volatile uint64_t part_h0;      // Q.put_h at rec_begin: where the packets of the part being recorded start
 static volatile uint64_t part_fence;   // the value of the part being recorded (rec_begin)
+static uint64_t r_sleeps;              // recorder: sleeps on wake_ev, read by rec_stats_log
 
 // Producer statistics, logged by rec_stats_log.
-static struct { uint64_t packets, words, drains, fallbacks, drain_ns, full_waits, parts, parts_on; } RS_;
+static struct { _Alignas(64) uint64_t packets, words, drains, fallbacks, drain_ns, full_waits, parts, parts_on, submits, wakes; } RS_;
 
 enum {
     R_PAD, R_BeginQuery, R_BeginRendering, R_BindDescriptorSets, R_BindIndexBuffer, R_BindPipeline, R_BindVertexBuffers2,
@@ -44,7 +68,7 @@ enum {
     R_SetRasterizerDiscardEnable, R_SetScissorWithCount, R_SetStencilCompareMask, R_SetStencilOp, R_SetStencilReference,
     R_SetStencilTestEnable, R_SetStencilWriteMask, R_SetViewportWithCount, R_WriteTimestamp, R_WriteTimestamp2,
     R_PushTpl, R_Push, R_VertexInput, R_Bool, R_PolygonMode, R_BlendEnable, R_BlendEquation, R_WriteMask, R_EndCb,
-    R_PassMark,
+    R_PassMark, R_BeginCb, R_Submit,
 };
 
 static uint64_t now_ns(void) { return plat_now_ns(); }
@@ -56,16 +80,34 @@ static void exec_packet(uint32_t op, void *p);
 static void rec_main(void *arg) {
     (void)arg;
     plat_thread_priority(rec_prio);
+    on_rec = 1;
+    perf_core_thread(PCORE_RECORDER);
     uint64_t t = 0;
     for (;;) {
-        uint64_t h = __atomic_load_n(&head, __ATOMIC_ACQUIRE);
+        uint64_t h = __atomic_load_n(&Q.head, __ATOMIC_ACQUIRE);
         if (t == h) {
-            for (int i = 0; i < 4096 && t == h; i++) _mm_pause(), h = __atomic_load_n(&head, __ATOMIC_ACQUIRE);
+            // Poll for the next packets before sleeping: a sleep costs the producer an event set and the recorder
+            // an OS wake-up. The poll is timed (TOTK_REC_SPIN_US, default 60) because a pause takes 8-150 cycles
+            // depending on the CPU: 4096 of them are ~140 us on a Coffee Lake i7 and a few us on Zen 4. With fewer than
+            // 6 logical processors it stays 4096 pauses. Savage Wind Recomp's recorder polls 60 us the same way.
+            if (rec_spin_ns && (!rec_spin_ab || rt_ab())) {
+                uint64_t t0 = now_ns();
+                for (unsigned i = 1; t == h; i++) {
+                    _mm_pause();
+                    h = __atomic_load_n(&Q.head, __ATOMIC_ACQUIRE);
+                    if (!(i & 31) && now_ns() - t0 >= rec_spin_ns) break;
+                }
+            } else {
+                for (int i = 0; i < 4096 && t == h; i++) _mm_pause(), h = __atomic_load_n(&Q.head, __ATOMIC_ACQUIRE);
+            }
             if (t == h) {
-                __atomic_store_n(&sleeping, 1, __ATOMIC_SEQ_CST);
-                h = __atomic_load_n(&head, __ATOMIC_SEQ_CST);
-                if (t == h) plat_event_wait_ns(&wake_ev, 2000000);   // a missed wake costs at most 2 ms
-                __atomic_store_n(&sleeping, 0, __ATOMIC_RELAXED);
+                __atomic_store_n(&Q.sleeping, 1, __ATOMIC_SEQ_CST);
+                h = __atomic_load_n(&Q.head, __ATOMIC_SEQ_CST);
+                if (t == h) {
+                    __atomic_add_fetch(&r_sleeps, 1, __ATOMIC_RELAXED);
+                    plat_event_wait_ns(&wake_ev, 2000000);   // a missed wake costs at most 2 ms
+                }
+                __atomic_store_n(&Q.sleeping, 0, __ATOMIC_RELAXED);
                 continue;
             }
         }
@@ -77,9 +119,9 @@ static void rec_main(void *arg) {
             uint32_t op = (uint32_t)(w & 0xffff), n = (uint32_t)(w >> 32);
             if (op != R_PAD) exec_packet(op, &ring[t % RING_WORDS + 1]);
             t += n;
-            if ((k & 63) == 63) __atomic_store_n(&tail, t, __ATOMIC_RELEASE);
+            if ((k & 63) == 63) __atomic_store_n(&Q.tail, t, __ATOMIC_RELEASE);
         }
-        __atomic_store_n(&tail, t, __ATOMIC_RELEASE);
+        __atomic_store_n(&Q.tail, t, __ATOMIC_RELEASE);
     }
     return;
 }
@@ -88,72 +130,115 @@ static void rec_main(void *arg) {
 
 static void wake(void) {
     __atomic_thread_fence(__ATOMIC_SEQ_CST);
-    if (__atomic_load_n(&sleeping, __ATOMIC_RELAXED)) plat_event_set(&wake_ev);
+    if (__atomic_load_n(&Q.sleeping, __ATOMIC_RELAXED)) plat_event_set(&wake_ev), RS_.wakes++;
 }
 
+// Room for need words: the tail read last leaves it in all but about one packet in 16 MiB of them.
 static void wait_space(uint64_t need) {
-    if (put_h + need - __atomic_load_n(&tail, __ATOMIC_ACQUIRE) <= RING_WORDS) return;
+    if (Q.put_h + need - Q.tail_seen <= RING_WORDS) return;
+    Q.tail_seen = __atomic_load_n(&Q.tail, __ATOMIC_ACQUIRE);
+    if (Q.put_h + need - Q.tail_seen <= RING_WORDS) return;
     RS_.full_waits++;
-    __atomic_store_n(&head, put_h, __ATOMIC_RELEASE);
+    Q.pub_h = Q.put_h;
+    __atomic_store_n(&Q.head, Q.put_h, __ATOMIC_RELEASE);
     wake();
-    while (put_h + need - __atomic_load_n(&tail, __ATOMIC_ACQUIRE) > RING_WORDS) _mm_pause();
+    while (Q.put_h + need - (Q.tail_seen = __atomic_load_n(&Q.tail, __ATOMIC_ACQUIRE)) > RING_WORDS) _mm_pause();
 }
 
 // Room for a packet of bytes of arguments; the caller fills it and calls pub().
 static void *put(uint32_t op, size_t bytes) {
-    uint64_t n = 1 + (bytes + 7) / 8, pos = put_h % RING_WORDS;
+    uint64_t n = 1 + (bytes + 7) / 8, pos = Q.put_h % RING_WORDS;
     if (n > RING_WORDS / 4) rt_fatal("vk_rec: a packet of %llu bytes", (unsigned long long)bytes);
     if (pos + n > RING_WORDS) {
         uint64_t pad = RING_WORDS - pos;
         wait_space(pad);
         ring[pos] = (uint64_t)R_PAD | pad << 32;
-        put_h += pad, pos = 0;
+        Q.put_h += pad, pos = 0;
     }
     wait_space(n);
     ring[pos] = (uint64_t)op | n << 32;
     // Packers reserve room for optional arrays they may leave unwritten (BeginRendering's depth and stencil); with
     // TOTK_REC_CHECK those words would still hold handles from the ring's previous lap.
     if (rec_chk) memset(&ring[pos + 1], 0, (n - 1) * 8);
-    put_h += n;
+    Q.put_h += n;
     RS_.packets++, RS_.words += n;
     return &ring[pos + 1];
 }
 
+// After a packet: head moves once Q.pub_words words wait, so the recorder reads a line the producer is still filling
+// once per batch instead of once per packet. Whatever the producer waits for publishes everything first (wait_space,
+// rec_drain), and the packets that begin and end a part go out at once (pub_now), so the recorder begins and ends
+// command buffers as early as before; TOTK_REC_CHECK publishes every packet, since rec_check_handle searches the ring
+// up to head.
+// The batch is 64 words (512 bytes) where the recorder polls before it sleeps, 8192 (64 KiB) where it does not (fewer
+// than 6 usable logical processors, rec_spin_ns): there it is mostly asleep when a batch goes out, and each publish
+// cost the producer an event set and the recorder a wake-up. Savage Wind Recomp measured that at 6.7% of its render
+// thread's busy time on 4 cores and moved to 64 KiB there (runtime/nvn/vulkan/record.c rec_init, publish_every).
+// TOTK_REC_PUB=<words> sets the batch; =ab: 64 words in the A blocks of TOTK_AB, 8192 in the B blocks.
+static int pub_ab;
+static void pub_now(void) {
+    Q.pub_h = Q.put_h;
+    __atomic_store_n(&Q.head, Q.put_h, __ATOMIC_RELEASE);
+    if (__atomic_load_n(&Q.sleeping, __ATOMIC_RELAXED)) plat_event_set(&wake_ev), RS_.wakes++;
+}
+
 static void pub(void) {
-    __atomic_store_n(&head, put_h, __ATOMIC_RELEASE);
-    if (__atomic_load_n(&sleeping, __ATOMIC_RELAXED)) plat_event_set(&wake_ev);
+    uint64_t w = pub_ab ? (rt_ab() ? 8192 : 64) : Q.pub_words;
+    if (Q.put_h - Q.pub_h >= w) pub_now();
 }
 
 void rec_drain(void) {
-    if (!rec_thread || __atomic_load_n(&tail, __ATOMIC_ACQUIRE) == put_h) return;
+    if (!rec_thread || __atomic_load_n(&Q.tail, __ATOMIC_ACQUIRE) == Q.put_h) return;
     uint64_t t0 = now_ns();
-    __atomic_store_n(&head, put_h, __ATOMIC_RELEASE);
+    Q.pub_h = Q.put_h;
+    __atomic_store_n(&Q.head, Q.put_h, __ATOMIC_RELEASE);
     wake();
-    for (int i = 0; __atomic_load_n(&tail, __ATOMIC_ACQUIRE) != put_h; i++)
+    for (int i = 0; __atomic_load_n(&Q.tail, __ATOMIC_ACQUIRE) != Q.put_h; i++)
         if (i < 20000) _mm_pause();
         else plat_yield();
     RS_.drains++, RS_.drain_ns += now_ns() - t0;
 }
 
-void rec_begin(VkCommandBuffer cb, uint64_t fence) {
+static int rec_own(void) {
+    if (end_mode < 0) end_mode = rt_knob("REC_END", "1");
+    return end_mode;
+}
+
+// vkBeginCommandBuffer (one submission) and the breadcrumb, on the thread that records cb.
+static void begin_now(VkCommandBuffer cb, int crumb, uint64_t n) {
+    VkCommandBufferBeginInfo bi = {.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,
+                                   .flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
+    vkBeginCommandBuffer(cb, &bi);
+    if (crumb) vk_crumb(cb, crumb, n);
+}
+
+// The command buffer the following packets go into; begin: the recorder begins it (TOTK_REC_END=1).
+typedef struct { VkCommandBuffer cb; int begin, crumb; uint64_t n; } A_BeginCb;
+
+void rec_begin(VkCommandBuffer cb, uint64_t fence, int crumb, uint64_t n) {
     static int mode = -1;
     if (mode < 0) {
-        // Default off on AMD GPUs. With the recorder those drivers lost the device or hung the GPU at startup on
-        // RDNA 2, 3 and 4, under AMD's Windows driver and under RADV (issues #5, #40, #277 and #283: a black screen,
-        // or VK_ERROR_DEVICE_LOST with a breadcrumb naming a part of a replay submission, and vk_recorder = 0 gets
-        // past it). Keeping objects until their submission completed and ending a part from the recorder
-        // (rec_end_cb) fixed the white scene on an RX 6700 XT, but not this: 0.2.8 turned the recorder on for every
-        // GPU and the AMD reports came back the same day. TOTK_VK_RECORDER=1 turns it on for an A/B run.
-        const char *set = cfg_get("TOTK_VK_RECORDER");
-        mode = rt_knob("VK_RECORDER", vk_caps.amd ? "0" : "1");
-        if (!mode && vk_caps.amd && !(set && *set))
-            rt_log("vk_rec: no recorder thread on AMD GPUs (TOTK_VK_RECORDER=1 turns it on)");
+        // On by default on every GPU. 0.2.8 turned it on for AMD too, and some RDNA 2, 3 and 4 systems lost the device
+        // or hung the GPU at startup (issues #5, #40, #277 and #283: a black screen, or VK_ERROR_DEVICE_LOST with a
+        // breadcrumb naming a part of a replay submission), so 0.3.x kept it off on AMD. Since then the recorder
+        // begins, ends and submits its own parts (TOTK_REC_END, which ended a GPU hang on AMD at split submissions),
+        // and AMD players who turned it on reported more FPS, so it is on again; TOTK_VK_RECORDER=0 turns it off
+        // where the device is still lost.
+        mode = rt_knob("VK_RECORDER", "1");
         if (mode) {
             plat_event_init(&wake_ev, 0, 0);
             rec_prio = plat_thread_priority_get();
             rec_chk = rt_knob("REC_CHECK", "0");   // debug switches, read before the recorder starts
             const char *d = cfg_get("TOTK_REC_DELAY_US");
             rec_delay_ns = d && *d && atoi(d) > 0 ? (uint64_t)atoi(d) * 1000 : 0;
+            const char *sp = cfg_get("TOTK_REC_SPIN_US");
+            rec_spin_ab = sp && !strcmp(sp, "ab");
+            int spin_us = sp && *sp && !rec_spin_ab ? atoi(sp) : 60;
+            rec_spin_ns = spin_us > 0 && plat_cpu_usable() >= 6 ? (uint64_t)spin_us * 1000 : 0;
+            const char *pw = cfg_get("TOTK_REC_PUB");   // words per publish (pub); 0: every packet
+            pub_ab = !rec_chk && pw && !strcmp(pw, "ab");
+            Q.pub_words = rec_chk ? 0 : pw && *pw && !pub_ab ? (uint64_t)atoi(pw) : rec_spin_ns ? 64 : 8192;
+            if (Q.pub_words > RING_WORDS / 8) Q.pub_words = RING_WORDS / 8;
             rec_thread = plat_thread_new(rec_main, NULL, "vk recorder");
             if (!rec_thread) {
                 rt_log("vk_rec: no recorder thread: the replay makes its Vulkan calls itself");
@@ -162,17 +247,34 @@ void rec_begin(VkCommandBuffer cb, uint64_t fence) {
                 rt_log("vk_rec: the replay's Vulkan commands are made by a recorder thread%s",
                        mode == 2 ? " in the B blocks (TOTK_AB)" : "");
                 if (rec_chk) rt_log("vk_rec: TOTK_REC_CHECK: deferred and destroyed objects are searched in the ring");
+                if (rec_spin_ns)
+                    rt_log("vk_rec: the recorder polls %llu us before it sleeps%s (TOTK_REC_SPIN_US)",
+                           (unsigned long long)(rec_spin_ns / 1000), rec_spin_ab ? " in the B blocks" : "");
+                else
+                    rt_log("vk_rec: the recorder polls 4096 pauses before it sleeps (TOTK_REC_SPIN_US)");
+                if (pub_ab) rt_log("vk_rec: head published every 64 words, 8192 in the B blocks (TOTK_REC_PUB=ab)");
+                else rt_log("vk_rec: head published every %llu words (TOTK_REC_PUB)", (unsigned long long)Q.pub_words);
                 if (rec_delay_ns) rt_log("vk_rec: TOTK_REC_DELAY_US: the recorder waits %llu us before each batch",
                                          (unsigned long long)(rec_delay_ns / 1000));
             }
         }
     }
     RS_.parts++;
-    if (!RT_KNOB_ON(mode)) return;
+    int on = RT_KNOB_ON(mode), own = on && rec_own();
+    if (!own) {
+        // The submitting thread begins cb. The part pool is the recorder's while it has packets (an A/B switch
+        // between parts): it finishes them first.
+        rec_drain();
+        begin_now(cb, crumb, n);
+        if (!on) return;
+    }
     RS_.parts_on++;
-    rec_cb = cb;   // the recorder is idle: rec_end drained it
-    __atomic_store_n(&part_h0, put_h, __ATOMIC_RELEASE);
+    __atomic_store_n(&part_h0, Q.put_h, __ATOMIC_RELEASE);
     __atomic_store_n(&part_fence, fence, __ATOMIC_RELEASE);
+    A_BeginCb *a = put(R_BeginCb, sizeof *a);
+    *a = (A_BeginCb){cb, own, crumb, n};
+    pub_now();
+    rec_cb = cb;
     rec_live = 1;
 }
 
@@ -182,42 +284,111 @@ void rec_end(void) {
     rec_live = 0;
 }
 
-// TOTK_REC_END (default 1): the thread that recorded the part also ends it, after the breadcrumb crumb (kind 0: none)
-// that vk_sync.c writes at the end of a submission. On an RX 6700 XT (driver 26.10.1, issue #5) parts the recorder
-// recorded and the submitting thread ended broke when a submission was split (vk_sync.c vk_sync_split): the GPU hung
-// or the scene turned green, then white. The handoff in rec_drain orders the two threads, so Vulkan allows the old
-// split; the driver did not cope with it. TOTK_REC_END=0 brings the old split back.
-// The line is written only when a part was recorded by the recorder: with the recorder off (the default on AMD) the
+// TOTK_REC_END (default 1): a part the recorder records is the recorder's from its begin to its submission: it begins
+// it, makes every command, ends it after the breadcrumb crumb (kind 0: none) that vk_sync.c writes at the end of a
+// submission, and submits it. The submitting thread touches the part pool only while the recorder is idle. That is
+// LAS Recomp's and Savage Wind Recomp's model (runtime/nvn/vulkan/record.h: command buffers allocated once, then only
+// the recorder begins, records, ends and submits them). Before, the submitting thread began and submitted parts the
+// recorder recorded and, before that, also ended them. The handoff in rec_drain orders the two threads, so Vulkan
+// allows that split, but on an RX 6700 XT (driver 26.10.1, issue #5) parts ended by the other thread broke when a
+// submission was split (vk_sync.c vk_sync_split): the GPU hung or the scene turned green, then white. Moving the end
+// fixed that; begin and submit followed. TOTK_REC_END=0 brings the old split back: the submitting thread begins,
+// ends and submits every part.
+// The line is written only when a part was recorded by the recorder: with the recorder off (TOTK_VK_RECORDER=0) the
 // switch changes nothing, and reports read an unconditional line as proof that the recorder was running (#315).
 typedef struct { int kind; uint64_t n; } A_EndCb;
 int rec_end_cb(VkCommandBuffer cb, int crumb, uint64_t n) {
-    static int mode = -1, said;
-    if (mode < 0) mode = rt_knob("REC_END", "1");
+    static int said;
+    int own = rec_own();
     if (rec_live && cb == rec_cb && !said) {
         said = 1;
-        rt_log("vk_rec: a part the recorder records is ended by %s (TOTK_REC_END=%d)",
-               mode ? "the recorder" : "the submitting thread", mode);
+        rt_log("vk_rec: a part the recorder records is begun, ended and submitted by %s (TOTK_REC_END=%d)",
+               own ? "the recorder" : "the submitting thread", own);
     }
-    if (!rec_live || cb != rec_cb || !mode) {
+    if (!rec_live || cb != rec_cb || !own) {
         rec_end();
         if (crumb) vk_crumb(cb, crumb, n);
         return 0;
     }
     A_EndCb *a = put(R_EndCb, sizeof *a);
     *a = (A_EndCb){crumb, n};
+    pub_now();
+    rec_live = 0;
+    sub_pending = 1;
+    return 1;
+}
+
+// At most two command buffers and one signal: the shape vk_sync.c submits.
+typedef struct { uint32_t ncb, nsig; VkCommandBufferSubmitInfo cbi[2]; VkSemaphoreSubmitInfo sig; } A_Submit;
+VkResult rec_submit(const VkSubmitInfo2 *si) {
+    if (!sub_pending) {
+        vk_queue_lock();
+        VkResult r = vkQueueSubmit2(vk_ctx()->gfx_queue, 1, si, VK_NULL_HANDLE);
+        vk_queue_unlock();
+        return r;
+    }
+    sub_pending = 0;
+    if (si->commandBufferInfoCount > 2 || si->signalSemaphoreInfoCount > 1 || si->waitSemaphoreInfoCount || si->pNext)
+        rt_fatal("vk_rec: a submission of %u command buffers, %u waits and %u signals the recorder cannot carry",
+                 si->commandBufferInfoCount, si->waitSemaphoreInfoCount, si->signalSemaphoreInfoCount);
+    A_Submit *a = put(R_Submit, sizeof *a);
+    memset(a, 0, sizeof *a);
+    a->ncb = si->commandBufferInfoCount, a->nsig = si->signalSemaphoreInfoCount;
+    for (uint32_t i = 0; i < a->ncb; i++) a->cbi[i] = si->pCommandBufferInfos[i];
+    if (a->nsig) a->sig = si->pSignalSemaphoreInfos[0];
+    __atomic_store_n(&q_subs, q_subs + 1, __ATOMIC_RELEASE);
+    RS_.submits++;
+    // TOTK_REC_ASYNC=1 (default): the submitting thread goes on while the recorder catches up with the part and submits
+    // it; it waited here for 0.29 ms a frame in the town on a Ryzen 7900X, 0.34 ms limited to 4 cores. The queue order
+    // the present, uploads, readbacks and device waits rely on holds through vk_queue_lock (rec_queue_wait). The values
+    // the part signals are counted as issued at once (vk_sync.c): waiting for one on the host before its submission is
+    // legal for a timeline semaphore. =0: waits here for the submission, as before; =ab with the A/B blocks.
+    if (rec_async < 0) rec_async = rt_knob("REC_ASYNC", "1");
+    if (RT_KNOB_ON(rec_async)) {
+        pub_now();
+        return __atomic_load_n(&r_sub_err, __ATOMIC_ACQUIRE);
+    }
     pub();
     rec_drain();
-    rec_live = 0;
-    return 1;
+    return r_sub_res;
+}
+
+static void submit_run(const A_Submit *a) {
+    VkSubmitInfo2 si = {.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO_2, .commandBufferInfoCount = a->ncb,
+                        .pCommandBufferInfos = a->cbi, .signalSemaphoreInfoCount = a->nsig,
+                        .pSignalSemaphoreInfos = a->nsig ? &a->sig : NULL};
+    vk_queue_lock();
+    VkResult r = vkQueueSubmit2(vk_ctx()->gfx_queue, 1, &si, VK_NULL_HANDLE);
+    vk_queue_unlock();
+    r_sub_res = r;
+    if (r != VK_SUCCESS && __atomic_load_n(&r_sub_err, __ATOMIC_RELAXED) == VK_SUCCESS)
+        __atomic_store_n(&r_sub_err, r, __ATOMIC_RELEASE);
+    __atomic_store_n(&r_subs, r_subs + 1, __ATOMIC_RELEASE);
+}
+
+void rec_queue_wait(void) {
+    if (!rec_thread || on_rec) return;
+    uint64_t want = __atomic_load_n(&q_subs, __ATOMIC_ACQUIRE);
+    if (__atomic_load_n(&r_subs, __ATOMIC_ACQUIRE) >= want) return;
+    uint64_t t0 = now_ns();
+    for (int i = 0; __atomic_load_n(&r_subs, __ATOMIC_ACQUIRE) < want; i++)
+        if (i < 20000) _mm_pause();
+        else plat_yield();
+    __atomic_add_fetch(&RQ_.n, 1, __ATOMIC_RELAXED);
+    __atomic_add_fetch(&RQ_.ns, now_ns() - t0, __ATOMIC_RELAXED);
 }
 
 void rec_stats_log(void) {
     if (!(rt_trace & RT_TRACE_PERF) || !RS_.parts) return;
     rt_log("vk_rec: %llu of %llu parts recorded by the recorder: %llu packets (%.1f MiB), %llu drains (%llu for a call "
-           "without a packet) waited %.2f ms in all, ring full %llu times",
+           "without a packet) waited %.2f ms in all, ring full %llu times, %llu submissions made by the recorder, "
+           "recorder slept %llu times and was woken %llu times, queue users waited for its submissions %llu times (%.2f ms)",
            (unsigned long long)RS_.parts_on, (unsigned long long)RS_.parts, (unsigned long long)RS_.packets,
            RS_.words * 8 / 1048576.0, (unsigned long long)RS_.drains, (unsigned long long)RS_.fallbacks,
-           RS_.drain_ns / 1e6, (unsigned long long)RS_.full_waits);
+           RS_.drain_ns / 1e6, (unsigned long long)RS_.full_waits, (unsigned long long)RS_.submits,
+           (unsigned long long)__atomic_exchange_n(&r_sleeps, 0, __ATOMIC_RELAXED), (unsigned long long)RS_.wakes,
+           (unsigned long long)__atomic_exchange_n(&RQ_.n, 0, __ATOMIC_RELAXED),
+           __atomic_exchange_n(&RQ_.ns, 0, __ATOMIC_RELAXED) / 1e6);
     memset(&RS_, 0, sizeof RS_);
 }
 
@@ -235,7 +406,7 @@ static const char *const kind_name[REC_KINDS] = {"image", "view", "buffer"};
 static volatile int32_t chk_seen[2], chk_named, chk_logged, chk_hz[2][2][REC_KINDS][64];   // [destroy][made][kind][op]
 void rec_check_handle(int kind, uint64_t handle, int destroy, uint64_t tl, uint64_t gpu) {
     if (!rec_chk || !handle || kind < 0 || kind >= REC_KINDS || !__atomic_load_n(&rec_live, __ATOMIC_RELAXED)) return;
-    uint64_t h = __atomic_load_n(&head, __ATOMIC_ACQUIRE), p = __atomic_load_n(&part_h0, __ATOMIC_ACQUIRE);
+    uint64_t h = __atomic_load_n(&Q.head, __ATOMIC_ACQUIRE), p = __atomic_load_n(&part_h0, __ATOMIC_ACQUIRE);
     uint64_t fence = __atomic_load_n(&part_fence, __ATOMIC_ACQUIRE);
     if (p > h || h - p > RING_WORDS / 2) return;   // a part this large is not searched
     plat_inc32(&chk_seen[destroy != 0]);
@@ -246,7 +417,7 @@ void rec_check_handle(int kind, uint64_t handle, int destroy, uint64_t tl, uint6
         if (op != R_PAD)
             for (uint32_t i = 1; i < n; i++) {
                 if (ring[(p + i) % RING_WORDS] != handle) continue;
-                int made = p < __atomic_load_n(&tail, __ATOMIC_ACQUIRE);
+                int made = p < __atomic_load_n(&Q.tail, __ATOMIC_ACQUIRE);
                 if (!destroy && tl >= fence) { plat_inc32(&chk_named); return; }
                 plat_inc32(&chk_hz[destroy][made][kind][op & 63]);
                 if (plat_inc32(&chk_logged) <= 64)
@@ -280,6 +451,8 @@ void rec_check_log(void) {
 }
 
 #define OWNED(cb) (rec_live && (cb) == rec_cb)
+
+int rec_owns(VkCommandBuffer cb) { return OWNED(cb); }
 // A call the ring takes no packet for: everything queued first, then the call here.
 #define FALLBACK() (rec_drain(), RS_.fallbacks++)
 
@@ -642,7 +815,14 @@ TARGET_CMD(rec_write_mask, R_WriteMask, PFN_vkCmdSetColorWriteMaskEXT, VkColorCo
 // ---- the recorder's side
 
 static void exec_packet(uint32_t op, void *p) {
-    VkCommandBuffer cb = rec_cb;
+    if (op == R_BeginCb) {
+        A_BeginCb *a = p;
+        r_cb = a->cb;
+        if (a->begin) begin_now(r_cb, a->crumb, a->n);
+        return;
+    }
+    if (op == R_Submit) { submit_run(p); return; }
+    VkCommandBuffer cb = r_cb;
     switch (op) {
 #define F(name, ...) case R_##name: { A_##name *a = p; vkCmd##name(cb, __VA_ARGS__); break; }
     F(BeginQuery, a->p, a->q, a->f)

runtime/vk_rec.h

@@ -1,11 +1,14 @@
-// The recorder thread (vk_rec.c): the Vulkan calls the replay makes into its command buffer are made by a thread of
-// their own. vk_sync.c brackets each part of a submission with rec_begin (after vkBeginCommandBuffer) and rec_end
-// (before vkEndCommandBuffer). In between, every vkCmd* on that command buffer copies its arguments into a packet of a
-// ring, in call order, and the recorder makes the real call in that order; calls into any other command buffer (uploads,
-// the window, probes) are made at once. A call the ring has no packet for (a structure with a pNext chain) waits for
-// the recorder to empty the ring and is then made at once on the calling thread. rec_end returns once the recorder made
-// every call. vk_sync.c ends a part through rec_end_cb: the recorder makes the last calls into a part it recorded,
-// vkEndCommandBuffer included; submitting and the command pool stay on the submitting thread.
+// The recorder thread (vk_rec.c): the Vulkan calls the replay makes into its command buffers are made by a thread of
+// their own. vk_sync.c begins each part of a submission (and the mirror prologue) with rec_begin, ends it with
+// rec_end_cb and submits it with rec_submit. In between, every vkCmd* on that command buffer copies its arguments
+// into a packet of a ring, in call order, and the recorder makes the real call in that order; calls into any other
+// command buffer (uploads, the window, probes) are made at once. A call the ring has no packet for (a structure with a
+// pNext chain) waits for the recorder to empty the ring and is then made at once on the calling thread. With
+// TOTK_REC_END=1 (default) the recorder also begins, ends and submits the parts it records, so a part's command buffer
+// never changes threads. rec_submit queues the submission and returns at once (TOTK_REC_ASYNC=1, default; 0: once the
+// recorder submitted it); every other user of the queue waits in vk_queue_lock until the recorder submitted what was
+// queued before (rec_queue_wait), so queue order stays the order of the calls. The submitting thread allocates from
+// the part pool only while the recorder is idle (vk_sync.c claim_slot).
 //
 // Every vkCmd* the runtime calls through vk.h goes through the rc_* wrappers below (the macros at the end); the
 // extension commands called through function pointers have rec_* wrappers of their own. TOTK_VK_RECORDER=0 makes every
@@ -15,12 +18,22 @@
 // MIT, Copyright (c) 2026 Peiikai): the driver's work behind the commands depends on nothing the replay decides next.
 #pragma once
 
-void rec_begin(VkCommandBuffer cb, uint64_t fence);   // the submitting thread, sub_lock held; fence: the part's value
+// The submitting thread, sub_lock held: vkBeginCommandBuffer (one submission) on cb and the breadcrumb crumb (0: none,
+// vk_crumbs.c) with number n, made by the recorder when it records the part, else at once; fence: the part's value.
+void rec_begin(VkCommandBuffer cb, uint64_t fence, int crumb, uint64_t n);
 void rec_end(void);                   // waits for the recorder; no-op when rec_begin did not start it
-// The end of part cb: the breadcrumb crumb (0: none, vk_crumbs.c), then vkEndCommandBuffer, both made by the recorder
-// when it records cb, and the wait for it (1); otherwise rec_end and the breadcrumb, and the caller ends cb (0).
+// The end of part cb: the breadcrumb crumb, then vkEndCommandBuffer, both queued for the recorder when it records cb
+// (1: rec_submit hands the part over); otherwise rec_end and the breadcrumb, and the caller ends cb (0).
 int rec_end_cb(VkCommandBuffer cb, int crumb, uint64_t n);
+// vkQueueSubmit2 of si (at most two command buffers, one signal, no wait, no fence) under vk_queue_lock: by the
+// recorder after a part it ended, else at once. Queued for the recorder, it returns the first failure of an earlier
+// submission the recorder made (VK_SUCCESS if none), since this one's result is not known yet.
+VkResult rec_submit(const VkSubmitInfo2 *si);
 void rec_drain(void);                 // waits until the recorder made every call queued so far
+// vk_queue_lock, before it takes the lock (not on the recorder, not when the thread holds the lock already): waits
+// until the recorder submitted every submission queued so far.
+void rec_queue_wait(void);
+int rec_owns(VkCommandBuffer cb);     // the recorder makes cb's calls: a direct call into cb follows rec_drain
 void rec_stats_log(void);             // TOTK_TRACE=perf: packets, bytes, drains and waits since the last call
 // TOTK_REC_CHECK=1 (debug, recorder on): vk_memory.c reports each object it defers (destroy 0, stamped tl) or destroys
 // (destroy 1) with the value the GPU reached (gpu); a packet of the part being recorded that still names it is logged

runtime/vk_sync.c

@@ -56,7 +56,7 @@ static VkQueryPool ts_pool;
 static int ts_state;                       // 0 not set up, 1 on, -1 off
 static double ts_period;                   // ns per tick
 static uint64_t ts_mask;                   // valid bits of a timestamp
-static uint32_t ts_next;                   // submitting thread (sub_lock): next pair
+static uint32_t ts_next;                   // next pair (atomic: parts recorded on other threads take pairs too)
 static volatile uint8_t ts_busy[TS_PAIRS]; // written until the completion thread read the pair
 static int ts_cur = -1;                    // the pair of the part being recorded, -1 untimed
 static uint64_t ts_last_end;               // completion thread: end of the latest part read (ticks)
@@ -175,6 +175,12 @@ static void ts_read_cb(void *arg) {   // completion thread, in submission order
     free(t);
 }
 
+static int ts_take(void) {
+    uint32_t p = __atomic_fetch_add(&ts_next, 1, __ATOMIC_RELAXED) % TS_PAIRS;
+    if (__atomic_exchange_n(&ts_busy[p], 1, __ATOMIC_ACQUIRE)) { plat_inc64(&ts_missed); return -1; }
+    return (int)p;
+}
+
 // At the start of a command buffer (sub_lock held): the pair it times, -1 untimed.
 static int ts_open(VkCommandBuffer cb) {
     if (ts_state != 1 || !perf_gpu_timing()) {
@@ -182,10 +188,9 @@ static int ts_open(VkCommandBuffer cb) {
         ts_setup(vk_ctx());   // the overlay turned full: timestamps from now on
         if (ts_state != 1) return -1;
     }
-    uint32_t p = ts_next % TS_PAIRS;
-    if (__atomic_load_n(&ts_busy[p], __ATOMIC_ACQUIRE)) { plat_inc64(&ts_missed); return -1; }
-    ts_next++;
-    ts_busy[p] = 1;
+    int t = ts_take();
+    if (t < 0) return -1;
+    uint32_t p = (uint32_t)t;
     vkCmdResetQueryPool(cb, ts_pool, 2 * p, 2);
     vkCmdWriteTimestamp(cb, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, ts_pool, 2 * p);
     return (int)p;
@@ -203,6 +208,17 @@ static void ts_close(VkCommandBuffer cb, int pair, uint64_t fence) {
     vk_completion_add(fence, ts_read_cb, t);
 }
 
+int vk_sync_ts_open(VkCommandBuffer cb) {
+    if (__atomic_load_n(&ts_state, __ATOMIC_ACQUIRE) != 1 || !perf_gpu_timing()) return -1;
+    int p = ts_take();
+    if (p < 0) return -1;
+    vkCmdResetQueryPool(cb, ts_pool, 2 * (uint32_t)p, 2);
+    vkCmdWriteTimestamp(cb, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, ts_pool, 2 * (uint32_t)p);
+    return p;
+}
+
+void vk_sync_ts_close(VkCommandBuffer cb, int pair, uint64_t fence) { ts_close(cb, pair, fence); }
+
 // The pair of the part being recorded.
 static void ts_begin(VkCommandBuffer cb) { ts_cur = ts_open(cb); }
 
@@ -309,7 +325,7 @@ void vk_sync_start(void) {
         if (reserved < now) reserved = now;
         stop = 0;
         plat_mutex_unlock(&st_lock);
-        vk_mem_note_submit(now);
+        vk_mem_note_fence(now);
         completion_thread = plat_thread_new(completion_main, c, "vk completion");
         if (!completion_thread) rt_fatal("vk: cannot start the completion thread");
         __atomic_store_n(&started, 1, __ATOMIC_RELEASE);
@@ -328,6 +344,7 @@ static int claim_slot(VkCtx *c, uint64_t fence) {
             slots[i].fence = fence;
             return i;
         }
+    rec_drain();   // the part pool is the recorder's while it has packets (vk_rec.h): it is idle for the allocation
     CmdSlot *n = realloc(slots, sizeof *n * (size_t)(nslots + 1));
     if (!n) rt_fatal("vk: out of memory (command buffers)");
     slots = n;
@@ -348,48 +365,72 @@ static CmdSlot *take_slot(VkCtx *c, uint64_t fence) {
 // submitted right before the part that needs them and timed like a part; VK_NULL_HANDLE when there are none. After the
 // part was recorded, sub_lock held; may move slots[].
 // First brings the shadow storage up to date with the CPU's writes, which needs no command buffer.
-static VkCommandBuffer prologue(VkCtx *c, uint64_t fence) {
-    vk_mem_shadow_sync();   // shadow storage: the CPU's writes go in before the part is submitted
-    if (!vk_mem_mirror_pending()) return VK_NULL_HANDLE;
+// direct: the calling thread records, ends and submits it itself while the part being replayed stays the recorder's
+// (vk_sync_part, which the caller made the recorder idle for).
+static VkCommandBuffer prologue(VkCtx *c, uint64_t fence, int direct) {
+    // Shadow storage: the CPU's writes go in before the part is submitted (TOTK_SYNC_AHEAD: queued for the GPU or
+    // made after the GPU finished the readers the game may think done, vk_memory.c).
+    vk_mem_shadow_sync(fence);
+    if (!vk_mem_mirror_pending() && !vk_mem_shadow_deferred()) return VK_NULL_HANDLE;
     int i = claim_slot(c, fence);   // before reading slots: the claim may move it
     VkCommandBuffer cb = slots[i].cb;
-    VkCommandBufferBeginInfo bi = {.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,
-                                   .flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
-    vkBeginCommandBuffer(cb, &bi);
+    if (direct) {
+        VkCommandBufferBeginInfo bi = {.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,
+                                       .flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
+        vkBeginCommandBuffer(cb, &bi);
+    } else {
+        rec_begin(cb, fence, 0, 0);   // a command buffer of the part pool: the recorder's like the parts
+    }
     int pair = ts_open(cb);
+    vk_mem_shadow_prologue(cb, fence);   // before the mirror copies, which read the shadow these bring up to date
     vk_mem_mirror_prologue(cb);
     ts_close(cb, pair, fence);
-    vkEndCommandBuffer(cb);
+    if (direct || !rec_end_cb(cb, 0, 0)) vkEndCommandBuffer(cb);
     return cb;
 }
 
+// A chunk of a chunked replay (vk_exec.c chunks_submit), while the submission's own part is open on the recorder. The
+// recorder begins, records, ends and submits the parts it owns (vk_rec.c rec_end_cb), so the prologue stays off it:
+// the recorder is made idle first, as the part pool allows one thread at a time, and the prologue is recorded, ended
+// and submitted here without taking the recorder's part away.
+void vk_sync_part(VkCommandBuffer cb) {
+    VkCtx *c = vk_ctx();
+    vk_upload_flush(0);
+    rec_drain();
+    VkCommandBuffer pro = prologue(c, cur_fence, 1);
+    VkCommandBufferSubmitInfo cbi[2] = {{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO, .commandBuffer = pro},
+                                        {.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO, .commandBuffer = cb}};
+    VkSubmitInfo2 si = {.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO_2, .commandBufferInfoCount = pro ? 2 : 1,
+                        .pCommandBufferInfos = pro ? cbi : cbi + 1};
+    vk_queue_lock();
+    VkResult r = vkQueueSubmit2(c->gfx_queue, 1, &si, VK_NULL_HANDLE);
+    vk_queue_unlock();
+    if (r != VK_SUCCESS) {
+        vk_device_fault_report(r);
+        rt_fatal("vk: vkQueueSubmit2 of a part of value %llu failed (%d)", (unsigned long long)cur_fence, (int)r);
+    }
+}
+
 VkCommandBuffer vk_sync_split(void) {
     VkCtx *c = vk_ctx();
     VkCommandBuffer cb = slots[cur_slot].cb;
     ts_end(cb, cur_fence);
     if (!rec_end_cb(cb, 0, 0)) vkEndCommandBuffer(cb);   // every command of the part is in cb, ended by its recorder
     vk_upload_flush(0);   // the uploads this part asked for precede it in queue order
-    VkCommandBuffer pro = prologue(c, cur_fence);
+    VkCommandBuffer pro = prologue(c, cur_fence, 0);
     VkCommandBufferSubmitInfo cbi[2] = {{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO, .commandBuffer = pro},
                                         {.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO, .commandBuffer = cb}};
     VkSubmitInfo2 si = {.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO_2, .commandBufferInfoCount = pro ? 2 : 1,
                         .pCommandBufferInfos = pro ? cbi : cbi + 1};
     ts_submitted();
-    vk_queue_lock();
-    VkResult r = vkQueueSubmit2(c->gfx_queue, 1, &si, VK_NULL_HANDLE);
-    vk_queue_unlock();
+    VkResult r = rec_submit(&si);   // by the recorder when it recorded the part
     if (r != VK_SUCCESS) {
         vk_device_fault_report(r);
         rt_fatal("vk: vkQueueSubmit2 of a part of value %llu failed (%d)", (unsigned long long)cur_fence, (int)r);
     }
-    vk_mem_note_submit(cur_fence);   // objects destroyed from here on may be in use by the part just submitted
     CmdSlot *s = take_slot(c, cur_fence);   // the submitted part keeps its slot until the value is reached
-    VkCommandBufferBeginInfo bi = {.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,
-                                   .flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
-    vkBeginCommandBuffer(s->cb, &bi);
-    vk_crumb(s->cb, VK_CRUMB_SUBMIT, cur_fence);
+    rec_begin(s->cb, cur_fence, VK_CRUMB_SUBMIT, cur_fence);
     ts_begin(s->cb);
-    rec_begin(s->cb, cur_fence);
     return s->cb;
 }
 
@@ -411,14 +452,13 @@ uint64_t vk_sync_submit(const uint64_t *handles, int n) {
     vk_mem_dirty_submit();   // TOTK_TRACE=perf: what a mirror would upload before this submit
     uint64_t fence = __atomic_load_n(&issued, __ATOMIC_ACQUIRE) + 1;
     cur_fence = fence;
+    // Before the first object is resolved: what any thread destroys from here on waits for this value (vk_memory.c
+    // defer), since the recording may already name it.
+    vk_mem_note_fence(fence);
     CmdSlot *s = take_slot(c, fence);
-    VkCommandBufferBeginInfo bi = {.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,
-                                   .flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
     uint64_t t0 = perf_begin();
-    vkBeginCommandBuffer(s->cb, &bi);
-    vk_crumb(s->cb, VK_CRUMB_SUBMIT, fence);
+    rec_begin(s->cb, fence, VK_CRUMB_SUBMIT, fence);   // begun by the recorder when it records the part
     ts_begin(s->cb);
-    rec_begin(s->cb, fence);
     int recorded = vk_exec_record(s->cb, handles, n, fence);
     s = &slots[cur_slot];   // the last part (vk_sync_split)
     if (recorded) ts_end(s->cb, fence);
@@ -428,7 +468,7 @@ uint64_t vk_sync_submit(const uint64_t *handles, int n) {
     perf_replay(t0);
     // After the recording: the uploads it asked for (and every earlier one) precede it in queue order.
     vk_upload_flush(0);
-    VkCommandBuffer pro = prologue(c, fence);
+    VkCommandBuffer pro = prologue(c, fence, 0);
     s = &slots[cur_slot];
     VkCommandBufferSubmitInfo cbi[2] = {{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO, .commandBuffer = pro},
                                         {.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO, .commandBuffer = s->cb}};
@@ -439,15 +479,12 @@ uint64_t vk_sync_submit(const uint64_t *handles, int n) {
                         .pCommandBufferInfos = pro ? cbi : cbi + 1, .signalSemaphoreInfoCount = 1,
                         .pSignalSemaphoreInfos = &sig};
     ts_submitted();
-    vk_queue_lock();
-    VkResult r = vkQueueSubmit2(c->gfx_queue, 1, &si, VK_NULL_HANDLE);
-    vk_queue_unlock();
+    VkResult r = rec_submit(&si);   // by the recorder when it recorded the part
     if (r != VK_SUCCESS) {
         vk_device_fault_report(r);
         rt_fatal("vk: vkQueueSubmit2 of value %llu failed (%d)", (unsigned long long)fence, (int)r);
     }
     s->fence = recorded ? fence : 0;
-    vk_mem_note_submit(fence);
     plat_mutex_lock(&st_lock);
     __atomic_store_n(&issued, fence, __ATOMIC_RELEASE);
     plat_cond_broadcast(&cv_work);
@@ -458,8 +495,11 @@ uint64_t vk_sync_submit(const uint64_t *handles, int n) {
     return fence;
 }
 
-void vk_completion_add(uint64_t fence, void (*fn)(void *), void *arg) {
-    plat_mutex_lock(&st_lock);
+static _Thread_local VkDeferred *deferred;
+
+void vk_completion_defer(VkDeferred *d) { deferred = d; }
+
+static void queue_put(uint64_t fence, void (*fn)(void *), void *arg) {   // st_lock held
     if (qhead + qn == qcap) {
         if (qhead) {
             memmove(queue, queue + qhead, sizeof *queue * (size_t)qn);
@@ -475,9 +515,34 @@ void vk_completion_add(uint64_t fence, void (*fn)(void *), void *arg) {
     // The completion thread sleeps only while nothing is issued beyond done: a callback for a later value (the replay's,
     // for the submission it records: counter reports, freed resources) becomes due through the submit, which wakes it.
     if (fence <= done) plat_cond_broadcast(&cv_work);
+}
+
+void vk_completion_add(uint64_t fence, void (*fn)(void *), void *arg) {
+    VkDeferred *d = deferred;
+    if (d) {
+        if (d->n == d->cap) {
+            d->cap = d->cap ? d->cap * 2 : 64;
+            d->items = realloc(d->items, sizeof(Pending) * (size_t)d->cap);
+            if (!d->items) rt_fatal("vk: out of memory (completion queue)");
+        }
+        ((Pending *)d->items)[d->n++] = (Pending){fence, fn, arg};
+        return;
+    }
+    plat_mutex_lock(&st_lock);
+    queue_put(fence, fn, arg);
     plat_mutex_unlock(&st_lock);
 }
 
+void vk_completion_flush(VkDeferred *d) {
+    plat_mutex_lock(&st_lock);
+    for (int i = 0; i < d->n; i++) {
+        const Pending *p = &((Pending *)d->items)[i];
+        queue_put(p->fence, p->fn, p->arg);
+    }
+    plat_mutex_unlock(&st_lock);
+    d->n = 0;
+}
+
 static int fence_wait_impl(uint64_t value, uint64_t timeout_ns);
 
 int vk_sync_fence_wait(uint64_t value, uint64_t timeout_ns) {

runtime/vk_texture.c

@@ -46,6 +46,8 @@ typedef struct Tex {
     uint32_t wgen;                // page generation of the last upload check
     uint32_t checked;             // submit_epoch of the last check from a draw binding
     int uploaded, released;
+    uint32_t rev_epoch, rev_pos;  // the state scan made it live in that submission at that op (vk_texture_revive)
+    uint32_t own_epoch, own_pos;  // a chunk made it GPU-owned in that submission at that op (vk_texture_gpu_written)
     uint32_t n_up;                // uploads of this entry (vk_texture_debug)
     int last_ch;                  // last vk_mem_changed result of make_fresh: 1 written, 0 clean, -1 not tracked
     struct Tex *next_at;          // other entries at the same storage address
@@ -61,6 +63,11 @@ typedef struct Tex {
     uint8_t *snap;                // a texture that fills no page: its storage bytes when the GPU write was recorded
     uint32_t wt_n;                // WriteTexels into the storage since the last upload
     struct Tex *rel_prev, *rel_next;   // while released: the list of released entries, oldest first
+    uint64_t rel_ms;              // while released: when (coarse_ms)
+    uint64_t rel_bytes;           // while released: its own memory, counted in rel_bytes_total
+    uint64_t use_seq;             // order of the last creation or take-back among all entries
+    uint64_t rel_seq;             // while released: use_seq of the newest entry placed before this release; an entry
+                                  // with a later use_seq on overlapping storage replaced this one's bytes
 } Tex;
 
 // A slot keeps what vk_texture_by_id resolved for it, so a draw binding skips the key lookup and the address chain: the
@@ -89,6 +96,7 @@ static SampSlot *sslots;
 static int nsslots;
 static nvn_hmap samplers = {.klen = sizeof(samp_par)};
 static int nreleased;
+static uint64_t rel_bytes_total;   // own memory of the released entries
 static Tex *rel_head, *rel_tail;   // released entries in release order (sweep_released drops from the head)
 static uint64_t n_uploads, n_reuploads;
 // Re-uploads since the last perf line (tex_lock): all, and those of textures a WriteTexels wrote since their last upload.
@@ -118,10 +126,12 @@ static uint64_t n_checks;    // draw-binding storage checks (make_fresh_snap) si
 // which emptied every cache keyed on the global generations.
 #define AGEN_N 4096
 static uint32_t agen[AGEN_N];
+static uint32_t arev[AGEN_N];   // per storage address (hashed): the submit_epoch of the state scan's last revival there
 // Per slot id, for vk_texture_slot_token without the lock: the registration serial (bits 16-47; 0: never registered)
 // and the address generation index (bits 0-15), one word so a reader gets a consistent pair. Fixed-size (slot ids are
 // 20 bits), unlike tslots, which moves when it grows.
 static uint64_t *sinfo;
+static uint32_t *sepoch;   // per slot: the submit_epoch its entry was checked in (vk_texture_slot_checked), 0 none
 static uint32_t reg_serial;
 
 static uint32_t agen_idx(uint64_t addr) { return (uint32_t)(addr * 0x9E3779B97F4A7C15ull >> 52); }
@@ -139,11 +149,17 @@ static void tex_init(void *p) {
     __atomic_store_n(&tex_ready, 1, __ATOMIC_RELEASE);
     return;
 }
+static _Thread_local int tex_depth;   // how often this thread holds tex_lock (entry_of releases it for a wait)
+static _Thread_local uint32_t tex_settles;   // entry_of waits of this thread (vk_texture_by_id_locked)
 static void lock(void) {
     if (!__atomic_load_n(&tex_ready, __ATOMIC_ACQUIRE)) plat_once(&tex_once, tex_init, NULL);
     plat_rmutex_lock(&tex_lock);
+    tex_depth++;
+}
+static void unlock(void) {
+    tex_depth--;
+    plat_rmutex_unlock(&tex_lock);
 }
-static void unlock(void) { plat_rmutex_unlock(&tex_lock); }
 
 static uint32_t umax(uint32_t a, uint32_t b) { return a > b ? a : b; }
 static uint32_t mip(uint32_t v, unsigned lv) { return umax(v >> lv, 1); }
@@ -443,8 +459,19 @@ static Tex *create_tex(const tex_t *t, const TexKey *k, int defer_init) {
         }
     }
     if (!img) {
-        VmaAllocationCreateInfo ai = {.usage = VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE};
-        VkResult r = vmaCreateImage(c->vma, &ci, &ai, &img, &e->alloc, NULL);
+        VkResult r = rt_oom_forced(RT_OOM_IMAGE) ? VK_ERROR_OUT_OF_DEVICE_MEMORY
+                                                 : vk_texture_image(&ci, &img, &e->alloc);
+        // Out of memory: one more try after the headroom blocks and the finished deferred objects are freed
+        // (vk_oom_release, which takes cmd_lock after the texture lock held here); released textures go at the next
+        // submit. No wait: the texture is asked for again at its next use, while a wait would hold the texture lock.
+        if (r == VK_ERROR_OUT_OF_HOST_MEMORY || r == VK_ERROR_OUT_OF_DEVICE_MEMORY) {
+            static volatile int32_t n_retried;
+            vk_oom_release();
+            VkResult again = vk_texture_image(&ci, &img, &e->alloc);
+            log_limited(&n_retried, "image creation failed (%d), tried again after a memory release: %s (%ux%ux%u fmt %#x)",
+                        (int)r, again == VK_SUCCESS ? "created" : "failed again", w, h, depth, d->format);
+            r = again;
+        }
         if (r != VK_SUCCESS) {
             log_limited(&n_failed, "image creation failed (%d): nvn fmt %#x %ux%ux%u target %d levels %u layers %u", (int)r,
                         d->format, w, h, depth, d->target, levels, layers);
@@ -524,7 +551,11 @@ static void own_remove(Tex *e) {
     plat_mutex_unlock(&owned_lock);
 }
 
+static uint64_t own_bytes(const Tex *e);
+
 static void rel_append(Tex *e) {
+    e->rel_bytes = own_bytes(e);
+    rel_bytes_total += e->rel_bytes;
     e->rel_next = NULL;
     e->rel_prev = rel_tail;
     if (rel_tail) rel_tail->rel_next = e;
@@ -533,6 +564,8 @@ static void rel_append(Tex *e) {
 }
 
 static void rel_unlink(Tex *e) {
+    rel_bytes_total -= e->rel_bytes;
+    e->rel_bytes = 0;
     if (e->rel_prev) e->rel_prev->rel_next = e->rel_next;
     else rel_head = e->rel_next;
     if (e->rel_next) e->rel_next->rel_prev = e->rel_prev;
@@ -540,6 +573,46 @@ static void rel_unlink(Tex *e) {
     e->rel_prev = e->rel_next = NULL;
 }
 
+// A clock read per submit, for stamps on paths that run many times a frame.
+static uint64_t coarse_ms;
+static uint64_t use_counter;
+
+// How long released entries wait until the game takes them back, split into those holding GPU-written content and the
+// others (TOTK_TRACE=perf, vk_texture_released_log): where transient textures end and the tail that can go begins.
+enum { AGE_1S, AGE_5S, AGE_30S, AGE_120S, AGE_LONG, AGE_N };
+static const char *const age_name[AGE_N] = {"<1s", "<5s", "<30s", "<120s", ">=120s"};
+static uint64_t takeback_n[2][AGE_N];
+static int age_bucket(uint64_t ms) {
+    return ms < 1000 ? AGE_1S : ms < 5000 ? AGE_5S : ms < 30000 ? AGE_30S : ms < 120000 ? AGE_120S : AGE_LONG;
+}
+
+// Keys of entries the pressure path dropped, to count the ones the game asks for again soon after: a drop that buys
+// nothing and, for GPU-written content, loses it. Direct-mapped, a lost record only undercounts.
+#define DROPPED_SLOTS 4096u
+static struct { uint64_t hash, ms; } dropped[DROPPED_SLOTS];
+static uint64_t n_recreated[2], n_recreated_written[2];   // index: within 10 s, later
+static uint64_t key_hash(const TexKey *k) {
+    uint64_t h = 0xcbf29ce484222325ull;
+    const uint8_t *b = (const uint8_t *)k;
+    for (size_t i = 0; i < sizeof *k; i++) h = (h ^ b[i]) * 0x100000001b3ull;
+    return h | 1;
+}
+static void dropped_note(const Tex *e) {
+    uint64_t h = key_hash(&e->key);
+    dropped[h >> 3 & (DROPPED_SLOTS - 1)].hash = h ^ (uint64_t)(e->pub.gpu_owned != 0) << 1;
+    dropped[h >> 3 & (DROPPED_SLOTS - 1)].ms = coarse_ms;
+}
+static void dropped_check(const TexKey *k) {
+    uint64_t h = key_hash(k);
+    unsigned i = h >> 3 & (DROPPED_SLOTS - 1);
+    uint64_t was = dropped[i].hash;
+    if (!was || (was & ~2ull) != (h & ~2ull)) return;
+    int later = coarse_ms - dropped[i].ms >= 10000;
+    n_recreated[later]++;
+    if (was != h) n_recreated_written[later]++;
+    dropped[i].hash = 0;
+}
+
 static void destroy_tex(Tex *e) {
     own_remove(e);
     tex_gen_bump(e->key.addr);
@@ -553,10 +626,16 @@ static void destroy_tex(Tex *e) {
             if (p->next_at == e) { p->next_at = e->next_at; break; }
     }
     if (e->released) nreleased--, rel_unlink(e);
+    // Another thread may hold &e->pub, taken under the lock and used after it (vk_texture_get, vk_texture_by_id: render
+    // targets for a whole pass, vk_exec.c resolve_target, vk_texture_storage_bound_locked writes through it). Freed
+    // here, that thread read a stale VkImage from freed memory, made views of it and cached them by that handle. The
+    // record goes with the image, once the submission being recorded is done; gone keeps views of it out of the caches.
+    e->pub.gone = 1;
     vk_exec_image_gone(e->pub.image);
     vk_defer_image(e->pub.image, e->alloc);
     free(e->snap);
-    free(e);
+    e->snap = NULL;
+    vk_defer_free(e);
 }
 
 // Released entries with another descriptor at the same address are replaced; past RELEASED_MAX released entries the
@@ -569,34 +648,173 @@ static void sweep_released(void) {
     while (nreleased > RELEASED_MAX && rel_head) destroy_tex(rel_head);
 }
 
-// TOTK_TEX_VRAM_PRESSURE (percent of the device-local budget, default 90; 0 = off): above it, each submit drops up to
-// PRESSURE_STEP of the oldest released entries, so a 4 GiB card does not keep RELEASED_MAX released images (each ASTC
-// one decoded to RGBA8) while the heap spills into system memory. A few per submit, for the reason above. Only entries
-// that come back intact are dropped: mirrored ones the GPU never wrote, re-uploaded from their storage. A render target
-// lives only in its image; dropping it brought the texture back empty (black flashes in a Depths boss fight, ~170
+// TOTK_TEX_VRAM_PRESSURE (percent of the device-local budget, default 90; 0 = off): above it, each budget query drops up
+// to PRESSURE_STEP released entries, oldest first, until the use is back under the line; a few at a time, for the reason
+// above. Two kinds come back without loss:
+//
+// Overwritten: the game placed another texture on the entry's storage after releasing it, so on the Switch its bytes are
+// gone too, and it has waited DEAD_AGE_MS. Transient render targets are taken back within a second or two (a stand-still
+// field window: about 19000 take-backs under 1 s, 2 later than that, none after 30 s), while their leftovers at
+// addresses the game's allocator moved on from stay released for good: 1.9 GiB of 2.0 GiB, nearly all of it render targets
+// no storage can refill. Not bound through a finalized slot for DEAD_QUIET_SUBMITS either: such a slot still samples the
+// entry (slot_resolve).
+//
+// Clean: mirrored ones the GPU never wrote, re-uploaded from their storage, at the cost of the upload, so after
+// CLEAN_AGE_MS: without the wait a fight re-created about 500 of them per 10 s. Any other render target lives only in its
+// image: dropping one that was merely old brought the texture back empty (black flashes in a Depths boss fight, ~170
 // "left uninitialised" aliases a minute).
+//
+// TOTK_TEX_RELEASED_MIB (default 256; 0 = no cap): the same drops run while the released entries hold more own memory than
+// that, under pressure or not. A 10-minute field run on the RX 9070 kept 818 MiB of them, 730 MiB with their storage
+// taken by a later texture, while 23579 of 23660 take-backs came within a second and none after 120 s. With the
+// cap the same route ended with 216 MiB of them, 1.1 GB less process commit and 1 GB less in VMA blocks; 15 dropped
+// textures were created again, and the slow frames stayed the same.
 #define PRESSURE_STEP 32
+#define CLEAN_AGE_MS 5000u
+#define DEAD_AGE_MS 30000u
+#define DEAD_QUIET_SUBMITS 600u
+#define DEAD_RETRY_QUERIES 8   // budget queries to skip after a search that found nothing
 static int vram_pressure = -1;
-static uint64_t n_pressure_drops;
+static uint64_t released_cap;
+static uint64_t n_pressure_drops, n_pressure_overwritten;
+
+static int tex_by_addr(const void *a, const void *b) {
+    const Tex *x = *(Tex *const *)a, *y = *(Tex *const *)b;
+    return x->pub.addr < y->pub.addr ? -1 : x->pub.addr > y->pub.addr;
+}
+
+// All entries sorted by storage address, with the highest storage end up to each one: the input of storage_taken.
+typedef struct { Tex **e; uint64_t *end; int n; } AddrIndex;
+
+static int addr_index(AddrIndex *x) {
+    x->n = 0;
+    x->e = malloc(sizeof *x->e * (size_t)(by_key.n + 1));
+    x->end = malloc(sizeof *x->end * (size_t)(by_key.n + 1));
+    if (!x->e || !x->end) { free(x->e), free(x->end); return 0; }
+    for (uint32_t i = 0; i < by_key.cap; i++)
+        if (by_key.vals[i]) x->e[x->n++] = by_key.vals[i];
+    qsort(x->e, (size_t)x->n, sizeof *x->e, tex_by_addr);
+    uint64_t top = 0;
+    for (int i = 0; i < x->n; i++) {
+        uint64_t end = x->e[i]->pub.addr + x->e[i]->t.L.size;
+        x->end[i] = top = end > top ? end : top;
+    }
+    return 1;
+}
+
+static void addr_index_free(AddrIndex *x) { free(x->e), free(x->end); }
+
+// A texture placed after the release of x->e[i] overlaps its storage.
+static int storage_taken(const AddrIndex *x, int i) {
+    const Tex *e = x->e[i];
+    for (int j = i; --j >= 0 && x->end[j] > e->pub.addr;)
+        if (x->e[j]->pub.addr + x->e[j]->t.L.size > e->pub.addr && x->e[j]->use_seq > e->rel_seq) return 1;
+    for (int j = i; ++j < x->n && x->e[j]->pub.addr < e->pub.addr + e->t.L.size;)
+        if (x->e[j]->use_seq > e->rel_seq) return 1;
+    return 0;
+}
+
+static uint64_t own_bytes(const Tex *e) {
+    VkCtx *c = vk_ctx();
+    if (!e->alloc || !c) return 0;
+    VmaAllocationInfo ai;
+    vmaGetAllocationInfo(c->vma, e->alloc, &ai);
+    return ai.size;
+}
+
+static int dead_candidate(const Tex *e) {
+    return e->alloc && (!e->checked || submit_epoch - e->checked >= DEAD_QUIET_SUBMITS);
+}
+
+// Drops overwritten entries until `need` bytes are free or the step is used up. The sort runs only under pressure, while
+// an old enough entry exists, and rests for a while when it finds nothing to drop. Returns the bytes freed.
+static uint64_t pressure_search_ns;   // longest search since the last mem: released line
+// step: PRESSURE_STEP, or OOM_STEP from oom_release, which neither waits out nor starts the rest.
+#define OOM_STEP 256
+static uint64_t drop_overwritten(uint64_t need, int *dropped, int step) {
+    static int rest;
+    int urgent = step > PRESSURE_STEP;
+    if (rest > 0 && !urgent) { rest--; return 0; }
+    if (!rel_head || coarse_ms - rel_head->rel_ms < DEAD_AGE_MS) return 0;
+    uint64_t t0 = plat_now_ns(), freed = 0;
+    AddrIndex x;
+    if (!addr_index(&x)) return 0;
+    Tex *cand[OOM_STEP];
+    int nc = 0;
+    for (Tex *e = rel_head; e && nc < step && coarse_ms - e->rel_ms >= DEAD_AGE_MS; e = e->rel_next) {
+        if (!dead_candidate(e)) continue;
+        int lo = 0, hi = x.n;   // the first entry at e's address, then e among those
+        while (lo < hi) {
+            int m = (lo + hi) / 2;
+            if (x.e[m]->pub.addr < e->pub.addr) lo = m + 1;
+            else hi = m;
+        }
+        while (lo < x.n && x.e[lo] != e) lo++;
+        if (lo < x.n && storage_taken(&x, lo)) cand[nc++] = e;
+    }
+    addr_index_free(&x);
+    for (int i = 0; i < nc && freed < need; i++) {
+        freed += own_bytes(cand[i]);
+        dropped_note(cand[i]);
+        destroy_tex(cand[i]);
+        (*dropped)++, n_pressure_drops++, n_pressure_overwritten++;
+    }
+    if (!nc && !urgent) rest = DEAD_RETRY_QUERIES;
+    uint64_t ns = plat_now_ns() - t0;
+    if (ns > pressure_search_ns) pressure_search_ns = ns;
+    return freed;
+}
+
+// Clean released entries, oldest first: mirrored ones the GPU never wrote, which come back from their storage.
+static uint64_t drop_clean(uint64_t need, uint64_t freed, int *dropped, int step) {
+    for (Tex *e = rel_head, *next; e && freed < need && *dropped < step && coarse_ms - e->rel_ms >= CLEAN_AGE_MS; e = next) {
+        next = e->rel_next;
+        if (!e->pub.mirrored || e->pub.gpu_owned || e->gpu_dirty) continue;
+        freed += own_bytes(e);
+        dropped_note(e);
+        destroy_tex(e);
+        (*dropped)++, n_pressure_drops++;
+    }
+    return freed;
+}
 
 static void sweep_pressure(void) {   // tex_lock held
     if (vram_pressure < 0) {
         const char *s = cfg_get("TOTK_TEX_VRAM_PRESSURE");
         vram_pressure = s && *s ? atoi(s) : 90;
         if (vram_pressure < 0 || vram_pressure > 100) vram_pressure = 90;
-        rt_log("vk: released textures under VRAM pressure: %s%d%%", vram_pressure ? "dropped above " : "off", vram_pressure);
+        const char *m = cfg_get("TOTK_TEX_RELEASED_MIB");
+        released_cap = (m && *m ? strtoull(m, NULL, 10) : 256) << 20;
+        rt_log("vk: released textures under VRAM pressure: %s%d%%; above %llu MiB of them: %s (TOTK_TEX_RELEASED_MIB)",
+               vram_pressure ? "dropped above " : "off", vram_pressure, (unsigned long long)(released_cap >> 20),
+               released_cap ? "dropped" : "no cap");
     }
-    if (!vram_pressure || !rel_head || submit_epoch & 7) return;   // the budget query every 8 submits
+    if (!rel_head || submit_epoch & 7) return;   // the budget query every 8 submits
+    uint64_t need = released_cap && rel_bytes_total > released_cap ? rel_bytes_total - released_cap : 0;
     uint64_t used, budget;
     int exact;
-    if (!vk_mem_vram(&used, &budget, &exact) || used <= budget / 100 * (uint64_t)vram_pressure) return;
-    int dropped = 0;
-    for (Tex *e = rel_head, *next; e && dropped < PRESSURE_STEP; e = next) {
-        next = e->rel_next;
-        if (!e->pub.mirrored || e->pub.gpu_owned || e->gpu_dirty) continue;
-        destroy_tex(e);
-        dropped++, n_pressure_drops++;
+    if (vram_pressure && vk_mem_vram(&used, &budget, &exact)) {
+        uint64_t line = budget / 100 * (uint64_t)vram_pressure;
+        if (used > line && used - line > need) need = used - line;
     }
+    if (!need) return;
+    int dropped = 0;
+    uint64_t freed = drop_overwritten(need, &dropped, PRESSURE_STEP);
+    drop_clean(need, freed, &dropped, PRESSURE_STEP);
+}
+
+// rt_oom_request (an allocation failed somewhere, oom.c): the same two kinds as sweep_pressure, as many as OOM_STEP
+// whatever the budget says, and the headroom blocks. Never GPU-written content that lives only in its image. The memory
+// comes back once the submission being recorded is done (destroy_tex defers the images). Texture lock held.
+static void oom_release(void) {
+    int dropped = 0;
+    uint64_t freed = drop_overwritten(UINT64_MAX, &dropped, OOM_STEP);
+    int overwritten = dropped;
+    freed = drop_clean(UINT64_MAX, freed, &dropped, OOM_STEP);
+    vk_headroom_release();
+    static volatile int32_t n_logged;
+    log_limited(&n_logged, "out of memory: %d released textures dropped (%d overwritten, %d clean), %.0f MiB, %d kept",
+                dropped, overwritten, dropped - overwritten, freed / 1048576.0, nreleased);
 }
 
 // ---------------------------------------------------------------- uploads
@@ -673,15 +891,20 @@ static void astc_check(const Tex *e, const uint8_t *raw, const VkAstcRegion *ar,
 // large ASTC array decodes to RGBA8 in one go: a staging buffer of 676682336 bytes failed (-2) on an Intel UHD with
 // 8 GB of shared memory and the game closed before the title screen.
 #define UPLOAD_PART (32ull << 20)   // fits the staging pool (vk_staging takes up to half of its 64 MiB block)
+// The open upload batch is submitted between chunks (and parts of the CPU path) once its staging and scratch pass this:
+// on fight2's first world frame one batch took hundreds of MiB, and the staging and scratch pools grew by 64 MiB blocks
+// on the replay thread. Submitting earlier keeps the queue order (the batch still precedes the part that samples the
+// images) and lets the GPU finish and return the buffers sooner.
+#define UPLOAD_HELD_MAX (48ull << 20)
 
-// Copies the whole storage into the image (every level and layer the image has).
-static void upload(Tex *e) {
+static int upload_gpu(Tex *e);
+
+// The CPU side of upload: deswizzles (and decodes or expands) into staging buffers, then copies them into the image.
+static void upload_cpu(Tex *e) {
     static volatile int32_t n_bad, n_undecoded, n_split;
     const tex_t *t = &e->t;
     const nvn_texdesc *d = &t->d;
     int bw, bh, bpb = nvn_fmt_bpb(d->format, &bw, &bh);
-    if (!bpb || !t->mem) return;
-    uint64_t p0 = perf_begin(), u0 = kern_now_ns();
     int linear = nvn_tex_is_linear(d), is3d = d->target == NVN_TEXTURE_TARGET_3D, one_d = e->pub.type == VK_IMAGE_TYPE_1D;
     uint32_t levels = linear ? 1 : e->pub.levels, layers = linear ? 1 : e->pub.layers;
     int astc = e->f.astc_bw != 0, rgb = e->f.expand_rgb;
@@ -762,6 +985,7 @@ static void upload(Tex *e) {
         }
         barrier(cb, VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_WRITE_BIT, ALL_RW);
         vk_upload_end();
+        if (!acheck && vk_upload_held() > UPLOAD_HELD_MAX) vk_upload_flush(0);
     }
     if (parts > 1)
         log_limited(&n_split, "texture %#llx fmt %#x %ux%u, %u levels, %u layers: %.1f MiB staged in %d parts",
@@ -778,7 +1002,201 @@ static void upload(Tex *e) {
     free(raw_copy);
     free(ar);
     free(rg);
+}
+
+// Block-linear storage in an imported pool, deswizzled by the GPU (vk_deswizzle.c): the replay thread only records
+// the dispatches, the image copies and (ASTC) the decode into the upload batch the CPU path also uses, which runs before
+// the part that samples the image. The GPU reads the pool when the batch runs, as the Switch's GPU reads it at
+// execution; a CPU write after make_fresh's write-watch reset still marks the pages and brings another upload. Each
+// chunk of regions gets a device-local scratch buffer of at most DESWZ_CHUNK bytes (one region, when larger), freed
+// once the batch completes. TOTK_DESWIZZLE=check compares every chunk with the CPU deswizzle (waits for the GPU).
+// 0 when not handled, before anything is recorded: the CPU path takes the texture.
+#define DESWZ_CHUNK (32ull << 20)
+
+// Uploads by path since the last perf summary (TOTK_TRACE=perf): 0 GPU, else why the CPU took it.
+enum { UP_GPU, UP_MODE, UP_LINEAR, UP_RGB, UP_ASTC_CPU, UP_BPB, UP_NOT_LIVE, UP_GEOM, UP_LIMITS, UP_SCRATCH, UP_WHY };
+static const char *const up_why_name[UP_WHY] = {"gpu", "cpu mode", "linear", "rgb", "astc on cpu", "bytes per block",
+                                                "not an imported pool", "layout", "device limits", "scratch"};
+static uint64_t up_n[UP_WHY], up_bytes[UP_WHY], up_ns[UP_WHY];
+static int up_why;   // upload_gpu's reason for returning 0 (texture lock held)
+
+static int upload_gpu(Tex *e) {
+    static volatile int32_t n_scratch_fail, n_undecoded, n_bad;
+    static volatile int64_t n_checked, n_differ;
+    const tex_t *t = &e->t;
+    const nvn_texdesc *d = &t->d;
+    int mode = vk_deswz_mode();
+    up_why = mode == VK_DESWZ_CPU ? UP_MODE : nvn_tex_is_linear(d) ? UP_LINEAR : e->f.expand_rgb ? UP_RGB : UP_GPU;
+    if (up_why) return 0;
+    int astc = e->f.astc_bw != 0, agpu = astc && vk_astc_gpu();
+    if (astc && (!agpu || vk_astc_mode() == VK_ASTC_CHECK)) return up_why = UP_ASTC_CPU, 0;   // or ASTC's check mode
+    int bw, bh, bpb = nvn_fmt_bpb(d->format, &bw, &bh);
+    if (!bpb || 16 % bpb) return up_why = UP_BPB, 0;   // the rows of the linear copy are padded to 16 bytes
+    VkMemSpan sp;
+    if (!vk_mem_resolve(e->pub.addr, t->L.size, &sp) || !vk_mem_span_live(&sp)) return up_why = UP_NOT_LIVE, 0;
+    int one_d = e->pub.type == VK_IMAGE_TYPE_1D, is3d = d->target == NVN_TEXTURE_TARGET_3D;
+    uint32_t levels = e->pub.levels, layers = e->pub.layers, n = levels * layers, k = 0;
+    VkDeswzRegion *dr = calloc(n, sizeof *dr);
+    VkBufferImageCopy *rg = calloc(n, sizeof *rg);
+    VkAstcRegion *ar = agpu ? calloc(n, sizeof *ar) : NULL;
+    if (!dr || !rg || (agpu && !ar)) rt_fatal("vk: out of memory (texture upload regions)");
+    int ok = 1;
+    VkDeviceSize max_dst = DESWZ_CHUNK;
+    for (uint32_t lv = 0; ok && lv < levels; lv++)
+        for (uint32_t ly = 0; ok && ly < layers; ly++) {
+            uint32_t row_bytes, rows, slices, gbpb;
+            uint64_t base;
+            unsigned th, td;
+            uint32_t w = mip(d->width, lv), h = one_d ? 1 : mip(d->height, lv), sl = is3d ? mip(d->depth, lv) : 1;
+            if (nvn_deswizzle_geom(d, lv, ly, &row_bytes, &rows, &slices, &base, &th, &td, &gbpb) || gbpb != (uint32_t)bpb ||
+                base >= t->L.size || rows != (h + bh - 1) / bh || slices != sl) {
+                ok = 0;
+                break;
+            }
+            uint64_t bhg = 1ull << th, bdg = 1ull << td, gobs_x = (row_bytes + 63) / 64;
+            uint64_t blocks_y = (rows + 8 * bhg - 1) / (8 * bhg), blocks_z = (slices + bdg - 1) / bdg;
+            uint64_t segs = gobs_x * blocks_y * blocks_z * (32ull << (th + td)), room = (t->L.size - base) / 16;
+            if (segs > room) segs = room;
+            uint32_t pitch16 = (row_bytes + 15) / 16;
+            VkDeviceSize out = (VkDeviceSize)pitch16 * 16 * rows * slices;
+            if (segs > UINT32_MAX || gobs_x * blocks_y > UINT32_MAX) { ok = 0; break; }
+            if (out > max_dst) max_dst = out;
+            dr[k] = (VkDeswzRegion){.src = base, .segs = (uint32_t)segs, .pitch16 = pitch16, .rows = rows, .slices = slices,
+                                    .th = th, .td = td, .gobs_x = (uint32_t)gobs_x, .blocks_y = (uint32_t)blocks_y};
+            rg[k] = (VkBufferImageCopy){.bufferRowLength = pitch16 * 16 / (uint32_t)bpb * (uint32_t)bw,
+                                        .bufferImageHeight = rows * (uint32_t)bh,
+                                        .imageSubresource = {e->pub.aspect, lv, ly, 1}, .imageExtent = {w, h, sl}};
+            if (agpu) ar[k] = (VkAstcRegion){.w = w, .h = h, .slices = sl, .level = lv, .layer = ly};
+            k++;
+        }
+    if (!ok || k != n || !vk_deswz_fits(sp.offset, t->L.size, dr, (int)n, max_dst)) {
+        free(dr), free(rg), free(ar);
+        return up_why = !ok || k != n ? UP_GEOM : UP_LIMITS, 0;
+    }
+    VkCtx *c = vk_ctx();
+    VkCommandBuffer cb = vk_upload_begin();
+    // Earlier GPU writes into the pool (copies, shaders) before the reads; the image's earlier uses before its copy.
+    barrier(cb, ALL_RW, VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT | VK_PIPELINE_STAGE_2_COPY_BIT,
+            VK_ACCESS_2_SHADER_STORAGE_READ_BIT | VK_ACCESS_2_TRANSFER_WRITE_BIT);
+    for (uint32_t k0 = 0, k1; k0 < n; k0 = k1) {
+        VkDeviceSize size = 0;
+        for (k1 = k0; k1 < n; k1++) {
+            VkDeviceSize out = (VkDeviceSize)dr[k1].pitch16 * 16 * dr[k1].rows * dr[k1].slices;
+            if (k1 > k0 && size + out > DESWZ_CHUNK) break;
+            dr[k1].dst = size, rg[k1].bufferOffset = size;
+            if (agpu) ar[k1].src = size;
+            size += out;
+        }
+        VkBuffer sb;
+        VmaAllocation sa;
+        if (vk_scratch_buffer(size, &sb, &sa) != VK_SUCCESS) {
+            log_limited(&n_scratch_fail, "texture %#llx: deswizzle scratch of %llu bytes failed%s",
+                        (unsigned long long)e->pub.addr, (unsigned long long)size,
+                        k0 ? "; its later levels left as they were" : "; deswizzled on the CPU");
+            if (!k0) {
+                vk_upload_end();
+                free(dr), free(rg), free(ar);
+                return up_why = UP_SCRATCH, 0;
+            }
+            break;
+        }
+        if (!vk_deswz_record(cb, sp.buffer, sp.offset, t->L.size, sb, size, dr + k0, (int)(k1 - k0))) {
+            log_limited(&n_bad, "texture %#llx: deswizzle of levels %u-%u not recorded; left as they were",
+                        (unsigned long long)e->pub.addr, k0, k1 - 1);   // vk_deswz_fits passed: not expected
+            vk_defer_buffer(sb, sa);
+            break;
+        }
+        barrier(cb, VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT, VK_ACCESS_2_SHADER_STORAGE_WRITE_BIT,
+                VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_READ_BIT);
+        if (agpu) {
+            uint64_t a0 = perf_begin();
+            if (!vk_astc_record(cb, sb, size, ar + k0, (int)(k1 - k0), e->f.astc_bw, e->f.astc_bh, e->f.astc_srgb,
+                                e->f.astc_bc, e->pub.image, e->pub.aspect, VK_NULL_HANDLE))
+                log_limited(&n_undecoded, "ASTC texture %#llx (fmt %#x %ux%u) not decoded on the GPU: left as it was",
+                            (unsigned long long)e->pub.addr, d->format, d->width, d->height);
+            perf_end(PERF_ASTC, a0);
+        } else {
+            vkCmdCopyBufferToImage(cb, sb, e->pub.image, VK_IMAGE_LAYOUT_GENERAL, k1 - k0, rg + k0);
+        }
+        if (mode == VK_DESWZ_CHECK) {   // the linear copy back to the host, compared with the CPU deswizzle
+            VkBuffer rb;
+            VmaAllocation ra;
+            const uint8_t *rmap = vk_staging(size, &rb, &ra);
+            VkBufferCopy cp = {0, 0, size};
+            vkCmdCopyBuffer(cb, sb, rb, 1, &cp);
+            barrier(cb, VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_WRITE_BIT, VK_PIPELINE_STAGE_2_HOST_BIT,
+                    VK_ACCESS_2_HOST_READ_BIT);
+            vk_defer_buffer(sb, sa);
+            vk_upload_end();
+            vk_upload_flush(1);
+            vmaInvalidateAllocation(c->vma, ra, 0, VK_WHOLE_SIZE);
+            for (uint32_t i = k0; i < k1; i++) {
+                const VkDeswzRegion *r = &dr[i];
+                size_t pitch = (size_t)r->pitch16 * 16, bytes = pitch * r->rows * r->slices;
+                uint8_t *cpu = malloc(bytes);
+                if (!cpu) rt_fatal("vk: out of memory (deswizzle check)");
+                uint32_t row_bytes = 0, rows, slices, gbpb;
+                uint64_t base;
+                unsigned th, td;
+                nvn_deswizzle_geom(d, rg[i].imageSubresource.mipLevel, rg[i].imageSubresource.baseArrayLayer, &row_bytes,
+                                   &rows, &slices, &base, &th, &td, &gbpb);
+                nvn_deswizzle(d, rg[i].imageSubresource.mipLevel, rg[i].imageSubresource.baseArrayLayer, t->mem, cpu, pitch);
+                uint64_t bad = 0, first = UINT64_MAX;
+                for (uint64_t row = 0; row < (uint64_t)r->rows * r->slices; row++)
+                    if (memcmp(cpu + row * pitch, rmap + r->dst + row * pitch, row_bytes)) {
+                        bad++;
+                        if (first == UINT64_MAX) first = row;
+                    }
+                free(cpu);
+                plat_add64(&n_checked, 1);
+                if (bad) {
+                    plat_add64(&n_differ, 1);
+                    rt_log("vk: deswizzle check: texture %#llx fmt %#x %ux%u level %u layer %u: %llu of %u rows differ, the "
+                           "first %llu (th %u td %u, %u rows of %u bytes, %u slices)", (unsigned long long)e->pub.addr,
+                           d->format, d->width, d->height, rg[i].imageSubresource.mipLevel,
+                           rg[i].imageSubresource.baseArrayLayer, (unsigned long long)bad, r->rows * r->slices,
+                           (unsigned long long)first, r->th, r->td, r->rows, row_bytes, r->slices);
+                }
+            }
+            vmaDestroyBuffer(c->vma, rb, ra);
+            if ((n_checked & 1023) == 0 || n_checked < 2)
+                rt_log("vk: deswizzle check: %lld regions compared, %lld differ", (long long)n_checked, (long long)n_differ);
+            cb = vk_upload_begin();
+            barrier(cb, ALL_RW, VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT | VK_PIPELINE_STAGE_2_COPY_BIT,
+                    VK_ACCESS_2_SHADER_STORAGE_READ_BIT | VK_ACCESS_2_TRANSFER_WRITE_BIT);
+            continue;
+        }
+        vk_defer_buffer(sb, sa);
+        if (k1 < n && vk_upload_held() > UPLOAD_HELD_MAX) {   // submit what is recorded, go on in a new batch
+            barrier(cb, VK_PIPELINE_STAGE_2_COPY_BIT | VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT,
+                    VK_ACCESS_2_TRANSFER_WRITE_BIT | VK_ACCESS_2_SHADER_STORAGE_WRITE_BIT, ALL_RW);
+            vk_upload_end();
+            vk_upload_flush(0);
+            cb = vk_upload_begin();
+            barrier(cb, ALL_RW, VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT | VK_PIPELINE_STAGE_2_COPY_BIT,
+                    VK_ACCESS_2_SHADER_STORAGE_READ_BIT | VK_ACCESS_2_TRANSFER_WRITE_BIT);
+        }
+    }
+    barrier(cb, VK_PIPELINE_STAGE_2_COPY_BIT | VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT,
+            VK_ACCESS_2_TRANSFER_WRITE_BIT | VK_ACCESS_2_SHADER_STORAGE_WRITE_BIT, ALL_RW);
+    vk_upload_end();
+    if (vk_upload_held() > UPLOAD_HELD_MAX) vk_upload_flush(0);
+    free(dr), free(rg), free(ar);
+    return 1;
+}
+
+// Copies the whole storage into the image (every level and layer the image has): on the GPU where it can
+// (upload_gpu), else on the CPU.
+static void upload(Tex *e) {
+    const tex_t *t = &e->t;
+    const nvn_texdesc *d = &t->d;
+    int bw, bh;
+    if (!nvn_fmt_bpb(d->format, &bw, &bh) || !t->mem) return;
+    uint64_t p0 = perf_begin(), u0 = kern_now_ns();
+    int why = UP_GPU;
+    if (!upload_gpu(e)) why = up_why, upload_cpu(e);
     uint64_t dt = kern_now_ns() - u0;
+    up_n[why]++, up_bytes[why] += t->L.size, up_ns[why] += dt;
     if (e->uploaded) {
         n_reuploads++, perf_count(PERF_CNT_TEX_REUP);
         ru_n++, ru_bytes += t->L.size, ru_ns += dt;
@@ -879,10 +1297,20 @@ static int alias_compatible(const Tex *e, const Tex *src) {
     return eb && eb == sb && ew == sw && eh == sh;
 }
 
-// src holds content the storage bytes do not: the GPU wrote it and, for a mirrored texture, it was not written back
-// and the CPU has not rewritten the storage since (the new entry's upload then reads the newer bytes).
-static int alias_has_gpu_content(const Tex *src) {
-    if (!src->pub.gpu_owned) return 0;
+// Whether e is live at op pos of a chunked replay: the state scan is ahead of the chunks, and an entry it revived at a
+// later op is still released for this one. Two 4K targets at one address, RGBA8 and R8 with memory of their own, were
+// swapped whenever the scan got past the second's SET_RT before a chunk sampling the storage got its turn, and large
+// parts of the scene flickered. UINT32_MAX: the state now.
+static int live_at(const Tex *e, uint32_t pos) {
+    return !e->released && !(e->rev_epoch == submit_epoch && e->rev_pos > pos);
+}
+
+// src holds content the storage bytes do not, at op pos of a chunked replay (UINT32_MAX: the state now): the GPU wrote
+// it and, for a mirrored texture, it was not written back and the CPU has not rewritten the storage since (the new
+// entry's upload then reads the newer bytes). A chunk at a later op that rendered src first does not count: in replay
+// order that render comes after the new entry's first use.
+static int alias_has_gpu_content(const Tex *src, uint32_t pos) {
+    if (!src->pub.gpu_owned || (src->own_epoch == submit_epoch && src->own_pos > pos)) return 0;
     if (!src->pub.mirrored) return 1;
     uint32_t now;
     return src->gpu_dirty && !storage_rewritten(src, src->gpu_gen, &now);
@@ -936,10 +1364,45 @@ static int alias_copy(Tex *e, const Tex *src) {
     return 1;
 }
 
+static void entry_live(Tex *e) {
+    if (!e->released) return;
+    takeback_n[e->pub.gpu_owned != 0][age_bucket(coarse_ms - e->rel_ms)]++;
+    e->use_seq = ++use_counter;
+    e->released = 0, nreleased--, rel_unlink(e), tex_gen_bump(e->key.addr);
+}
+
+// A live colour entry at e's storage of e's size, as of op pos: the new entry may take its content (alias_copy).
+static int alias_candidate(const TexKey *k, const tex_t *t, uint32_t pos) {
+    for (const Tex *p = nvn_hm_get(&by_addr, &k->addr); p; p = p->next_at)
+        if (live_at(p, pos) && !p->f.depth && !p->f.stencil && p->t.L.size == t->L.size) return 1;
+    return 0;
+}
+
 static Tex *entry_of(const tex_t *t) {
     TexKey k;
     make_key(t, &k);
     Tex *e = nvn_hm_get(&by_key, &k);
+    uint32_t pos = vk_exec_replay_pos();
+    // A chunk of a chunked replay makes the entry as of its own op: whether it takes the content of a texture at its
+    // storage (alias_copy) depends on what the chunks before it render there, and those may still be recording on
+    // other threads. Taken from a render they had not recorded yet, the new image got the storage's old bytes instead:
+    // on an RX 7800 XT the grass near Link was missing in 3 of 29 launches with 4 threads (a map texture drawn once
+    // per tile, kept from then on). With a live colour texture of its size at the storage, the chunk waits until the
+    // chunks before it are recorded, the texture lock released for the wait (as often as this thread holds it).
+    if (!e && pos != UINT32_MAX && tex_alias != 0 && alias_candidate(&k, t, pos) && vk_exec_chunk_behind()) {
+        tex_t own = *t;   // t may point into tslots, which a registration moves while the lock is released
+        t = &own;
+        int d = tex_depth;
+        for (int i = 0; i < d; i++) unlock();
+        vk_exec_chunk_settle();
+        for (int i = 0; i < d; i++) lock();
+        tex_settles++;
+        if ((e = nvn_hm_get(&by_key, &k))) {   // another chunk made it meanwhile
+            entry_live(e);
+            e->t = *t;
+            return e;
+        }
+    }
     if (!e) {
         if (tex_alias < 0) {
             tex_alias = cfg_bool("TOTK_TEX_ALIAS");
@@ -947,6 +1410,8 @@ static Tex *entry_of(const tex_t *t) {
         }
         if (!(e = create_tex(t, &k, tex_alias))) return NULL;
         perf_count(PERF_CNT_TEX_NEW);
+        dropped_check(&k);
+        e->use_seq = ++use_counter;
         // released entries of another descriptor at this address are replaced by the new one; a live one holding
         // GPU-written content is the alias source, preferring the one that differs only in swizzle (same_but_swizzle)
         int alias = 0;
@@ -955,8 +1420,9 @@ static Tex *entry_of(const tex_t *t) {
             nx = p->next_at;
             if (p->released) { destroy_tex(p); continue; }
             alias = 1;
+            if (!live_at(p, pos)) continue;   // the scan revives it at a later op: released at this chunk's
             if (tex_alias && (!src || (same_but_swizzle(&p->key, &k) && !same_but_swizzle(&src->key, &k))) &&
-                alias_compatible(e, p) && alias_has_gpu_content(p))
+                alias_compatible(e, p) && alias_has_gpu_content(p, pos))
                 src = p;
         }
         if (alias) perf_count(PERF_CNT_TEX_ALIAS);
@@ -982,7 +1448,7 @@ static Tex *entry_of(const tex_t *t) {
         nvn_hm_put(&by_key, &k, e);
         tex_gen_bump(k.addr);
     }
-    if (e->released) e->released = 0, nreleased--, rel_unlink(e), tex_gen_bump(k.addr);
+    entry_live(e);
     e->t = *t;
     return e;
 }
@@ -1008,6 +1474,25 @@ const VkTexImage *vk_texture_get(const tex_t *t) {
     return e ? &e->pub : NULL;
 }
 
+// The replay's state scan (vk_exec.c): the entry vk_texture_get would return, neither created nor made current; the
+// recording does that. It is revived here, in replay order, and remembers the op it happened at: a chunk may bind a
+// finalized target's slot before the chunk that renders it reached its SET_RT, or long after the scan went past a later
+// target at the same storage, and slot_resolve has to answer as the replay would at the chunk's own op (live_at).
+const VkTexImage *vk_texture_revive(const tex_t *t, uint32_t pos) {
+    if (!t || !t->mem) return NULL;
+    TexKey k;
+    make_key(t, &k);
+    lock();
+    Tex *e = nvn_hm_get(&by_key, &k);
+    if (e && e->released) {
+        e->rev_epoch = submit_epoch, e->rev_pos = pos;
+        __atomic_store_n(&arev[agen_idx(k.addr)], submit_epoch, __ATOMIC_RELEASE);
+        entry_live(e);
+    }
+    unlock();
+    return e ? &e->pub : NULL;
+}
+
 // Render targets the CPU reads without NVN's invalidate (vk_texture_write_back_small); small_pending: one may be dirty.
 // wb_limit: targets up to this size (bytes) always go back when the CPU waits for a fence, TOTK_WB_FENCE_LIMIT, read
 // once. A larger one goes back unless the GPU wrote it in each of the last WB_RUN frames (wb_wanted). wb_trace:
@@ -1040,6 +1525,10 @@ void vk_texture_gpu_written(const tex_t *t) {
     Tex *e = entry_of(t);
     if (e) {
         make_fresh(e);   // a partial render keeps what the storage held before
+        if (!e->pub.gpu_owned) {   // entry_of's alias choice of a chunk at an earlier op leaves this render out
+            uint32_t pos = vk_exec_replay_pos();
+            e->own_epoch = pos == UINT32_MAX ? 0 : submit_epoch, e->own_pos = pos;
+        }
         e->pub.gpu_owned = 1;
         uint64_t f = vk_exec_recording_fence();
         if (f > e->wr_fence) e->wr_fence = f;
@@ -1073,6 +1562,8 @@ void vk_texture_release(const tex_t *t) {
     Tex *e = nvn_hm_get(&by_key, &k);
     if (e && !e->released) {
         e->released = 1;
+        e->rel_ms = coarse_ms;
+        e->rel_seq = use_counter;
         nreleased++;
         rel_append(e);
         tex_gen_bump(k.addr);
@@ -1108,7 +1599,9 @@ void vk_texture_register(int id, const tex_t *t, const view_t *v) {
     }
     if (!sinfo) {
         uint64_t *p = calloc(1 << 20, sizeof *p);
-        if (!p) rt_fatal("vk: out of memory for texture slot tokens");
+        uint32_t *q = calloc(1 << 20, sizeof *q);
+        if (!p || !q) rt_fatal("vk: out of memory for texture slot tokens");
+        __atomic_store_n(&sepoch, q, __ATOMIC_RELEASE);
         __atomic_store_n(&sinfo, p, __ATOMIC_RELEASE);
     }
     TexSlot *s = &tslots[id];
@@ -1124,15 +1617,15 @@ void vk_texture_register(int id, const tex_t *t, const view_t *v) {
     unlock();
 }
 
-// The entry slot s resolves to, and the view and has_view a caller gets with it (the slot's, with the descriptor's
-// swizzle when the entry is another descriptor at the same storage). NULL when the slot names no texture.
-static Tex *slot_resolve(TexSlot *s, view_t *view, int *has_view) {
+// The entry slot s resolves to at op pos, and the view and has_view a caller gets with it (the slot's, with the
+// descriptor's swizzle when the entry is another descriptor at the same storage). NULL when the slot names no texture.
+static Tex *slot_resolve(TexSlot *s, view_t *view, int *has_view, uint32_t pos) {
     *view = s->v;
     *has_view = s->has_view;
     TexKey k;
     make_key(&s->t, &k);
     Tex *e = nvn_hm_get(&by_key, &k);
-    if (!e || e->released) {   // MNG hle_nvn.c: a finalized slot follows the live texture at its storage
+    if (!e || !live_at(e, pos)) {   // MNG hle_nvn.c: a finalized slot follows the live texture at its storage
         // Prefer the live entry that differs from the slot descriptor only in its swizzle: transient render targets
         // share storage, and the newest one at the address is usually another size. The map screen samples its
         // 2250x1080 map target through an (R,G,B,1) descriptor and followed a 960x60 target at the same address.
@@ -1145,13 +1638,14 @@ static Tex *slot_resolve(TexSlot *s, view_t *view, int *has_view) {
         Tex *own = e, *any = NULL;
         e = NULL;
         for (Tex *p = nvn_hm_get(&by_addr, &k.addr); p; p = p->next_at) {
-            if (p->released || linear_format(p->key.d.format) != linear_format(k.d.format)) continue;
+            if (!live_at(p, pos) || linear_format(p->key.d.format) != linear_format(k.d.format)) continue;
             if (!any) any = p;
             if (same_but_swizzle(&p->key, &k)) { e = p; break; }
         }
         if (!e) e = any ? any : own;
     }
-    if (!e) e = s->t.mem ? entry_of(&s->t) : NULL;
+    TexSlot copy;   // entry_of may release the lock (a chunk's wait), and a registration then moves tslots
+    if (!e && s->t.mem) copy = *s, s = &copy, e = entry_of(&s->t);
     // A followed texture is another descriptor at the same storage: its image holds the content, but sampling takes the
     // slot descriptor's swizzle. The UI renders text into RGBA8 and samples it through a (A,A,A,A) descriptor; with the
     // render target's RGBA swizzle the text blur saw R = 1 everywhere and drew solid boxes behind tips and labels.
@@ -1193,14 +1687,38 @@ const VkTexImage *vk_texture_by_id(int id, view_t *view, int *has_view) {
 const VkTexImage *vk_texture_by_id_locked(int id, view_t *view, int *has_view) {
     if (id < 0 || id >= ntslots || !tslots[id].used) return NULL;
     TexSlot *s = &tslots[id];
-    uint32_t g = agen[sinfo[id] & 0xffff];
+    uint32_t idx = sinfo[id] & 0xffff, g = agen[idx];
+    uint32_t pos = arev[idx] == submit_epoch ? vk_exec_replay_pos() : UINT32_MAX;
+    if (pos != UINT32_MAX) {   // what the storage holds depends on the op: chunks at other ops share no answer in s
+        view_t v;
+        int hv;
+        Tex *f = slot_resolve(s, &v, &hv, pos);
+        s = &tslots[id];   // slot_resolve may have released the lock
+        if (f && (!f->uploaded || f->checked != submit_epoch)) {
+            make_fresh_snap(f);
+            f->checked = submit_epoch;
+        }
+        if (view) *view = has_view ? v : s->v;
+        if (has_view) *has_view = view ? hv : s->has_view;
+        return f ? &f->pub : NULL;
+    }
     if (!s->ce || s->cgen != g) {   // resolved again: an entry at its address was created, dropped or released, or the
         view_t v;                   // slot registered (s->ce cleared); an empty result is not kept: image creation may
         int hv;                     // succeed later
-        s->ce = slot_resolve(s, &v, &hv);   // entry_of may create an entry and bump g: g read before it
+        uint32_t settles = tex_settles;
+        Tex *r = slot_resolve(s, &v, &hv, UINT32_MAX);   // entry_of may create an entry and bump g: g read before it
+        s = &tslots[id];
+        if (settles != tex_settles) {   // the lock was released for a wait: the slot may be another texture by now
+            if (r && (!r->uploaded || r->checked != submit_epoch)) make_fresh_snap(r), r->checked = submit_epoch;
+            if (view) *view = has_view ? v : s->v;
+            if (has_view) *has_view = view ? hv : s->has_view;
+            return r ? &r->pub : NULL;
+        }
+        s->ce = r;
         s->cview = v, s->chas_view = (uint8_t)hv;
         s->cgen = g;
         s->cepoch = 0;
+        __atomic_store_n(&sepoch[id], 0, __ATOMIC_RELAXED);
     }
     Tex *e = s->ce;
     // Bindings of one submit share one check, answered from the submit's write-watch snapshot (guest writes during a
@@ -1212,6 +1730,7 @@ const VkTexImage *vk_texture_by_id_locked(int id, view_t *view, int *has_view) {
             e->checked = submit_epoch;
         }
         s->cepoch = submit_epoch;
+        __atomic_store_n(&sepoch[id], submit_epoch, __ATOMIC_RELEASE);
     }
     if (view) *view = has_view ? s->cview : s->v;
     if (has_view) *has_view = view ? s->chas_view : s->has_view;
@@ -1233,10 +1752,26 @@ int vk_texture_slot_swz(int id, uint8_t swz[4]) {
 
 uint32_t vk_texture_epoch(void) { return __atomic_load_n(&submit_epoch, __ATOMIC_RELAXED); }
 
+int vk_texture_slot_moves(int id) {
+    const uint64_t *si = __atomic_load_n(&sinfo, __ATOMIC_ACQUIRE);
+    if (!si || id < 0 || id >= (1 << 20)) return 0;
+    uint64_t v = __atomic_load_n(&si[id], __ATOMIC_RELAXED);
+    return __atomic_load_n(&arev[v & 0xffff], __ATOMIC_ACQUIRE) == __atomic_load_n(&submit_epoch, __ATOMIC_RELAXED);
+}
+
+int vk_texture_slot_checked(int id) {
+    const uint32_t *e = __atomic_load_n(&sepoch, __ATOMIC_ACQUIRE);
+    return e && id >= 0 && id < (1 << 20) &&
+           __atomic_load_n(&e[id], __ATOMIC_ACQUIRE) == __atomic_load_n(&submit_epoch, __ATOMIC_RELAXED);
+}
+
 void vk_texture_new_submit(void) {
     lock();
     if (!++submit_epoch) submit_epoch = 1;
+    coarse_ms = plat_now_ms();
     sweep_pressure();
+    if (__atomic_load_n(&rt_oom_req, __ATOMIC_RELAXED) && __atomic_exchange_n(&rt_oom_req, 0, __ATOMIC_ACQUIRE))
+        oom_release();
     static uint64_t log_at;
     uint64_t f = __atomic_load_n(&gfx_counters.frames, __ATOMIC_RELAXED) / 60;
     if ((rt_trace & RT_TRACE_PERF) && f != log_at && tex_alias > 0) {
@@ -1249,15 +1784,25 @@ void vk_texture_new_submit(void) {
                (unsigned long long)f * 60, (unsigned long long)ru_n, ru_bytes / 1048576.0, ru_ns / 1e6,
                (unsigned long long)ru_wt_n, ru_wt_bytes / 1048576.0, ru_wt_ns / 1e6);
         ru_n = ru_bytes = ru_ns = ru_wt_n = ru_wt_bytes = ru_wt_ns = 0;
+        char ub[512];
+        size_t uo = 0;
+        for (int i = 0; i < UP_WHY; i++)
+            if (up_n[i] && uo < sizeof ub)
+                uo += (size_t)snprintf(ub + uo, sizeof ub - uo, "%s %s %llu (%.1f MiB, %.2f ms)", uo ? "," : "",
+                                       up_why_name[i], (unsigned long long)up_n[i], up_bytes[i] / 1048576.0, up_ns[i] / 1e6);
+        if (uo) rt_log("perf: frame %llu: texture uploads by path:%s", (unsigned long long)f * 60, ub);
+        memset(up_n, 0, sizeof up_n), memset(up_bytes, 0, sizeof up_bytes), memset(up_ns, 0, sizeof up_ns);
         uint64_t ck, calls, untr;
         vk_mem_snap_stats(&ck, &calls, &untr);
         rt_log("perf: frame %llu: binding storage checks %llu (texture %llu), write-watch snapshot queries %llu, untracked %llu",
                (unsigned long long)f * 60, (unsigned long long)ck, (unsigned long long)n_checks, (unsigned long long)calls,
                (unsigned long long)untr);
         n_checks = 0;
-        rt_log("perf: frame %llu: released textures kept %d, dropped under VRAM pressure %llu", (unsigned long long)f * 60,
-               nreleased, (unsigned long long)n_pressure_drops);
-        n_pressure_drops = 0;
+        rt_log("perf: frame %llu: released textures kept %d (%.0f MiB), dropped under VRAM pressure or the cap %llu "
+               "(%llu overwritten)", (unsigned long long)f * 60, nreleased, rel_bytes_total / 1048576.0,
+               (unsigned long long)n_pressure_drops,
+               (unsigned long long)n_pressure_overwritten);
+        n_pressure_drops = n_pressure_overwritten = 0;
     }
     unlock();
 }
@@ -1551,7 +2096,12 @@ static void write_back_n(Tex **v, int n) {
     if (np) vk_upload_flush(1);
     for (int i = 0; i < np; i++) {
         WbPart *q = &p[i];
-        uint8_t *buf = malloc(q->g.img * (size_t)q->g.nz);
+        size_t bytes = q->g.img * (size_t)q->g.nz;
+        uint8_t *buf = malloc(bytes);
+        if (!buf) {   // waits for memory with the texture lock held, after the release that needs no submit
+            vk_oom_release();
+            buf = rt_realloc_wait(NULL, bytes, "vk: texture write-back buffer");
+        }
         if (!buf) rt_fatal("vk: out of memory for a texture write-back");
         texels_read_take(q->e, &q->g, &q->t, buf);
         storage_texels(&q->e->t, &q->g, &q->r, buf, 1);
@@ -1963,6 +2513,74 @@ void vk_texture_census(VkTexCensus *s) {
     unlock();
 }
 
+// TOTK_TRACE=perf, with mem: vram: what the released entries in own memory hold and how long they waited.
+void vk_texture_released_log(uint64_t frame) {
+    VkCtx *c = vk_ctx();
+    if (!c) return;
+    enum { CL_CLEAN, CL_BACKED, CL_DIRTY, CL_POOLS, CL_N };
+    uint64_t cl_n[CL_N] = {0}, cl_b[CL_N] = {0}, age_b[AGE_N] = {0}, dead_n = 0, dead_b = 0, dead_dirty_b = 0;
+    uint64_t bound_n = 0, bound_b = 0, ded_n = 0, ded_b = 0;
+    typedef struct { uint32_t w, h, fmt, n; uint64_t b; } Group;
+    Group gr[256];
+    int ng = 0;
+    lock();
+    AddrIndex x = {0};
+    int indexed = addr_index(&x);
+    for (int i = 0; indexed && i < x.n; i++) {
+        Tex *e = x.e[i];
+        if (!e->released || !e->alloc) continue;
+        VmaAllocationInfo2 ai2;
+        vmaGetAllocationInfo2(c->vma, e->alloc, &ai2);
+        const VmaAllocationInfo ai = ai2.allocationInfo;
+        if (ai2.dedicatedMemory) ded_n++, ded_b += ai.size;
+        int cl = !e->pub.mirrored ? CL_POOLS : e->gpu_dirty ? CL_DIRTY : e->pub.gpu_owned ? CL_BACKED : CL_CLEAN;
+        cl_n[cl]++, cl_b[cl] += ai.size;
+        if (cl != CL_CLEAN) age_b[age_bucket(coarse_ms - e->rel_ms)] += ai.size;
+        if (e->checked && submit_epoch - e->checked < 600) bound_n++, bound_b += ai.size;
+        if (storage_taken(&x, i)) dead_n++, dead_b += ai.size, dead_dirty_b += cl != CL_CLEAN ? ai.size : 0;
+        int g = 0;
+        while (g < ng && (gr[g].w != e->t.d.width || gr[g].h != e->t.d.height || gr[g].fmt != (uint32_t)e->t.d.format)) g++;
+        if (g == ng && ng < 256) gr[ng++] = (Group){e->t.d.width, e->t.d.height, (uint32_t)e->t.d.format, 0, 0};
+        if (g < ng) gr[g].n++, gr[g].b += ai.size;
+    }
+    if (indexed) addr_index_free(&x);
+    const double M = 1048576.0;
+    rt_log("mem: released frame %llu: own memory: clean %llu %.0f MiB, GPU-written with storage current %llu %.0f MiB, "
+           "GPU-written image only %llu %.0f MiB, not mirrored %llu %.0f MiB; all but clean by age <1s %.0f <5s %.0f <30s %.0f "
+           "<120s %.0f older %.0f MiB; storage taken by a later texture %llu %.0f MiB (%.0f not clean); bound through a slot "
+           "in the last 600 submits %llu %.0f MiB; in dedicated memory, returned to the driver when dropped, %llu %.0f MiB; "
+           "longest search %.2f ms",
+           (unsigned long long)frame, (unsigned long long)cl_n[CL_CLEAN], cl_b[CL_CLEAN] / M, (unsigned long long)cl_n[CL_BACKED],
+           cl_b[CL_BACKED] / M, (unsigned long long)cl_n[CL_DIRTY], cl_b[CL_DIRTY] / M, (unsigned long long)cl_n[CL_POOLS],
+           cl_b[CL_POOLS] / M, age_b[AGE_1S] / M, age_b[AGE_5S] / M, age_b[AGE_30S] / M, age_b[AGE_120S] / M,
+           age_b[AGE_LONG] / M, (unsigned long long)dead_n, dead_b / M, dead_dirty_b / M, (unsigned long long)bound_n,
+           bound_b / M, (unsigned long long)ded_n, ded_b / M, pressure_search_ns / 1e6);
+    pressure_search_ns = 0;
+    for (int k = 0; k < 2; k++) {
+        char line[256];
+        size_t o = 0;
+        for (int a = 0; a < AGE_N; a++) {
+            o += (size_t)snprintf(line + o, sizeof line - o, " %s %llu", age_name[a], (unsigned long long)takeback_n[k][a]);
+            takeback_n[k][a] = 0;
+        }
+        rt_log("mem: released frame %llu: taken back after,%s:%s", (unsigned long long)frame, k ? " GPU-written" : " others", line);
+    }
+    rt_log("mem: released frame %llu: dropped under pressure and created again: within 10 s %llu (%llu GPU-written), later %llu (%llu)",
+           (unsigned long long)frame, (unsigned long long)n_recreated[0], (unsigned long long)n_recreated_written[0],
+           (unsigned long long)n_recreated[1], (unsigned long long)n_recreated_written[1]);
+    n_recreated[0] = n_recreated[1] = n_recreated_written[0] = n_recreated_written[1] = 0;
+    for (int top = 0; top < 4; top++) {
+        int best = -1;
+        for (int g = 0; g < ng; g++)
+            if (gr[g].n && (best < 0 || gr[g].b > gr[best].b)) best = g;
+        if (best < 0) break;
+        rt_log("mem: released frame %llu: largest %d: %ux%u fmt %#x: %u entries %.0f MiB", (unsigned long long)frame, top + 1,
+               gr[best].w, gr[best].h, gr[best].fmt, gr[best].n, gr[best].b / M);
+        gr[best].n = 0;
+    }
+    unlock();
+}
+
 void vk_texture_shutdown(void) {
     VkCtx *c = vk_ctx();
     lock();
@@ -1979,8 +2597,8 @@ void vk_texture_shutdown(void) {
     by_addr = (nvn_hmap){.klen = 8};
     samplers = (nvn_hmap){.klen = sizeof(samp_par)};
     free(tslots), free(sslots);
-    tslots = NULL, sslots = NULL, ntslots = nsslots = 0, nreleased = 0;
+    tslots = NULL, sslots = NULL, ntslots = nsslots = 0, nreleased = 0, rel_bytes_total = 0;
     rel_head = rel_tail = NULL;
-    if (sinfo) memset(sinfo, 0, sizeof *sinfo << 20);   // no slot is registered: tokens 0
+    if (sinfo) memset(sinfo, 0, sizeof *sinfo << 20), memset(sepoch, 0, sizeof *sepoch << 20);   // no slot is registered: tokens 0
     unlock();
 }

runtime/vk_usnap.c

@@ -11,11 +11,13 @@
 //   that submission's write-back, the mirror holds them until then.
 // - A GPU write within the submission (UpdateUniformBuffer, a storage buffer a shader writes, a buffer copy or fill,
 //   an image-to-buffer copy: vk_usnap_drop) drops the copies it touches from then on, so later draws read the result.
+//   A chunked replay (vk_exec.c) records the ops out of order: its scan drops at the position of the op
+//   (vk_usnap_drop_at, scan_writes for the shaders' stores), and a chunk finds a copy only at a position before that.
 // - The arena goes back to a free list once the submission's fence is done (completion thread).
 // The design follows MacNativeGames' TOTK_UBO_SNAPSHOT (games/totk/third_party/d2r_gfx/gfx/nvn/nvn_mtl.m usnap_take,
 // usnap_drop; MIT, Copyright (c) 2026 jhaizhou); the code is ours.
 // The copy is taken where vk_exec_record starts: at the submit call when the replay runs on the submitting thread, later
-// with TOTK_REPLAY_ASYNC. TOTK_UBO_SNAPSHOT=0 binds live memory as before; =ab alternates (TOTK_AB).
+// with TOTK_REPLAY_ASYNC; a chunked replay takes it on a recording thread while its scan runs. TOTK_UBO_SNAPSHOT=0 binds live memory as before; =ab alternates (TOTK_AB).
 #include <stdlib.h>
 #include <string.h>
 
@@ -42,7 +44,7 @@ typedef struct Arena {
 } Arena;
 
 typedef struct { uint64_t lo, hi; } Rng;
-typedef struct { uint64_t lo, hi, off; int dropped; } Reg;
+typedef struct { uint64_t lo, hi, off, drop_at; } Reg;   // drop_at: dropped from this position on; UINT64_MAX not
 typedef struct { uint64_t lo, hi, fence; } Wrote;
 
 static struct {
@@ -62,6 +64,7 @@ static struct {
 
 static Arena *free_arenas;
 static PlatMutex free_lock = PLAT_MUTEX_INIT;
+static PlatMutex drop_lock = PLAT_MUTEX_INIT;   // threads recording chunks of one submission drop concurrently
 
 static void *grow(void *p, int *cap, size_t elem) {
     int n = *cap ? *cap * 2 : 1024;
@@ -279,7 +282,7 @@ void vk_usnap_begin(const uint64_t *handles, int n, uint64_t fence) {
             continue;
         }
         if (U.nreg == U.creg) U.reg = grow(U.reg, &U.creg, sizeof *U.reg);
-        U.reg[U.nreg++] = (Reg){lo, hi, total, 0};
+        U.reg[U.nreg++] = (Reg){lo, hi, total, UINT64_MAX};
         total += (hi - lo + ALIGN - 1) & ~(uint64_t)(ALIGN - 1);
     }
     if (!U.nreg) return;
@@ -313,29 +316,38 @@ static int first_above(uint64_t a) {
     return lo;
 }
 
-int vk_usnap_find(uint64_t addr, uint64_t n, uint64_t align, VkDescriptorBufferInfo *out) {
+int vk_usnap_find(uint64_t addr, uint64_t n, uint64_t at, uint64_t align, VkDescriptorBufferInfo *out) {
     if (!U.on || !n) return 0;
     int i = first_above(addr);
     if (i >= U.nreg) return 0;
     const Reg *r = &U.reg[i];
-    if (r->dropped || addr < r->lo || addr + n > r->hi) return 0;
+    if (__atomic_load_n(&r->drop_at, __ATOMIC_RELAXED) <= at || addr < r->lo || addr + n > r->hi) return 0;
     uint64_t off = r->off + (addr - r->lo);
     if (off % align) return 0;
     *out = (VkDescriptorBufferInfo){U.arena->buf, off, n};
-    U.hits++;
+    __atomic_fetch_add(&U.hits, 1, __ATOMIC_RELAXED);
     return 1;
 }
 
-void vk_usnap_drop(uint64_t addr, uint64_t n) {
+static void drop(uint64_t addr, uint64_t n, uint64_t at) {
     if (!U.mode || !n) return;
+    plat_mutex_lock(&drop_lock);
     if (U.nwr == U.cwr) U.wr = grow(U.wr, &U.cwr, sizeof *U.wr);
     U.wr[U.nwr++] = (Wrote){addr, addr + n, U.fence};
-    if (!U.on) return;
-    for (int i = first_above(addr); i < U.nreg && U.reg[i].lo < addr + n; i++)
-        if (!U.reg[i].dropped) U.reg[i].dropped = 1, U.drops++, U.gen++;
+    if (U.on)
+        for (int i = first_above(addr); i < U.nreg && U.reg[i].lo < addr + n; i++)
+            if (at < U.reg[i].drop_at) {
+                if (U.reg[i].drop_at == UINT64_MAX) U.drops++;
+                __atomic_store_n(&U.reg[i].drop_at, at, __ATOMIC_RELAXED);
+                __atomic_add_fetch(&U.gen, 1, __ATOMIC_RELAXED);
+            }
+    plat_mutex_unlock(&drop_lock);
 }
 
-uint64_t vk_usnap_gen(void) { return U.gen; }
+void vk_usnap_drop(uint64_t addr, uint64_t n) { drop(addr, n, 0); }
+void vk_usnap_drop_at(uint64_t addr, uint64_t n, uint32_t at) { drop(addr, n, at); }
+
+uint64_t vk_usnap_gen(void) { return __atomic_load_n(&U.gen, __ATOMIC_RELAXED); }
 
 // the host copy where there is one (DLSS), else the device copy
 static uint8_t *host_copy(void) { return U.arena->cpu ? U.arena->cpu : U.arena->map; }
@@ -344,7 +356,7 @@ const uint8_t *vk_usnap_map(void) { return U.arena ? host_copy() : NULL; }
 
 const uint8_t *vk_usnap_bytes(uint64_t addr, uint64_t n) {
     VkDescriptorBufferInfo bi;
-    return vk_usnap_find(addr, n, 1, &bi) ? host_copy() + bi.offset : NULL;
+    return vk_usnap_find(addr, n, UINT32_MAX, 1, &bi) ? host_copy() + bi.offset : NULL;
 }
 
 // The arenas back on the free list (vk_exec_draw_shutdown: the device is idle, every completion ran).

tests/cheats_vm_check.py

@@ -0,0 +1,312 @@
+"""The data cheat VM of runtime/cheats.c and the opcode tables of tools/cheats.py, without game files.
+
+tools/cheats.py: opcode lengths and which opcodes the VM runs (vm_ok), and how a whole cheat is classified.
+runtime/cheats.c, compiled into a harness with stubs for the rest of the runtime: the same cheats are classified and
+measured opcode by opcode (op_len must give the lengths tools/cheats.py gives), then run on a fake main image and heap:
+opcode C0 with every operand type, width and condition, its else and the skip over a failed block (nested, and with
+operand words that look like 20000000), and C1-C3.
+
+Run with .venv: python tests/cheats_vm_check.py [--cc <clang.exe>]
+"""
+import argparse
+import os
+import subprocess
+import sys
+import tempfile
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(ROOT / 'tools'))
+import cheats  # noqa: E402
+
+# Issue #973: two cheats for 1.1.0 whose only opcode the VM lacked was C046B400 FFFFFFFF (if r[B] as u32 != -1).
+REPORTED = '''
+[Zonai Devices 98]
+580F0000 04646C60
+580F1000 00000138
+580F1000 000011A8
+300C0000 0000003C
+989BF000
+540B1000 00000000
+C046B400 FFFFFFFF
+640F1000 00000000 00000062
+21000000
+780F0000 00000008
+20000000
+310C0000
+[All Materials Collected 98]
+580F0000 04646C60
+580F1000 00000138
+580F1000 00000F28
+300C0000 000001FE
+989BF000
+540B1000 00000000
+C046B400 FFFFFFFF
+640F1000 00000000 00000062
+21000000
+780F0000 00000008
+20000000
+310C0000
+'''
+
+# (words, kind, note substring): every C0 operand type and width, C1-C3, and what stays unsupported.
+CLASSIFY = [
+    ('C0150010 00000010', 'data', ''),            # X=0 memory + offset, 1 byte
+    ('C025011B', 'data', ''),                     # X=1 memory + offset register, 2 bytes
+    ('C04512B0 00000010', 'data', ''),            # X=2 register + offset
+    ('C08513BC', 'data', ''),                     # X=3 register + offset register
+    ('C0151400 00000005', 'data', ''),            # X=4 static, 1 byte: one value word
+    ('C0851400 00000001 00000002', 'data', ''),   # X=4 static, 8 bytes: two value words
+    ('C04515B0', 'data', ''),                     # X=5 other register
+    ('C1030A10', 'data', ''),
+    ('C2100C00', 'data', ''),
+    ('C3000801', 'data', ''),
+    ('C0451600', 'unsupported', 'not supported'),   # X=6 is no operand type
+    ('C0351400 00000005', 'unsupported', 'bad width'),
+    ('C4000000', 'unsupported', 'not supported'),
+    ('FF000000', 'unsupported', 'not supported'),
+    ('FF100000', 'unsupported', 'not supported'),
+]
+
+
+class StubMain:
+    """No code in main: tools/cheats.py code_writes sees no instruction writes."""
+
+    def in_code(self, off):
+        return False
+
+
+def py_classify(words):
+    c = cheats.Cheat('t', False, 1)
+    c.words = words
+    cheats.classify(StubMain(), c)
+    return c.kind, c.note
+
+
+def words_of(text):
+    return [int(w, 16) for w in text.split()]
+
+
+def check_python():
+    for c in cheats.parse(REPORTED, 'mod'):
+        kind, note = py_classify(c.words)
+        assert (kind, note) == ('data', ''), f'tools/cheats.py: {c.name}: {kind} {note}'
+    for text, kind, note in CLASSIFY:
+        got = py_classify(words_of(text))
+        assert got[0] == kind and note in got[1], f'tools/cheats.py: {text}: {got}, want {kind} {note}'
+        if kind == 'data':
+            assert [len(w) for _, w in cheats.ops(words_of(text))] == [len(words_of(text))], text
+    print('cheats_vm_check: tools/cheats.py ok', flush=True)
+
+
+HARNESS = r'''
+#include <assert.h>
+#include "cheats.c"
+
+// ---- the rest of the runtime, as far as cheats.c reaches it
+static uint8_t heap[0x1000] __attribute__((aligned(4096)));
+static uint8_t image[0x1000] __attribute__((aligned(4096)));
+const char *cfg_get(const char *name) { (void)name; return NULL; }
+const char *cfg_value(const char *name) { (void)name; return ""; }
+void cfg_set(const char *name, const char *value) { (void)name; (void)value; }
+const RtConfig *rt_config(void) { static RtConfig c; return &c; }
+const RtGuestLayout *mem_layout(void) { static RtGuestLayout l; l.heap.base = (uintptr_t)heap; return &l; }
+uint32_t mem_query(uint64_t a, RtMemoryInfo *out) {
+    memset(out, 0, sizeof *out);
+    if (a < (uintptr_t)heap || a >= (uintptr_t)heap + sizeof heap) return 1;
+    out->base = (uintptr_t)heap, out->size = sizeof heap, out->state = 1, out->perm = 3;
+    return 0;
+}
+void rt_fatal(const char *fmt, ...) { (void)fmt; abort(); }
+void rt_log(const char *fmt, ...) { (void)fmt; }
+void prof_write(void) {}
+int ldr_count(void) { return 0; }
+const RtLoaded *ldr_module(int i) { (void)i; return NULL; }
+int rt_patch_set(const char *name) { (void)name; return -1; }
+int rt_patch_enabled(int set) { (void)set; return 0; }
+void rt_patch_switch(int set, int on) { (void)set; (void)on; }
+void input_poll(uint64_t *b, int32_t *lx, int32_t *ly, int32_t *rx, int32_t *ry) { *b = 0, *lx = *ly = *rx = *ry = 0; }
+void mods_scan(void) {}
+int mods_count(void) { return 0; }
+ModInfo *mods_at(int i) { (void)i; return NULL; }
+int mods_exefs_pending(uint32_t *key) { (void)key; return 0; }
+int plat_exe_path(char *out, size_t cap) { (void)out; (void)cap; return 0; }
+PlatFile *plat_file_open(const char *path) { (void)path; return NULL; }
+void plat_file_close(PlatFile *f) { (void)f; }
+int64_t plat_file_pread(PlatFile *f, void *buf, uint32_t n, uint64_t off) { (void)f; (void)buf; (void)n; (void)off; return -1; }
+uint8_t *plat_file_read(const char *path, uint64_t *size) { (void)path; (void)size; return NULL; }
+int plat_file_write(const char *path, const void *data, size_t n) { (void)path; (void)data; (void)n; return 0; }
+int plat_file_exists(const char *path) { (void)path; return 0; }
+int plat_fs_unlink(const char *path) { (void)path; return 0; }
+int plat_fs_list(const char *dir, int (*cb)(const char *, int, uint64_t, void *), void *ctx) { (void)dir; (void)cb; (void)ctx; return 0; }
+int plat_spawn(const char *const *argv, const char *cwd) { (void)argv; (void)cwd; return 0; }
+_Noreturn void plat_exit(int code) { exit(code); }
+int plat_mem_state(uint64_t addr, uint64_t limit, uint64_t *end) { (void)addr; *end = limit; return PLAT_PROT_RW; }
+int plat_mem_protect(uint64_t addr, uint64_t n, int prot) { (void)addr; (void)n; (void)prot; return 1; }
+
+#include "cases.h"
+
+// The last cheat of text, parsed as cheats_init parses a file.
+static Cheat *load(const char *text) {
+    char *s = strdup(text);
+    parse(s, "t");
+    free(s);
+    return &g_cheats[g_n - 1];
+}
+
+static uint32_t u32(const uint8_t *p) { uint32_t v; memcpy(&v, p, 4); return v; }
+static uint64_t u64(const uint8_t *p) { uint64_t v; memcpy(&v, p, 8); return v; }
+
+// Marks after the condition: 1 at heap+0x100 when it held, else 2 at heap+0x104; 3 at heap+0x108 after the end.
+#define MARKS " 04100000 00000100 00000001 21000000 04100000 00000104 00000002 20000000 04100000 00000108 00000003"
+
+// Runs setup + cond + MARKS; want: 1 the condition holds, 0 it fails.
+static void cond(const char *setup, const char *op, int want) {
+    char text[1024];
+    snprintf(text, sizeof text, "[c]\n%s %s%s\n", setup, op, MARKS);
+    memset(heap + 0x100, 0, 0x10);
+    run(load(text), 0);
+    uint32_t held = u32(heap + 0x100), other = u32(heap + 0x104), end = u32(heap + 0x108);
+    if (held != (want ? 1u : 0u) || other != (want ? 0u : 2u) || end != 3) {
+        fprintf(stderr, "%s %s: marks %u %u %u, want %d\n", setup, op, held, other, end, want);
+        abort();
+    }
+}
+
+int main(void) {
+    g_main = (uintptr_t)image, g_main_size = sizeof image, g_code_end = g_main;   // no code: nothing is a code cheat
+    for (int k = 0; k < (int)(sizeof cases / sizeof *cases); k++) {
+        Cheat *c = load(cases[k].text);
+        CheatKind kind = classify(c);
+        if (kind != cases[k].kind || !strstr(c->note, cases[k].note)) {
+            fprintf(stderr, "classify %s: %d (%s), want %d (%s)\n", cases[k].text, kind, c->note, cases[k].kind, cases[k].note);
+            abort();
+        }
+        for (int i = 0, j = 0; kind == CHEAT_DATA && i < c->n; j++) {
+            int len = op_len(c->words + i, c->n - i);
+            if (len != cases[k].lens[j]) {
+                fprintf(stderr, "op_len %s, opcode %d: %d, tools/cheats.py %d\n", cases[k].text, j, len, cases[k].lens[j]);
+                abort();
+            }
+            i += len;
+        }
+    }
+
+    char hb[64], hb10[64];   // r[B] = the heap's address, r[B] = the heap + 0x10
+    snprintf(hb, sizeof hb, "400B0000 %08X %08X", (unsigned)((uintptr_t)heap >> 32), (unsigned)(uintptr_t)heap);
+    snprintf(hb10, sizeof hb10, "400B0000 %08X %08X", (unsigned)((uintptr_t)heap >> 32), (unsigned)((uintptr_t)heap + 0x10));
+    uint64_t v = 0x1122334455667705ull;
+    memcpy(heap + 0x10, &v, 8);   // heap+0x10: 05 77 66 55 44 33 22 11
+    memcpy(image + 0x20, &v, 8);
+
+    // X=0 memory + offset: heap (M=1) and main (M=0), each width
+    cond("400A0000 00000000 00000005", "C015A010 00000010", 1);
+    cond("400A0000 00000000 00000005", "C016A010 00000010", 0);
+    cond("400A0000 00000000 00007705", "C025A010 00000010", 1);
+    cond("400A0000 00000000 55667705", "C045A000 00000020", 1);
+    cond("400A0000 11223344 55667705", "C085A000 00000020", 1);
+    cond("400A0000 11223344 55667706", "C085A000 00000020", 0);
+    // X=1 memory + offset register
+    cond("400A0000 00000000 55667705 400C0000 00000000 00000010", "C045A11C", 1);
+    cond("400A0000 00000000 55667705 400C0000 00000000 00000014", "C045A11C", 0);
+    // X=2 register + offset, X=3 register + offset register
+    char setup[256];
+    snprintf(setup, sizeof setup, "400A0000 00000000 55667705 %s", hb);
+    cond(setup, "C045A2B0 00000010", 1);
+    cond(setup, "C045A2B0 00000011", 0);
+    snprintf(setup, sizeof setup, "400A0000 00000000 55667705 %s 400C0000 00000000 00000010", hb);
+    cond(setup, "C045A3BC", 1);
+    snprintf(setup, sizeof setup, "400A0000 00000000 00000005 %s 400C0000 00000000 00000010", hb10);
+    cond(setup, "C015A3BC", 0);   // heap+0x20 holds 0
+    // an address the guest cannot read compares as 0, as dmnt's read leaves its zeroed value
+    cond("400A0000 00000000 00000000 400B0000 00000000 00000000", "C045A2B0 00000000", 1);
+    cond("400A0000 00000000 00000001 400B0000 00000000 00000000", "C045A2B0 00000000", 0);
+    // X=4 static: the register and the value cut to the width
+    cond("400A0000 FFFFFFFF FFFFFF05", "C015A400 00000005", 1);
+    cond("400A0000 FFFFFFFF FFFFFF05", "C015A400 12345605", 1);
+    cond("400A0000 FFFFFFFF FFFF7705", "C025A400 00007705", 1);
+    cond("400A0000 FFFFFFFF 55667705", "C045A400 55667705", 1);
+    cond("400A0000 00000001 00000002", "C085A400 00000001 00000002", 1);
+    cond("400A0000 00000001 00000002", "C085A400 00000000 00000002", 0);
+    // X=5 other register, cut to the width
+    cond("400A0000 00000000 00000005 400B0000 00000001 00000005", "C045A5B0", 1);
+    cond("400A0000 00000000 00000005 400B0000 00000001 00000005", "C085A5B0", 0);
+    // the six conditions, register 5 against 4, 5 and 6; condition 0 or 7 never holds
+    static const int want[8][3] = {{0, 0, 0}, {1, 0, 0}, {1, 1, 0}, {0, 0, 1}, {0, 1, 1}, {0, 1, 0}, {1, 0, 1}, {0, 0, 0}};
+    for (int c = 0; c < 8; c++)
+        for (int k = 0; k < 3; k++) {
+            char op[32];
+            snprintf(op, sizeof op, "C04%XA400 %08X", c, 4 + k);
+            cond("400A0000 00000000 00000005", op, want[c][k]);
+        }
+    // a failed block skips a nested C0 with its else and end, and C0 operand words that read 20000000 / 21000000
+    cond("400A0000 00000000 00000005",
+         "C046A400 00000005 C045A400 00000005 04100000 00000100 00000009 21000000 20000000"
+         " C085A400 21000000 20000000 20000000 C045A010 20000000 20000000 C045A2B0 20000000 20000000", 0);
+    cond("400A0000 00000000 00000005", "C045A400 00000005 C046A400 00000005 21000000 20000000", 1);
+
+    // C1: save r[A] in slot 3, clear r[A], restore slot 3 into r[B]; clear slot 3 and restore it into r[C]
+    load("[s]\n400A0000 00000000 0000002A C1030A10 C10A0030 C10B0300 C1030020 C10C0300"
+         " A4A00410 00000110 A4B00410 00000114 A4C00410 00000118\n");
+    memcpy(heap + 0x110, "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF", 12);
+    run(&g_cheats[g_n - 1], 0);
+    assert(u32(heap + 0x110) == 0 && u32(heap + 0x114) == 0x2A && u32(heap + 0x118) == 0);
+    // C2: save A and B, clear A, restore A only; clear the saved B, restore B
+    load("[m]\n400A0000 00000000 00000007 400B0000 00000000 00000008 C2100C00 C2300C00 C2000400 C2200800 C2000800"
+         " A4A00410 00000110 A4B00410 00000114\n");
+    memset(heap + 0x110, 0xFF, 8);
+    run(&g_cheats[g_n - 1], 0);
+    assert(u32(heap + 0x110) == 7 && u32(heap + 0x114) == 0);
+    // C3: r[1] into static register 0x80, which stays across runs; static register 0x05 (none sets it) into r[2]
+    load("[st]\n40010000 00000000 00000033 40020000 00000000 00000044 C3000801 C3000052 A8200410 00000120\n");
+    memset(heap + 0x120, 0xFF, 8);
+    run(&g_cheats[g_n - 1], 0);
+    assert(g_static[0x80] == 0x33 && u64(heap + 0x120) == 0);
+    // saved registers start at 0 in each run, like the registers
+    load("[z]\nC10A0300 A8A00410 00000128\n");
+    memset(heap + 0x128, 0xFF, 8);
+    run(&g_cheats[g_n - 1], 0);
+    assert(u64(heap + 0x128) == 0);
+
+    puts("cheats_vm_check: runtime/cheats.c ok");
+    return 0;
+}
+'''
+
+KIND = {'data': 'CHEAT_DATA', 'unsupported': 'CHEAT_UNSUPPORTED'}
+
+
+def cases_h():
+    """The C table: every classify case and each reported cheat, with tools/cheats.py's kind and opcode lengths."""
+    rows = [(' '.join(f'{w:08X}' for w in c.words), 'data', '') for c in cheats.parse(REPORTED, 'mod')] + CLASSIFY
+    out = ['static const struct { const char *text; CheatKind kind; const char *note; int lens[64]; } cases[] = {']
+    for text, kind, note in rows:
+        try:
+            lens = [len(w) for _, w in cheats.ops(words_of(text))]
+        except ValueError:
+            lens = []
+        out.append(f'    {{"[t]\\n{text}\\n", {KIND[kind]}, "{note}", {{{", ".join(map(str, lens)) or "0"}}}}},')
+    return '\n'.join(out + ['};', ''])
+
+
+def check_c(cc):
+    with tempfile.TemporaryDirectory(prefix='totk-cheats-') as tmp:
+        folder = Path(tmp)
+        (folder / 'check.c').write_text(HARNESS)
+        (folder / 'cases.h').write_text(cases_h())
+        exe = folder / ('check.exe' if os.name == 'nt' else 'check')
+        subprocess.run([cc, '-O1', '-g', '-Wall', '-Wextra', '-Wno-unused-function', '-I', str(folder),
+                        '-I', str(ROOT / 'runtime'), str(folder / 'check.c'), '-o', str(exe)], check=True)
+        subprocess.run([str(exe)], check=True)
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument('--cc', default='clang')
+    args = ap.parse_args()
+    check_python()
+    check_c(args.cc)
+
+
+if __name__ == '__main__':
+    main()

tests/exefs_patches_test.py

@@ -0,0 +1,258 @@
+"""tools/exefs_patches.py on modules it makes itself: code writes become sites, data writes become runs of the data file
+as emulators apply a patch file before the dynamic linker (plain bytes, DT_RELR words rebased, writes into words an
+explicit-addend relocation fills dropped), the writes it refuses (outside the image, into the linker's tables or a
+lazily bound pointer), the status and data files, and the check after the lift against the bytes recomp/gen.py lists in
+folded.txt, with gen.py run on a module with one jump table. No game files.
+
+Run with .venv (lz4 and numpy, as recomp/): python tests/exefs_patches_test.py
+"""
+import struct
+import subprocess
+import sys
+import tempfile
+import unittest
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent.parent
+sys.path.insert(0, str(ROOT / 'tools'))
+import exefs_patches as ep  # noqa: E402
+
+BUILD_ID = bytes.fromhex('0123456789abcdef') + bytes(24)
+BID_NAME = '0123456789ABCDEF'   # an IPS file name may leave the build id's trailing zeros out
+MOV_W8_100, MOV_W8_200, RET = 0x52802008, 0x52804008, 0xD65F03C0
+R_RELATIVE, R_GLOB_DAT, R_JUMP_SLOT = 1027, 1025, 1026
+
+
+def nso(text, ro, data, bss=0x1000):
+    """An uncompressed NSO with .text at 0, .rodata at 0x1000 and .data at 0x2000."""
+    segs = [(0x0, text), (0x1000, ro), (0x2000, data)]
+    hdr = bytearray(0x100)
+    hdr[0:4] = b'NSO0'
+    off = 0x100
+    for i, (mem, d) in enumerate(segs):
+        struct.pack_into('<IIII', hdr, 0x10 + 0x10 * i, off, mem, len(d), bss if i == 2 else 0)
+        struct.pack_into('<I', hdr, 0x60 + 4 * i, len(d))
+        off += len(d)
+    hdr[0x40:0x60] = BUILD_ID
+    return bytes(hdr) + b''.join(d for _, d in segs)
+
+
+def put(buf, off, data):
+    buf[off:off + len(data)] = data
+
+
+def reloc_module():
+    """fn at 0x8 (mov w8, #0x100; ret); .rodata: constants at 0x1000, MOD0 at 0x1100 and the relocation tables at
+    0x1200 (RELA: a RELATIVE word at 0x2100, a GLOB_DAT word at 0x2108), 0x1300 (JMPREL: a JUMP_SLOT word at 0x2110)
+    and 0x1400 (RELR: the word at 0x2118, addend 0x30); .data: .dynamic at 0x2000, plain bytes at 0x2200."""
+    text = struct.pack('<4I', 0x14000002, 0x1100, MOV_W8_100, RET)
+    ro = bytearray(0x500)
+    put(ro, 0x000, b'constant bytes!!')
+    put(ro, 0x100, b'MOD0' + struct.pack('<6i', 0x2000 - 0x1100, 0, 0, -0x1100, -0x1100, 0))
+    put(ro, 0x200, struct.pack('<QQq', 0x2100, R_RELATIVE, 0x8) + struct.pack('<QQq', 0x2108, R_GLOB_DAT, 0))
+    put(ro, 0x300, struct.pack('<QQq', 0x2110, R_JUMP_SLOT, 0))
+    put(ro, 0x400, struct.pack('<Q', 0x2118))
+    data = bytearray(0x300)
+    dyn = [(7, 0x1200), (8, 48), (23, 0x1300), (2, 24), (0x24, 0x1400), (0x23, 8), (0, 0)]
+    put(data, 0x000, b''.join(struct.pack('<QQ', t, v) for t, v in dyn))
+    put(data, 0x118, struct.pack('<Q', 0x30))
+    put(data, 0x200, b'ABCDEFGH')
+    return nso(text, bytes(ro), bytes(data))
+
+
+# A module with one jump table (clang's compressed form), assembled from:
+#        b fn                         ; 0x0
+#        .word mod0                   ; 0x4: the MOD0 offset
+#   fn:  cmp w0, #3                   ; 0x8
+#        b.hi Ldefault
+#        adrp x8, table               ; table = 0x1020: .byte 0, 2, 4, 6
+#        add x8, x8, #0x20
+#        adr x9, L0
+#        ldrb w10, [x8, w0, uxtw]
+#        add x9, x9, x10, lsl #2
+#        br x9
+#   L0..L3: mov w0, #10/20/30/40; ret ; Ldefault: mov w0, #0; ret
+# .rodata: 32 bytes of text before the table, 21 after it, MOD0 at 0x1040 and a RELATIVE relocation to fn (the
+# lifter's function seed); .data: .dynamic (DT_RELA) and the relocated word.
+JT_TEXT = bytes.fromhex(
+    '02000014401000001f0c0071e8010054080000b008810091890000100a49603829090a8b20011fd640018052c0035fd680028052'
+    'c0035fd6c0038052c0035fd600058052c0035fd600008052c0035fd6')
+JT_RO = bytes.fromhex(
+    '756e746f7563686564206279746573206265666f726520746865207461626c6500020406627974657320616674657220746865'
+    '207461626c65000000000000004d4f4430c00f00000000000000000000c0efffffc0efffff00000000000000003020000000000000'
+    '03040000000000000800000000000000')
+JT_DATA = struct.pack('<7Q', 7, 0x1060, 8, 24, 0, 0, 0)   # DT_RELA, DT_RELASZ, DT_NULL; the word the RELATIVE fills
+
+
+def ips32(writes):
+    """An IPS32 file of {image offset: bytes} (offsets count the NSO header, as emulators read them)."""
+    body = b''.join((off + ep.NSO_HEADER).to_bytes(4, 'big') + len(d).to_bytes(2, 'big') + d
+                    for off, d in sorted(writes.items()))
+    return b'IPS32' + body + b'EEOF'
+
+
+class Classify(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        cls.tmp = tempfile.TemporaryDirectory()
+        path = Path(cls.tmp.name) / 'main'
+        path.write_bytes(reloc_module())
+        cls.m = ep.Main(path)
+        cls.md = ep.MainData(cls.m)
+
+    @classmethod
+    def tearDownClass(cls):
+        cls.tmp.cleanup()
+
+    def run_file(self, writes, md=True):
+        return ep.classify(self.m, self.md if md else None, self.m.nso.build_id, Path(f'{BID_NAME}.ips'),
+                           ips32(writes))
+
+    def test_relocated_words(self):
+        self.assertEqual(self.md.rela, {0x2100: R_RELATIVE, 0x2108: R_GLOB_DAT, 0x2110: R_JUMP_SLOT})
+        self.assertEqual(self.md.relr, {0x2118})
+
+    def test_code_only(self):
+        kind, note, sites, runs, dropped = self.run_file({0x8: struct.pack('<I', MOV_W8_200)})
+        self.assertEqual((kind, note, runs, dropped), ('code', '', [], 0))
+        self.assertEqual(sites, {0x8: [MOV_W8_200]})
+
+    def test_data_only(self):
+        kind, note, sites, runs, dropped = self.run_file({0x1000: b'\xAA\xBB', 0x2200: b'XY'})
+        self.assertEqual((kind, sites, dropped), ('data', {}, 0))
+        self.assertEqual(runs, [(0x1000, b'\xAA\xBB', False), (0x2200, b'XY', False)])
+
+    def test_code_and_data(self):
+        kind, _, sites, runs, _ = self.run_file({0x8: struct.pack('<I', MOV_W8_200), 0x1004: b'\x01'})
+        self.assertEqual(kind, 'code')
+        self.assertEqual(sites, {0x8: [MOV_W8_200]})
+        self.assertEqual(runs, [(0x1004, b'\x01', False)])
+
+    def test_same_bytes_are_nothing(self):
+        kind, _, _, runs, _ = self.run_file({0x1000: b'constant'})
+        self.assertEqual((kind, runs), ('empty', []))
+
+    def test_explicit_addend_words_change_nothing(self):
+        kind, _, _, runs, dropped = self.run_file({0x2100: b'\x11\x22\x33\x44', 0x2108: b'\x55'})
+        self.assertEqual((kind, runs, dropped), ('empty', [], 5))
+
+    def test_relr_word_is_rebased(self):
+        kind, _, _, runs, _ = self.run_file({0x2118: b'\x31'})
+        self.assertEqual(kind, 'data')
+        self.assertEqual(runs, [(0x2118, struct.pack('<Q', 0x31), True)])
+
+    def test_long_runs_are_split(self):
+        _, _, _, runs, _ = self.run_file({0x2200: bytes(i % 255 + 1 for i in range(ep.RUN_MAX)) + b'\xFF'})
+        self.assertEqual([(o, len(d)) for o, d, _ in runs], [(0x2200, ep.RUN_MAX), (0x2200 + ep.RUN_MAX, 1)])
+
+    def test_refused_writes(self):
+        for writes, why in (({0x2110: b'\x01'}, 'binds lazily'), ({0x2000: b'\x01'}, 'linker\'s tables'),
+                            ({0x1208: b'\x01'}, 'linker\'s tables'), ({0x4000: b'\x01'}, 'outside main\'s image'),
+                            ({-0x80: b'\x01'}, 'outside main\'s image')):
+            kind, note, sites, runs, _ = self.run_file({0x8: struct.pack('<I', MOV_W8_200), **writes})
+            self.assertEqual((kind, sites, runs), ('unsupported', {}, []), note)
+            self.assertIn(why, note)
+
+    def test_data_without_a_data_file(self):
+        kind, note, _, _, _ = self.run_file({0x1000: b'\xAA'}, md=False)
+        self.assertEqual(kind, 'unsupported')
+        self.assertIn('only code patches are supported', note)
+
+    def test_pchtxt(self):
+        text = (f'@nsobid-{BID_NAME}\n@flag offset_shift 0x100\n@enabled\n'
+                f'00001000 AABB\n00002200 "XY"\n@disabled\n00002204 CC\n')
+        kind, _, _, runs, _ = ep.classify(self.m, self.md, self.m.nso.build_id, Path('x.pchtxt'), text.encode())
+        self.assertEqual(kind, 'data')
+        self.assertEqual(runs, [(0x1000, b'\xAA\xBB', False), (0x2200, b'XY', False)])
+
+
+class Tool(unittest.TestCase):
+    """The command line: build, then the check after the lift."""
+
+    def setUp(self):
+        self.tmp = tempfile.TemporaryDirectory()
+        self.d = Path(self.tmp.name)
+        (self.d / 'exefs').mkdir()
+        (self.d / 'patches.txt').write_text('# no sets of the port\n')
+
+    def tearDown(self):
+        self.tmp.cleanup()
+
+    def mod_file(self, mod, writes):
+        f = self.d / 'mods' / mod / 'exefs' / f'{BID_NAME}.ips'
+        f.parent.mkdir(parents=True, exist_ok=True)
+        f.write_bytes(ips32(writes))
+
+    def tool(self, *args):
+        r = subprocess.run([sys.executable, str(ROOT / 'tools' / 'exefs_patches.py'), '--exefs', str(self.d / 'exefs'),
+                            *map(str, args)], capture_output=True, text=True)
+        self.assertEqual(r.returncode, 0, r.stdout + r.stderr)
+        return r.stdout
+
+    def build(self):
+        self.tool('--mods', self.d / 'mods', '--patches', self.d / 'patches.txt', '--out', self.d / 'out.txt',
+                  '--status', self.d / 'exefs.status', '--data', self.d / 'exefs.data')
+        status = {f[0]: f for f in (line.split('\t') for line in (self.d / 'exefs.status').read_text().splitlines())}
+        data = [line.split('\t') for line in (self.d / 'exefs.data').read_text().splitlines()
+                if not line.startswith('#')]
+        return status, data
+
+    def key(self, mod):
+        return ep.fnv(f'{mod}/exefs/{BID_NAME}.ips'.encode())
+
+    def test_build_and_check(self):
+        (self.d / 'exefs' / 'main').write_bytes(reloc_module())
+        self.mod_file('Code', {0x8: struct.pack('<I', MOV_W8_200), 0x2200: b'Q'})
+        self.mod_file('Data', {0x1000: b'\xAA', 0x2118: b'\x31'})
+        self.mod_file('Bad', {0x2110: b'\x01'})
+        status, data = self.build()
+        code, dat, bad = self.key('Code'), self.key('Data'), self.key('Bad')
+        self.assertEqual([status[k][2] for k in (code, dat, bad)], ['code', 'data', 'unsupported'])
+        self.assertEqual([status[k][4] for k in (code, dat)], ['data 1', 'data 9'])   # bytes its data lines hold
+        self.assertIn(f'set mod_{code.lower()} TOTK_MODPATCH_{code} 0', (self.d / 'out.txt').read_text())
+        self.assertEqual(sorted((f[0], f[2], f[3], f[4:]) for f in data),
+                         sorted([(code, '2200', '51', []), (dat, '1000', 'aa', []),
+                                 (dat, '2118', '3100000000000000', ['base'])]))
+        # after the lift: .rodata bytes the lifted code read make the Data file unsupported; the Code file's data is
+        # in .data, which the lifter never reads
+        folded = self.d / 'folded.txt'
+        folded.write_text('# test\n1000 1001\n')
+        self.tool('--status', self.d / 'exefs.status', '--data', self.d / 'exefs.data', '--folded', folded)
+        status2 = {f[0]: f for f in (line.split('\t') for line in (self.d / 'exefs.status').read_text().splitlines())}
+        self.assertEqual(status2[code][2], 'code')
+        self.assertEqual(status2[dat][2:4], ['unsupported', '-'])
+        self.assertIn('jump table', status2[dat][4])
+        left = [line.split('\t')[0] for line in (self.d / 'exefs.data').read_text().splitlines()
+                if not line.startswith('#')]
+        self.assertEqual(left, [code])
+
+    def test_check_without_the_lift_list(self):
+        (self.d / 'exefs' / 'main').write_bytes(reloc_module())
+        self.mod_file('Ro', {0x1000: b'\xAA'})
+        self.mod_file('Rw', {0x2200: b'Q'})
+        self.build()
+        self.tool('--status', self.d / 'exefs.status', '--data', self.d / 'exefs.data', '--folded',
+                  self.d / 'missing.txt')
+        status = {f[0]: f[2] for f in (line.split('\t') for line in (self.d / 'exefs.status').read_text().splitlines())}
+        self.assertEqual((status[self.key('Ro')], status[self.key('Rw')]), ('unsupported', 'data'))
+
+    def test_jump_table_through_the_lift(self):
+        """recomp/gen.py lists the jump table's bytes in folded.txt; a write into them is refused, one beside them
+        is kept."""
+        (self.d / 'exefs' / 'main').write_bytes(nso(JT_TEXT, JT_RO, JT_DATA))
+        r = subprocess.run([sys.executable, str(ROOT / 'recomp' / 'gen.py'), '--exefs', str(self.d / 'exefs'),
+                            '--out', str(self.d / 'gen'), '--modules', 'main', '--jobs', '1'],
+                           capture_output=True, text=True)
+        self.assertEqual(r.returncode, 0, r.stderr)
+        folded = self.d / 'gen' / 'main' / 'folded.txt'
+        self.assertEqual(ep.read_ranges(folded), [(0x1020, 0x1024)])
+        self.mod_file('Table', {0x1022: b'\x00'})
+        self.mod_file('Beside', {0x1024: b'B'})
+        self.build()
+        self.tool('--status', self.d / 'exefs.status', '--data', self.d / 'exefs.data', '--folded', folded)
+        status = {f[0]: f[2] for f in (line.split('\t') for line in (self.d / 'exefs.status').read_text().splitlines())}
+        self.assertEqual((status[self.key('Table')], status[self.key('Beside')]), ('unsupported', 'data'))
+
+
+if __name__ == '__main__':
+    unittest.main()

tests/installer_gui_test.py

@@ -377,13 +377,19 @@ class DpiTest(unittest.TestCase):
         code = ('import tkinter\n'
                 'from installer import dpi, gui\n'
                 'root = tkinter.Tk()\n'
-                'print(dpi.awareness(), dpi.scale(), root.tk.call("tk", "scaling"), gui.PAD, gui.WRAP)\n'
+                'print(dpi.awareness(), dpi.scale(), root.tk.call("tk", "scaling"), gui.PAD, gui.WRAP, '
+                'root.winfo_screenwidth())\n'
                 'root.destroy()\n')
         r = subprocess.run([sys.executable, '-c', code], cwd=ROOT, capture_output=True, text=True, timeout=120)
         self.assertEqual(r.returncode, 0, r.stderr[-2000:])
-        awareness, scale, scaling, pad, wrap = r.stdout.split()
+        awareness, scale, scaling, pad, wrap, width = r.stdout.split()
         self.assertGreaterEqual(int(awareness), 1)   # 1 system aware, 2 per-monitor aware
-        self.assertAlmostEqual(float(scaling), float(scale) * 96 / 72, places=6)
+        # Tk's win/tkWinX.c TkWinDisplayChanged rounds the nominal screen width to whole millimetres.
+        # Its generic/tkCmds.c ScalingCmd converts that width back to pixels per point, so a 120 DPI display
+        # can report 120.118 DPI. Keep the exact comparison, accounting for that round trip.
+        width_mm = int(25.4 * int(width) / (float(scale) * 96) + 0.5)
+        expected_scaling = 25.4 * int(width) / (72 * width_mm)
+        self.assertAlmostEqual(float(scaling), expected_scaling, places=6)
         self.assertEqual((int(pad), int(wrap)), (round(14 * float(scale)), round(620 * float(scale))))
 
 

tests/installer_test.py

@@ -315,6 +315,75 @@ class LiftKeyTest(unittest.TestCase):
         self.assertEqual(self.key(), before)
 
 
+class LiftRunTest(unittest.TestCase):
+    """Lift.run with fake tools: the ExeFS patch stage writes the status and data files, the lift lists the bytes it
+    read, and the data writes are checked against that list after it; a failed lift or check drops both files."""
+
+    def setUp(self):
+        tmp = tempfile.TemporaryDirectory()
+        self.addCleanup(tmp.cleanup)
+        base = Path(tmp.name)
+        self.dest, temp = base / 'install', base / 'temp'
+        self.dest.mkdir()
+        temp.mkdir()
+        layout = SimpleNamespace(recomp=ROOT / 'recomp', tools=ROOT / 'tools',
+                                 patches=lambda version: base / 'patches-v143.txt')
+        self.ctx = SimpleNamespace(dest=self.dest, temp=temp, layout=layout, keys={'lift': 'k'},
+                                   state={'version': '1.4.3'}, reporter=Reporter(out=io.StringIO()))
+        self.addCleanup(self.ctx.reporter.close)
+        mod = self.dest / 'mods' / 'SQ' / 'exefs'
+        mod.mkdir(parents=True)
+        (mod / '277178B7DBA1B6D4.ips').write_bytes(b'IPS32EEOF')
+        self.calls, self.fail = [], set()
+        saved = steps.run_tool
+        self.addCleanup(lambda: setattr(steps, 'run_tool', saved))
+        steps.run_tool = self.run_tool
+
+    def run_tool(self, ctx, step, script, args, parse=None, cwd=None):
+        args = [str(a) for a in args]
+        name = Path(script).name + (' --folded' if '--folded' in args else '')
+        self.calls.append((name, args))
+        log = ctx.temp / 'tool.log'
+        log.write_text('')
+        if name in self.fail:
+            return 1, log
+        if name == 'exefs_patches.py':
+            for flag in ('--status', '--data', '--out'):
+                p = Path(args[args.index(flag) + 1])
+                p.parent.mkdir(parents=True, exist_ok=True)
+                p.write_text(flag)
+        elif name == 'gen.py':
+            out = Path(args[args.index('--out') + 1]) / 'main'
+            out.mkdir(parents=True, exist_ok=True)
+            (out / 'folded.txt').write_text('# test\n')
+        return 0, log
+
+    def outputs(self):
+        return [p.is_file() for p in (steps.exefs_status(self.ctx), steps.exefs_data(self.ctx))]
+
+    def test_data_file_and_the_check_after_the_lift(self):
+        steps.Lift().run(self.ctx)
+        self.assertEqual([c[0] for c in self.calls], ['exefs_patches.py', 'gen.py', 'exefs_patches.py --folded'])
+        build = self.calls[0][1]
+        self.assertEqual(build[build.index('--data') + 1], str(steps.exefs_data(self.ctx)))
+        check = self.calls[2][1]
+        self.assertEqual(check[check.index('--folded') + 1], str(self.ctx.temp / 'gen' / 'main' / 'folded.txt'))
+        self.assertEqual(check[check.index('--status') + 1], str(steps.exefs_status(self.ctx)))
+        self.assertEqual(self.outputs(), [True, True])
+
+    def test_failed_lift_drops_the_data_file(self):
+        self.fail = {'gen.py'}
+        with self.assertRaises(steps.InstallError):
+            steps.Lift().run(self.ctx)
+        self.assertEqual(self.outputs(), [False, False])
+
+    def test_failed_check_drops_both_files(self):
+        self.fail = {'exefs_patches.py --folded'}
+        steps.Lift().run(self.ctx)
+        self.assertEqual(self.outputs(), [False, False])
+        self.assertIn('warn.exefs_skipped', self.ctx.reporter.out.getvalue())
+
+
 class IniTest(unittest.TestCase):
     EXAMPLE = '[display]\n# vk_present = mailbox\n\n[language]\n# locale = en-US\n# lang = en-US\n# x = 1\n'
 
@@ -420,6 +489,29 @@ class CleanupTest(unittest.TestCase):
         self.assertEqual(self.run_cleanup(), 'cleanup.romfs_unreadable')
 
 
+class FreeSpaceTest(unittest.TestCase):
+    def test_missing_folders_use_the_nearest_existing_parent(self):
+        with tempfile.TemporaryDirectory() as d:
+            parent = Path(d)
+            path = parent / 'missing' / 'install'
+            with mock.patch.object(steps.shutil, 'disk_usage', return_value=SimpleNamespace(free=1234)) as usage:
+                self.assertEqual(steps.free_bytes(path), (1234, parent.stat().st_dev))
+            usage.assert_called_once_with(parent)
+            self.assertFalse(path.parent.exists())
+
+    def test_missing_root_reports_the_filesystem_error(self):
+        path = Path(ROOT.anchor) / 'missing' / 'install'
+        root = Path(path.anchor)
+        error = FileNotFoundError(2, 'No such file or directory', str(root))
+        # Bound the failed existence checks so the old parent walk cannot hang the test suite.
+        with mock.patch.object(Path, 'exists', side_effect=[False] * (len(path.parents) + 1)):
+            with mock.patch.object(steps.shutil, 'disk_usage', side_effect=error) as usage:
+                with self.assertRaises(FileNotFoundError) as raised:
+                    steps.free_bytes(path)
+        self.assertIs(raised.exception, error)
+        usage.assert_called_once_with(root)
+
+
 class BaseGameCheckTest(unittest.TestCase):
     """Issue #261: an update NSP chosen as the base game was reported as "not Tears of the Kingdom"."""
     BASE, UPDATE = '0100F2C0115B6000', '0100F2C0115B6800'

tests/mouse_direct_check.py

@@ -18,7 +18,7 @@ const int rt_game_version = TEST_GAME_VERSION;
 static int raw_x, raw_y, takes, captured=1, foreground=1, allowed=1, mode=1, patched=1, selecting;
 static float ratio=0.5f;
 static uint64_t now=1000000000;
-static GuestFn hook, activity_hook, begin_hook, angles_hook, hand_hook, load_hook;
+static GuestFn hook, activity_hook, begin_hook, angles_hook, hand_hook, load_hook, state_hook;
 const char *cfg_get(const char *key) { (void)key; return mode ? "direct" : "stick"; }
 const char *cfg_value(const char *key) { return cfg_get(key); }
 double cfg_double(const char *key) { (void)key; return 1; }
@@ -40,13 +40,14 @@ void *rt_patch_page(void) { return ∶ }
 void rt_register_host(uint64_t addr, GuestFn fn) {
     if(addr==HOST_MOUSE_DIRECT) hook=fn; else if(addr==HOST_MOUSE_ACTIVITY) activity_hook=fn;
     else if(addr==HOST_MOUSE_ORBIT_BEGIN) begin_hook=fn; else if(addr==HOST_MOUSE_ORBIT_ANGLES) angles_hook=fn;
-    else if(addr==HOST_MOUSE_HAND) hand_hook=fn; else if(addr==HOST_MOUSE_LOAD) load_hook=fn; else abort();
+    else if(addr==HOST_MOUSE_ORBIT_STATE) state_hook=fn; else if(addr==HOST_MOUSE_HAND) hand_hook=fn; else if(addr==HOST_MOUSE_LOAD) load_hook=fn; else abort();
 }
 void plat_mutex_lock(PlatMutex *m) { (void)m; }
 void plat_mutex_unlock(PlatMutex *m) { (void)m; }
 void rt_log(const char *fmt, ...) { (void)fmt; }
 float cam_speed_applied(void) { return 1; }
-uint64_t input_buttons_last(void) { return 0; }
+static uint64_t buttons;
+uint64_t input_buttons_last(void) { return buttons; }
 int mem_guest_readable(uint64_t a, uint64_t n) { (void)a; (void)n; return 0; }   // no player: on-foot gains
 static float scalar(__uint128_t v) { float f; memcpy(&f,&v,4); return f; }
 #if TEST_GAME_VERSION == 143
@@ -109,6 +110,7 @@ int main(void) {
     uint64_t input_address=(uintptr_t)target; memcpy(camera+0x20,&input_address,8);
     for(int rate=0;rate<3;rate++) {
         memset(orbit_seen,0,sizeof orbit_seen);
+        guest_float_store((uintptr_t)camera+0x15c,0);
         ratio=30/rates[rate]; c.x[20]=(uintptr_t)target; c.x[21]=0; c.v[9]=c.v[10]=0;
         raw_x=100; raw_y=-50; hook(&c);
         guest_float_store((uintptr_t)camera+0x108,30); guest_float_store((uintptr_t)camera+0x120,40);
@@ -118,16 +120,16 @@ int main(void) {
         c.x[20]=(uintptr_t)camera; c.sp=(uintptr_t)frame; angles_hook(&c);
         assert(fabsf(guest_float((uintptr_t)frame+0x44)-(30-50*DEG_PER_COUNT))<0.0001f);
         assert(fabsf(guest_float((uintptr_t)frame+0x48)-(40-100*DEG_PER_COUNT))<0.0001f);
-        assert(fabsf(guest_float((uintptr_t)camera+0x15c)-(30-50*DEG_PER_COUNT))<0.0001f);
+        assert(guest_float((uintptr_t)camera+0x15c)==0);
         c.x[0]=(uintptr_t)camera; begin_hook(&c); angles_hook(&c);
         assert(fabsf(guest_float((uintptr_t)frame+0x48)-(40-100*DEG_PER_COUNT))<0.0001f); // one sample cannot repeat next update
-        // The next input frame is stationary and the game's parameter update rewrites the old default target.
-        // Restoring the mouse's desired target makes the camera filter stay at the new pitch, not spring to zero.
+        // The native target remains available for the distance controller.
+        // Holding pitch happens at the final output, after the native filter.
         c.x[20]=(uintptr_t)target; c.v[9]=c.v[10]=0; raw_x=raw_y=0; hook(&c);
         guest_float_store((uintptr_t)camera+0x15c,0);
         c.x[0]=(uintptr_t)camera; begin_hook(&c);
         float current=guest_float((uintptr_t)camera+0x108), goal=guest_float((uintptr_t)camera+0x15c);
-        assert(fabsf(goal-(30-50*DEG_PER_COUNT))<0.0001f && fabsf((current+goal)*0.5f-(30-50*DEG_PER_COUNT))<0.0001f);
+        assert(goal==0 && fabsf(current-(30-50*DEG_PER_COUNT))<0.0001f);
         // A later game filter overwrites the pitch after the begin hook. Neither a stationary frame nor the next
         // tiny mouse delta may integrate from that recentered result.
         guest_float_store((uintptr_t)camera+0x108,0);
@@ -137,7 +139,7 @@ int main(void) {
         guest_float_store((uintptr_t)frame+0x44,0);
         c.x[20]=(uintptr_t)camera; angles_hook(&c);
         assert(fabsf(guest_float((uintptr_t)frame+0x44)-(30-50*DEG_PER_COUNT))<0.0001f);
-        assert(fabsf(guest_float((uintptr_t)camera+0x15c)-((30-50*DEG_PER_COUNT)-19))<0.0001f);
+        assert(guest_float((uintptr_t)camera+0x15c)==0);
         assert(fabsf(guest_float((uintptr_t)camera+0x108)-((30-50*DEG_PER_COUNT)-14))<0.0001f);
         for(int i=0;i<100;i++) {
             guest_float_store((uintptr_t)frame+0x44,0); angles_hook(&c);
@@ -161,16 +163,76 @@ int main(void) {
         guest_float_store((uintptr_t)camera+0x108,3);   // the game left the pitch at 3 + the offset 14
         c.x[20]=(uintptr_t)target; c.v[9]=c.v[10]=0; hook(&c);
         c.x[0]=(uintptr_t)camera; begin_hook(&c);
-        assert(fabsf(guest_float((uintptr_t)camera+0x15c)-(17-14-5))<0.0001f);
+        assert(guest_float((uintptr_t)camera+0x15c)==0);
         guest_float_store((uintptr_t)frame+0x44,0);
         c.x[20]=(uintptr_t)camera; angles_hook(&c);
         assert(fabsf(guest_float((uintptr_t)frame+0x44)-17)<0.0001f);
         guest_float_store((uintptr_t)camera+0x15c,0);   // the game's parameter update writes its default target
         c.x[20]=(uintptr_t)target; hook(&c);
         c.x[0]=(uintptr_t)camera; begin_hook(&c);
-        assert(fabsf(guest_float((uintptr_t)camera+0x15c)-(17-14-5))<0.0001f);
+        assert(guest_float((uintptr_t)camera+0x15c)==0);
+        // Aim/close-up handoff: the native camera can replace its parameter
+        // reference on every frame. Mouse pitch hold must never overwrite it,
+        // nor restore a stale reference when a controller takes over later.
+        for(int i=0;i<20;i++) {
+            float native_target=5+i, native_radius=2+i*0.1f;
+            guest_float_store((uintptr_t)camera+0x15c,native_target);
+            guest_float_store((uintptr_t)camera+0x164,native_radius);
+            guest_float_store((uintptr_t)camera+0x130,native_radius-0.5f);
+            c.x[20]=(uintptr_t)target; c.v[9]=c.v[10]=0; hook(&c);
+            c.x[0]=(uintptr_t)camera; begin_hook(&c);
+            assert(guest_float((uintptr_t)camera+0x15c)==native_target);
+            guest_float_store((uintptr_t)frame+0x44,0); // native pitch filter
+            c.x[20]=(uintptr_t)camera; angles_hook(&c);
+            assert(fabsf(guest_float((uintptr_t)frame+0x44)-17)<0.0001f);
+            assert(guest_float((uintptr_t)camera+0x15c)==native_target);
+            assert(guest_float((uintptr_t)camera+0x164)==native_radius);
+            assert(guest_float((uintptr_t)camera+0x130)==native_radius-0.5f);
+        }
+        float pad_pitch=0.5f; c.x[20]=(uintptr_t)target;
+        memcpy(&c.v[10],&pad_pitch,4); hook(&c);
+        c.x[0]=(uintptr_t)camera; begin_hook(&c);
+        assert(guest_float((uintptr_t)camera+0x15c)==24);
+        c.v[10]=0;
         memset(camera+0x88,0,4); memset(camera+0x10c,0,4); memset(camera+0x11c,0,4);
     }
+    // The native manual-orbit branch keeps the distance it inherited from
+    // aiming. Mouse-owned orbit must select the native automatic-distance
+    // transition, before initialization; controller and scripted modes keep
+    // their own state. The host hook never writes a distance or position.
+    {
+        assert(state_hook);
+        c.x[20]=(uintptr_t)target; c.x[21]=0; c.v[9]=c.v[10]=0;
+        raw_x=1; raw_y=0; hook(&c);
+        c.x[0]=(uintptr_t)camera; begin_hook(&c);
+        c.x[20]=(uintptr_t)camera; angles_hook(&c);
+        uint32_t manual=4, override=0;
+        memcpy(camera+0x180,&manual,4); memcpy(camera+0x188,&override,4);
+        c.x[10]=4; expected=c; state_hook(&c);
+        expected.x[8]=0; expected.x[10]=3;
+        assert(!memcmp(&expected,&c,sizeof c));
+        uint32_t unchanged; memcpy(&unchanged,camera+0x180,4); assert(unchanged==4);
+        // The game's transition initializer, rather than this hook, owns the state.
+        for(int state=0;state<=5;state++) if(state!=4) {
+            c.x[10]=state; state_hook(&c); assert(c.x[10]==(uint64_t)state);
+        }
+        buttons=1ull<<B_ZR; c.x[10]=4; state_hook(&c); assert(c.x[10]==4);
+        buttons=1ull<<B_R; state_hook(&c); assert(c.x[10]==4); buttons=0;
+        uint8_t forced=0x40; memcpy(target+0x15c,&forced,1);
+        state_hook(&c); assert(c.x[10]==4); forced=0; memcpy(target+0x15c,&forced,1);
+        uint64_t recenter=0x20; memcpy(target+0x148,&recenter,8);
+        state_hook(&c); assert(c.x[10]==4); recenter=0; memcpy(target+0x148,&recenter,8);
+        override=1; memcpy(camera+0x188,&override,4);
+        state_hook(&c); assert(c.x[10]==4 && c.x[8]==1);
+        override=0; memcpy(camera+0x188,&override,4);
+        // Any real controller input retains the native manual-distance behaviour.
+        c.x[20]=(uintptr_t)target; float pad=0.25f;
+        memcpy(&c.v[9],&pad,4); c.v[10]=0; hook(&c);
+        c.x[20]=(uintptr_t)camera; c.x[10]=4; state_hook(&c); assert(c.x[10]==4);
+        c.x[20]=(uintptr_t)target; c.v[9]=c.v[10]=0; hook(&c);
+        c.x[20]=(uintptr_t)camera; c.x[10]=4; now+=150000000ull;
+        state_hook(&c); assert(c.x[10]==4); now++;
+    }
     // A gamepad-only step (no mouse motion, no pitch hold) leaves the camera input and the angles as the game made them.
     memset(orbit_seen,0,sizeof orbit_seen);
     c.x[20]=(uintptr_t)target; c.x[21]=0; raw_x=raw_y=0; c.v[9]=c.v[10]=0;
@@ -233,7 +295,7 @@ int main(void) {
     assert(takes==before && raw_x==42); selecting=0; raw_x=0;
     puts("direct mouse: register contract, routing, linearity, frame ratio and capture checks passed");
     puts("direct mouse: final orbit angles, single-count motion, equal axes and held-object routing passed");
-    puts("direct mouse: persistent pitch target, radial cursor hold and gamepad-only steps untouched passed");
+    puts("direct mouse: pitch hold with native distance reference, radial cursor hold and gamepad-only steps untouched passed");
     puts("direct mouse: final-pitch recenter rejection, explicit handoff and bounded quick-list steps passed");
     // A shake: two direction changes of swings of at least 10 degrees within 0.35 s, then the stick follows the mouse
     // side by side and lets go when it rests; nothing plays on after the mouse
@@ -437,7 +499,7 @@ int main(void) {
 #else
 // 1.2.1: the same hooks with that version's registers and stack slots (MouseLayout layout_121) and its load sites
 int main(void) {
-    assert(hook && activity_hook && begin_hook && angles_hook && hand_hook && load_hook);
+    assert(hook && activity_hook && begin_hook && angles_hook && hand_hook && load_hook && !state_hook);
     (void)takes;
     Cpu c, expected;
     unsigned char target[0x500]={0}, camera[0x300]={0}, hand[0x300]={0};