Mirrors/totk-pc-recomp
1
0 Fork 0
mirror of https://github.com/valentderah/totk-pc-recomp.git synced 2026-11-11 01:41:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

Keep the repository to what players and contributors use; move the agent setup and working notes out of git

Browse Source 
docs/building.md merges the two toolchain docs and the README build steps, and lists the developer checks. The
Maxwell reference moves to docs/maxwell.md. architecture.md loses the milestone table, controls.md describes timed
script lines and input recording, and the tester guide includes shader preparation.

The agent skills, hooks, playbook, backlog and the study notes on upstream projects are the maintainer's local setup:
.claude/, notes/ and CLAUDE.local.md are ignored now, and tools/worktree.py links them into task worktrees. AGENTS.md
keeps the project rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Valiantsin DzerakhandClaude Opus 5.5 committed 2026-10-27 13:12:50 +03:00
1 parent 97d3762ba1
commit e3e72dfb44
24 files changed
+212 -2483
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.claude/settings.json
-16
View File
@@ -1,16 +0,0 @@
{
"hooks": {
"PreToolUse": [
{
"matcher": "Bash|PowerShell",
"hooks": [
{
"type": "command",
"shell": "bash",
"command": "grep -qiE \"taskkill[^|;&]*/IM[^|;&]*totk|Stop-Process[^|;&]*-Name[^|;&]*totk|pkill[^|;&]*totk|killall[^|;&]*totk\" && echo \"{\\\"hookSpecificOutput\\\":{\\\"hookEventName\\\":\\\"PreToolUse\\\",\\\"permissionDecision\\\":\\\"deny\\\",\\\"permissionDecisionReason\\\":\\\"Killing totk.exe by name stops other agents and benchmark runs. Stop only a totk.exe you started, by PID (taskkill //PID <pid> //F).\\\"}}\" || true"
}
]
}
]
}
}
.claude/skills/private-build/SKILL.md
-77
View File
@@ -1,77 +0,0 @@
---
name: private-build
description: Run a task in its own git worktree and build dir, commit on task/<tag>, hand back a branch. Use before editing runtime/ or tools/ code, building totk.exe, or cleaning up a finished task.
---
# Task worktree: start, build, commit, hand back, clean up
Several agents change `runtime/` and `tools/` at once and the coordinator merges their branches, so every task gets a
git worktree `.worktrees/<tag>` on branch `task/<tag>` and a build dir `build/v121-<tag>`. `<tag>` is a short name for
your task.
## 1. Start
```
.venv/Scripts/python tools/worktree.py new <tag>
```
Work and build only inside `.worktrees/<tag>`. The worktree holds tracked files. `build/`, `generated/`, `refs/`,
`toolchain/`, `.venv/`, `.venv-py314/` and `config.local.toml` are links to the main checkout, so everything written
there is shared: `generated/` and `recomp/` belong to the coordinator. A needed lifter or generator change is reported
in the hand-back, never made. `tools/` changes are made in the worktree like any other file.
## 2. Build
```
export PATH=/c/msys64/ucrt64/bin:$PATH && .venv/Scripts/python tools/build_game.py --runtime runtime --out build/v121-<tag>
```
The build reuses `build/v121`'s generated objects: it hardlinks them into `obj/` and compiles only `runtime/*.c`, so a
first build takes about 10 s. It does so when `build/v121` is up to date and every header the generated code includes
is byte-identical in your `runtime/`. If you changed such a header (`cpu.h`, ...) or the main build is stale, it prints
`compiling all generated C: <why>` and rebuilds everything (about 200 s on every core). `--full-build` forces that. A
`.c` change is incremental either way. After every build check the exe's mtime: a failed build leaves the old
`totk.exe` in place, and you would test old code.
Run the game with the `run-game` skill. `totk.exe --probe-kernel`, `--probe-vkexec`, `--probe-nvn-tex` and
`--probe-shader` pass before you hand back.
## 3. Commit
Commit on `task/<tag>` as often as you like.
- Run `python tools/check_content.py --staged` before each commit.
- Messages are plain English sentences, no type prefix, no `Co-Authored-By` trailer.
- `main` stays untouched: you never commit to it, merge into it or push anything.
Keep notes in `build/v121-<tag>/NOTES.md` as you go. When you stop unfinished, write `build/v121-<tag>/HANDOFF.md`
(under 80 lines): findings with paths, what is ruled out, the leading hypothesis, the exact next step, temporary debug
edits (file:line). Both stay uncommitted.
## 4. Hand back
A fix that costs frame time is not done: keep new work off per-draw and per-pixel paths unless the semantics need it.
The coordinator measures `fight.txt` with `--quiet` before merging.
A change on the render path also gets one look: a field `--shot` of your build next to one of `build/v121` at the same
frame. Runs diverge, so small differences prove nothing, but gross regressions show (grass gone, colours off, black
areas) and block the hand-back.
Then:
1. Merge or rebase `main` into `task/<tag>` until your branch merges into `main` cleanly. Rebuild and rerun the four
probes on the result.
2. Report: the branch name, the commit list (`git log main..task/<tag> --oneline`), what each change costs per draw,
shader or frame, the probe results, and any lifter or generator change you need.
## 5. Clean up (coordinator)
After the merge, or when the task is dropped:
1. Copy `*.md` and any helper scripts from `build/v121-<tag>/` to `build/archive/v121-<tag>/`.
2. `.venv/Scripts/python tools/worktree.py remove <tag> --delete-branch` (it unlinks the links before git removes the
worktree, refuses a dirty worktree or an unmerged branch; `--force` overrides).
3. Delete `build/v121-<tag>/`.
Tasks waiting to be continued keep their worktree and build dir. `build/v121` (the main build) and `build/v121-pf`
(shared shader and pipeline caches) are never deleted.
.claude/skills/run-game/SKILL.md
-61
View File
@@ -1,61 +0,0 @@
---
name: run-game
description: Run a TotK build into gameplay from a backed-up save. Use when you need the game in-world (screenshot, perf measurement, frame dump, scripted input) or need to pick a save scene.
---
# Run a build into gameplay
One command loads a save and plays scripted input; it never writes the user's profile (`build/profile`):
```
.venv/Scripts/python tools/run_game.py <build dir> --save <backup> --frames N [--script tools/input/<f>.txt] [--shot out.png] [--env K=V] [--quiet] [--tag name]
```
It copies the profile, swaps in the save from `build/save-backup/`, writes the log to `<build dir>/logs/<tag>.log` and
prints `<log>: rc=<code>`. The game window opens behind the user's windows without taking focus.
## Saves
| `--save` | Scene after `load_save.txt` (gameplay from about frame 1500) |
|---|---|
| `user-save-2026-10-21-field` | Hyrule field, Link in tall grass |
| `user-save-2026-10-21-monsters` | Moblins and Zonai constructs attack at once; use with `--script tools/input/fight.txt` |
| `user-save-2026-10-21` | Hyrule field facing the castle, rainy; distant sky over the castle |
| `user-save-2026-10-23-pillars` | Hyrule field by standing stone pillars, about 17:40, a Zonai construct beside Link; use with `--script tools/input/backrun.txt` (runs backward from 1550: phantom rocks, a one-frame glitch at about 1605) |
| `user-save-2026-10-24-brightbloom` | Cave, about 20:40: a Brightbloom plant at the lower left (its bulb sits in its cup; it floated beside the stem before ddd0ce8), teal Moblin and Bokoblin ahead or Zonai constructs (runs differ) |
| `user-save-2026-10-22-island` | Sky island edge; use with `--script tools/input/island.txt` (runs off, skydives, opens the paraglider with B at frame 1975; X does nothing in a dive) |
`--frames` counts presents. Pick `1500 + the frames you need in-world`.
At most 3 runs go at once across all agents (`TOTK_MAX_RUNS`, lock files in `build/run-slots/`): more exhaust device
memory. `run_game.py` waits for a free slot, so start every game run through it; probes (`--probe-*`) run directly.
## Options that change results
- `--shot` builds pipelines synchronously: use it for pictures, never for timing.
- `--quiet` waits until no other `totk.exe` or `ninja.exe` runs. Use it for every timing you report.
- Input scripts: lines `<frame> <button> [hold]` or `<frame> LSTICK|RSTICK x y [hold]`, names in `docs/controls.md`.
A new script starts with the four A presses of `tools/input/load_save.txt`. The Purah Pad map opens with
`<frame> MINUS 4`.
## Recordings of the user's play
`TOTK_INPUT_RECORD=<absolute path>` writes the merged input as script lines; `TOTK_SHOTS=lo-hi:step` with
`TOTK_SHOTS_DIR=<absolute path>` saves 480x270 frames. The user's sessions live in `build/recordings/` (`<name>.txt` +
`<name>-shots/`). Replay one with `--script build/recordings/<name>.txt`; runs can diverge (weather, positions), so check
the shots of your replay before relying on it.
## Flight recorder
`TOTK_FLIGHT=1` keeps the last 600 frames (`TOTK_FLIGHT_FRAMES`): interval, render-thread blocked time, event counts
and a 320x180 thumbnail each. F8 in the game window, a frame over `TOTK_FLIGHT_SPIKE_MS` (100; one dump per 10 s) or
`TOTK_FLIGHT_AT=<frame>,...` writes `build/flight/<time>-f<frame>/` (`frames.tsv`, `f<frame>.png`) 30 frames later;
the log line `flight: dump <dir>` names it. `python tools/flight_view.py <dir>` lists the frames around the worst
interval and the counts that stand out, and writes `contact.png`. Costs 132 MB of host memory for the thumbnails.
## Running totk.exe yourself
Set `TOTK_ASSUME_FOCUS=1` (an unfocused window stalls the game), `TOTK_INPUT=none` and `TOTK_BACKGROUND_WINDOW=1`.
Other agents run the game at the same time: stop only a `totk.exe` you started, by PID.
Measuring, frame dumps and debugging: `docs/agent-playbook.md`.
.gitignore
+9 -8
View File
@@ -47,12 +47,16 @@ __pycache__/
/tools/xlate/bin/
/tools/xlate/obj/
# Third-party agent skills, restored from skills-lock.json with `npx skills`
/.agents/skills/reverse-engineer-anything/
/.claude/skills/reverse-engineer-anything/
# Coding-agent setup and working notes (skills, hooks, playbook, backlog, research notes, milestone plans): local to
# the maintainer's machine. tools/worktree.py links .claude and notes into task worktrees.
/.claude/
/.agents/
/CLAUDE.local.md
/skills-lock.json
/notes/
/docs/milestones/
# Local agent and editor settings
/.claude/settings.local.json
# Editor settings
.vs/
.vscode/
.idea/
@@ -65,8 +69,5 @@ Thumbs.db
desktop.ini
.DS_Store
# Milestone plans and task splits: working notes for local agents, not published
/docs/milestones/
# Agent task worktrees (tools/worktree.py)
/.worktrees/
AGENTS.md
+11 -42
View File
@@ -1,6 +1,6 @@
# TotK Recomp
Static recompilation of *Tears of the Kingdom* (Switch, ARM64) into a native Windows x86-64 exe with a Vulkan renderer. Plan: [docs/architecture.md](docs/architecture.md).
Static recompilation of *Tears of the Kingdom* (Switch, ARM64) into a native Windows x86-64 exe with a Vulkan renderer. Design: [docs/architecture.md](docs/architecture.md). Toolchain, build commands and tests: [docs/building.md](docs/building.md).
## Game content boundary
@@ -13,52 +13,21 @@ This repository is public. Game data lives outside it and every derived file sta
## Reusing code
`refs/` holds upstream checkouts at the commits the research notes cite (`tools/fetch_refs.sh` recreates it). Read-only.
`refs/` holds the upstream checkouts (`tools/fetch_refs.sh` recreates it). Read-only.
- Copy only MIT, BSD, ISC, zlib or Apache-2.0 code, keep its header, and add a row to `THIRD_PARTY_NOTICES.md`.
- yuzu, suyu, OpenSmash-Ultimate and other GPL code: read for behaviour, write our own.
- `tools/vendor/` stays byte-identical to upstream; Windows changes go in `tools/extract.py`.
## Research notes
## Code
Read the note before working on its area; each cites `file:line` in `refs/`.
- Lifter, generator, Windows memory layer, differential tests: [docs/research/lasrecomp.md](docs/research/lasrecomp.md)
- TotK host behaviour, NVN, shaders, extraction, version tables, patches: [docs/research/macnativegames.md](docs/research/macnativegames.md)
- Other ports and recompilers: [docs/research/prior-art.md](docs/research/prior-art.md)
- Sizes, function and instruction counts, OS touch-points of the 1.2.1 modules: [docs/research/totk-v121-inventory.md](docs/research/totk-v121-inventory.md)
- Maxwell GPU semantics our Vulkan path must match, TotK bugs other ports and emulators fixed: [docs/research/maxwell.md](docs/research/maxwell.md)
- Windows host costs (timers, file I/O and flushes, thread sampling, VMA allocation, bit flips, run-to-run spread): read before adding a wait, file access or allocation on a game or render thread, or comparing frame times: [docs/research/windows-host.md](docs/research/windows-host.md)
- Symptoms other ports already solved (artifacts, crashes, pacing, scheduling) and whether we have the fix; search it before investigating any symptom: [docs/research/port-lessons.md](docs/research/port-lessons.md)
## Toolchain
Setup and verify commands: [docs/toolchain-windows.md](docs/toolchain-windows.md), [docs/toolchain-msys2-python.md](docs/toolchain-msys2-python.md).
- Build C with MSYS2 UCRT64 Clang (`C:\msys64\ucrt64\bin`). Generated C uses GNU extensions, so `cl.exe` is out.
- A 32-bit winlibs `cmake`/`ninja` sits early on the user PATH. Call CMake and Ninja from `C:\msys64\ucrt64\bin` by full path or put that directory first.
- Two Python environments: `.venv` (3.12) for the lifter and LAS Recomp tools, `.venv-py314` for MacNativeGames' extraction scripts, which need `compression.zstd`.
- Game extraction: `.venv-py314\Scripts\python tools\extract.py` writes verified ExeFS and RomFS to `build/extracted/v0` (1.0.0) and `build/extracted/v121` (1.2.1, the primary target). `tools/vendor/` stays byte-identical to upstream; Windows changes go in `tools/extract.py`.
- Lifter (`recomp/`) and CPU headers (`runtime/`) start from LAS Recomp. Every lifter change passes the differential tests against Unicorn, all run with `.venv`:
- `tests\diff\lift.py`: instructions sampled from the extracted 1.2.1 modules (one per opcode signature, then random); `--coverage` lists every word the lifter rejects or turns into a trap.
- `tests\diff\simd.py`: the whole FP/SIMD encoding space; needs no game files.
- `tests\diff\atomic.py`: exclusive load/store sequences and ARMv8.1 atomics; needs no game files.
- `tests\diff\func.py`: whole leaf functions as the generator emits them, loops and jump tables included.
- `tests\excl_mt.py`: the exclusive monitor helpers of `cpu.h` under contention and ABA, on several threads; needs no game files.
- Game code build (`.venv`): `recomp\gen.py --exefs build\extracted\v121\exefs --out generated\v121 --patches recomp\patches-v121.txt` writes the C (about 15 s). The patch table holds the 60 fps and real-time frame step code patches, switchable at run time (`recomp/patches.py`); `.venv-py314\Scripts\python tools\make_romfs_patches.py` writes their RomFS half to `build\v121\romfs-patches`. `tools\build_game.py` compiles and links `build\v121\totk.exe` with Ninja (about 200 s clean, incremental after). The AMD driver cache that makes prepared shaders fast is keyed to the exe path and holds a limited amount: preparing another build dir can evict this one's libraries (40-220 ms per new effect in play); `totk.exe --warm-shaders` rebuilds them from the shader cache (seconds when they are still cached). `recomp/layout.py` alone decides module bases. Jump tables are resolved by `recomp/jumptables.py`; after changing it, `tests\jumptables_check.py` must report 0 disagreements with its Unicorn oracle (on 1.4.3 too: `--exefs build\extracted\v143\exefs`).
- Game version 1.4.3: the same commands with `v143` (`--patches recomp\patches-v143.txt`, `build_game.py --version v143`, `make_romfs_patches.py --version v143`). Its builds default to `build\profile-v143`; `run_game.py` runs it on copies of the 1.2.1 save backups.
- Shader census (`.venv-py314`): `tools\shader_census.py --version v121 [--translate]` extracts every Maxwell shader to `build/v121/shaders/` and runs it through `build\xlate\totk_xlate.exe` and `spirv-val`; failure groups go to `failures.txt`. `tools\xlate_smoke.py` checks the translator alone, without game files.
- Shader preparation (`.venv-py314`, once per build dir and translator): `tools\prepare_shaders.py build\v121` translates every game shader's default variant and the variants of the MacNativeGames shared shader list (read from `refs/` at run time, with the rows its history dropped: `tools/shader_list.py`) into the build's shader cache and builds their stage libraries into the driver cache (`totk.exe --prepare-shaders`; 1.4.3: 29,587 variants, 7 min on 24 threads, 1.7 GB); play then translates and compiles only variants neither covers.
- Open work with its evidence: [docs/backlog.md](docs/backlog.md).
- Installer (`installer/`, `tools/make_release.py`), which builds the game on a player's machine: [docs/installer.md](docs/installer.md).
- Running, measuring or frame-dumping the game, save backups, task worktrees and their builds: [docs/agent-playbook.md](docs/agent-playbook.md).
- Project skills (`.claude/skills/`): `private-build` before changing runtime/ or tools/ code and building totk.exe (one git worktree per task); `run-game` to run a build into gameplay from a save.
- hactool: `toolchain/hactool/hactool.exe`. From an MSYS2 shell pass `-k <keys>` explicitly; the shell's `HOME` hides the default key location.
- Ghidra 12.1.2 with SwitchLoader: `toolchain/ghidra/ghidra_12.1.2_PUBLIC`. Game modules (NSO) open here, headless or GUI.
- REA (`reverse-engineer-anything` skill + MCP server) analyses our own x86-64 builds. On Windows its Ghidra provider admits PE files only, so NSO analysis goes through Ghidra directly. Register the MCP server once per clone, at local scope:
```powershell
claude mcp add --scope local rea -e "GHIDRA_INSTALL_DIR=<repo>\toolchain\ghidra\ghidra_12.1.2_PUBLIC" -e "JAVA_HOME=<jdk-21>" -- cmd.exe /d /c npx -y rea-agents@6.7.0 mcp
```
- Build C with MSYS2 UCRT64 Clang (`C:\msys64\ucrt64\bin`). Generated C uses GNU extensions, so `cl.exe` is out. Call Ninja from that directory by full path or put it first on PATH.
- Two Python environments: `.venv` (3.12) for the lifter and the build, `.venv-py314` for the extraction and shader scripts, which need `compression.zstd`.
- Generated C is never edited by hand: change the generator (`recomp/`).
- Every lifter change passes the differential tests against Unicorn ([docs/building.md](docs/building.md), "Checks"). After changing `recomp/jumptables.py`, `tests\jumptables_check.py` reports 0 disagreements on 1.2.1 and 1.4.3.
- `recomp/layout.py` alone decides module bases.
- Graphics bugs: [docs/maxwell.md](docs/maxwell.md) lists the Maxwell behaviour the Vulkan path must match and the TotK bugs other ports fixed.
- Comments say what the code does and why. Cite code by file plus function and what it does; a line number may follow as a hint.
## Writing
README.md
+7 -77
View File
@@ -16,7 +16,7 @@ The Switch code is lifted from ARM64 to C ahead of time, compiled with Clang, an
## Status
This is an early build: version 1.2.1 starts, loads saves and runs in the world at 60 fps with keyboard, mouse or gamepad input and with sound, and it has been tested on AMD, NVIDIA and Intel GPUs. Visual bugs and crashes are expected; the known ones are listed in [docs/backlog.md](docs/backlog.md). No release has been published yet, and macOS, Linux, other game versions and an in-game settings menu are not supported yet.
This is an early build: version 1.2.1 starts, loads saves and runs in the world at 60 fps with keyboard, mouse or gamepad input and with sound, and it has been tested on AMD, NVIDIA and Intel GPUs. Visual bugs and crashes are expected. No release has been published yet, and macOS, Linux, other game versions and an in-game settings menu are not supported yet.
To report a problem, open an issue and follow [CONTRIBUTING.md](CONTRIBUTING.md). Testers: [docs/tester-guide.md](docs/tester-guide.md) says what to send back.
@@ -32,86 +32,18 @@ To report a problem, open an issue and follow [CONTRIBUTING.md](CONTRIBUTING.md)
## How this project is made
Most of the code and documentation here is written by AI coding agents (Claude Code) under the maintainer's direction. The maintainer sets the goals, plays every build, decides what is merged and reviews the result; the agents read the upstream code, write and test the changes and measure them. [AGENTS.md](AGENTS.md) and [docs/agent-playbook.md](docs/agent-playbook.md) are the rules they work under. The same standards apply to every contribution, whoever or whatever wrote it: [CONTRIBUTING.md](CONTRIBUTING.md).
Most of the code and documentation here is written by AI coding agents (Claude Code) under the maintainer's direction. The maintainer sets the goals, plays every build, decides what is merged and reviews the result; the agents read the upstream code, write and test the changes and measure them. [AGENTS.md](AGENTS.md) holds the rules they work under. The same standards apply to every contribution, whoever or whatever wrote it: [CONTRIBUTING.md](CONTRIBUTING.md).
## Documentation
- Players and testers: [controls](docs/controls.md), [tester guide](docs/tester-guide.md).
- How it works: [architecture](docs/architecture.md), with research notes on the upstream projects, Maxwell GPU behaviour and what other ports learned in [docs/research/](docs/research/).
- What is being worked on: [backlog](docs/backlog.md).
- Building: [toolchain-windows.md](docs/toolchain-windows.md), [toolchain-msys2-python.md](docs/toolchain-msys2-python.md), and the [installer](docs/installer.md) that builds the game on a player's machine.
- Coding agents: [AGENTS.md](AGENTS.md) and [docs/agent-playbook.md](docs/agent-playbook.md).
- How it works: [architecture](docs/architecture.md), and the [Maxwell GPU behaviour](docs/maxwell.md) the Vulkan back end has to match.
- Building: [building](docs/building.md), and the [installer](docs/installer.md) that builds the game on a player's machine.
- Coding agents: [AGENTS.md](AGENTS.md).
## Building from source
Windows 10 or later, an x86-64 CPU with AVX2, and a GPU with Vulkan 1.3 and `VK_EXT_external_memory_host`. Allow about 60 GB for the work folder. Extraction wants about 55 GB free while it runs and keeps about 35 GB afterwards.
Install the toolchain from [docs/toolchain-windows.md](docs/toolchain-windows.md) and [docs/toolchain-msys2-python.md](docs/toolchain-msys2-python.md): MSYS2 UCRT64 (Clang, Ninja, SDL3, Vulkan, shaderc), Python 3.12 and 3.14, the Vulkan SDK, the .NET 10 SDK, Visual Studio Build Tools, and hactool. A 32-bit `cmake` or `ninja` earlier on `PATH` will be picked up by accident; the build calls the MSYS2 copies itself.
From the repository root, in PowerShell:
1. Python environments (once):
```powershell
uv venv --python 3.12 .venv
uv pip install --python .venv/Scripts/python.exe -r tools/requirements.txt
uv venv --python 3.14 .venv-py314
uv pip install --python .venv-py314/Scripts/python.exe -r tools/requirements-py314.txt
```
2. Copy `config.example.toml` to `config.local.toml` and set the paths to your base game, the 1.2.1 update and `prod.keys`. `config.local.toml` is ignored by git. The tools read the keys file themselves.
3. Extract and verify:
```powershell
.venv-py314\Scripts\python tools\extract.py
```
This writes `build\extracted\v0` (base 1.0.0) and `build\extracted\v121` (1.2.1). A hash mismatch stops the run.
4. Shader translator (once; no game files):
```powershell
.venv\Scripts\python tools\build_xlate.py
```
This publishes `build\xlate\totk_xlate.exe`.
5. Lift the 1.2.1 modules, then compile and link:
```powershell
.venv\Scripts\python recomp\gen.py --exefs build\extracted\v121\exefs --out generated\v121 --patches recomp\patches-v121.txt
.venv-py314\Scripts\python tools\make_romfs_patches.py
.venv\Scripts\python tools\build_game.py
```
`--patches` builds in the 60 fps and frame-step patches; `make_romfs_patches.py` writes their RomFS half to `build\v121\romfs-patches`. The link copies `SDL3.dll`, `libiconv-2.dll` and `libopus-0.dll` next to the exe.
6. Prepare the shaders (once per build). It needs `refs\MacNativeGames` (`tools\fetch_refs.sh`) for the shared shader list:
```powershell
.venv-py314\Scripts\python tools\prepare_shaders.py build\v121
```
Without this step the first launch compiles shaders and pipelines during play and stutters until the caches fill.
7. Package a portable folder (optional):
```powershell
.venv\Scripts\python tools\package_build.py
```
This writes `build\v121-portable`, which runs on another of your machines without MSYS2, Python or .NET. `romfs.bin` stays out unless you pass `--with-romfs`; otherwise put it in `romfs\` next to the exe or point `TOTK_ROMFS` at its folder. After copying, run `totk.exe --prepare-shaders` once in the new folder, because the driver keeps its pipeline cache per exe path. The folder holds game code and data derived from your game: it is for your own machines only, never share it.
8. Run:
```powershell
build\v121\totk.exe
```
Keep the window in the foreground; an unfocused window stalls the game. Close the window to quit.
`build\v121\totk.exe --selftest` checks the build without game files.
Windows 10 or later, an x86-64 CPU with AVX2, a Vulkan 1.3 GPU and about 60 GB of disk. [docs/building.md](docs/building.md) lists the toolchain (MSYS2 UCRT64 Clang, Python 3.12 and 3.14, .NET 10, the Vulkan SDK, hactool) and the steps: extract and verify your game files, build the shader translator, lift the modules to C, compile and link `build\v121\totk.exe`, prepare the shaders.
Settings live in `totk.ini` next to `totk.exe`; copy [totk.ini.example](totk.ini.example), which lists them, and edit it. A `TOTK_*` environment variable of the same name overrides the file (`TOTK_FPS_LIMIT=30` beats `fps_limit = 60`). Saves go to `build\profile`, created on first use. Everything the build writes stays under `build\` and `generated\`, which git ignores.
@@ -136,8 +68,6 @@ This project stands on two MIT codebases and the projects they came from. Code t
- **[SavageWindRecomp](https://github.com/Revivedx/SavageWindRecomp)** (MIT), a static recompilation of *Breath of the Wild*, is the model for preemption at indirect calls and a reference for frame pacing and the platform split.
- The Switch homebrew community's documentation ([Switchbrew](https://switchbrew.org)).
Research notes on each of them: [docs/research/](docs/research/).
## Legal
TotK Recomp is an independent, non-commercial fan project. It is not affiliated with, endorsed by or sponsored by Nintendo. *The Legend of Zelda*, *Tears of the Kingdom* and *Nintendo Switch* are trademarks of Nintendo.
@@ -148,7 +78,7 @@ No game files, game code, assets, shaders, fonts or keys are included in this re
The project's own code is under the [MIT License](LICENSE). The license covers this project's code only and grants nothing for Nintendo's game, which you supply yourself.
Third-party code keeps its own license. The full list, with the commit each copy was taken from, is [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). A local build links or ships these pieces (versions are the MSYS2 UCRT64 packages the toolchain doc was written against; a newer package from the same source is fine):
Third-party code keeps its own license. The full list, with the commit each copy was taken from, is [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). A local build links or ships these pieces (versions are the MSYS2 UCRT64 packages [docs/building.md](docs/building.md) was verified with; a newer package from the same source is fine):
| Component | Version | License | How it is used |
|---|---|---|---|
docs/agent-playbook.md
-149
View File
@@ -1,149 +0,0 @@
# Agent playbook: running, measuring and debugging the game
Read this before you run the game, take a frame dump or measure performance. This page holds the conventions; the
`--help` of each tool holds its options.
## Work in a task worktree
Start, build, hand back and clean up a task with the `private-build` skill (`.claude/skills/private-build/SKILL.md`): a git worktree on `task/<tag>` and a build dir `build/v121-<tag>`.
The pipeline cache, `pipeline-keys.bin`, `shader-cache`, `shader-programs.bin` and `shader-answers.bin` live next to
each `totk.exe`, so a task build starts cold: seed it by copying `build/v121`'s five before timing anything. A build seeded
from an older one stalls on pipeline and shader builds, and its slow frames say nothing about the change.
## Running
Run a build into gameplay with the `run-game` skill (`.claude/skills/run-game/SKILL.md`): `tools/run_game.py`, the save
backups and their scenes, input scripts. `build/profile` is the user's real profile: never write it. Other agents run
the game at the same time: stop only a `totk.exe` you started, by PID; `taskkill /IM totk.exe` closes every instance.
## Measuring
Timings are only comparable on a quiet machine. Pass `--quiet` to take the machine-wide lock `build/.gpu-run.lock`
(held until the game exits, freed by the OS if the caller dies, holder pid and build in the file), then wait until no
other `totk.exe` or `ninja.exe` runs. The log's first and last lines record the count either way.
- `--env TOTK_TRACE=perf,sched`, then `python tools/perf_summary.py <log> <lo> <hi>`: fps, max frame and the
render-thread buckets per 60 frames. Per-thread run, core, host, lock and addr shares come per 120 presents.
A second `perf: frame N: gpu avg` line per window has the GPU busy time per frame (timestamp queries per submission
part, `vk_sync.c`; upload batches not included), fence waits of the render thread and of other threads, VRAM use and
budget (`VK_EXT_memory_budget`), process RAM and `bound: gpu|cpu|pace` (pace: the frame waited for pacing 15% of the
time or more; gpu: the GPU was busy 80% of the interval or more).
- The pool census (`mem: census`, `vk_memory.c` `vk_mem_census`) is logged at frame 1800, with `perf` every 1800
frames: pools by kind and flags, and the bytes bound or copied per frame by pool class since frame 1200. Also at
frame 600. `mem: class` (once at start) says whether the GPU counts as uma or discrete and lists its heaps and types.
- With `perf`, `mem: dirty` every 60 frames: what a VRAM mirror of the buffer-bound pools would upload (pages the CPU
wrote per frame and per submit, eager and lazy, by pool class), FlushMappedRange calls and bytes, and the mirror
size. The dirty count costs about 1.1 ms of CPU per frame on the submitting thread here (3 ms on an 8th-gen i7);
`perf_summary.py` summarises it.
- `TOTK_PROFILE=<path> TOTK_PROFILE_FRAMES=lo-hi`: sampling profiler (`runtime/prof_win.c`), read with
`tools/prof_report.py <profile> <totk.map>`. With `TOTK_PROFILE_RENDER=1` it samples the render thread only and keeps
call chains: the report adds inclusive time per function and the exe callers of the driver and system DLLs.
- `tools/regress.py <build> [--base <build>] [--routes fight,rock,island,field] [--repeat 3] [--cold] [--validation]
[--out dir]`: runs the routes through `run_game.py --quiet`, scores logs and `TOTK_SHOTS` frames
(`tools/regress_score.py`, tested by `tests/regress_score_test.py`) and prints median and worst per metric. Run the candidate alone and judge it against
the targets: max fps, no slow frames (`perf: slow frame`), no black or flat frames, no new validation errors.
`--base` (a diff with `ok`, `better`, `worse`) only when a question needs a side-by-side; do not rerun a base build
by habit. Glitches are found by looking at the shots and dumps, not by the diff. `--cold` empties only the
pipeline cache. Each route writes `<out>/<build>/<route>-worst.png` (the 6 worst frames) and `route_failed` marks a
run that ended in game over, crashed or stopped early: its numbers do not count. One run takes 2 to 4 minutes.
- Baselines (Ryzen 9 7900X, RX 9070, 1600x900, 60 fps patches and real-time step on, 2026-10-23):
- field: 59.4 fps mean, 60 median, replay 9.5 ms per frame;
- `fight.txt`: 59.1 fps mean, 60 median, replay 13.6 ms, 0 CPU-side slow frames; the slow frames left are
pipeline and shader-translation waits.
- `--validation` runs work in the world (1800 frames checked). The layer prints each message kind 10 times, so it
gives no totals; the known messages left are in the backlog, item 6.
## Frame dumps
`TOTK_DUMP=<frame> TOTK_DUMP_DIR=build/captures/<name>` writes per-pass PNGs, `passes.txt` and `images.txt`
(`runtime/vk_dump.c`). `passes.txt` lists bound textures with swizzles, storage buffers with their c0 slots, and
copies and clears. `TOTK_DUMP_BUF=addr:size[:off:stride:w:h]` dumps a guest buffer.
- A draw's argument lines come before its pass header.
- Pass numbers shift between runs. Find a pass by its draw count instead: the g-buffer pass has about 1990 draws.
RenderDoc cannot capture: it hides `VK_EXT_external_memory_host` ([controls.md](controls.md), "RenderDoc capture").
## Other GPUs
The RX 9070 is the only GPU agents can run; the build also has to run on NVIDIA and older AMD (Vega, wave64 only). New
Vulkan code queries the feature, extension, format or limit it uses and keeps a working path or a logged fallback when
it is missing (`vk_pipeline.c` provoking vertex and subgroup size pinning are examples). Never hard-code what this GPU
reports: alignments, subgroup size, depth formats, `maxPushDescriptors`.
Numbers from this machine do not carry over. The user's other machines (the test machines of
[backlog.md](backlog.md)) measured:
- GTX 1050 Ti (4 GiB, 214 MiB BAR, no ReBAR; i7 8th-10th gen) and RTX 3090 Ti: GPU time 110-120 ms per frame in the
world against 4.3 ms here, because the GPU reads imported host pools over PCIe. The VRAM mirror (cd6f490) brings
the 1050 Ti to 10-11 ms; that machine is now CPU-bound at 25-45 fps (replay 25-30 ms per frame, backlog item 8).
- The same write-watch scan costs about 3x the CPU time it costs here.
- NVIDIA and AMD desktop GPUs have no ASTC: textures decode in a compute shader (`TOTK_ASTC`). Before that, CPU
decodes made 0.4-0.7 s load frames on the 1050 Ti.
- `cannot alias its pool memory` fills its 16-line log limit on NVIDIA and on the RX 9070 alike (1920x1080, 1600x900,
800x450 render targets); a suspect for the black patches of backlog item 2.
A run on another machine goes through the user: hand over a build that starts with `TOTK_TRACE=perf` (the
environment, or `trace = perf` in its `totk.ini`) and writes stderr to a log file, and ask for the log. The user
cannot run scripted input there.
## Checks before handing back
`totk.exe --probe-kernel`, `--probe-vkexec`, `--probe-nvn-tex` and `--probe-shader` run from the build directory and
must pass. CPU-header changes also need `tests\diff\simd.py`, `lift.py` and `func.py` (AGENTS.md).
`tools\build_game.py` hashes every object against `obj/objects.sha256` before it links and recompiles a mismatch (the
faulty RAM flipped bits in stored objects once). The line `objects verified: N, recompiled after
a hash mismatch: M` should show M of 0; a nonzero M means rerun the probes. A task build checks the shared objects against
the main build's manifest, so build `build/v121` first.
## Bug classes already fixed
Check these classes before you re-investigate a symptom that looks like one of them.
- A colour target outside the fragment shader's output map must keep its texel, as on Maxwell: write mask 0 from
`vk_pipeline.c` `target_wmask`. AMD wrote undefined values there (lime silhouettes in grass); zero-filling it
instead wiped the half-res depth (sky rectangles).
- The CPU copies GPU-rendered textures between pools with memcpy (grass tile maps): `nvn_api_core.c` `ov_memcpy`
writes them back first. Other CPU reads of GPU output are not covered yet. A write-back skips a texture whose
storage the CPU rewrote after the GPU render (`vk_texture.c` `write_back`); writing it anyway put stale grass maps
into the far terrain for a second (texture flicker on a hit).
- The CPU reads GPU-rendered textures in CPU-cached pools: `nvnMemoryPoolInvalidateMappedRange` writes them back
(`nvn_ext.c` `nvn_pool_invalidate`; TotK invalidates its 260x260 R16 grass maps and 1x1 targets), and once the CPU
waits for a fence, targets up to 64 KiB rendered before it go back too (`vk_texture.c`
`vk_texture_write_back_small`): the 128x128 cut map is read with no invalidate. Without it a cut left the grass
standing, with no blades and no insects.
- A one-shot draw (a pass none of whose targets the previous frame rendered) waits for its shader and pipeline instead
of being skipped: `vk_exec_draw.c` `prepare_draw`, `TOTK_PIPE_SKIP`. Skipping left the cloud and fog volumes empty
for the session. Its stage libraries build on idle workers in parallel (`vk_pipeline.c` `lib_wait`).
- Storage buffers bound by Ryujinx first-use order instead of the NVN c0 slot: `vk_exec_draw.c` `ssbo_slot`. It showed
as tile rectangles.
- 3D colour targets rejected: `vk_exec.c` `resolve_target`. The grading LUT was never written, so the dialog
background and the loading map were black.
- Primitive restart ignored: `vk_exec_draw.c` `restart_for`. It showed as streaks on cliffs.
- The slot-follow path picked the newest texture at shared storage: `vk_texture.c` `vk_texture_by_id` now prefers
`same_but_swizzle`. It showed as a streaky map screen.
## Lessons from other ports
Before you form a hypothesis about a symptom, follow "Before you investigate a symptom" in
[research/port-lessons.md](research/port-lessons.md): evidence first, then a search of the lessons and the upstream
notes. `refs/nxnative/docs/agents/05-debugging.md` has recipes for intermittent glitches, racy crashes and memory growth.
## Context budget
Read output through a filter; the full text stays on disk.
- Game logs run to megabytes: `grep` the lines you need, `tail -20` for the end, `perf_summary.py ... | grep SUMMARY`
for timings.
- Generated C (`generated/v121`) runs to hundreds of thousands of lines per file: `grep -n` the function, then
`sed -n` its range.
- Look at screenshots downscaled: paste several into one 1280x720 grid with PIL and read the grid.
- Long-running commands go to the background with their output in a file; read the file's last lines when it ends.
- Research notes (`docs/research/`) run to 40 KB each: `grep -n` the symptom or the subsystem, then read that
section.
- Read each screenshot once; write what it shows into the notes instead of opening it again.
- Keep notes in `build/v121-<tag>/NOTES.md` from the first finding, so a handoff costs one write.
After about 40 tool calls (a call costs about 2.4k tokens of context), write `build/v121-<tag>/HANDOFF.md` (under 80 lines) and stop. Include:
findings with file paths, what is ruled out, the leading hypothesis, the exact next step, and the temporary debug edits
(file:line). A fresh agent continues from it.
docs/architecture.md
+5 -20
View File
@@ -1,6 +1,6 @@
# Architecture
How the Switch build of *Tears of the Kingdom* becomes a native Windows x86-64 program. Each choice below cites the research note it rests on: [lasrecomp.md](research/lasrecomp.md), [macnativegames.md](research/macnativegames.md), [prior-art.md](research/prior-art.md).
How the Switch build of *Tears of the Kingdom* becomes a native Windows x86-64 program. The upstream projects it builds on are listed under Credits in the [README](../README.md).
## Starting points
@@ -24,20 +24,20 @@ MacNativeGames cannot run on x86-64 as is: its CPU path executes ARM64 code nati
- ExeFS and RomFS extraction: MacNativeGames' Python extractor (Python 3.14 for `compression.zstd`, `.venv-py314`).
- Update merging: hactool at `1d64a83`, with MacNativeGames' BKTR patches (`toolchain/hactool`).
- Verification: MacNativeGames' `expected-v*.json` hashes of ExeFS, RomFS and fonts.
- Primary version: 1.2.1, the update most players have. MacNativeGames supports it with the same build path as the others. 1.0.0, 1.1.2 and 1.2.1 share `sdk`, `subsdk0` and `rtld` byte for byte, so the lifted SDK modules and the host's SDK boundary serve all three; only `main` differs. 1.4.3 replaced all three modules (new SDK 20.5.22, RELR relocations, audio renderer REV15) and is a second target on the same build path: `--version v143`, `recomp/patches-v143.txt` (state in backlog item 39).
- Primary version: 1.2.1, the update most players have. MacNativeGames supports it with the same build path as the others. 1.0.0, 1.1.2 and 1.2.1 share `sdk`, `subsdk0` and `rtld` byte for byte, so the lifted SDK modules and the host's SDK boundary serve all three; only `main` differs. 1.4.3 replaced all three modules (new SDK 20.5.22, RELR relocations, audio renderer REV15) and is a second target on the same build path: `--version v143`, `recomp/patches-v143.txt`.
- Windows driver: `tools/extract.py` runs the vendored extractors (`tools/vendor/macnativegames/`) and identifies the update by the SHA-256 of its `main`. hactool has no Darwin `funopen` here, so the base Program NCA is copied out of the NSP before the BKTR merge (about 17 GB of temporary space).
### 2. CPU: lift ARM64 to C
- Modules: `main`, `sdk`, `subsdk0`. `rtld` is dropped and linking is done by the host, as MacNativeGames does.
- Base: LAS Recomp's hand-written decoder (`lifter.py`) and NEON/FP lowering (`simd_c.py`, `simd.h`), MIT. Output: one C function per guest function, direct calls for `BL`, an address-keyed table for indirect branches.
- Rewritten for TotK scale (see lasrecomp.md, "Scaling risks"):
- Rewritten for TotK scale:
- generator runs in parallel processes, with bounded memory per worker;
- jump tables resolved by our own symbolic pass instead of Unicorn, which keeps GPL code out of the shipped generator;
- `LDXP`/`STXP` (128-bit exclusive pairs) supported;
- no 64 MB limit on `main`; `subsdk*` loaded;
- guest registers kept in C locals where liveness allows, instead of memory.
- Coverage oracle: MacNativeGames' NSO rewriter lists every site where game code touches the OS (SVCs, TLS reads, counter reads, `x18`, exclusive windows, jump tables): 68,968 sites in 1.4.3 `main`; 1.2.1 counts are in [totk-v121-inventory.md](research/totk-v121-inventory.md). The lifter must handle every one.
- Coverage oracle: MacNativeGames' NSO rewriter lists every site where game code touches the OS (SVCs, TLS reads, counter reads, `x18`, exclusive windows, jump tables): 68,968 sites in 1.4.3 `main`. The lifter must handle every one.
- Memory: guest address equals host address, all below 2^39 (IPC encodes 39-bit addresses). The heap, stacks and the alias region are reserved with `MEM_WRITE_WATCH`, so the graphics back end can ask which pages the CPU wrote since the last upload (`runtime/memory.c`).
- Indirect branches go through `rt_dispatch` (`runtime/dispatch.c`), which looks the target up in the address table and tail-calls it.
- Testing: LAS Recomp's differential harness, every lifted instruction class checked against Unicorn (test-only, never linked into the game).
@@ -65,8 +65,6 @@ Every guest thread is a host thread. TotK assumes Horizon's scheduler: three app
- Host work gives the core back. Queue submits, presents, GPU waits and shader translation run without a core (`NVN_BLOCKING`, `kern_core_away`), so a thread blocked in the driver does not hold up guest threads.
- `TOTK_FREE_CORES=1` turns the gate off, for experiments.
Other ports and what they changed about scheduling: [port-lessons.md](research/port-lessons.md), rows S1-S6.
### 4. Graphics
- NVN front end from MacNativeGames (`nvn_api_*.c`, `nvn_obj.h`, texture layout and deswizzle). The game records command buffers through it; nothing reaches the GPU until the game submits.
@@ -81,7 +79,7 @@ Other ports and what they changed about scheduling: [port-lessons.md](research/p
- Draw state. A draw whose program, bound state and textures repeat reuses the previous draw's shaders and skips rebuilding its descriptor sets; descriptor sets are written through update templates (`vk_exec_draw.c`, `vk_pipeline.c`).
- Allocation: Vulkan Memory Allocator. Debugging: Vulkan validation layers, per-pass frame dumps (`TOTK_DUMP`, `vk_dump.c`) and a flight recorder (`TOTK_FLIGHT`). RenderDoc cannot capture, because it hides `VK_EXT_external_memory_host` from the application.
Maxwell behaviour the back end has to match, and the TotK bugs other ports fixed: [maxwell.md](research/maxwell.md).
Maxwell behaviour the back end has to match, and the TotK bugs other ports fixed: [maxwell.md](maxwell.md).
Why Vulkan rather than D3D12 or wgpu:
@@ -97,16 +95,3 @@ Why Vulkan rather than D3D12 or wgpu:
- Analysis: Ghidra with the Switch loader, Capstone, `llvm-objdump`.
- Players: the installer ([installer.md](installer.md)) builds the game code on the player's machine with a bundled llvm-mingw, so no recompiled code is ever distributed.
## Milestones
| | Goal | Done when | Status |
|---|---|---|---|
| M0 | Toolchain and repository | Every tool in the toolchain docs passes its verify command | Done |
| M1 | Input | The user's dump extracts and matches `expected-v1.0.0.json` and `expected-v1.2.1.json`; sizes and function counts of `main`, `sdk`, `subsdk0` are recorded | Done |
| M2 | Lifter at scale | All three modules lift to C, compile and link into one exe; differential tests pass | Done |
| M3 | Host boots | The exe reaches the first NVN call with threads, memory and RomFS working | Done |
| M4 | Playable | Vulkan back end draws the title screen and the world; input and audio work | Done |
| M5 | Renderer foundation | Fewer stutters and visual glitches, a render thread with room for 60 fps | In progress |
| M6 | Release shape | Build-on-player-machine packaging, Linux, 60 fps without dips | Installer done; Linux open |
Open problems, in the order they are worked on: [backlog.md](backlog.md).
docs/backlog.md
-229
View File
@@ -1,229 +0,0 @@
# Backlog
The open work, in priority order: no artifacts and no crashes first, then 60 fps without dips, then other GPUs and
platforms. Each item states its evidence and the number or observation that decides it. Maxwell item numbers refer to
[research/maxwell.md](research/maxwell.md), lesson ids (S1, G3, ...) to [research/port-lessons.md](research/port-lessons.md).
Remove an item when it is merged or dropped.
Test machines: A, Ryzen 9 7900X + RX 9070 (development); B, laptop with a GTX 1050 Ti (4 GiB, no ReBAR) and an 8th- to
10th-generation Core i7; C, Core i7 of the same generations with Intel UHD 630. An RTX 3090 Ti was measured once. Only
machine A runs scripted routes; B and C send logs from the user's play.
## Bugs
1. **Yunobo vanishes.** He fades out of the main view about 45 frames after his ability is triggered (the ball fades
over 15 frames, then is gone); his shadow-map draws stay. Ruled out: occlusion counts, CPU readback of GPU targets,
missing NVN entry points, GPU-side resources. The game itself stops issuing his main-view draws. Leading hypothesis:
game logic fades the model alpha (material c11[33].w) and drops it from the draw queues on a CPU-side input
(camera-near fade, GfxColFade, CancelDraw distance or an AI fade). Next: a cheap check with the 60 fps and
real-time patches off; then find his c11 buffer, watch writes to c11[33].w and walk up from the writer in Ghidra.
Repro: the user's recorded session `yunobo-cast2` from save `user-save-2026-10-24-yunobo1`, 1360 frames.
2. **Black patches for a moment on machine B**, every time the Purah Pad opens (right stick click, keyboard C). Not
the VRAM mirror: a verify run that caught the glitch compared 4.45 million mirror pages with 0 mismatches. Leading
hypothesis: a new texture descriptor over a GPU-rendered texture fails `alias_compatible` and starts as a
zero-cleared image in its own memory (`vk_texture.c` `entry_of`, `create_tex`, `init_image`). The run's two largest
`left uninitialised` counts (33, 41) fall in its last seconds, where the glitch was. `cannot alias its pool memory`
fills its 16-line log limit on NVIDIA and on the RX 9070 alike. Next: per-alias diagnostic lines and a
`TOTK_TEX_FILL=magenta` knob, then a user run. Same class as yuzu PR 10418 ("areas black for a frame on camera
changes", Maxwell 2.3 / check 7).
3. **Phantom objects and the flat far field**: for one frame a rock or log stands in place of pillars, or a surface
fills the sky, most often while running or falling from a sky island. The game's producer rewrites its per-object
uniform blocks 14-24 ms after the submit that reads them, with no fence; NVN starts the GPU within microseconds,
our replay takes longer. A guard in `nvn_api_core.c` holds the producer until the newest submit has completed;
whether it removes every case is waiting for play. Any replay or frame-overlap change must keep that guarantee.
Scheduling experiments so far: port-lessons S1, S4.
4. **Lantern texture bug**: waiting for the user's description or a cropped screenshot.
5. **Fire on grass deals no damage and grass does not burn.** The CPU reads GPU-written data in GPU-only pools that
are never written back (Maxwell 4.1). Recheck first: GPU textures are now written back on pool invalidates and
fence waits (01889c7, which brought grass cutting back).
6. **Validation messages left** in world runs: `cp_t_tcb_20` samples an R32F view with depth compare in
`vkCmdDispatch` (VUID-vkCmdDispatch-None-06479; needs a D32 copy of the 4x4 R32F placeholder, Maxwell check 20);
"VkDescriptorSet 0x0 does not have a binding for vp_c1" in `vkCmdDrawIndexed`; a fragment write to Location 0
with no colour attachment.
7. **To check in play**: F11, menus, movies, rumble and gyro; the game's actions on the keyboard layout and the order
of the D-pad keys 1-4. Gyro: the user's logs show only Xbox pads, which have no gyro. With a DualSense or Pro
Controller, check Steam Input (it presents pads as a virtual Xbox pad without sensors) and the game's own
motion-aiming option.
## Frame time and freezes
8. **Replay on weaker CPUs.** With the VRAM mirror, machine B renders the GPU part in 10-11 ms, but the replay takes
25-30 ms per frame (5.2 ms on machine A), so 25-45 fps. A `sched` run rules out core contention: the render thread
spends 79-88% of the wall time on the CPU inside `nvnQueueSubmitCommands`, guest threads never preempt it and the
OS takes about 5%. Mirror bookkeeping explains 2-3 ms. Suspects: NVIDIA driver cost per command, mirror and texture
checks inside the replay. Since then the descriptor build skip, update templates and the shader memo cut the
field replay on machine A from 8.87 to 7.99 ms. Next: a render-thread profile from machine B
(`TOTK_PROFILE_RENDER=1`; the profile is written when the window closes).
9. **Replay thread** (`TOTK_REPLAY_ASYNC=1`, merged off by default). On machine A no fps gain: fights are bound by
the game threads, field render busy time drops from 10 to 1.3 ms. Where the replay is longer than the frame
(machine B) there is nothing to overlap: the game's main thread waits each frame for the previous frame's GPU
work. Next: a user run on machine B with `replay_async = 1`.
10. **Texture uploads in gameplay**, now the largest cause of long frames. Step 1 splits the cost (image and memory
creation, CPU deswizzle, staging, copies, locks) before choosing between a worker-thread upload, GPU deswizzle,
image pooling or a per-frame budget. Dropped after measuring: narrower guest-written marks for `WriteTexels` and
box-only image updates (re-uploads after `WriteTexels` are 0.003-0.05% of frame time).
11. **Asynchronous virtual-core preemption** (`TOTK_ASYNC_PREEMPT`, merged on by default; port-lessons S7). It cuts
the waits it targets (`TOTK_CORE_DIAG`: game-class waits of 1 ms or more 53 / 41 / 27 to 4 / 10 / 18 in field /
fight / a larger fight), but A/B timings show no mean-fps gain: fight frames over 25 ms 62 without, 96 with, over
four pairs (58.7 / 58.2 fps, inside run-to-run noise). One of three field runs without it froze 7 s with the render
thread blocked on nothing; no run with it did. Next: user play with it on, and `TOTK_ASYNC_PREEMPT=0` first on any
new hang or stutter. If fights stay worse, try a shorter first spin (`AP_FIRST_NS`, 250 us) or mode 2.
12. **Driver cache re-warm.** The AMD driver keys its pipeline library cache by exe path, so a moved folder stutters
until `totk.exe --prepare-shaders` runs again. Branch `task/rewarm` (not merged, never run in game) stamps
`shader-prepared.bin` with device, driver and exe path and re-warms at start on a mismatch. Check whether NVIDIA
and older AMD drivers cache the same way. Needed before machines B and C get an installer build. 2026-10-27: a
relink keeps the cache (`totk.exe --warm-shaders` on a relinked exe: 29,587 variants in 1.8 s), but its size is
limited: preparing a new build dir (28,408 cold libraries) left `build 143` cold, and a library a draw needed took
40-220 ms in user play instead of 0.2 ms. `--warm-shaders` re-warms without translating.
13. **Shared shader list.** Newer MacNativeGames lists collapse variants that differ only in texture kind into `"*"`
rows, which our runtime cannot match (it translates for the exact kind); `tools/prepare_shaders.py` reads `"*"` as
2D. Option: match `"*"` rows at run time, as MacNativeGames' `prepare_shader_seed.py` does.
14. **Freeze census**: a 10-15 minute user session with `TOTK_FLIGHT=1`, `TOTK_TRACE=perf` and `TOTK_INPUT_RECORD`,
ranking every frame over 50 ms by cause. Decides items 15-17.
15. `TOTK_PIPE_SKIP=fullscreen` as the default. On a cold larger fight: 0 frames over 100 ms, 1% low 33.9 to 41.7 fps.
Risk: one frame without a full-screen pass the first time its pipeline appears.
16. **ASTC decode budget.** The largest batch, the 169 M texel 8x8 array at boot, scales to about 375 ms of GPU time on
a GTX 1050 Ti. Split it by mip level with `minLod` if runs on machine B (`TOTK_ASTC_TIME=1`) show decodes in play.
17. **ASTC to BC7 re-encode on the GPU** for 4 GB cards: the same array is 677 MB as RGBA8 against 42 MB of blocks.
18. **Spin-wait gates** (MacNativeGames a038e6b, from nxnative 066f2af / 6d34984, MIT): loops polling memory hand the
core to a queued thread after 50 us. Survey TotK 1.2.1 for such loops first (port-lessons S3).
19. **Over the GPU memory budget**, wait for running work and drain deferred releases before failing (MacNativeGames
4de7b4e). Matters on 4 GB cards.
20. **GPU memory over long sessions** (port-lessons P3), never measured. MacNativeGames' TotK reached its 5000 MB GPU
budget after 2 h 43 min (their issue #72) and grew from 4.5 to 6.25 GiB resident; their dead-serial hash sets
reached about 1 GiB after 3 h. Our textures are destroyed when their pool range goes or a new texture takes the
address (`vk_texture.c` `entry_of`, `vk_texture_range_gone`), pools at Finalize, Vulkan objects deferred.
`mem: vram` (every 600 frames with `TOTK_TRACE=perf`, `vk_memory.c` `vram_log`) splits device-local memory. A 16,000
frame uncapped fight on A (`build/v121-vram/NOTES.md`): pools flat (import 1160 MiB host, physical 111 MiB), mirrors
440 MiB, live textures 1635 MiB, released textures up to 825 MiB (1686 entries); 3.25 GB in all and flat from frame
9000, no leak. The released entries are the grower: TextureFinalize keeps the image (`vk_texture_release`) for the
finalize-and-init-again cycle of transient textures and for finalized slots that still name the storage, and
`sweep_released` drops all of them at once only past 4096 entries, a count, not bytes (about 2 GB at this mix).
The GTX 1050 Ti of machine B (budget 3620 MiB) reaches 3300-3386 MiB within 1-2 min of play, and from then on up to
394 pools stay imported (`mem: mirror` "stay imported: budget"), so the GPU reads them over PCIe.
Next:
- Count the frames from release to reuse of each released entry, in `mem: vram`, on the fight and the field: where
transient textures end and the tail that can go begins.
- Drop released entries oldest first, one by one, never all at once, and only under pressure (device-local use past
about 85% of the budget, before a mirror is refused); unchanged on large cards. A fixed cap below the transient
working set would recreate images every frame and lose GPU-rendered content (flicker, empty tile maps).
- Check on B: mirrors refused for the budget and fps before and after; then a 2-3 h session with travel.
- 2-4 GB cards: ASTC kept as RGBA8 is 4x its size; BC1/BC3 recompression (yuzu, `maxwell.md` 2.13) is the larger
saving.
21. **Later**: recording on several threads; descriptor buffers or bindless (touches the translator); CPU/GPU frame
overlap as LAS Recomp 13398126d (only after item 9, and it must keep the guard of item 3); an x86-64-v2 build for
CPUs without AVX2/FMA (SavageWindRecomp ae42014b4); render passes ended by uniform updates and clear-only work
(port-lessons P2); faster save loading; asynchronous texture write-back; menus, chests and cutscenes at 30 fps under
the 60 fps patch, as SavageWindRecomp does (`runtime/hle/framepace.c`, [prior-art.md](research/prior-art.md)
section E); a trimmed pipeline key (SavageWindRecomp).
## Correctness queue
Worked in this order.
22. Shader census for the open graphics lessons (port-lessons G3, G6), offline with no game run: how many TotK shaders
fetch from buffer textures and how many clears use a partial mask.
23. Image aliasing over GPU-only pool memory (Maxwell 2.3 / check 7): count distinct texture keys per address per
frame; the flight recorder counts new aliases.
24. `VK_KHR_index_type_uint8` (Maxwell check 9): drops the CPU widening of 130-170 byte-index draws per frame.
25. Texture gather rounding on AMD (Maxwell 2.10): yuzu's sub-pixel offset at `ImageGather` fixed BotW shadow edges.
26. Horizon scheduling fidelity (port-lessons S1, S2, S3, S5, S6): wake a better-priority thread on the signaller's core
at SVC return; pin ideal-core -2 and main threads to core 0. Count each case with `TOTK_TRACE=svc,sched` first;
fight fps must not drop.
27. Line width 4.5 (Maxwell check 14): dynamic `wideLines`.
28. `vk_memory.c` `defer()` (unverified): an object destroyed while frame F is recorded is stamped with F-1 and may be
freed before F's commands that use it are submitted.
29. Masked clears (Maxwell 1.3 / check 3): a fix was written and parked; merge one if the census (item 22) finds a
partial-mask clear.
30. GPU timestamps (port-lessons T1): find whether dynamic resolution reads the report difference before writing real
GPU times per report.
31. MacNativeGames since our pin `b75be8c` (245 commits): kernel sync `dab728a` (S6), NVN texture sizes `3c482a7`,
CPU reads of small render targets `8a29e0a` (item 5), border colour `29dd997`. Add the lessons, move the pin and fix
the `refs/MacNativeGames` line citations in `docs/`.
32. Maxwell sources not read yet (maxwell.md section 8).
## Release and platforms
Each platform item gets a written design first, agreed with the maintainer.
33. **Release**: fix what stops play on the test machines, then publish the repository and a first installer release
([installer.md](installer.md)). Runs on more GPUs (NVIDIA, Vega, weaker CPUs, [tester-guide.md](tester-guide.md))
go into one table: GPU, driver, CPU, game version, fps in field and fight, hitches, errors and crashes, with a log
per machine.
34. **Platform layer**: no Win32 outside `runtime/platform/`. `tools/check_platform.py` counts 1479 uses in 41 files
and fails on `astc_probe.c`, `gfx_abi_probe.c`, `input.h`, `motion_fusion.c`, `movie_probe.c`, `ov_movie.c`,
`swkbd_win.c`, `vk_astc.c` and `vk_exec_draw.c`.
35. **Linux**: `runtime/platform/linux.c`, split as SavageWindRecomp splits its windows, linux and macos files. Write
tracking (`host.h` `mem_written_pages`, `mem_freeze` / `mem_thaw` / `mem_frozen`, used by the VRAM mirror and the
texture checks): freezing maps to `mprotect` plus a `SIGSEGV` handler; a kernel write (`read()` into guest memory)
returns `EFAULT` instead of faulting, so the same thaw-before-read sites apply. Written pages: the `PAGEMAP_SCAN`
ioctl with write-protect matching (kernel 6.7+, made for Wine's `GetWriteWatch`); on older kernels, write-protect
the mirrored and texture-bound pages after each sync and mark them in the fault handler (Ryujinx), uploading pools
the game rewrites every frame whole instead of tracking them. Soft-dirty (`/proc/self/clear_refs`) is process-wide
and slow, a last resort. Under Proton the Windows build already works: Wine emulates both.
Frame time fixes of 2026-10 (merge 2b1e295) and what each needs on Linux:
- Short SleepThread spins (`kernel.c` `sleep_ns`, `SLEEP_SPIN_NS`): Windows-only cause, the high-resolution
waitable timer wakes 0.3-0.5 ms late. `clock_nanosleep` wakes within the timer slack (50 us default,
`prctl(PR_SET_TIMERSLACK)` lowers it): spin below that or not at all, and measure with `TOTK_TRACE=sched`
("sleep asked / took").
- Background save Commit (`fs_host.c`), pipeline cache autosave thread (`vk_pipeline.c`), shader record writes
outside the lookup lock (`gfx_shader.c`), `.use` masks read at start: portable as written; they keep file I/O
and flushes off the game and render threads, which any OS needs. On Linux the Commit's FlushFileBuffers become
`fsync` of each file and directory, and the renames `rename()` plus an `fsync` of the parent.
- Recording copy index by 64 KiB block (`nvn_api_cmd.c`), world clock input (`frame_rate.c`, `input_win.c`):
portable logic, no OS calls.
- Slow-present profiler (`prof_win.c`, `TOTK_PROFILE_SLOW`): Windows-only (SuspendThread, GetThreadContext,
QueryThreadCycleTime, RtlVirtualUnwind). On Linux use `perf record -g` or a `SIGPROF` sampler; `tstall` has no
direct equivalent.
- VMA block allocation on the render thread (`vk_texture.c` `create_tex`, 8-11 ms per 256 MiB block on AMD
Windows): measure again on RADV before working around it.
36. **Steam Deck**: RADV offers Vulkan 1.3 and `VK_EXT_external_memory_host`; 4 Zen 2 cores, so expect 30-40 fps.
37. **Launcher**: one exe for Windows and Linux, SDL3 and Dear ImGui (SavageWindRecomp `runtime/ui/launcher.mm`), with
Play, Settings, Game files and Quit. Settings that need a restart go there, and it is the recovery path when a
setting stops the boot. The installer already covers checking, extracting, building and preparing shaders.
## Later
38. **In-game menu**. F1, Esc or Minus+Plus opens it; the game gets no input while it is open. Tabs Screen, Game,
Sound, Controls, Menu and Advanced show option rows over the `cfg` table (`runtime/cfg.c`), with the description of
the selected row beside them, English and Russian text from `runtime/ui_text.c`. Every change goes to `totk.ini`,
keeping the user's comments; a key set in the environment shows greyed out, a key read only at start says "after
restart". Advanced holds every remaining key by group, open the log folder, copy system information
(`runtime/sysinfo.c`) and reset of all settings behind a confirmation. `TOTK_PERF_OVERLAY` (F10 cycles): `fps` one
line, `full` a panel of fixed width from `runtime/perfmon.c` (CPU, GPU and memory load through PDH, temperature,
power and clock through NVML on NVIDIA only, frame times with a graph, replay, GPU time and the render thread's
waits). Dear ImGui is drawn after the blit into the swapchain image (`runtime/ui/menu.cpp`). On machine A the
swapchain gets fifo for the default mailbox in 30 of 31 logs; the Presentation row says so.
Left to do:
1. Not yet checked by hand: keyboard and gamepad navigation (the long Advanced page, the reset confirmation), Copy
system info and Open log folder, Quit, input right after closing, `totk.ini` locked by another program.
2. AMD GPU temperature, power and clock (ADLX); the overlay shows "-" for them on machine A.
3. Later pages: button remapping (LAS Recomp `runtime/ui/controls_ui.mm`), AMD FSR 1 or bilinear scaling of the
900p image, gyro sensitivity.
Rendering above the game's 900p is a separate study (a game-side resolution patch, as Switch mods do, or scaling
every render target ourselves); MacNativeGames has neither. The software keyboard could move onto
SavageWindRecomp's `osk.mm`.
39. **Game version 1.4.3**. Builds, boots and plays: the user played it from their own saves on 2026-10-26 without
problems; the field save runs at 60 fps with game speed 1.000, REV15 audio without underruns, 0 unimplemented NVN
calls, 0 validation errors; every probe passes on 1.4.3 and on 1.2.1; the jump-table check reports 0 disagreements
on both; the installer takes either update. Saves carry over both ways. `prepare_shaders.py` seeds every shader's
default variant, as MacNativeGames' seed does, and reads the list with the V rows its history dropped
(`tools/shader_list.py`; the current list alone has 134 rows of 1.4.3 shaders): 29,587 variants, among them all
1,095 that the field and intro runs met in play. Open:
1. MacNativeGames types vertex inputs and colour outputs by the bound format, so one variant serves any format;
ours still differ by those types (347 of 19,468 variants met on 1.2.1). No frame-time gain, only fewer
translations on first sight.
2. The installer end-to-end test (`tests/installer_e2e.py`) runs only on 1.2.1.
3. MacNativeGames' 1.4.3 output, render and fixed resolution patches (`game_patches_v143.c`) are not ported; 1.2.1
has none of them either.
4. The opening cutscene (Zelda under Hyrule Castle) shows hard light bands on 1.4.3 and on 1.2.1 alike; whether
that is the game's look or ours is not checked.
40. **Community ExeFS patches** (`.ips`, `.pchtxt`, each for one game version): apply them through the generator's
patch table (`recomp/patches.py`). RomFS mods already load (`romfs.c`, `TOTK_ROMFS_MODS`). Code-module mods
(exlaunch `subsdk9`) need porting one by one.
41. **Frame rates above 60 and VRR.**
42. **Map tour**: teleport Link across the map by writing position, time of day and weather in guest memory, to fill
the caches on a first run and to serve as an automatic test (crashes, validation, fps per region).
docs/building.md
+155
View File
@@ -0,0 +1,155 @@
# Building from source
Windows 10 or later, an x86-64 CPU with AVX2, and a GPU with Vulkan 1.3 and `VK_EXT_external_memory_host`. Allow about
60 GB for the work folder. Extraction wants about 55 GB free while it runs and keeps about 35 GB afterwards.
Players who only want to play can use the [installer](installer.md) instead: it builds the game on their machine
without this toolchain.
## Toolchain
Versions are the ones the build was verified with; newer patch releases are fine.
| Tool | Version | Install | Used for |
|---|---|---|---|
| MSYS2 UCRT64 | clang/lld 23.1.3 | msys2.org installer to `C:\msys64`, packages below | compiling the generated C and the runtime |
| Visual Studio Build Tools | 2026, C++ workload, Windows SDK 10.0.26100 | winget `Microsoft.VisualStudio.BuildTools` with `--add Microsoft.VisualStudio.Workload.VCTools --includeRecommended` | the linker .NET NativeAOT calls |
| .NET SDK | 10.0 | winget `Microsoft.DotNet.SDK.10` | the shader translator (`totk_xlate.exe`) |
| Vulkan SDK | 1.4.363 | winget `KhronosGroup.VulkanSDK` | validation layers, `vulkaninfo` |
| uv | 0.13 | standalone installer from astral.sh | the two Python environments |
| hactool | 1.4.0 @ `1d64a83` | built from source, see below | NCA and update extraction |
MSYS2 packages, after `pacman -Syuu` (run it twice; the first pass updates the core and exits):
```sh
pacman -S --needed --noconfirm mingw-w64-ucrt-x86_64-{clang,lld,ninja,gcc,make,pkgconf,sdl3,opus,libvpx,vulkan-devel,\
vulkan-memory-allocator,vulkan-validation-layers,shaderc,spirv-tools} base-devel git unzip
```
`ffmpeg` from MSYS2 is a GPLv3 build and is never linked; the movies go through libvpx.
PATH:
- Add `C:\Program Files (x86)\Microsoft Visual Studio\Installer` to the user PATH: NativeAOT needs `vswhere.exe`, or
the publish fails at the link step (MSB3073).
- Another `cmake` or `ninja` earlier on PATH (a 32-bit winlibs MinGW, for example) is picked up by accident. The build
scripts call Ninja from `C:\msys64\ucrt64\bin` themselves; from a shell, put `C:\msys64\ucrt64\bin` first.
### hactool
`tools/extract.py` runs `toolchain\hactool\hactool.exe`. Build it from SciresM/hactool at `1d64a83` in an MSYS2 UCRT64
shell (`config.mk`: `CC=gcc EXEEXT=.exe`, static link, `make -j8`) with two changes:
1. The portable part of MacNativeGames' `build_app.py` `build_hactool()`: the `MNG_TITLE_KEYS` variable, and the two
BKTR fixes in `nca.c` for the subsection table padding and the subsection layout of newer cumulative updates.
2. A manifest with `<activeCodePage>UTF-8</activeCodePage>` embedded through a `.rc` file, so paths under a non-ASCII
user profile reach it intact.
hactool reads `-k <prod.keys>`; MSYS2 shells set `HOME`, which hides its default key location, so pass `-k` there.
## Build
From the repository root, in PowerShell.
1. Python environments (once). `.venv` runs the lifter and the build; `.venv-py314` runs the extraction scripts, which
need the stdlib `compression.zstd` of Python 3.14:
```powershell
uv venv --python 3.12 .venv
uv pip install --python .venv/Scripts/python.exe -r tools/requirements.txt
uv venv --python 3.14 .venv-py314
uv pip install --python .venv-py314/Scripts/python.exe -r tools/requirements-py314.txt
```
2. Copy `config.example.toml` to `config.local.toml` and set the paths to your base game, the 1.2.1 update and
`prod.keys`. `config.local.toml` is ignored by git. The tools read the keys file themselves.
3. Extract and verify. This writes `build\extracted\v0` (1.0.0) and `build\extracted\v121` (1.2.1); a hash mismatch
stops the run:
```powershell
.venv-py314\Scripts\python tools\extract.py
```
4. Shader translator (once; no game files). This publishes `build\xlate\totk_xlate.exe`:
```powershell
.venv\Scripts\python tools\build_xlate.py
```
5. Lift the modules to C, write the RomFS half of the frame-rate patches, then compile and link `build\v121\totk.exe`:
```powershell
.venv\Scripts\python recomp\gen.py --exefs build\extracted\v121\exefs --out generated\v121 --patches recomp\patches-v121.txt
.venv-py314\Scripts\python tools\make_romfs_patches.py
.venv\Scripts\python tools\build_game.py
```
Lifting takes about 15 s, a clean compile about 200 s on 24 threads; later builds recompile only what changed.
`--patches` builds in the 60 fps and real-time frame step patches, switchable at run time. The link copies
`SDL3.dll`, `libiconv-2.dll` and `libopus-0.dll` next to the exe.
6. Prepare the shaders (once per build folder and translator). It needs `refs\MacNativeGames` (`tools\fetch_refs.sh`)
for the shared shader list:
```powershell
.venv-py314\Scripts\python tools\prepare_shaders.py build\v121
```
Without this step the first launch compiles shaders and pipelines during play and stutters until the caches fill.
The AMD driver keeps its pipeline cache per exe path and holds a limited amount: preparing another build folder can
evict this one's. `totk.exe --warm-shaders` rebuilds it from the shader cache.
7. Run. Keep the window in the foreground; an unfocused window stalls the game.
```powershell
build\v121\totk.exe
```
Settings live in `totk.ini` next to `totk.exe` ([totk.ini.example](../totk.ini.example) lists them). Saves go to
`build\profile`.
### Portable folder
`.venv\Scripts\python tools\package_build.py` writes `build\v121-portable`, which runs on another of your machines
without MSYS2, Python or .NET. `romfs.bin` stays out unless you pass `--with-romfs`; otherwise put it in `romfs\` next to
the exe or point `TOTK_ROMFS` at its folder. Run `totk.exe --prepare-shaders` once in the copied folder. The folder
holds code and data derived from your game: for your own machines only.
### Game version 1.4.3
Add the 1.4.3 update to `updates` in `config.local.toml`; `extract.py` writes `build\extracted\v143`. Then the same
commands with `v143`: `gen.py --exefs build\extracted\v143\exefs --out generated\v143 --patches recomp\patches-v143.txt`,
`make_romfs_patches.py --version v143` and `build_game.py --version v143`.
## Checks
Before a commit:
- `python tools/check_content.py --staged`: nothing game-derived is staged. CI runs the same check;
`git config core.hooksPath .githooks` runs it before each local commit.
- `build\v121\totk.exe --selftest`: layout, dispatch table and host-function registry, without game files.
The lifter (`recomp/`) and the CPU headers (`runtime/cpu.h`, `runtime/simd.h`) are checked against Unicorn. Every lifter
change passes these, all run with `.venv`:
| Test | Checks | Game files |
|---|---|---|
| `tests\diff\lift.py` | instructions sampled from the extracted modules; `--coverage` lists every word the lifter rejects or traps | yes |
| `tests\diff\simd.py` | the whole FP/SIMD encoding space | no |
| `tests\diff\atomic.py` | exclusive load/store sequences and ARMv8.1 atomics | no |
| `tests\diff\func.py` | whole leaf functions as the generator emits them, loops and jump tables included | yes |
| `tests\excl_mt.py` | the exclusive monitor helpers under contention and ABA, on several threads | no |
| `tests\jumptables_check.py` | `recomp/jumptables.py` against a Unicorn oracle; must report 0 disagreements, on 1.4.3 too | yes |
Runtime probes, each a mode of `totk.exe` (the list is at the top of `runtime/main.c`): `--probe-kernel`,
`--probe-vkexec`, `--probe-nvn-tex` and `--probe-shader` cover most runtime changes.
Shaders: `.venv-py314\Scripts\python tools\shader_census.py --version v121 --translate` extracts every Maxwell shader
to `build\v121\shaders\` and runs it through the translator and `spirv-val`; failure groups go to `failures.txt`.
`tools\xlate_smoke.py` checks the translator alone, without game files.
## Reverse engineering (optional)
- Ghidra 12.1.2 with Adubbz's Ghidra-Switch-Loader in `toolchain\ghidra\` (JDK 21) opens the game modules (NSO).
- `tools\fetch_refs.sh` clones the upstream projects this port builds on into `refs\`, read-only.
docs/controls.md
+6 -2
View File
@@ -51,14 +51,18 @@ The stick direction bits (`StickLLeft` to `StickRDown`) are derived from the sti
## Scripted input
`TOTK_INPUT_SCRIPT=<file>` replays presses by presented-frame number (`gfx_counters.frames`). It is ORed with real input and still applies under `TOTK_INPUT=none`. One event per line, `#` starts a comment:
`TOTK_INPUT_SCRIPT=<file>` replays presses. It is ORed with real input and still applies under `TOTK_INPUT=none`. One event per line, `#` starts a comment:
```
<frame> <button> [hold_frames] # A B X Y L R ZL ZR PLUS MINUS LEFT UP RIGHT DOWN LCLICK RCLICK; hold defaults to 4
<frame> LSTICK|RSTICK <x> <y> [hold_frames] # -32767..32767, up is +y
@<ms> <button> [hold_ms] # timed on the world clock
@<ms> LSTICK|RSTICK <x> <y> [hold_ms]
```
Each event logs once (`input: script frame N: A`). `tools/input/load_save.txt` loads the first save from the title screen.
`<frame>` counts presented frames. `@<ms>` counts milliseconds of the world clock: game time stepped since the end of the first save load (the player leaves the title screen's origin), so a timed event lands at the same game time at any frame rate and load length. A later load, such as fast travel, still shifts what follows. Each event logs once (`input: script frame N: A`). `tools/input/load_save.txt` loads the first save from the title screen.
`TOTK_INPUT_RECORD=<file>` (`input_record` in `totk.ini`, or Input recording in the menu) writes the merged input in the same format, one line per button press and per stick segment: events before the world clock starts by frame, the rest timed. The file replays as a script, which makes it the repro to attach to a bug report.
## RenderDoc capture (blocked)
docs/installer.md
+2 -2
View File
@@ -112,7 +112,7 @@ totk.ini.example, VERSION the default ini; the release string (git describe of
...`), which prints one JSON progress event per line. Tests and CI use the CLI.
- Paths may hold any Unicode characters (a Cyrillic user name puts the temp and install folders under it). Every exe
the setup runs or builds reads its arguments and paths as UTF-8 through an `activeCodePage` manifest: hactool
(our patch, [toolchain-msys2-python.md](toolchain-msys2-python.md)), totk.exe (`runtime/totk.rc`), ninja (upstream). The Python steps write
(our patch, [building.md](building.md)), totk.exe (`runtime/totk.rc`), ninja (upstream). The Python steps write
their files and logs as UTF-8, and `build_game.py` quotes paths with spaces on its Ninja command lines.
- A game file whose data fails its hash checks or ends early stops the setup with `check.input_damaged`, which names
the file and asks to copy it again (`extract.DamagedFile`). A failed merge checks the base Program NCA alone to tell
@@ -121,5 +121,5 @@ totk.ini.example, VERSION the default ini; the release string (git describe of
## Later
- Re-warm the driver cache on start after a move, GPU or driver change ([backlog.md](backlog.md) item 12).
- Re-warm the driver cache on start after a move, GPU or driver change.
- NSZ and XCZ input. Linux release.
docs/research/maxwell.md → docs/maxwell.md
+2 -2
View File
@@ -211,13 +211,13 @@ Grep-level look on 2026-10-22, statuses updated 2026-10-26; "saw" is what the gr
21. **Report timestamps: unit and instant.** `write_counter` (`vk_exec_xfer.c`) writes host `ns * 12 / 625` (19.2 MHz
ticks) at fence completion for every report; `nvnDeviceGetTimestampInNanoseconds` converts back with the same factor
(`nvn_ext.c`). DK3D: GPU ticks convert at 625/384 ns (614.4 MHz), TIMESTAMP is written when the ROP has retired
the earlier work, TIMESTAMP_TOP when the command processor reaches it (4.2). Two consequences for port-lessons T1:
the earlier work, TIMESTAMP_TOP when the command processor reaches it (4.2). Two consequences for the engine's GPU time (dynamic resolution and speed follow it):
(a) if the 1.2.1 consumer scales raw report ticks with its own constant rather than the NVN conversion call, our
19.2 MHz unit is 32x off from the hardware's; find that consumer before fixing the unit; (b) the delta needs
`vkCmdWriteTimestamp2` with `TOP_OF_PIPE` for TIMESTAMP_TOP and `BOTTOM_OF_PIPE` for TIMESTAMP, per submission
segment, converted to whichever unit (a) shows. **Mismatch (instant); unit unverified.**
## 8. Sources not read yet (TODO)
## 8. Sources not read yet
- Mesa NVK (`src/nouveau/vulkan`, MIT): how it maps Vulkan provoking vertex, depth clip, restart, feedback loops onto B197.
- envytools rnndb `graph/gf100_3d.xml`, `tsc.xml`, `tic.xml` (MIT): field-level TIC/TSC meanings (LOD snap, trilinear opt).
docs/research/lasrecomp.md
-447
View File
@@ -1,447 +0,0 @@
# LASRecomp study (reference for the TotK port)
Source: `refs/LASRecomp`, commit `da35b83` (2026-10-19, single squashed commit), version `beta_4`
(`runtime/ui/launcher.mm:88`). Paths below are relative to `refs/LASRecomp`.
TL;DR: a clean, small static recompiler (~2.8k lines of Python), a runtime that recompiles on the player's
machine, and an NVN-at-API-level HLE renderer for Vulkan. The CPU pipeline and the Windows platform layer can be
reused with targeted fixes. The kernel, services, NVN HLE, Maxwell shader translator and Vulkan backend cover only
what Link's Awakening calls, so for TotK they need large extensions or replacement.
## 0. Size
| Area | Lines | Reuse for TotK |
|---|---|---|
| Recompiler (`tools/gen.py lifter.py simd_c.py nso.py analyze.py sym.py`) | 2,779 | yes, with fixes |
| Recompiler tests (`tools/test_lift.py test_simd.py` + harnesses) | 865 | yes |
| Runtime core (cpu/dispatch/kernel/ipc/loader/simd/platform/main/paths/settings/romfs/gamelib/gamebuild) | 6,338 | mostly |
| Services (`runtime/services/*.c`) | 2,452 | partly |
| Generic HLE (input, bindings, movie, nvn_boot, pl, swkbd, vi, heaps, framepace) | 2,024 | partly |
| NVN core (`runtime/nvn/objects.c cmdbuf.c tegra.c formats.c astc.c output_scale.c` + headers) | 2,319 | base only |
| Maxwell translator (`runtime/nvn/maxwell/`) | 2,268 | base only |
| Vulkan backend (`runtime/nvn/vulkan/`) | 6,019 | base only |
| Metal backend (`runtime/nvn/*.m`) | 3,892 | no (Windows target) |
| LA-specific: `tools/patches.py`, `warp_table.py` | 958 | no |
| LA-specific HLE (`runtime/hle/{chamber,digs,doorjump,escape,flags_ext,hookshot,keysanity,miniboss,mirror,pets,pot_glow,qol,rescue,shopsanity,souls,warp,widescreen}.c`, `runtime/rando.c`) | 4,593 | no |
| `runtime/nvn/texpatch.c` (LA button-prompt texture edits) | 1,043 | no |
| UI (`runtime/ui/*`, launcher + randomizer UI + overlays) | 6,479 | launcher concept only |
| Randomizer tools (`tools/rando/*.py`) | 8,489 | no |
## 1. CPU pipeline
### Loading
- `tools/nso.py:12-41`: NSO0 header, 3 segments LZ4-decompressed (`lz4.block`), flat image, MOD0 → dynamic section,
dynsym via `DT_HASH` nchain, RELA + JMPREL relocations.
- Runtime loader `runtime/loader.c`: its own LZ4 decoder (`:17-38`), loads into the guest window at fixed bases,
applies `R_RELATIVE/ABS64/GLOB_DAT/JUMP_SLOT` (`:143-173`). Imports resolve in this order: host HLE implementation
by symbol name, then a guest export, then an "unimplemented" stub at a fake address (`:131-141`).
- Only two modules: `main` @ `0x7000000000`, `sdk` @ `0x7004000000` (`runtime/loader.c:177`, `runtime/rt.h:73-74`,
also hard-coded in `Makefile:191-192`, `runtime/gamebuild.c:304`, `tools/sym.py:15`). `rtld` and `subsdk*` are
ignored. rtld's job (module init order, `nn::init::Start` arguments) is done by the host (`runtime/main.c:16-36`).
**The main image (text, rodata, data and bss) must fit in 64 MB**, or `SDK_BASE` has to move.
### Decoding
- **Capstone is not used by the lifter.** `tools/lifter.py` is a hand-written AArch64 decoder (op0 split at
`:180-194`). Capstone is used only in `analyze.py` (instruction census), `sym.py` (disassembly for debugging) and the
tests.
- Unsupported encodings raise `Unsupported` and become `rt_udf(c, pc)` traps in the output (`tools/gen.py:60-69`), so a
build never fails on an unknown instruction; it crashes at run time if the instruction executes.
- Gaps that matter for TotK:
- Exclusive pair `LDXP/STXP/LDAXP/STLXP` (`o1=1`) raises Unsupported (`lifter.py:391-392`). 128-bit lock-free code
(tagged pointers) will trap.
- Register-offset atomics (LSE) are not supported (`lifter.py:445-446`). That is fine: the Switch's Cortex-A57 is
ARMv8.0.
- `simd_c.py:1-6`: ARMv8.0 + Crypto + CRC32 only. FP16 arithmetic, RDM, dot product, FCMA and JSCVT are rejected
(not present on A57).
### Function, CFG and jump-table discovery (`tools/gen.py`)
- Seeds (`boundaries`, `:71-83`): `.eh_frame_hdr` table (`analyze.py:28-46`), exported FUNC symbols, and targets of
`R_AARCH64_RELATIVE` relocations that land in `.text` (vtables, callbacks).
- More seeds (`:519-546`): every `BL` target, `ADR` targets, and `ADRP`+`ADD` pairs within 8 instructions.
- Fixpoint (`:547-561`): a function is the contiguous range `[start, next_start)`. `scan()` (`:298-328`) collects
branches. A branch target outside the range becomes a new start. Repeat until no new starts appear. Results are
cached per `(start, end)`.
- Jump tables (`:98-277`): **Unicorn** single-steps the code between the bounds check (`cmp #N; b.hi/b.ls`) and the
`br`, once for each index `0..N-1` (at most 4096). It first replays the `ADRP/ADR/ADD` instructions from the
function entry so the table base is known.
- Unresolved computed `br`: `tables[a]='all'` (`:319-322`).
- Any function with a table gets a label on every instruction and an `L_dispatch` switch over all of its instruction
addresses (`:378-382`, `:459-463`). This is the fallback when emulation missed cases.
- Indirect `br` from a load (tail call through a pointer) → `rt_dispatch`.
### Emitted C
One C function per guest function, `void f_<addr>(Cpu *c)`. Templates are in `gen.py:355-466` and `lifter.py`.
Representative shape, assembled from those templates:
```c
#include "cpu.h"
#include "simd.h" /* --simd=c (x86 hosts) */
void f_7000123460(Cpu *c);
void f_7000123400(Cpu *c) {
FN_ENTER(0x7000123400ull); /* profiler hook, one predictable branch */
{ uint64_t ea = c->sp - 32; ST64(ea, c->x[29]); ST64(ea + 8, c->x[30]); c->sp = ea; } /* stp pre-index */
c->x[29] = (uint64_t)(c->sp + 0x0ull); /* mov x29, sp */
(void)(subs64(c, c->x[0], 0x10ull)); /* cmp x0, #16 (flags into c->nzcv) */
if (COND_HI(c)) { goto L_7000123440; }
c->x[30] = 0x7000123418ull; f_7000123460(c); /* bl: direct host call */
c->pc = c->x[8]; c->x[30] = 0x7000123420ull; rt_dispatch(c); /* blr x8 */
SV_FOP_S(FOP_ADD, 1, 0, VREG(vf32, 1), VREG(vf32, 2), (vf32){}); /* fadd v0.4s, v1.4s, v2.4s (simd_c.py:437-469) */
switch (c->x[9]) { case 0x7000123430ull: goto L_7000123430; /*...*/ default: break; }
c->pc = c->x[9]; goto L_dispatch; /* br x9 through a resolved jump table */
L_7000123440:;
{ uint64_t ea = c->sp; uint64_t a = LD64(ea), b = LD64(ea + 8); c->sp = ea + 32;
c->x[29] = (uint64_t)(a); c->x[30] = (uint64_t)(b); } /* ldp post-index */
return; /* ret */
L_dispatch:
switch (c->pc) { /* every instruction address of this function */ default: break; }
__attribute__((musttail)) return rt_dispatch(c);
}
```
- `b` to another function: `__attribute__((musttail)) return f_X(c);` (`gen.py:370-376`).
- `ret` is a plain C `return;`. The model assumes strict call/return discipline: `x30` is never checked. Guest
setjmp/longjmp, fibers, or code that changes LR before `ret` would break. LA needs none of these (no handling exists
in the runtime).
- Patch hooks:
- `PATCHES[addr]` replaces single instructions (`gen.py:408-412`, `tools/patches.py`).
- `RANDO_CODE` emits seed-selected instruction variants (`gen.py:397-407`).
- Output is chunked at ≤1500 functions or ~1.5 MB of C per file. A function with >100 KB of C gets its own file.
Unchanged chunks are not rewritten, so `make` stays incremental (`gen.py:480-490`, `:499-501`, `:577-608`).
- `<mod>_table.c` holds sorted `fn_addrs[]` and `fn_ptrs[]` (`gen.py:613-625`).
### Indirect calls and branches (`runtime/dispatch.c`)
- `rt_dispatch` (`:160-164`):
1. Per-thread direct-mapped cache, 4096 entries keyed by `pc>>2`.
2. On a miss, `dispatch_slow` (`:134-157`): HLE fake-address range, then a linear scan of overrides, then a binary
search of the two sorted tables (`:115-128`).
3. Always `musttail`, so chains of indirect jumps don't grow the host stack.
- HLE functions live at fake addresses `HLE_BASE + 16*i`, at most 4096 of them (`:14-57`).
- Unknown address: `FATAL`.
- `main` → `sdk` calls go through PLT/GOT slots (relocated function pointers), so every SDK call from the game is an
indirect dispatch.
### FP/SIMD
- On arm64 hosts: raw `.inst` passthrough in inline asm, V regs loaded from and stored to `Cpu` around each batch of up
to 24 instructions (`lifter.py:680-728`).
- On x86 (Windows): `--simd=c` (`gen.py:503-504`). `simd_c.py` decodes each encoding at lift time into calls to
`always_inline` helpers in `runtime/simd.h` and `simd_fp.h` with constant fields, so clang folds the switches.
- Semantics follow the ARM pseudocode (`simd.h:1-8`): NaN propagation order, FPCR.DN/FZ/RMode, saturating FP→int,
fused FMA, ARM estimate tables. Not modelled: FPSR cumulative bits and QC, trap enables.
- Requires x86-64-v3 (AVX2, FMA, BMI2). Checked at startup in `runtime/platform/windows.c:143-149`.
- FPCR/FPSR live in `Cpu` (`cpu.h:17-18`). All V regs live in `c->v[32]` in memory (`cpu.h:20`).
- Tested by differential fuzzing against Unicorn. `tools/test_simd.py` enumerates the whole FP/SIMD encoding space;
`tools/test_lift.py` covers the integer path one instruction at a time.
### Exclusive monitor and atomics (`runtime/cpu.h:73-97`)
- `ldxN` records `(addr, value)` in `Cpu`. `stxN` does a CAS against the recorded value: ABA-tolerant, standard
practice.
- `LDAR/STLR` → `__atomic` acquire/release.
- `CLREX` → `excl_addr = ~0` (`lifter.py:325-326`).
- `DMB/DSB` → `__atomic_thread_fence(SEQ_CST)` (`lifter.py:327`).
- `DC ZVA` → 64-byte `memset` (`:330-331`). Other cache maintenance is a no-op.
### System registers, TLS, timing
- `TPIDRRO_EL0` = `c->tpidrro`, a per-thread 0x200-byte TLR in `TLS_BASE` (`runtime/kernel.c:201-221`).
- `TPIDR_EL0` is user-writable (`lifter.py:337, 351`). musl TLS is handled by the guest SDK itself.
- `CNTFRQ` = 19.2 MHz. `CNTVCT/CNTPCT` → `rt_ticks()`.
- `DCZID` = 4. `CTR_EL0` = `0x8444c004` (`lifter.py:335-346`).
### Memory model (`runtime/rt.h:68-84`, `runtime/platform/windows.c:13-82`)
- **Identity mapping: guest address == host address.** `LD64(a)` is `*(u64u*)(uintptr_t)a` (`cpu.h:53-71`). There is
no base register and no offset.
- Window `0x70_0000_0000 .. 0x73_2000_0000` (12.5 GB):
- modules
- `HLE_BASE 0x7080000000`
- `TLS_BASE 0x7090000000`
- `HLE_HEAP 0x7092000000` (224 MB)
- `ALIAS 0x7100000000` (4 GB)
- `STACK 0x7200000000` (1 GB, `kernel.h:7-8`)
- `HEAP 0x7240000000` (3.25 GB max)
- Windows: the window is reserved by `VirtualAlloc(MEM_RESERVE | MEM_WRITE_WATCH)` in a priority-101 constructor
(`runtime/main.c:37-52`). If the range is taken, the process re-executes itself, up to 20 times.
- Regions are committed on map, and a vectored exception handler lazily commits 64 KB on the first stray access
(`windows.c:21-38`).
- `MEM_WRITE_WATCH` feeds the Vulkan vertex/index cache (`windows.c:40-63`).
- `svcMapMemory` copies the data instead of aliasing it (`kernel.c:580-589`).
- No page protection emulation, no MMIO.
### Stack and threads
- Guest stack: in guest memory (`c->sp`).
- Host stack: every guest `BL` is a host C call, so guest recursion runs on the host stack. Each guest thread gets a
256 MB host stack (`kernel.c:226`), reserved rather than committed on Windows (README credit #107).
- One guest thread = one pthread (winpthreads on Windows), started by `thread_main` (`kernel.c:187-199`).
- Priorities are stored, core masks are ignored (`kernel.c:642`). There is no scheduler.
- One global `g_klock` guards all kernel state. Each thread has its own cond var.
- In-place reboot: `longjmp` to `thread_main` at safe points (`kernel.c:173-185`).
## 2. Runtime / HLE
### Kernel (`runtime/kernel.c`, 868 lines)
| SVC | Behaviour |
|---|---|
| 0x01 | SetHeapSize |
| 0x02, 0x03 | No-op OK |
| 0x04, 0x05 | Map/Unmap by copy |
| 0x06 | QueryMemory |
| 0x07 | ExitProcess |
| 0x08–0x10 | Threads |
| 0x11–0x17 | Events, shared memory, transfer memory |
| 0x18, 0x19 | WaitSynchronization, Cancel |
| 0x1A–0x1D | ArbitrateLock/Unlock, CondVar |
| 0x1E | GetSystemTick |
| 0x1F | ConnectToNamedPort (only `sm:`) |
| 0x21, 0x22 | SendSyncRequest (with user buffer) |
| 0x24–0x27 | |
| 0x29 | GetInfo (fixed values) |
| 0x2C, 0x2D | Map/UnmapPhysicalMemory |
| 0x34, 0x35 | WaitForAddress, SignalToAddress |
| 0x45 | CreateEvent |
| anything else | `FATAL` (`:785`) |
- Handle table: 4096 handles (`kernel.c:30`).
### IPC (`runtime/ipc.c`, 246 lines, plus `ipc.h`)
- HIPC/CMIF parsing and serialization.
- Services register through a linker-section table, `SERVICE("name", handler)`. On PE/COFF it uses the
`.svc$a/$m/$z` trick (`ipc.c:8-17`).
- An unknown service gets a logging stub that returns OK with 0x40 zero bytes (`ipc.c:19-24, 34-41`).
- Implemented services:
| Service | Notes |
|---|---|
| `sm:` | |
| `acc:u0` | |
| `appletOE` | no applets (`swkbd.c` HLEs the keyboard) |
| `apm` | |
| `lm` | |
| `audren:u` | CPU mixer modelled on Ryujinx's AudioRenderer: PCM16/ADPCM, resampling, sub-mixes → SDL3 audio stream. Effects and splitters not implemented (`audio.c:14-18, 875`) |
| `audout:u` | |
| `fsp-srv` | RomFS as a raw image plus overlays; save data and SD card mapped to host dirs. `romfs_build.c` builds RomFS images |
| `hid`, `hid:sys` | shared memory filled from SDL3 keyboard/gamepad (`runtime/hle/input.c`, `bindings.c`) |
| `set`, `set:sys` | |
| `pctl*` | |
| `aoc:u` | |
| `time:u/a/s` | |
### Function-level HLE
`HLE_FUNC("mangled", fn)` binds guest imports by name (`runtime/hle.h:150-166`):
- `vi`/`nv` display init (`hle/vi.c`)
- `nn::pl` shared fonts
- `movie::` (nnMedia) on FFmpeg (`hle/movie.c`, 796 lines)
- swkbd
- heap wrappers
- `nvnBootstrapLoader` / `nvnDeviceGetProcAddress` (`hle/nvn_boot.c`)
`hle_override(addr, fn)` replaces functions by address, but only on the dispatch path (indirect calls).
`runtime/hle/hle_symbols.txt` is empty.
### Recompile on the player's machine (`runtime/gamebuild.c`, `runtime/gamelib.c`)
- The release contains no game code.
- On first run, the start screen does three things:
1. Runs the PyInstaller-frozen `gen.py` (`tools/lasgen.spec`).
2. Compiles the chunks with bundled `zig cc` as they appear. Jobs = `min(cores, RAM_GB-2)`
(`runtime/ui/launcher.mm:283-285`): budget about 1 GB RAM per compile job.
3. Links `lasgame.dll` with `zig lld-link`.
- The DLL exports only `lasgame_tables()` (`tools/gamelib_entry.c`).
- It calls back into the exe through an import library. The exe is linked with `--export-all-symbols --out-implib`
(`Makefile:104-106`).
- `GAMELIB_ID` (a hash of the generator, the headers and the flags) forces a rebuild when they change
(`Makefile:111-114`).
- The UI says the build takes "a few minutes" for LA (`launcher.mm:282`).
## 3. Graphics
### NVN on Windows
- NVN is HLE'd at the **API level**, not the GPU command level. Every `nvn*` proc address resolves to a host stub
(`hle/nvn_boot.c`). About 495 `NVN(...)` entry points exist (`objects.c` 371, `cmdbuf.c` 124).
- Command buffers are recorded as `CmdRec` records into guest command memory (`cmdbuf.c:1-3`), then executed by the
Vulkan backend (`runtime/nvn/vulkan/exec.c`, 3.8k lines).
- Vulkan 1.3 is required (dynamic rendering: `VkPipelineRenderingCreateInfo`, `exec.c:1513`). Allocation uses VMA.
- Missing NVN entry points silently `return 0` (`dispatch.c:69-75`).
### Shaders
- Own Maxwell SM5.x decoder in `runtime/nvn/maxwell/translate.c` (2.2k lines). Pipeline: emit MSL, token-rewrite to
GLSL 4.60, compile with shaderc (runtime, Apache-2.0) to SPIR-V (`translate.c:1-12`, `shader.h:1-27`).
- Branches become a `switch(pc)` state machine with explicit SSY/PBK/PCNT stacks.
- Handled opcodes: BAR BFE BFI BRA BRK BRX CONT CSETP EXIT F2F F2I FADD FADD32I FCMP FFMA FFMA32I FLO FMNMX FMUL
FMUL32I FSET FSETP FSWZADD I2F I2I IADD IADD3 IADD32I ICMP IMNMX IMUL IMUL32I IPA ISCADD ISCADD32I ISET ISETP KIL
LD(attr) LDC LDG/STG LDL/STL LOP LOP3 LOP32I MOV MOV32I MUFU P2R PBK PCNT POPC PRMT PSET PSETP R2P S2R SEL SHF SHFL
SHL SHR SSY ST(attr) SYNC TEX TEXS TLDS TXQ XMAD.
- **Missing** (`fail()` at `translate.c:1624`):
- shared memory LDS/STS
- atomics ATOM/ATOMS/RED
- surface load/store SULD/SUST
- TLD, TLD4 (gather), TXD, TMML
- VOTE
- double precision
- CSET
- bindless TEX.B (`translate.c:760`)
- Graphics pipelines are VS+FS only (`stageCount = 2`, `exec.c:1516`, also `:1092, :3269, :3330`), plus compute. No geometry or tessellation.
- Pipeline caching:
- SPIR-V disk cache `cache/spirv/<hash>.spv` (`vulkan/shaders.c:20-54`).
- `VkPipelineCache` saved to `cache/vk_pipelines.bin` (`vulkan/device.c:217-248`).
- One translator thread plus N pipeline worker threads (`exec.c:566-640`). Draws are skipped until their pipeline is
ready (`exec.c:2685`), so new areas show pop-in.
### Textures
- Tegra block-linear deswizzle on the CPU (`runtime/nvn/tegra.c`).
- BCn formats map natively (`vulkan/textures.c:92-104`).
- **ASTC is decoded on the CPU to RGBA8** (`runtime/nvn/astc.c`, LDR 2D only, bit-exact with astcenc;
`vulkan/textures.c:299-308, 548-555`).
### Other
- Resolution scaling, SMAA and FSR1 options (`output_scale.c`, `smaa_data.c`).
- Write-watch vertex cache (`exec.c:2048-2150`).
## 4. Windows toolchain
### Dev build
- MSYS2 UCRT64 (`tools/setup_windows.ps1`): clang, lld, make, SDL3, FFmpeg, vulkan-devel, shaderc, spirv-tools, VMA,
validation layers, nasm.
- Python 3.12 from python.org, with venv `tools/venv` (capstone, unicorn, lz4).
- Build commands: `make gen` then `make -j` (`docs/BUILD-windows.md`).
### Flags (`Makefile`)
| What | Flags |
|---|---|
| Generated code | `clang -O1 -fno-strict-aliasing -Wno-return-type -Wno-unused-label -fvisibility=hidden -Iruntime -D_FILE_OFFSET_BITS=64 -march=x86-64-v3` (`:39, :91-92`) |
| Runtime | `-O2 -g -fno-strict-aliasing -Wall ...` (`:117`) |
| Exe link | `clang++ -fuse-ld=lld -Wl,--stack,16777216 ... -lvulkan-1 -lshaderc_shared -Wl,--export-all-symbols -Wl,--out-implib` (`:93-95, :104`) |
| Game DLL link | `clang -shared -fuse-ld=lld build/lasrecomp.dll.a` (`:105`) |
### Player build
- zig 0.17.0 `zig cc` with `-mcpu=x86_64_v3`, linked with `zig lld-link @rt/link.rsp`.
- The zig runtime is prebuilt and the bundle slimmed by `tools/zig_slim.sh`; the exe is fetched by
`tools/fetch_zig.sh`.
- The `lasgen.exe` PyInstaller bootloader needs Control Flow Guard cleared, because Unicorn longjmps out of its JIT
(`tools/pe_no_cfg.py`).
### No MSVC dependence
- Target is the mingw-w64 / UCRT ABI.
- The generated C needs GNU/Clang extensions: `__int128`, `vector_size`, `__builtin_shufflevector/convertvector`,
`musttail`, statement expressions, inline asm on arm64. So `cl.exe` cannot compile it; clang-cl would need checking.
### Unknown figures
- Generated file count, C size and compile time are not recorded in the repo.
- Chunking bounds them at 1.5 MB per file. LA main `.text` is about 16.5–23 MB (code patch at `0x7001071220`, rodata
literal at `0x70016afba4`, `tools/patches.py`). That is roughly 4–6 M instructions, so LA is not "small".
- Measure TotK with `python tools/nso.py <exefs>/main` before estimating.
## 5. Game-specific vs generic
- LA-specific, around 22k lines in total:
- `tools/patches.py` (687 lines, 60 hard-coded `0x70…` addresses: widescreen, warp, rando hooks)
- `tools/warp_table.py`, `tools/rando/*` (8.5k)
- `runtime/hle/*` LA files (4.6k), `runtime/rando.c`
- `runtime/nvn/texpatch.c` (1k)
- most of `runtime/ui/*` (6.5k)
- `third_party/LAS-Randomizer`
- hard-coded addresses in `runtime/nvn/cmdbuf.c`, `runtime/hle/movie.c`, `runtime/fnprof.c`
(`grep -l 'main+0x\|0x70[0-9a-f]{8}' runtime`)
- Generic, around 18k lines: recompiler, cpu/dispatch/loader/kernel/ipc, services, platform layer, NVN core, Maxwell
translator, Vulkan backend, ASTC, tegra, generic HLE.
- About 45% of the non-third-party code is LA-only. Of the generic part, the CPU side is reusable almost as-is, while
the GPU/kernel/service side holds only what LA calls.
## 6. Scaling risks for TotK (concrete)
1. **Python generator memory and time.**
- `Module.cache` keeps one `Insn` object (with C strings) per instruction (`gen.py:60-69`), and `scanned` keeps
label sets per function. At 10M+ instructions that is several GB of RAM. Lifting is single-process pure Python.
- Fix: stream functions, drop caches after emission, parallelize lifting across processes (functions are
independent once boundaries are fixed).
2. **Jump-table emulation cost.** Each table index replays address materialization from the function entry
(`gen.py:159-161, 257-264`), so cost is O(entries × function length) per table, through Unicorn single-steps from
Python. Large switch-heavy functions (state machines, AI scripts, Havok) will be slow.
3. **"All labels" fallback** (`gen.py:378-382, 459-463`). Any function with a jump table gets a label on every
instruction and a switch over all of them. Clang -O1 compile time and RAM grow badly, and register promotion dies.
This drives the ~1 GB per compile job budget. Needs a precise table resolver (data-flow on index register and table
base, no emulation) so the fallback is the exception.
4. **Codegen quality.**
- Guest regs live in `Cpu` in memory. `LD/ST` use `may_alias` types plus `-fno-strict-aliasing` (`cpu.h:53-71`,
`Makefile:39`), so clang must assume every guest store can clobber `c->x[]`/`c->v[]` and reload them.
- SIMD helpers carry NaN/FZ checks per op.
- Built at `-O1`.
- Fine for LA at 60 fps; TotK is CPU-bound on Switch.
- Options: cache guest regs in C locals with explicit spill at calls and dispatch; put `Cpu` behind
`restrict`/noalias; FPCR-specialized fast paths.
5. **Fixed layout.** `SDK_BASE = MAIN_BASE + 64 MB` (`rt.h:73-74`, `Makefile`, `gamebuild.c:304`), only 2 modules
(`loader.c:177`), `g_modules[4]`. Make it data-driven (N modules, bases computed from sizes).
6. **Dispatch.**
- 4096-entry per-thread direct-mapped cache; binary search on a miss; linear `overrides` scan on every slow path
(`dispatch.c:145-146`).
- TotK's virtual-heavy code and many threads will miss more.
- Better: a perfect hash or a dense table indexed by `(addr - text_lo) >> 2` (4 bytes × text/4 ≈ text-size
memory).
7. **Missing ISA.** `LDXP/STXP` (`lifter.py:391-392`) trap at run time. Add them via `cmpxchg16b` (x86-64-v3 has it).
Also log a census of `Unsupported` per build (`gen.py:66`).
8. **Call model.** `ret` = C return; guest `BL` = host call. Guest setjmp/longjmp, `nn::os` fibers, or exception
unwinding would break. Audit TotK's SDK imports for `setjmp`, `longjmp`, `__cxa_throw`, `nn::os::Fiber`.
9. **Kernel.**
- One global lock; no scheduler; core masks ignored.
- Unimplemented SVCs are FATAL. The SVC list is LA-scoped: no MapProcessMemory, CreateCodeMemory,
SetThreadActivity, GetThreadContext, etc.
- Handle cap 4096.
- TotK has many worker threads (job system); expect contention on `g_klock`.
10. **GPU.**
- Translator lacks shared memory, atomics, image load/store, gathers, TXD/TMML, bindless and VOTE, and has no
geometry/tess stages. TotK compute and deferred passes will hit these.
- NVN coverage is only what LA calls, and missing calls silently return 0.
- CPU ASTC decode plus RGBA8 upload is 4× VRAM and a large CPU cost on a heavily ASTC game. Use a GPU compute
ASTC decoder.
- Draw skipping on pipeline misses means visible pop-in.
11. **Audio.** No effects or splitters (`audio.c:875`), so TotK reverb/delay will be missing.
12. **Toolchain cost.** Chunk compile at ~1 GB per job. 64-job cap. Single lld link of one DLL. Expect link RAM and
time to scale with object size; consider splitting the game DLL or using LTO-free thin archives.
## 7. Licensing
| Component | License | Where | Impact |
|---|---|---|---|
| LASRecomp code | MIT (`LICENSE`, © 2026 Peiikai) | all of `tools/`, `runtime/` | Reusable with attribution |
| Capstone 5.0.9 | BSD-3 | analyze/sym/tests only | Fine |
| Unicorn 2.1.4 | **GPL-2.0** (Python bindings BSD) | `gen.py` jump tables, tests | Build-time helper only (separate process). Shipping a frozen generator that bundles Unicorn makes that helper a GPL distribution. Replacing it with an own resolver removes the issue |
| Keystone 0.9.2 | **GPL-2.0** | `tools/rando/collect_patches.py` only | Not needed for TotK |
| oead / evfl | **GPL-2.0+** | randomizer only | Not needed |
| LAS-Randomizer | MIT | randomizer | Not needed |
| python-lz4 / LZ4 | BSD | nso.py | Fine |
| Dear ImGui, VMA, SMAA, AMD FSR1 | MIT | UI, renderer | Fine |
| SDL3 | zlib | window/input/audio | Fine |
| FFmpeg 9.0.2 | LGPL-2.1+ (release built LGPL-only, `tools/build_deps_windows.sh`) | movies | Dynamic link only |
| shaderc | Apache-2.0 | GLSL→SPIR-V | Fine |
| zig 0.17.0 / LLVM | MIT / Apache-2.0 w/ LLVM exception | player-side compiler | Fine |
| PyInstaller | GPL-2.0+ w/ bootloader exception | freezing helpers | Fine |
| GCC runtime, winpthreads | GPL-3 w/ runtime exception, MIT/BSD | Windows DLLs | Fine |
| Baloo 2 font | OFL-1.1 | start screen | Fine |
| Ryujinx (audio design reference, `audio.c:14-18`) | MIT | design only | Fine |
envytools docs were used for the shader translator. For the TotK shader work, the Ryujinx shader translator (MIT, C#)
is a license-compatible reference. yuzu's `shader_recompiler` is GPL-2.0+, so code cannot be copied from it into an MIT
codebase.
## 8. Recommendation for the TotK port
### Reuse with fixes
- `nso.py`
- `lifter.py` (add LDXP/STXP)
- `simd_c.py` + `simd.h` + `simd_fp.h` (tested; add fast paths later)
- `cpu.h`, `dispatch.c` (replace the lookup structure), `loader.c` (N modules)
- `platform/windows.c` (reserve + lazy commit + write watch is directly applicable)
- `ipc.c` framework, `services/fs.c` + `romfs_build.c`, `services/time.c`, `hid_set.c`, `hle/input.c`
- `nvn/tegra.c`, `nvn/astc.c` (as the reference/CPU fallback)
- Test harnesses `test_lift.py`, `test_simd.py`, `test_translate.*`, `test_astc.*`
- The recompile-on-user-machine distribution model (`gamebuild.c`, `zig_slim.sh`)
### Rewrite or replace
- `gen.py` driver: scalable, parallel, streaming, precise jump tables without Unicorn
- Register caching in codegen
- Kernel scheduler and locking
- Most of the NVN HLE and Vulkan backend
- Maxwell translator: needs a full SM5.x IR with compute/atomics/images/bindless/gathers
- GPU ASTC
- Audio effects
### Drop
- `patches.py` contents, rando, LA HLE, LA UI, Metal backend
docs/research/macnativegames.md
-587
View File
@@ -1,587 +0,0 @@
# MacNativeGames / d2r-mobile / nxnative: research notes
Scope: what the three reference ports do, what transfers to an x86-64 Windows port of TotK, what is Apple-only.
Checkouts (read-only, HEAD at time of study, 2026-10-20):
| Repo | Path | HEAD |
|---|---|---|
| MacNativeGames | `refs/MacNativeGames` | `b75be8c` |
| d2r-mobile | `refs/d2r-mobile` | `53e7c5b` (MacNativeGames copied its generic parts from `9f0c257`) |
| nxnative | `refs/nxnative` | `a35e964` |
All paths below are relative to `refs/MacNativeGames` unless prefixed.
## 0. Architecture in one paragraph
Not an emulator and not a recompiler. The game's three ARM64 NSOs (`main`, `sdk`, `subsdk0`; `rtld` is dropped and the
host links) are rewritten offline so they can run natively on Apple ARM64 (`games/totk/third_party/d2r/patch_nso.py`).
A C host maps them at original relative layout, resolves imports itself, and replaces Horizon at two levels:
(a) kernel SVCs via a "gate" trampoline, (b) whole `nn::*` SDK functions overridden by mangled name at relocation time.
NVN is replaced entirely: `nvnBootstrapLoader` is overridden, all 523 NVN entry points route to a host implementation
that records commands into game memory and replays them on Metal 4. Shaders: Maxwell code -> Ryujinx translator ->
SPIR-V -> SPIRV-Cross -> MSL -> Metal, cached on disk, partially pre-built at app build time.
Consequence for us: the CPU side of their work (rewriter, gate) is ARM64-host-only and useless on x86-64 as code, but it
is an exact, verified census of every place game code touches the OS. Everything above the CPU boundary (SVC
semantics, `nn::*` overrides, NVN command model, shader pipeline, TotK patches, data formats) is reusable knowledge, and
much of it is plain C.
## 1. Build pipeline: XCI/NSP + prod.keys -> app
Entry: `build_app.py` (1028 lines). Steps documented at `build_app.py:1-30` and `docs/how-it-works.md:73-125`.
### 1.1 Inputs
- Base game 1.0.0 only (`0100F2C0115B6000`), XCI or NSP. NSZ/XCZ base unsupported. Personalized tickets unsupported.
- Optional cumulative update NSP/NSZ via `--update`: 1.1.2 (v262144), 1.2.1 (v393216), 1.4.3 (v720896). No intermediate
update needed. DLC and combined packages unsupported (`README.md:66-67, 204`).
- `prod.keys`. NSP needs `--firmware <dir of .nca>` or `--shared-fonts <dir>` (NSP has no system fonts; XCI reads its
update partition) (`README.md:67, 100`).
- Python deps (`games/totk/requirements.txt`): `cryptography==50.0.2`, `lz4==4.4.5`, `capstone==5.0.9`, `nsz==5.0.0`.
Python 3.14 is required because the build uses stdlib `compression.zstd` (`requirements.txt:1`,
`scripts/locate_shaders.py:24`).
### 1.2 Code (step 1, `build_app.py:384-426`)
1. Container detection by header: XCI = HFS0 secure partition, NSP = PFS0 (`docs/how-it-works.md:75`).
2. `games/totk/scripts/extract_exefs.py`: own pure-Python reader, no hactool. NCA3, key-area or common-ticket AES-CTR,
hierarchical SHA-256 PFS0. Title key from ticket decrypted with titlekek from `prod.keys`, in memory only
(`extract_exefs.py:1-5, 93-146`). Ticket signature type `0x10004` is little-endian `04 00 01 00`
(`docs/how-it-works.md:132`).
3. ExeFS hashes compared to `expected-<ver>.json`. A different `main.npdm` is accepted if its ACID signature verifies
against Nintendo's public ACID modulus (`build_app.py:57-70, 146-171`).
4. `scripts/prepare_modules.py` -> ELF inventory; `third_party/d2r/patch_nso.py <nso> <out> <name>` for sdk, subsdk0,
main with `DM_RULESET=native DM_PRIVATE_GATES=0`; output hash must equal `rewritten_sha256` (`build_app.py:410-418`).
5. `scripts/prepare_runtime.py` binds the C loader to the verified images: emits `build/runtime_tables.h` (symbols,
relocations incl. RELR, gates) (`scripts/prepare_runtime.py:1-25`).
### 1.3 Data (step 2, `build_app.py:429-470`, `573-597`)
- Base RomFS: `scripts/extract_romfs.py`, decrypts the 17 GB RomFS into one flat `romfs.bin` + `index.json`, verified
through all six IVFC levels, then whole-image SHA-256 compared (`build_app.py:433-441`).
- Update merge (BKTR): hactool, pinned `https://github.com/SciresM/hactool.git` @ `1d64a83450e025622f3468c28fc4164dad2c5ef6`
(`build_app.py:44`). `build_hactool()` patches the fetched source before `make CC=clang` (`build_app.py:473-570`):
1. `MNG_TITLE_KEYS` env: private title-key file for both tickets, mode 0600, deleted after merge.
2. `MNG_BASE_RANGE_OFFSET/SIZE`: funopen-backed bounded view of the base NCA inside the original XCI/NSP, avoiding a
second 17 GB copy (Darwin `funopen`; Windows needs another mechanism or a temp copy).
3. Two BKTR layout checks relaxed: newer updates leave sector-aligned padding after the subsection table
(`build_app.py:542-565`).
Output via `--sectionN` (not `--romfs`, which does not export the BKTR section) (`docs/how-it-works.md:119`).
Driver: `scripts/extract_update_romfs.py`.
- Update version is chosen by exact ExeFS hashes, not file name; BKTR section marker identifies an update NCA
(`docs/how-it-works.md:117`).
- Shared fonts: `scripts/extract_shared_fonts.py`. Six BFTTF from system data titles starting `0100000000000811`;
BFTTF = big-endian words XORed with `key = word0 ^ 0x7f9a0218`, word1 = size (`extract_shared_fonts.py:19-28, 122-128`).
Other firmware fonts accepted if structurally valid.
- RomFS overlays from the player's files (manifest: path, size, sha256 overlay, sha256 original):
- `scripts/make_romfs_patches.py`: quick-menu capture timing (`UI/LayoutArchive/Common.Product.100.Nin_NX_NVN.blarc.zs`
anim keys moved to frame 0) and 1.0.0 game-over retry (`Event/EventFlow/DmF_SY_GameOver.bfevfl.zs`) (header lines 1-25).
- `scripts/make_output_res_overlay.py`: HeapDef for 2560x1440 (see section 6.3).
### 1.4 Shaders and packaging (steps 3-4)
- SPIRV-Cross @ `aa217aeb6c9f0ace7a0ab233b28807edf45eb165`, compiled from 11 sources with plain clang, no CMake
(`build_app.py:49-54, 600-612`).
- Prebuilt `games/totk/prebuilt/macos-arm64/totk_xlate` checked against `totk_xlate.sha256` (`build_app.py:615-630`).
- `scripts/prepare_shader_seed.py` extracts every shader from the verified RomFS + main module, translates default-bind
variants plus every `V` row from `assets/shader_list.tsv`; refuses to ship fewer than 8,000 (`docs/first-launch-shaders.md`).
1.4.3: 8,290 unique shaders, 24,871 runtime files, ~334 MiB seed.
- Optional: `metal` compiles MSL -> `.metallib` per source hash (`build_app.py:674`); `tools/warm_metal_cache.m`.
- Runtime binary built per version (`games/totk/Makefile:71-72, 341, 351, 361`: `-DTOTK_GAME_V112/V121/V143`), bundled,
codesigned, `--check` run.
### 1.5 Verification hashes (`games/totk/assets/expected-v*.json`)
| | 1.0.0 | 1.1.2 | 1.2.1 | 1.4.3 |
|---|---|---|---|---|
| `main` sha256 | `faf81f8a…a3eaab` | `f04fbebd…08d1ac` | `e5aa8d48…efcb6` | `e84a90d5…2dc9e` |
| `sdk` | `91573d44…4eea` | same | same | `1d187f29…e4ad` |
| `subsdk0` | `9d04c989…3f99` | same | same | `767c35bc…79b1` |
| `rtld` | `28317605…a44d` | same | same | `49308064…2817` |
| RomFS size | 17,069,139,640 | 17,051,988,992 | 17,052,047,360 | 17,052,625,232 |
| RomFS sha256 | `4751542f…dafcf` | `d98fde08…c43` | `c2beb1c2…7c11` | `797e66f2…19e9b` |
Full hashes in the JSON files. Fonts identical across versions. Base card image 17,536,620,544 bytes, program NCA
`9d827368992a2faafa9062aa5458b7ec.nca` (`expected-v1.0.0.json`). 1.0.0-1.2.1 share sdk/subsdk0/rtld; 1.4.3 replaced
all three (new SDK, RELR relocs, GNU-hash-only SDK modules) (`docs/how-it-works.md:125`). nxnative records 1.4.3 build IDs:
`main=277178b7dba1b6d4949a84778d4abc58 subsdk0=f39f4c431234c72eb520cfdd1001bbcf sdk=574284b40c06cf1821a44857939083be`,
SDK 20.5.22 (`refs/nxnative/dm/games/0100F2C0115B6000/game.tsv`).
These hashes are directly reusable for our own input validation.
## 2. NSO rewriter: every place game code touches the OS
File: `games/totk/third_party/d2r/patch_nso.py` (1036 lines, Python + capstone), plus `call_window.py`, `relr.py`.
Verification: `structck.py` (structural, independent llvm-objdump recount), `equiv.py` (Unicorn per-site differential
with mutation negative controls).
### 2.1 Why
Apple's kernel clobbers `x18` (platform register) and owns `tpidr_el0`; Horizon's TLR (`tpidrro_el0`), SVCs and the
system counter have no Darwin counterpart (`patch_nso.py:12-22`). iOS forbids writable code, so the rewrite is static and
never patched at runtime; thread-slot indices come from a read-only "key page" (`patch_nso.py:21, 245-248`).
### 2.2 What is rewritten (linear scan of `.text`, `patch_nso.py:609-619`)
| Instruction | Encoding mask | Replacement |
|---|---|---|
| `svc #n` | `w & 0xFFE0001F == 0xD4000001` | host gate, `arg = imm16` |
| `mrs xN, tpidr_el0` | `0xD53BD040` | load thread slot `KTLS` |
| `msr tpidr_el0, xN` | `0xD51BD040` | store `KTLS` |
| `mrs xN, tpidrro_el0` | `0xD53BD060` | load `KTLR` (Horizon ThreadLocalRegion ptr) |
| `mrs xN, cntpct_el0` | `0xD53BE020` | host gate (timer) |
| `mrs xN, cntvct_el0` | `0xD53BE040` | host gate (timer) |
| any `x18`/`w18` operand | capstone regex | load/modify/store slot `K18` via scratch regs |
`x18` handling variants: plain (`site_plain`, 331-341), conditional branch on x18 (`site_cond`, 343-358), `adr/adrp x18`
(`site_pcrel`, 360-369), `br x18` jump tables with landing pads (`site_br`, 371-391; 1.4.3 adds signed 32-bit offset
tables at five sites, `SIGNED_TABLES` 631-637 keyed by build ID `277178b7…`), `blr x18` call windows (`call_window.py`,
1.0.0 `main+0x9c9858`), LDXR/STXR exclusive windows containing x18 copied whole into the patch area so no stores land
inside the reservation (`find_windows`/`site_window`, 457-582). Function bounds from `.eh_frame_hdr` (584-605).
Local TotK fixes vs upstream: jump-table look-back 8 -> 32 instructions for `main+0x590574` (1.0.0)
(`third_party/d2r/README.md`, `patch_nso.py:649`); exclusive window fallthrough exit fix for `main+0x72afdc`,
`main+0x7aa630` (`patch_nso.py:560-562`).
Not rewritten in code but handled at runtime: `mrs CTR_EL0` and `dc civac` trap on Darwin and are emulated in the
SIGILL handler, SDK only (`games/totk/runtime/loader.c:370-374`).
The `CUTS`, `OBS`, `PARAMS` tables in the copied rewriter are D2R-only (keyed by D2R build ID `db0507f3…`,
`patch_nso.py:54-167, 916-928`); TotK uses none of them. TotK game patches live in the runtime (section 6).
### 2.3 Counts (OS touch-points)
- 1.2.1 main: 31,475 rewrite sites; sdk: 100 SVC/timer gates (`docs/v1.2.1-port.md`). sdk has 33 `cntpct` sites,
14 with destination x19..x23 (`runtime/gate.S` comment at the x0..x29 restore).
- 1.1.2: 55,018 site checks across modules (`docs/how-it-works.md:130`). 1.4.3: 68,968 semantic sites, 554,281
relocations applied at load (`docs/v1.4.3-port.md:20-21`).
- Game `main` itself makes almost no SVCs; the SVCs live in `sdk`. `main` touches the OS mostly through `x18`
(compiler uses it as a general register), TLS (`tpidr_el0`) and imported `nn::*` / `nvn*` functions.
### 2.4 Implication for an x86-64 recompiler
- `x18` is an ordinary GPR in Nintendo's ABI: a recompiler must model it as a normal register; no special handling.
- `tpidr_el0` (user TLS, written by the game) and `tpidrro_el0` (TLR, read-only) need per-guest-thread storage; TLR
layout: host allocates 0x200 bytes TLR, 8 KiB TLS, and stores a dummy pointer at TLR+0x1F8 (`loader.c:277-290`).
IPC message buffer is in the TLR (first 0x100 bytes, `runtime/ipc.c:16`).
- `cntpct_el0`/`cntvct_el0` must return 19.2 MHz ticks: host computes `ns * 12 / 625` (`loader.c:299-300`).
- Exclusive load/store pairs (LDXR/STXR, LDAXR/STLXR) appear in both game and SDK; a recompiler must lower them to
host CAS or equivalent with matching semantics.
- Jump tables via `br x18` with byte/halfword/signed-32 offsets exist; static recompilation must recover them
(the rewriter's recovery logic and the 1.4.3 table list are a ready reference).
- The SVC list in section 3.2 is the complete kernel surface actually used.
## 3. Horizon host
Files: `games/totk/runtime/` (9,856 lines total). Generic parts adapted from d2r-mobile `dm/host`.
### 3.1 Entry and gate
- `runtime/gate.S` (124 lines, ARM64 asm): switches to the per-thread host stack, saves x0-x30, q0-q31, NZCV, FPSR, FPCR
into an 800-byte frame, calls `runtime_dispatch(return_pc, regs)`, restores (x18 skipped).
- `runtime_dispatch` (`loader.c:296-304`): finds the gate by return address in `<module>.gates.tsv`; timer gates write
19.2 MHz ticks to the target register; SVC gates call `runtime_svc` (`services.c:304`).
- Host functions handed to guest code run on the thread's host stack (`runtime/hle_stack.c`, `hle_stack.S`, 4096 thunks).
### 3.2 SVCs implemented (complete list)
`runtime/services.c:304-382`, `runtime/threads.c:1-20, 354-560`, `runtime/ipc.c:169-210`. Unknown SVC = fatal.
| SVC | Name | Where | Notes |
|---|---|---|---|
| 0x01 | SetHeapSize | services.c:369 | 2 MiB granularity, cap 4 GiB; 1.4.3 sets ~2.2 GiB in nninitStartup |
| 0x02 | SetMemoryPermission | services.c:315 -> loader.c:66 | RELRO on module data, 4 KiB logical on 16 KiB host pages |
| 0x03 | SetMemoryAttribute | loader.c:91 | only the RELRO lock (mask=attr=16) |
| 0x04/0x05 | MapMemory/UnmapMemory | services.c:131 | stack moves; copy + protect source |
| 0x06 | QueryMemory | services.c:343 | synthesizes MemoryInfo for heap, alias, stack, modules |
| 0x08-0x0A | Create/Start/ExitThread | threads.c:354-378 | |
| 0x0B | SleepThread | threads.c:379 | 0/-1/-2 = yields |
| 0x0C/0x0D | Get/SetThreadPriority | threads.c:409-410 | prio 28..59 |
| 0x0E/0x0F | Get/SetThreadCoreMask | threads.c:411-418 | bookkeeping |
| 0x10 | GetCurrentProcessorNumber | threads.c:419 | virtual core |
| 0x11/0x12/0x17 | SignalEvent/ClearEvent/ResetSignal | threads.c:506 | |
| 0x15 | CreateTransferMemory | ipc.c:202 | |
| 0x16 | CloseHandle | threads.c:549 | |
| 0x18 | WaitSynchronization | threads.c:522 | |
| 0x19 | CancelSynchronization | threads.c:542 | |
| 0x1A/0x1B | ArbitrateLock/Unlock | threads.c:422-430 | tag = owner handle, bit 0x40000000 = waiters |
| 0x1C/0x1D | WaitProcessWideKeyAtomic/SignalProcessWideKey | threads.c:431-469 | condvars |
| 0x1F | ConnectToNamedPort | ipc.c:188 | only `sm:` |
| 0x21/0x22 | SendSyncRequest(+WithUserBuffer) | ipc.c:193 | in-process CMIF/HIPC/TIPC server |
| 0x25 | GetThreadId | threads.c:420 | |
| 0x26 | Break | services.c:377 | |
| 0x27 | OutputDebugString | services.c:378 | |
| 0x29 | GetInfo | services.c:317-338 | ids 0-8,10-18,20-23,28; 18 = program id `0x0100f2c0115b6000` |
| 0x2C/0x2D | MapPhysicalMemory/Unmap | services.c:339, 159-191 | alias region, 4 KiB page bitmap; LimitReached `0x10801` on pool exhaustion |
| 0x34/0x35 | WaitForAddress/SignalToAddress | threads.c:470-500 | |
| 0x45 | CreateEvent | threads.c:501 | |
Semantics follow Atmosphère mesosphère (KConditionVariable, KAddressArbiter, KSynchronization) (`threads.c:12`).
### 3.3 Services
Only one real IPC service is served in-process: `audren:u` (`runtime/audio_renderer.c:343`), plus `sm:` itself.
Everything else is replaced at the `nn::*` function level by mangled-name override during relocation
(`loader.c:209-228`: `runtime_observer` for JUMP_SLOT, then `runtime_override`, wrapped to run on the host stack).
~163 distinct symbols across:
| Area | File | Count | Examples |
|---|---|---|---|
| oe/aoc/pctl/err/diag/fs init | `services.c:289-302` | 24 | GetDesiredLanguage, GetOperationMode, GetPerformanceMode, aoc list = empty |
| fs | `fs_host.c`, `romfs.c` | 31+2 | MountRom, MountSaveData, MountCacheStorage, file/dir ops; saves = host dir |
| hid | `input.c`, `hid_feedback.c` | 16+13 | GetNpadStates (FullKey/Handheld/JoyDual/...), six-axis, vibration |
| applet/oe/nfp/err | `applet_host.c` | 31 | notification messages, launch params, nfp (amiibo) stubs |
| time | `time_host.c` | 13 | clocks, ToCalendarTime, ClockSnapshot (0xD0 layout) |
| codec | `opus_codec.c` | 7 | HardwareOpusDecoder -> libopus |
| movie | `movie_host.c` | vtable | `movie::Decoder` (subsdk0 = NintendoSDK_movie) -> libvpx VP9 |
| pl | `shared_font.c` | 3 | shared font address/size/state |
| swkbd | `swkbd_host.c` | 7 | ShowKeyboard -> host text entry |
| account | `account.c` | 6 | preselected user, nickname |
| vi | `frontend.m` | 5 | OpenDefaultDisplay, CreateLayer, GetNativeWindow, vsync event |
| os | `threads.c`, `services.c` | few | CreateSystemEvent, AllocateAddressRegion (observed) |
| nvn | `hle_nvn.c` | 523 | via `nvnBootstrapLoader` |
Startup: host calls the game's own `nn::init::Start(main_thread_handle, 0, rtld_cb, module_inits, module_finis)` in sdk
(`loader.c:470-477`); DT_INIT for each module is run from the callback (`loader.c:458-462`). nnMain return PC is a
version-table checkpoint.
TotK-specific vs generic: thread/sync/IPC/gate/stack code is generic (adapted from D2R). Memory layout constants,
`nn::*` override list (driven by what TotK `main` imports), audio REV11/REV15 renderer, movie decoder, fonts, swkbd hook
by PLT/GOT address are TotK-shaped. Roughly half of `runtime/` is game-agnostic Horizon emulation.
### 3.4 Threading model
- One guest thread = one host pthread running on the guest stack the SDK allocated (`threads.c:4-14`).
- All waits under one global mutex `G`, one condvar per thread; correctness first.
- Virtual cores: TotK keys per-core allocators/job queues by `GetCurrentProcessorNumber` without locks, so guest threads
must hold one of 3 virtual application cores while running guest code; core released only where Horizon would switch
(waits, sleep, yield, exit, blocking host calls). `TOTK_FREE_CORES=1` disables (`threads.c:120-150`). **Critical
correctness constraint for any port.**
- Guest threads run at `QOS_CLASS_USER_INTERACTIVE` (`threads.c:235`).
### 3.5 Memory layout
`runtime/services.c:18-123`, `docs/how-it-works.md:27-32`.
- All guest memory must be below 2^39: the SDK encodes IPC buffer addresses in 39 bits (HIPC descriptor bits 36..38).
- macOS 26: alias region fixed at `0x70_0000_0000`, size 64 GiB (real Horizon 39-bit size). Stack region 256 MiB, heap
region below it; 6 GiB low reservation searched from `0x3_0000_0000`.
- Fallback (macOS 27/iOS): one hole for stack+heap+alias, alias shrunk in 4 GiB steps down to 8 GiB; game uses < 3 GiB.
- Application pool reported: `0xCD500000` (~3.2 GiB, retail) + extra for 2K heaps (`services.c:60-63`).
- System resource 32 MiB; NPDM declares 24 MB system resource, so the game uses the alias region via MapPhysicalMemory
(`refs/nxnative/dm/games/0100F2C0115B6000/game.tsv`).
- Module images: page-aligned, main gets one extra writable page after its image for patch data (`loader.c:182-185`).
- Host page size 16 KiB asserted (`loader.c:489`); Windows has 4 KiB pages / 64 KiB allocation granularity, which makes
guest-exact 4 KiB permission tracking easier than on macOS.
### 3.6 TLS
Per thread: 8 KiB TLS block (`tpidr_el0` target), 0x200-byte TLR (`tpidrro_el0` target), dummy pointer at TLR+0x1F8,
256 KiB sigaltstack (`loader.c:269-290`). Slot storage = Darwin pthread keys read through `tpidrro_el0 + key*8`.
## 4. NVN layer (NVN on Metal)
Location: `games/totk/third_party/d2r_gfx/gfx/nvn/` and `gfx/tex/`. Mapping reference:
`refs/d2r-mobile/dm/gfx/nvn_metal_mapping.md` (78 lines, good one-page spec).
### 4.1 Files and size
| File | Lines | Role | Backend |
|---|---|---|---|
| `hle_nvn.c` | 2,727 | API front: 258 `NVN(...)` implementations, object semantics, command recording | agnostic C (uses mach for VM, pthread) |
| `totk_nvn_ext.c` | 275 | NVN 55 extras (debug labels, deferred binds, timestamps) | agnostic |
| `nvn_obj.h` | 203 | in-place object layouts + command record format | agnostic |
| `include/nvn.h` | 1,756 | NVN types/structs | agnostic |
| `nvn_protos.h`, `names.txt` | 577, 523 | prototypes, 523 entry names | agnostic |
| `nvn_thunks.S`, `nvn_entry.S` | 6,813, 79 | generated ARM64 counting thunks per entry | ARM64 asm, regenerate |
| `nvn_tex.c`/`.h` | 164 | texture layout sizes (deko3d-derived, zlib) | agnostic |
| `nvn_dirty.c` | 253 | write-protect dirty-page tracking of pool memory | OS VM API (mprotect/mach) |
| `totk_vpool.c` | 121 | virtual memory pools via `mach_vm_remap` | OS VM API |
| `nvn_hmap.h`, `nvn_pool_index.h`, `totk_serial_set.h`, `nvn_cube_group.h`, `nvn_pso_skey.h` ... | ~1,000 | containers, pool index, PSO state keys | agnostic |
| `nvn_mtl.m` | 13,696 | replay of command streams onto Metal 4, PSO cache, render passes, present | Metal / ObjC |
| `totk_mfx.m`, `totk_mfx_up.m` | 621 | MetalFX upscale + frame interpolation | Apple-only |
| `totk_shader_rt.m` | 279 | runtime shader translation client, cache, variants | Foundation + spawn |
| `nvn_refl.m` | 108 | reflection JSON -> bindings | Foundation |
| `fps_overlay.c`, `totk_sysload.c` | 428 | HUD via CoreText, IOKit load | Apple-only |
| `gfx/tex/nvn_deswizzle.c`, `nvn_bcdec.c`, `nvn_bc1astc.c` | ~270 | block-linear deswizzle, BC decode, BC1->ASTC | agnostic |
| `gfx/tex/nvn_deswizzle_gpu.m`, `nvn_bctc.metal`, `nvn_fmt_mtl.c` | ~980 | GPU deswizzle, BC transcode shaders, NVN->MTLPixelFormat | Metal |
Totals: `gfx/nvn` 29,295 lines (incl. 6,813 generated asm); Metal-specific ~15,000; backend-agnostic C ~6,000.
### 4.2 Model
`hle_nvn.c:1-20`, `nvn_metal_mapping.md`:
- `main` imports only `nvnBootstrapLoader`; host overrides it; the NVN driver in sdk never runs. All 523 names resolve to
counting thunks; unimplemented = fatal (`hle_nvn.c:54-66`). TotK requires NVN API 55, minor >= 15
(`hle_nvn.c:153`, check at `main+0xe9d614`). Device integer queries return Tegra X1 values (`hle_nvn.c:150-165`).
- GPU address = CPU address. Memory pools are game memory wrapped as zero-copy MTLBuffers (16 KiB-rounded,
overlapping allowed). Buffer address -> (MTLBuffer, offset) via interval table.
- Commands recorded into the game-provided command memory in `nvn_obj.h` format; replayed at `QueueSubmitCommands`.
CPU-visible side effects (event writes, counters, sync) happen after GPU completion.
- Sync object = submit fence value; `SyncWait` returns after the GPU reaches it (MTLSharedEvent).
- Textures: block-linear, deswizzled on upload; CPU writes detected by write-protect dirty tracking and re-uploaded.
TotK streams textures by CPU write + `nvnMemoryPoolFlushMappedRange` (`nxnative game.tsv: tex_flush_upload`).
- Window: game-owned swapchain textures presented by a present thread with `WindowSetCrop`.
- Shader bindings: driver constant buffers c0 (SSBO table) and c2 (texture/sampler handles) are filled by the replay
layer. TotK offsets: combined texture handles c2+0x20 (32 slots x 8 B), separate textures c2+0x168, samplers
c2+0x568, images c2+0x120; c0 SSBO tables vs=0x110 ps=0x510, compute c0 0x200 (group counts) / 0x210 (UBO table),
vertex point-size limit c0+0x610 (`tools/totk_xlate/Service.cs:26-37`); nxnative lists cs SSBO base 0x310 (`game.tsv`).
- Conditional rendering in GPU order (`nvn_mtl.m:1679`), occlusion queries via visibility buffer (`nvn_mtl.m:724`),
layered passes (`nvn_mtl.m:7252`), store-action lookahead (`nvn_mtl.m:3786-3830`), PSO prediction on submit thread
(`docs/pipeline-prediction-stalls.md`).
- `nvn_mtl.m` still contains D2R-only paths (shop drag image `8662`, 3D brightening `9948`, world freeze `10380`,
power-save `10426`): dead weight for TotK.
### 4.3 Retarget to Vulkan or D3D12
Reusable as-is or with small edits: `hle_nvn.c`, `nvn_obj.h`, `nvn.h`, `nvn_protos.h`, `names.txt`, `nvn_tex.c`,
`gfx/tex/*.c`, PSO state key logic, pool index, serial set, the c0/c2 slot conventions, reflection format.
Replace: `nvn_mtl.m` (13.7k lines of ObjC/Metal 4), MetalFX, CoreText overlay, `nvn_dirty.c`/`totk_vpool.c` VM calls
(`VirtualProtect` + vectored exception handler, `MapViewOfFile3`/placeholders for remap), the ARM64 thunk asm
(x86-64 or plain C function table).
Vulkan is the natural target: SPIR-V from the Ryujinx translator feeds it directly (drop SPIRV-Cross and MSL rewriting),
Ryujinx's own Vulkan backend is a working reference for Maxwell-on-Vulkan state mapping, and features Metal lacked
(quads via index emulation, depth clamp, geometry stages, subgroup ops in vertex stage, border colors) are mostly native.
Zero-copy "GPU address = CPU address" maps to `VK_EXT_external_memory_host` or to explicit buffer mirroring with dirty
tracking; on discrete GPUs (no unified memory) the pool-as-buffer model needs staging, which is the biggest design change.
D3D12 would need SPIR-V -> DXIL (spirv-cross HLSL or dxil-spirv); more work, no benefit.
Estimate: a Vulkan replay layer of similar scope to `nvn_mtl.m` (10-15k lines), reusing the front end unchanged.
## 5. Shader path
### 5.1 Translator
- `vendor/ryujinx`: Ryujinx 1.3.2 `Ryujinx.Graphics.Shader`, `Ryujinx.Common`, `Spv.Generator`, unmodified in-tree
(`vendor/ryujinx/README.md`).
- `games/totk/tools/totk_xlate/`: `Program.cs` (215, IGpuAccessor answering texture/attr/output types from a job file,
c1 constant segment needed for BRX jump tables), `Service.cs` (468, `--service` line protocol: JSON request ->
SPIR-V -> spirv-cross MSL + reflection JSON), `VertexCounterLowering.cs` (61).
- `publish_native.sh`: copies vendor/ryujinx, applies patches, `dotnet publish -r osx-arm64 -p:PublishAot=true`.
For Windows: `-r win-x64`. .NET SDK 10.0.401 used (`prebuilt/macos-arm64/README.md`).
- Patches (`tools/totk_xlate/patches/`), all to Ryujinx shader optimizer, all real correctness bugs, all relevant to any
SPIR-V consumer:
1. `0001-keep-call-arguments-in-variables`: don't propagate expressions into Call args (SPIR-V passes args by pointer).
2. `0002-keep-live-call-sources-when-removing-dead-node`: DCE must not strip sources of side-effecting ops (TotK
sun-visibility kernel `c7830fac` lost its atomicMin).
3. `0003-keep-side-effect-assignments-in-the-ast-optimizer`: unused-result atomics (Maxwell RED, ATOM to RZ) kept.
4. `0004-no-else-if-flattening`: else-if flattening gave shared merge blocks; SPIRV-Cross then dropped the trailing
else (fragment shader `1870878a` lost loops with atomic max + discard; 12 variants lost atomics, 83 changed discards).
Note: 0004 is a SPIRV-Cross interaction; check whether the Vulkan path needs it (probably harmless to keep).
- Exact-hash lowerings: visibility-counter vertex shader uses subgroup ballot + atomic; lowered to one atomic add per
passing invocation, restricted to code SHA-256 `EABE9FDD…24AD` (1.1.2/1.2.1) and `F5F4D231…1FF7B` (1.4.3)
(`VertexCounterLowering.cs:1-20`). Needed on Metal only because vertex-stage subgroup builtins were unavailable through
SPIRV-Cross; Vulkan supports subgroup ops in vertex shaders, so likely not needed.
- MSL post-processing in Service.cs: depth textures sampled as plain textures rewritten to `depth2d` etc.
(`Service.cs:39-46, 118+`), FP64 detection (`Service.cs:47`). Metal-only.
- Window origin upper-left, depth 0..1 (game sets via `nvnDeviceSetWindowOriginMode`/`SetDepthMode`); some shaders use
`NEAR_IS_MINUS_W` (`totk_shader_rt.m:29`).
### 5.2 Shader identity and extraction
- Key = SHA-256(first 0x880 bytes of NVN control section + code) (`totk_shader_rt.m:1`, `nvn_mtl.m:4962-4963`).
Control sections are 0x880 or 0x88c bytes; only 0x880 is hashed. Graphics code has a 0x30-byte NVN header
(+ 0x50-byte SPH inside code per Program.cs) (`scripts/locate_shaders.py`).
- Variant = shader + translator "answers" (texture slot types, image formats, attr/output component types); variants
reused only if answers match current bind state.
- Sources (`scripts/prepare_shader_seed.py:75-210`, `docs/first-launch-shaders.md` step 1):
- BNSH files (`BNSH` magic, `grsc` at +96) inside `Shader/*.bfsha.zs` (BFSHA; `material.bfsha` holds most),
`Effect/*.esetb`, UI layouts.
- agl `.sharcb` (`BAHS` magic) in `Lib/agl`, `Shader/ApplicationPackage`, `Lib/Terrain`, `Lib/gsys`: records
`<u64 code offset, u32 control size, i32 control offset>`; controls may be shared. Missing these = magenta title.
- BNSH embedded in `main`.
- RomFS `.zs` = zstd with dictionaries from `Pack/ZsDic.pack.zs` (SARC of dicts); SARC nesting up to 4 deep.
- Counts: 1.0.0 9,360; 1.1.2 8,286; 1.4.3 8,290 unique shaders, 0 offline translation failures.
### 5.3 Shared shader list and pipeline cache
- `games/totk/assets/shader_list.tsv`: 41,414 `V` (shader key + deps JSON), 1,557 `S` (libraries to warm), 63 `C`
(compute pipelines), 4,737 `R` (render pipelines, hex-encoded key), `L` locator rows (`docs/shader-list.md`).
Identifiers only, no game data. `t` flag = title screen set (1,445 rows across versions), `m` = background warm.
The `V` rows (shader hash + bind state) are backend-agnostic and directly reusable to pre-translate SPIR-V for Vulkan.
`R` rows encode Metal-pipeline-specific state keys and are not portable.
- Runtime cache: `~/Library/Application Support/MacNativeGames/TOTK/shader-cache`; missing variants appended to
`shader-list-contribution.txt`.
- First-compile stalls (100-900 ms) remain the main perf issue; long sessions showed footprint growth 4.5 -> 6.25 GiB
(`docs/v1.4.3-port.md:55-72`).
## 6. TotK-specific knowledge (CPU-approach independent)
### 6.1 Version address table (`games/totk/runtime/game_version.h`)
Keys are 1.0.0 main offsets; values per version. Generated by `scripts/map_version_sites.py` (instruction-window search
ignoring PC-relative fields; data via ADRP+LDR users; functions via callers) (`docs/v1.2.1-port.md`, "补丁地址").
| Role | 1.0.0 | 1.1.2 | 1.2.1 | 1.4.3 |
|---|---|---|---|---|
| present-interval controller clamp (fps60) | 0x8f6c60.. | 0x8f66e4.. | 0x8f273c.. | 0x2db204.. (`game_patches_v143.c:16-22`) |
| frame-step producer (realtime) | 0x7a83f4.. | 0x7f7a38.. | 0x7ede7c.. | 0x2daa54, 0x3e5658 (`game_patches_v143.c:74-75`) |
| global ratio | 0x8f68b8.. | 0x8f633c.. | 0x8f2394.. | in 0x2daa54 block |
| scene render size store | 0xc630bc | 0xcdd3e8 | 0xcc3f68 | 0x585e48 |
| output size getter call | 0xc630a4 | 0xcdd3d0 | 0xcc3f50 | 0x3dcb28 (display cfg +0x10/+0x508) |
| swapchain size const | 0xe996ac | 0xea7efc | 0xe7aa3c | 0xbdb578 |
| nnMain return checkpoint | 0xe995c4 | 0xea7e14 | 0xe7a954 | 0xbdb48c |
| forceSetMatrix | 0x69a18c | 0x70376c | 0x6ba86c | 0x45ce70 |
| scene slot (data) | 0x4663f08 | 0x4725058 | 0x4728538 | 0x3add238 |
| stale-ref crash site | 0x15ecb0c | 0x1627f10 | 0x1622bf0 | 0x1bd01c4 |
| ShowKeyboard PLT / GOT | 0x2aa4fa0 / 0x453f648 | 0x2b16100 / 0x46129e0 | 0x2b18000 / 0x4615d78 | 0x2bb79f0 / 0x398b348 |
| frame controller return | 0x8f6cbc | 0x8f6740 | 0x8f2798 | 0x2db2ec |
| actor name/pos/rot offsets | 0x210/0x2ac/0x2b8 | 0x218/0x2b4/0x2c0 | same | same |
| ultrawide camera/viewport | - | - | 0xcb4b10, 0x12521cc, 0x19b36f0, 0x1ecdf68 | 0x628c74, 0xb43460, 0x1ff04f4, 0x253fbb0 |
Exact original/replacement words per site: `game_patches.c:27-35, 121-126, 169-173, 298-304`;
`game_patches_v143.c:16-22, 74-75, 193-194, 260-263`. Every patch verifies original words before writing.
### 6.2 Frame timing (the 30 -> 60 fps unlock)
- Adaptive present-interval controller `main+0x8f6bec` (1.0.0), called once per frame from `main+0x8f68b4`, converts
measured frame work time (system object +0x118) into 60 Hz ticks clamped to [min, max]; result goes to
`nvnWindowSetPresentInterval` and to the variable frame-rate step `main+0x7a83e4` (ratio = interval*0.5, frame time =
interval/60). Shipping min = 2 (30 fps). Patch lowers min to 1 (7 instructions) (`game_patches.c:13-35`).
- Real-time step: replace the producer with two floats read from a data page (ratio = dt*30, dt seconds), updated each
present from a 5-sample moving average of present intervals, clamped to [1/120, 1/15] s (`game_patches.c:42-154`).
- 1.4.3: FrameRateMgr has two call sites fed from `0x3e5658` with a 0.5/1.0 coefficient selected by `sys+0x1da` bit 0;
replace the nominal 60/FPS value before the multiply, not the callee entry (`docs/v1.4.3-port.md:48`).
- nxnative alternative for 1.4.3 (`refs/nxnative/dm/tools/patch_nso.py:1037-1168`): interval picker `main+0x2db174`,
int fps at timing +0x50 read at `main+0x2daa54` and `main+0x3e562c`, float fps at +0x24; host chooses whole-vsync
steps; FPSLocker-derived fixes: D-pad menu fade `main+0x1fbc80`, swapchain argument `main+0xbe14a8` = 1, movie start/end
30 fps cap at `main+0x2b36100`/`main+0x2b33918`, view render scale at `main+0xd5c24`.
- Side effects of high fps: game-over "Retry" hung on 1.0.0 because `DmF_SY_GameOver` waits one frame after pausing;
fixed by RomFS overlay. 1.1.2+ use `DmF_SY_GameOver.110` (EventWaitUpdate), unaffected
(`scripts/make_romfs_patches.py` header). Quick-menu capture background black at 60 fps; fixed by moving bflan
visibility keys to frame 0.
- `TOTK_PCHTXT` loader accepts community emulator pchtxt patches (by `@nsobid`) (`game_patches.c:387-423`).
### 6.3 Resolution
- Scene render size from parameter tree (`main+0xc6325c`: handheld 1280x720, docked 1600x900), stored in renderer
+0xcc4/+0xcc8 (`game_patches.c:156-168`). 1.4.3 actual scene RT is 1600x896.
- Output/swapchain 1920x1080 constant before "NvInitializeGraphics" (`main+0xe996ac`); canvas from graphics params
(+0x38 output, +0x30 scene) (`game_patches.c:207-235`). 1.4.3: display config object +0x10 canvas, +0x508 crop,
+0x528 scale, pending bit 21 of +8 (`game_patches_v143.c:144-148`).
- Runtime mode switch path: post `nn::oe` message 30 (operation mode changed) + set renderer +0xca4 bit 0x200 ->
`main+0x1185310` handler relayouts (`game_patches.c:222-230`).
- 2560x1440 needs bigger heaps: `System/HeapSize/Product.Nin_NX_NVN.HeapDef.byml.zs`: `DynamicTextureAllocatorMem`
113,664 KiB (parent `Graphics` 201,728 KiB); scale by pixel ratio, grow Graphics and the app pool by the same amount
(+89,088 KiB at 1440p) (`scripts/make_output_res_overlay.py`, `game_patches.c:271-284`).
- UI canvas width cap, 19:9 aspect constants (D2R; not TotK). TotK ultrawide: patch every producer of the projection
aspect ratio (sead camera +0xb0), DefaultAspect 1.777778 in rodata only copied at camera creation
(`runtime/aspect_patch.h:1-13`).
- Final upscale shader hashes: EASU `a3a570eb…` (old), `ac58fc18…` (1.4.3, multiplies by 1/3.5 then 3.5, R11G11B10F)
(`docs/v1.4.3-port.md:39`).
### 6.4 Game data and engine facts
- Engine: sead / agl / gsys, NintendoWare G3d2 / Ui2d / Vfx2, Havok; `main` imports `nn::ro` (NRO loading)
(`refs/nxnative/dm/games/0100F2C0115B6000/game.tsv`). subsdk0 = NintendoSDK_movie (Android MediaCodec over NvMM).
- Heaps: `sead::ExpHeap` use list at +0xc8, heap self ptr +0x28, size +0x30 (`loader.c:394-419`).
- Movies: 28 WebM in `Event/Movie`, VP9 profile 0, 1920x1080, 30 fps, video only; NVDEC path aborts without a host
decoder; `movie::Decoder` 17 vtable methods, output format 4 = NV12; pacing by PTS required at >30 fps
(`runtime/movie_host.c:1-24`). Identical across 1.0.0-1.2.1.
- Audio: `audren:u` REV11 (Q14 biquads) and REV15 (1.4.3: 0x188-byte voice input, float biquads, 0xa8 splitter
destination) (`docs/v1.4.3-port.md:26`); effects (delay, reverb, compressor) ported from Ryujinx.Audio. Voice streams
are Opus via `nn::codec::HardwareOpusDecoder` with 8-byte big-endian header {payload size, final range}
(`runtime/opus_codec.c:1-5`). 1.4.3 English voice: 280,723 packets.
- Streamed audio bug: `content://Sound/Resource/Stream/Dm_OP_0001.bwav` has a double separator after the mount name;
path parser must skip repeated `/` (`docs/audio-troubleshooting.md`, `runtime/romfs_path.h`).
- Save format (1.1.2/1.2.1): 2,307,656 bytes, header `0x47E0F4`; 54 files in a profile (`docs/v1.2.1-port.md`).
- Language codes the game accepts via `GetDesiredLanguage`: ja, de, fr, es, it, nl, ru, ko, zh-Hans, pt (pt-BR on 1.4.3),
en-US (`services.c:246-272`).
- RomFS file counts: 1.1.2 304,471; 1.2.1 304,473; 1.4.3 304,243.
### 6.5 Known pitfalls (from docs and loader)
1. Virtual cores: per-core lock-free data in the game; must emulate 3 application cores (section 3.4).
2. 39-bit guest address limit for IPC (section 3.5).
3. MapPhysicalMemory must map only unmapped pages and leave mapped ones intact; TotK heaps grow by remapping overlapping
ranges. Pool exhaustion must return LimitReached (132 -> `0x10801`), not `0xd401` (abort) (`services.c:164-174`).
4. Texture auto block height: NVN truncates `height_in_blocks*1.5/8`; deko3d rounds up; wrong at boundaries (shrine warp
pad 160x160 BC3-sRGB shows black) (`docs/how-it-works.md:47-55`).
5. D16 load-clear vs fragment depth rounding differ (0.5 -> 32767 vs 32768); Ultrahand/Recall classification grid
(`docs/how-it-works.md:57-63`). Check on Vulkan too.
6. Aliased memory (virtual pools remapping physical pools, two pools on same CPU memory) needs alias-aware barriers
(`docs/how-it-works.md:67-71`).
7. Crash workarounds in game code: stale optional target reference at `GAME_ADDR(0x15ecb0c)` (object+0x6b0) after long
play (#57); tagged invalid reference `0xffff0000ffffffff` during save load (`loader.c:313-369`). A recompiler must
reproduce or patch these.
8. TotK reserves only 16 bytes for `NVNtextureView` on its stack (`main+0x823398`), host objects must fit NVN's reserved
sizes (`nvn_obj.h` FITS asserts).
9. Shader translator optimizer bugs (section 5.1).
10. Long-session metadata growth (dead-serial hash sets ~1 GiB after 3 h) (`docs/serial-history.md`).
11. Mods: RomFS-only overlay via manifest; ExeFS/IPS not supported (`README.md:200`).
## 7. Apple-only vs portable
| Component | Files | Status |
|---|---|---|
| NSO rewriter, gate, thread_enter, hle_stack asm, NVN thunks | `third_party/d2r/*.py`, `runtime/*.S`, `nvn_thunks.S` | ARM64-host-only. Useless for x86-64 execution; valuable as OS-touch census and verification method |
| Loader | `runtime/loader.c` | logic portable (reloc kinds 1027/257/1025/1026, RELR, symbol override); mmap/mach/CommonCrypto/signals Apple/POSIX |
| Kernel emulation | `runtime/threads.c`, `ipc.c`, `services.c` | portable C over pthreads + mach_vm; Windows needs VirtualAlloc2 placeholders, SRWLock/CondVar or winpthreads |
| `nn::*` overrides | `fs_host.c`, `romfs.c`, `time_host.c`, `account.c`, `applet_host.c`, `opus_codec.c`, `movie_host.c`, `shared_font.c`, `swkbd_host.c`, `hid_feedback.c`, `input.c`, `audio_renderer.c`, `audio_effects.c`, `audio_reverb.c` | portable C (libopus, libvpx); CommonCrypto SHA-256 to replace |
| Audio output | `audio_mixer.c` | AudioToolbox; replace with WASAPI/XAudio2/SDL |
| Input, haptics, motion | `gamepad.m`, `gamepad_feedback.m` | GameController/CoreHaptics/IOKit; replace with SDL3/XInput |
| Window, front end | `frontend.m`, `app/launcher.m`, `frontend_ios.m` | AppKit/UIKit |
| Graphics replay | `nvn_mtl.m`, `nvn_refl.m`, `totk_shader_rt.m`, `gfx/tex/*.m`, `.metal` | Metal 4 |
| Upscale / interpolation | `totk_mfx*.m` | MetalFX; replace with FSR/DLSS/XeSS or drop |
| HUD | `fps_overlay.c`, `totk_sysload.c` | CoreText/IOKit |
| Shader translator | `vendor/ryujinx`, `tools/totk_xlate` | portable C#; NativeAOT for win-x64 works |
| Build | `build_app.py`, `scripts/*.py` | mostly portable Python; Apple bits: `xcrun`, codesign, lsregister, APFS clone, `funopen` hactool hook |
## 8. Toolchain
- macOS 26+ on Apple silicon (Metal 4), Xcode command-line tools (`xcrun clang`, C11, `-fobjc-arc`).
- Homebrew: `opus`, `libvpx` (linked statically from `/opt/homebrew/opt/*/lib/*.a`), `python@3.14`.
- Python 3.14 (stdlib `compression.zstd`), venv with `cryptography`, `lz4`, `capstone`, `nsz`; dev: `unicorn==2.1.4`,
`Pillow`, `brotli` (`requirements-dev.txt`).
- .NET 10 SDK (10.0.401) only to rebuild `totk_xlate` (NativeAOT).
- Fetched at build: hactool @ `1d64a83`, SPIRV-Cross @ `aa217ae`.
- Optional: Xcode Metal toolchain for offline `.metallib`; iOS: Xcode 26, xcodegen, cmake, ninja.
- Single-instance "game slot" lock for all GPU tests (`runtime/game_slot.c`).
## 9. Licenses
From `THIRD_PARTY_NOTICES.md` and LICENSE files:
| Code | License |
|---|---|
| MacNativeGames (repo) | MIT (`LICENSE`) |
| `third_party/d2r` (rewriter, verification), `third_party/d2r_gfx` (NVN on Metal), `tools/totk_xlate` | MIT (d2r-mobile) |
| `nvn_tex.c` texture layout sizes | zlib (deko3d) |
| bcdec 0.985 | MIT or public domain |
| `vendor/ryujinx` (Shader, Common, Spv.Generator 1.3.2) | MIT |
| audio effects/mixer ported from Ryujinx.Audio | MIT (`third_party/licenses/Ryujinx-Audio.txt`) |
| SPIRV-Cross | Apache-2.0 |
| hactool (+ bundled mbedTLS) | ISC (+ Apache-2.0) |
| libopus, libvpx | BSD-3-Clause |
| d2r-mobile | MIT |
| nxnative | MIT (`refs/nxnative/LICENSE`); bundles libvpx, stb_vorbis, libdeflate, Vulkan headers, NVIDIA FLIP with their own licenses |
The TotK-Ultrawide community mod is GPL; MacNativeGames states none of its code is used (`runtime/aspect_patch.h:6`).
Everything we would reuse (host C, NVN front end, translator patches, addresses, hashes) is MIT/zlib/Apache: compatible
with any license we choose.
## 10. nxnative and d2r-mobile notes
- d2r-mobile `dm/host` is the generic origin (loader, `hle_svc.c`, `hle_thread.c`, `hle_table.c` for libc passthrough,
`HLE`/`HLE_FORCE` registration with mandatory reasons, `UNIMPL` named stubs). Offline shader route: everything
pre-translated to metallib, no runtime translation. `dm/gfx/nvn_metal_mapping.md` is the best concise NVN spec.
- nxnative generalizes to 14 games with per-game profiles (`dm/games/<title id>/game.tsv`), including TotK 1.4.3 and BotW
1.9.0 (shares sdk/subsdk0 build with TotK per its comment), Vulkan-on-Metal path for Hades (`dm/gfx/vk/hle_vk.m`),
LibHac-based `dm/tools/nx_extract` (.NET) for NSP + update layering. Its TotK profile has memory numbers
(`mem_alias 0x98000000`, heap `0x86000000`, app pool `0xCD500000`) and NVN constant-buffer bases. One-script packaging:
`scripts/build_ipa_from_nsp.py` (1,229 lines).
## 11. Takeaways for the Windows x86-64 port
1. The CPU strategy must change (no native ARM64), but the rewriter's site census (`sites.tsv`, `gates.tsv` per module)
is a precise list of every SVC, timer read, TLS access and x18 use; reuse `patch_nso.py`'s scanner as an analysis
tool for the recompiler front end.
2. The host boundary is mostly at `nn::*` function level, not IPC: a recompiler can link recompiled `main` against host
implementations of ~163 `nn::*` symbols plus 523 NVN entries, and either recompile `sdk` (for os/thread/heap code
that uses the 38 SVCs in section 3.2) or HLE more of it.
3. Port as C: `threads.c`, `ipc.c`, `services.c` memory logic, `fs_host.c`, `romfs.c`, `time_host.c`, audio renderer
and effects, opus, movie, fonts, swkbd, `hle_nvn.c` + `nvn_obj.h` + texture code.
4. Write a Vulkan replay to replace `nvn_mtl.m`; keep Ryujinx translator + the four patches, emit SPIR-V directly.
5. Reuse version hashes, address table, fps/realtime/resolution patch semantics, RomFS overlays, HeapDef math, shader
extraction formats and the shared shader list `V` rows.
docs/research/port-lessons.md
-129
View File
@@ -1,129 +0,0 @@
# Port lessons: symptoms other ports already solved
Each lesson maps a player-visible symptom to the cause another port found, and records whether our runtime has the fix.
Sources: nxnative (MIT, `refs/nxnative` at `d5b380b`, 2026-10-22; `docs/agents/03-bringup.md`, `04-graphics.md`,
`05-debugging.md`, `docs/development.md`, the TotK profile `dm/games/0100F2C0115B6000/game.tsv`) and MacNativeGames
([macnativegames.md](macnativegames.md)). nxnative runs TotK 1.4.3, so its `main+` addresses do not carry over to 1.2.1.
Maxwell semantics live in [maxwell.md](maxwell.md); a lesson that is also a Maxwell check names its number (`mx N` =
maxwell.md section 7 check N). Our side was checked by reading the code on 2026-10-22 and updated 2026-10-26. Our code is cited by file and function
or SVC number, so the citations survive edits; `file:line` citations point into `refs/`.
Status: **open** (missing or partial), **unknown** (not checked), **done**, **parked**.
## Before you investigate a symptom
1. Get the evidence first: the user's screenshot or recording, or a flight-recorder dump
([agent-playbook.md](../agent-playbook.md), "Frame dumps"). A guess without a captured frame costs a game run each.
2. Search the lessons for the symptom words: this file, maxwell.md sections 6-7, then the upstream notes:
`grep -rniE 'black|flicker' docs/research refs/nxnative/docs refs/MacNativeGames/docs`. Words that match well are
what the player sees (black, magenta, flicker, slow motion, missing, stale, streaks) and the object (sky, grass,
shadow, fog, HUD).
3. A **done** match: confirm our fix still holds before deriving anything new. An **open** match is the first
hypothesis.
4. Read the code the hypothesis depends on before any game run: the guest functions in Ghidra or `generated/v121`,
our runtime path. A hypothesis the code already rules out costs no run (2026-10-23: ten agent runs went to
scheduler and lock theories that one Ghidra read of the game's double-buffer protocol would have narrowed).
5. Then falsify what is left with the cheapest switch, dump or trace before writing a fix. A fix that changes timing
or scheduling also needs the slow-frame count of the same route next to the run without it.
## Keeping the lessons current
Upstream changes daily. `git -C refs/<name> fetch origin`, then `git log --format='%h %s' <pin>..origin/HEAD` lists the
new commits and `git diff <pin> origin/HEAD -- docs` their notes. Add lessons here, then move the pin in
`tools/fetch_refs.sh`. nxnative rewrote its history on 2026-10-22; when a pinned hash is gone, find it by commit subject.
## Open: timing and frame pacing
| id | symptom | cause and their fix | our status | next step |
|---|---|---|---|---|
| T1 | GPU-bound scenes run in slow motion; dynamic resolution never drops | The Zelda engine's GPU time is the `TIMESTAMP` report at the end of the frame minus `TIMESTAMP_TOP` at its start, read after the frame fence; the interval controller and dynamic resolution use it. One time per submission made it 0-6 ticks (BotW). Fix: each report gets the GPU start or end of its command segment (`gpu_timestamps segment`, `DM_NVN_GPU_TS`; 04-graphics.md) | open: every report gets the host time at fence completion (`vk_exec_xfer.c` `write_counter`), so the difference is microseconds. Counter word 0 is right (conditional clears need it). 1.2.1's interval controller does not read it: `[x1+0x118]` (`0x8f26c8`) is `GetSystemTick` minus the frame start (`f_70008f24bc`); dynamic resolution not checked | find a reader of the report difference; then `vkCmdWriteTimestamp` (top of pipe for `TIMESTAMP_TOP`) converted to 19.2 MHz ticks with calibrated timestamps |
| T2 | frame-rate counter wanders 50-60 | the vsync event and the window acquire each kept a 60 Hz clock and drifted apart; both follow one display tick now (03-bringup.md) | open: `gfx_window.c` `vsync_main` and the pacer of `nvn_api_obj.c` (`pace`, `present_period`) run separate clocks | derive acquire slots from the vsync thread's tick |
| T3 | a 30 Hz thread runs at 27 Hz, slow cutscenes | host sleep overshoot; `sleep_precise` | partly: sleeps up to 1 ms spin (`kernel.c` `sleep_ns`; 20 us sleeps took 460-515 us before, [windows-host.md](windows-host.md)); longer ones use the high-resolution waitable timer, 0.3-0.5 ms late | measure `SleepThread` at 5, 16 and 33 ms with `TOTK_TRACE=sched` |
## Open: thread scheduling
Horizon runs a better-priority thread at once and pins some threads to one core; a host runs them all in parallel.
A one-step ordering difference between two engine threads can show as one frame with inconsistent data.
| id | symptom | cause and their fix | our status | next step |
|---|---|---|---|---|
| S1 | state read one step early; use-after-free between threads | a woken better-priority thread pinned to the signaller's core runs before the signaller's next instruction (`dm_yield_to_core_peers`, `core_preempt`; 03-bringup.md) | open: the preempt flag is honoured at the holder's next SVC or indirect call (`dispatch.c` `dispatch_preempt`; MacNativeGames `threads.c:120-151` checks at SVCs), and a waiter interrupts a holder in game code after 250 us (S7), but a woken thread does not run before the signaller's next instruction. Tried 2026-10-23: a check at every host call made phantom rocks rarer (0/10 to 4/6 clean runs) but added 50-75 ms CPU-side hitches, so it was reverted; a nxnative-style waker handoff did not help | none until a symptom is traced to it alone |
| S2 | per-core data races, different thread order | `CreateThread` with ideal core -2 and the main thread get the default core and a mask of that core only; `SetThreadCoreMask` -2 means core 0 / mask 1 | open: mask 7 for negative cores and the main thread (`kernel.c` SVC 0x08 `CreateThread` and the main thread's `thr_new`); `SetThreadCoreMask` -2 keeps the old ideal core (`kernel.c` SVC 0x0F) | count -2 creations with `TOTK_TRACE=svc`, then pin |
| S3 | worker threads burn cores, late frames | 843 SVC-free spin loops in TotK `main`; nxnative's rewriter makes them poll 2 ms, then sleep | partly: a better-priority waiter takes the core at the spinning holder's next indirect call or interrupts it after 250 us (S7); a spin loop still burns a host core and keeps equal-priority waiters out | measure core-wait while a holder spins (sched trace `preempt` latency) |
| S4 | rare hang or lost wake | `SignalToAddress` type 2, count <= 0 with waiters: Ryujinx, yuzu and nxnative set `value - 2` | done: `kernel.c` SVC 0x35 writes `value - 1` and the count > 0 branch writes `nw <= count ? value - 1 : value`, both as mesosphere master `kern_k_address_arbiter.cpp`; 1.2.1 calls type 2 only with count 1, and `value - 2` did not change the phantom rocks (2026-10-23) | none |
| S5 | the main thread holds a core a worker needs | yield types 0 / -1 / -2 differ on Horizon; TotK's render-graph wait is a poll plus yield | unknown: all three hand the core only to an equal or better waiter (`kernel.c` SVC 0x0B) | trace which yield type 1.2.1's frame-end poll uses |
| S6 | lost or extra wakes | MacNativeGames `dab728a`: `SignalToAddress` wake-all, `SystemEvent` rebinding, targeted wakes | unknown | read the commit, compare with `kernel.c` |
| S7 | stutter: a better thread waits milliseconds for a core whose owner runs game code | MacNativeGames 8bf0893 (PR #488, on by default e4b4601): the waiter signals the owner, which gives the core up when interrupted in top-level game code; priority inheritance through guest mutexes; the SDK's preemption disable (TLR+0x100 / +0x102, SVC 0x36); without the last two their game hung at "Continue". 1.1.2: 58% of long core waits were this class; Mac worst-1% frame 18.4 to 10.6 ms | done: merged, on by default (`TOTK_ASYNC_PREEMPT`; SuspendThread and a register-saving stub, `kernel.c` `kern_ap_stub`, instead of a signal). `TOTK_CORE_DIAG=1` measured this class at 0.47 / 0.83 / 1.06 ms of thread-time per frame (field / fight / monsters2) with our indirect-call checkpoints, waits to 16+ ms; with the port 0.29 / 0.34 / 0.84 and the waits of 1 ms or more 53 / 41 / 27 to 4 / 10 / 18. 1.2.1's SDK has the disable protocol (sdk f_70051242a8, f_7005124360). A first stub clobbered the flags before saving them (`sub` before `pushfq`) and crashed at boot | A/B timings: [backlog](../backlog.md) item 11 |
## Open: graphics
| id | symptom | cause and their fix | our status | next step |
|---|---|---|---|---|
| G3 | light or decal lists, particle data read 0 | buffer textures read by 1D integer fetches; bound as 2D rows | open: buffer textures are not replayed and bind a stand-in (`vk_exec_draw.c` `bind_resources`) | census; Vulkan texel buffer views need no 2D trick |
| G4 | textures read as the dummy | bindless handles stored in the game's own uniform buffers (`tex_ubo`) | not in 1.2.1: 0 of 75k descriptors take a handle from cbuf >= 3 and no startup shader has a `_B` texture op (2026-10-22); only c2 handles are mapped (`tools/xlate/Program.cs` `TexSlot`) | if a later version needs it: nxnative `ubo_read` (`dm/gfx/nvn/nvn_mtl.m:4199-4215`) |
| G5 | garbage in feedback effects | new GPU textures start at zero | open: depth starts cleared, GPU-only colour images start undefined (`vk_texture.c` `init_image`; mx 7) | clear new GPU-only colour images |
| G6 | 50x50 black tiles, masks not cleared | clears with a partial channel or stencil mask | parked: a fix was written and not merged (mx 3, backlog item 29) | merge if the census shows partial-mask clears |
| G7 | garbage geometry on title or load screens | 8-bit indices written by a copy in the same submission read stale by CPU widening | open: CPU widening at record time (`vk_exec_draw.c` `op_draw_idx`; mx 9) | `VK_KHR_index_type_uint8` (backlog item 24) |
| G8 | cave stone blocks get no torch light | `ReadTexels` of a CPU-filled BC4 atlas returned the never-uploaded copy | open: ported, not checked on the 5120x5120 atlas | dump the cave save |
| G9 | scattered squares in albedo | G-buffer attaches one texture at two slots | unknown | validation on a G-buffer frame |
| G10 | stale 32x32 blocks where pre-pass and G-buffer disagree | identical position math must give identical depth (invariance) | unknown | grep a dumped VS SPIR-V for `Invariant` |
| G11 | characters stretched into streaks | integer vertex inputs vs the format's type | unknown | count `VUID-vkCmdDraw-Input-08734` |
| G12 | one flat ambient colour | a 1-layer 2D array sampled as 2D | unknown | read view creation in `vk_exec_draw.c` `get_view` |
| G13 | magenta streamed textures | sparse textures use a 64 KB tile layout | unknown | look for virtual pools in the NVN log |
| G14 | upside-down UI text | lower-left window origin | unknown: origin recorded (`nvn_api_core.c` `nvnDeviceSetWindowOriginMode`) | grep the NVN log for the origin |
## Open: performance
| id | symptom | cause and their fix | our status | next step |
|---|---|---|---|---|
| P2 | low fps outdoors | inline uniform updates and clear-only passes end render passes; deferring them cut TotK's attachment traffic from 1,100 to 700 MB a frame (`DM_NVN_UB_PARTIAL`, `DM_NVN_LAZY_CLEAR`, `cb_extent.tsv`) | open: `UpdateUniformBuffer` is a buffer copy between full barriers, which end the pass (`vk_exec_draw.c` `op_update_ub`) | count passes per frame and those ended by an update in a `TOTK_DUMP` |
| P3 | out-of-memory after minutes | dead textures re-entered tables | unknown | log VMA bytes and live images every 1000 frames |
## Open: debugging tools
| id | lesson | our status |
|---|---|---|
| D1 | Per-shader highlight, depth override and skip (`DM_NVN_HILITE`, `DEPTHOVR`) split "rejected by depth" from "never drawn" and bisect a glitch to one shader | open: `TOTK_DUMP` has none |
| D2 | First-use texture info with the count of non-zero guest bytes (`TEXINFO`, `TEXRAW`, `ZBLOCK`) shows a texture the CPU filled but we did not upload | open: dump lines carry address, mirror state and upload count only |
| D3 | Crash reports with a stack scan and the words at `*xN` | open: vectored handler (`main.c` `crash_handler`) |
## Done
Confirm the fix still holds before re-deriving any of these.
- Speed follows the present interval at 60 fps (T4): MacNativeGames' controller patch and real-time frame step as
generator patches (`recomp/patches-v121.txt`, `runtime/frame_rate.c`); the pacer ignores the game's interval while
the step is real-time (`nvn_api_obj.c` `present_period`).
- Driver constant bank c0 (G2, fog that never parts, flicker in tile rows): a c0 bank per stage with the dispatch
size at 0x200, point size at 0x610 and base instance at 0x644, `vk_exec_draw.c` `vk_c0_fill`.
- GPU idle while the replay records (P1): the replay is submitted in parts between passes every `TOTK_SPLIT_DRAWS`
(64) draws, only the last part signals, `vk_exec.c` `maybe_split`.
- Unwritten vertex outputs read (0,0,0,1) (no sun shadows in Link's Awakening): translator patch
`tools/xlate/patches/0006-initialise-unlinked-vertex-outputs.patch` (mx 2).
- Conditional clears on counter words; a timestamp report writes 0 in the counter word: `vk_exec_xfer.c`.
- CPU-filled textures without a flush (magenta cave walls): write-watched pool mirrors, `vk_memory.c` (`mem_written_pages`) and `vk_texture.c`. The same
path covers BotW's sky (tables memcpy'd into a CPU-cached pool, flushed, sampled from a zero texture: brown sky).
- Deferred texture binds read at execution (3D scene black): `nvn_api_cmd.c` `Bind*Deferred`.
- `nvnTextureCompare` compares the channel mapping (HUD prompt boxes, black stamina wheel): `nvn_api_obj.c` `nvnTextureCompare`.
- BC textures drawn through an RGBA16UI view (BC4 encoder): block-texel-view-compatible images, `vk_texture.c` `block_views_ok`.
- Fragment draws with no render target: `vk_exec.c` `resolve_rt`. Output types from attachment formats: `vk_pipeline.c`
`vk_pipeline_answers`.
- Block-height rounding: `nvn_tex.c`, the block-height pick (mx 15). Depth mode: `vk_pipeline.c` `create_graphics` (mx 16).
- Same-pass feedback sampling (magenta or dark 16x16 blocks, G15): feedback-loop layout and a by-region barrier, or a pass
restart without the extension (`vk_exec_draw.c` feedback loops, `TOTK_FEEDBACK`; mx 10).
- Cubes sampled as cube arrays, 1D textures, swizzles, sRGB views, scaled copies, targets outside the output map keep
texels: see the playbook's "Bug classes already fixed".
- `GetCommandMemoryUsed` follows the driver's sizes and resets at `AddCommandMemory`: `nvn_api_cmd.c` `nvnCommandBufferGetCommandMemoryUsed`. The game
sizes model heaps from it, so a larger host record crashes world loads.
- `nvnQueueFinish` waits for the GPU only: `nvn_api_core.c` `nvnQueueFinish`.
- Yield back-off: none (`kernel.c` SVC 0x0B); one that starts before 2 ms of yields drops TotK to 20 fps.
- Condition-variable waiters wake in arrival order within a priority: `kernel.c` (`kern_test_cv_old_order` restores the old order
for tests).
- The guest clock never freezes: `main.c` `rt_ticks`. Focus message 15 only under focus modes 1 and 2: `gfx_window.c`
`WM_ACTIVATEAPP`, `ov_system.c` `SetFocusHandlingMode`.
- Alias region and heap: `memory.c` (`HEAP_CAP`, `ALIAS_SIZE`). Adr-relative jump tables: `recomp/jumptables.py`.
- Scripted and recorded input by frame: `TOTK_INPUT_SCRIPT`, `TOTK_INPUT_RECORD`.
Not applicable to us: Metal and iOS limits (big-stack entry points, TBDR store order, encoder order, display-link rates,
efficiency cores), the NSO rewriter's retry blocks, `DT_RELR` (SDK 20 only), GC-title thread contexts, audio renderer
take-over (we serve `audren:u`).
docs/research/prior-art.md
-259
View File
@@ -1,259 +0,0 @@
# Prior art: Switch static recompilation and native ports (as of 2026-10-20)
Scope: prior art for porting TotK (0100F2C0115B6000, ARM64 NSO) to native Windows x86-64 by static recompilation.
Labels: **[confirmed]** = seen in a primary source (post JSON, repo, code). **[reported]** = press or secondary source only.
**[inference]** = my reading of the evidence.
## A. Another TotK PC port (announced 2026-10-15)
- **[reported]** A native PC build of TotK was shown on X on 2026-10-15: 32-59 fps on an RTX 4060 Ti, Vulkan, "worked
on for a couple of days". No game version, toolchain or source was published, and the screenshots were later
removed after a copyright report. Press: https://wccftech.com/zelda-tears-of-the-kingdom-unofficial-native-pc-port-sluggish-notrexed-release-2-4-weeks/
- **[inference]** The figures fit a static ARM64-to-C recompilation with an NVN layer on a portable graphics API. There
is nothing to reuse.
## B. Other projects
### Switch → native, static recompilation (closest prior art)
| Project | URL | Status | Lang / license | Relevance |
|---|---|---|---|---|
| **OpenSmash-Ultimate (ssbu-recomp)** | https://github.com/Denkishi/ssbu-recomp | Active (pushed 2026-10-20), 254 stars. SSBU 13.0.4/13.0.5 → **Windows x64**. Boots, plays full matches at 59-60 fps | C + Python tools, **GPL-3.0** | **Closest blueprint for our target.** Lifter `tools/lifter/recomp.py`: AArch64 → C, one C function per guest function, checked against Unicorn on 32,043 instruction cases. clang → game DLL plus one DLL per script module. Runtime: about 1,100 `nn*` entry points, threads, C++ exceptions through lifted code. NVN → Vulkan/GL (`nvn_vk*.c`). `mx_translate.c`: Maxwell shader → GLSL → SPIR-V. Pipelines prebuilt in background threads. Deterministic mode with a per-frame state hash. GPL: read for ideas; copying code makes our project GPL. |
| **LAS Recomp** | https://github.com/peiikai/LASRecomp (local: `refs/LASRecomp`) | Beta release, 1.37k stars. Link's Awakening v1.0.0 → Windows x64 (Vulkan), macOS (Metal), Linux | C runtime + Python lifter, **MIT** (own code) | **Most reusable code (MIT).** "Made almost exclusively with Claude Code." `tools/lifter.py` (728 lines): AArch64 → C; integer, memory and control flow in C; FP/SIMD lowered to C on x86 hosts by `simd_c.py` (1,156 lines, with `runtime/simd.h`). `gen.py`: whole-NSO generation; functions from `.eh_frame`; Unicorn for analysis; `--chunk`. `tools/test_lift.py` and `test_simd.py`. Runtime: `kernel.c`, `ipc.c`, `services/` (acc, am, audio, fs, hid, time), `nvn/` with a Maxwell translator, Vulkan backend (GLSL via shaderc), ASTC decode. Windows: guest window at `0x70_0000_0000` reserved with `VirtualAlloc`, committed on demand through a vectored exception handler. Toolchain: MSYS2 UCRT64 clang/lld; zig for building on the player's PC. Dev tooling worth copying: `perf_sweep.py` (warp to each room, per-room fps table), `shot.sh`, `frames.sh`, `explore_input.sh`, `fnprof.c`. |
| **SavageWindRecomp** | https://github.com/Revivedx/SavageWindRecomp (local: `refs/SavageWindRecomp`) | Alpha 2 (2026-10-21). Breath of the Wild **1.8.2** to Windows x64 and Linux, Vulkan. Great Plateau at 60 fps; nobody has played past it | C runtime + Python lifter, **MIT** (`LICENSE:1-4`: Revivedx, plus Peiikai for LAS Recomp) | LAS Recomp fork for the engine TotK descends from (sead, ksys, agl; NVN 55.15). Kernel core gate, NVN command recording, submission queue, frame pacing and Vulkan backend were reworked for BOTW; its dated bring-up log is the useful part. Findings: section E. |
| **suyu v0.0.4 "Recompiler mode"** | https://github.com/suyu-emu/suyu-v0.0.4 | Archived after the final release (Sep 2026). No published perf data; users report crashes | C++, **GPL-2.0+** | `src/core/recompiler/arm64_to_c.h` (4.5k lines, header-only): "AArch64 → portable C, N64Recomp-style, GuestContext. Every instruction translates or emits a runtime fallback." Bundled with the Eden/yuzu HLE plus GPU. Useful as a second AArch64→C reference and for its fallback design. Weakest of the three. Coverage: https://xenospectrum.com/en/suyu-static-recompiler-performance-evidence/ |
| **bgyss/SwitchRecomp** | https://github.com/bgyss/SwitchRecomp (local: `refs/SwitchRecomp`) | Specs plus scaffolding. Last push 2026-03, 88 stars | Rust; Cargo says MIT OR Apache-2.0 but the repo has no LICENSE file | See section C. Documentation value only. |
| Mario Kart 8 DX recomp | listed at https://allrecomps.com | "Limited race tests", Windows, no release | ? | Watch only. |
### Switch → Apple native ARM64 execution (no CPU translation; HLE and graphics are reusable)
These do not recompile. They patch the instructions macOS/iOS cannot run (x18, TPIDRRO, `svc`, counter registers) and
run the guest code directly. For an x86 port, the relevant parts are the **Horizon HLE host, the NVN reimplementation,
TotK-specific data and the bring-up playbooks**.
| Project | URL | Notes |
|---|---|---|
| **MacNativeGames** (TotK) | https://github.com/ly05010419/MacNativeGames (local: `refs/MacNativeGames`) | **MIT**, 2.06k stars, active. TotK **1.0.0, 1.1.2, 1.2.1, 1.4.3** on Apple silicon and iOS. Reported about 90 fps on M4 after shader warm-up. NVN → Metal 4 with dirty-page tracking. Shaders: Ryujinx `Ryujinx.Graphics.Shader` (patched, NativeAOT `totk_xlate`) → SPIR-V → SPIRV-Cross → MSL. Shared shader list: 26,832 variants. `docs/how-it-works.md` documents guest memory below 2^39 (IPC buffer addresses use 39 bits), the alias region, NVN block-height rounding (deko3d's heuristic is wrong at some sizes), TotK frame-pacing and 60 fps patches, and per-version expected hashes. **Best TotK-specific reference for HLE and NVN behaviour.** Press: https://www.notebookcheck.net/Zelda-Tears-of-the-Kingdom-Mac-port-runs-at-90-fps-topping-Switch-2-and-PC-rival.1427448.0.html |
| **nxnative** | https://github.com/seedds/nxnative (local: `refs/nxnative`) | **MIT**. 39 game profiles on 2026-10-22 (about 10 new a day), TotK 1.4.3 and BotW at 60 fps. `docs/agents/01..05` is an agent playbook for porting a new Switch title: extract, survey (SDK/NVN version, SVC and import census), bring-up failure catalogue, NVN graphics, debugging; `docs/development.md` has per-game findings. **Best source for project skills and symptom lessons**; what applies to us is in [port-lessons.md](port-lessons.md). The history was rewritten on 2026-10-22. |
| d2r-mobile | https://github.com/jhaizhou-ops/d2r-mobile (local: `refs/d2r-mobile`) | MIT. Origin of the NSO rewriter, Horizon host and NVN-on-Metal design. |
| totk-mobile | https://github.com/OJZen/totk-mobile | Apache-2.0, 73 stars. TotK 1.4.3 only, follows the d2r-mobile design. |
### Decompilation
| Project | URL | Notes |
|---|---|---|
| zeldaret/botw | https://github.com/zeldaret/botw | BotW 1.5.0 decomp, C++, no license file, active (2026-10-19), progress at https://botw.link/progress. Not near 100% (about 40 MB executable). Useful for TotK: shared EPD engine (sead, ksys, agl), so class layouts and names carry over to some extent. Matching decomp is not a viable route for TotK. |
### Design references from other consoles
| Project | URL | Notes |
|---|---|---|
| N64Recomp | https://github.com/N64Recomp/N64Recomp | MIT, 8.3k stars. Canonical "instruction → C over a context struct" model; the suyu engine copies it. |
| XenonRecomp / XenosRecomp | https://github.com/hedge-dev/XenonRecomp , https://github.com/hedge-dev/XenosRecomp | MIT. PPC → C++ recompiler plus Xenos shader → HLSL. Patterns: function-table dispatch for indirect calls, jump-table detection config, mid-asm hooks. UnleashedRecomp (GPL-3.0) is the full-game example. |
| ReXGlue SDK | https://github.com/rexglue/rexglue-sdk | 1.1k stars, active 2026-10. X360 recomp runtime built on a Xenia-derived runtime. Model for "one SDK, many titles". |
| Mario Kart Wiicompiled | https://www.dsogaming.com/news/unofficial-pc-port-of-mario-kart-wii-coming-in-august-2026/ | Wii (PPC) static recomp, AI-assisted, beta 2026-08. |
### Emulators (HLE and GPU references; most are under DMCA pressure)
| Project | URL | Notes |
|---|---|---|
| Ryujinx / Ryubing | GitHub `Ryubing/Ryujinx` returns 404; `ryujinx-mirror` was blocked by DMCA on 2025-03-14. Ryubing's own Git host (not verified) | MIT C#. `Ryujinx.Graphics.Shader` (Maxwell → SPIR-V/GLSL) is the shader translator MacNativeGames and nxnative ship. HLE services are a semantics reference. Reported to be in maintenance mode in 2026. |
| Eden | https://git.eden-emu.dev/eden-emu/eden | yuzu descendant, GPL. Reported as the most active fork; suyu 0.0.4 is based on it. Hit by DMCA in Feb 2026 ([reported]). |
| Citron, Sudachi | (off GitHub) | Citron is alive off GitHub; Sudachi is mostly dead ([reported], low-quality sources such as https://tech-insider.org/eden-vs-ryubing-best-switch-emulator-2026/). |
| dynarmic | https://github.com/lioncash/dynarmic (archived 2024), forks https://github.com/azahar-emu/dynarmic and in-tree in Eden | 0BSD. A64 decoder tables and IR are a semantics reference for the lifter (flags, FP/SIMD corner cases). |
| oaknut | https://github.com/merryhime/oaknut | MIT, archived 2024. AArch64 emitter/encoder, not a decoder. Low relevance. |
### Generic lifters
| Project | URL | Notes |
|---|---|---|
| remill | https://github.com/lifting-bits/remill | Apache-2.0, active 2026-08. AArch64 → LLVM IR. Per-instruction semantics as C++ functions. Candidate for an LLVM-IR route or as a semantics oracle. Its AArch64 SIMD coverage is incomplete (from memory, not checked against the current tree). |
| mcsema | https://github.com/lifting-bits/mcsema | AGPL-3.0, archived 2022. Do not use. |
| RetDec | https://github.com/avast/retdec | MIT, LLVM decompiler. Analysis aid only. |
| Unicorn | https://www.unicorn-engine.org | Differential-test oracle; LASRecomp and ssbu-recomp both use it. |
### What the field converged on (all three working Switch recomps)
1. AArch64 → **C** (not LLVM IR). One C function per guest function, chunked source files, compiled by clang.
2. Function discovery from `.eh_frame`, `bl` targets and relocations. Indirect calls go through a dispatch table
keyed by guest address.
3. Guest memory at its original addresses, reserved up front (`0x70_0000_0000` window). Addresses must stay below 2^39.
4. Per-instruction differential testing against Unicorn.
5. HLE of `nn::` services and SVCs in the host runtime, not kernel emulation.
6. NVN reimplemented at the API level. Maxwell shaders translated at load time (own translator or Ryujinx's).
Pipelines prewarmed.
7. Dev loop: scripted input, warp sweeps, per-scene fps tables, frame dumps, function profiler.
## C. refs/SwitchRecomp assessment
Layout: `specs/` (35 draft specs SPEC-000..270), `docs/` (35 notes and schemas), `crates/` (8 crates, about 15.9k lines
of Rust including tests), `skills/` (7 Codex skills), `scripts/` (capture, ffmpeg, Ghidra export, AWS). There is one
commit in the local clone, dated 2026-02-17.
### Crates: usable? No.
Not compiled here: `cargo` is not installed and installing it was out of scope. Dependencies are pure Rust (clap,
serde, toml, sha2, lz4_flex, thiserror), so it would most likely build and its tests would pass, but nothing in it does
real work.
- `recomp-isa`: a toy interpreter covering 12 ops (`MovImm`, `Add`, `Sub`, `Cmp`, shifts, `Ldr/StrImm`, `Ret`) with
registers in a `BTreeMap`.
- `recomp-pipeline/src/homebrew/lift.rs` (1.2k lines): decodes about 15 integer encoding classes (movz/movk, adr,
add/sub imm/reg, logical reg, ldr/str unsigned imm, b, bl, b.cond, cbz/cbnz, tbz/tbnz, br, ret, nop). It has no
LDP/STP, pre/post-index, FP/SIMD, exclusives or atomics, system registers or `svc` lowering. That covers a negligible
fraction of a real game's .text.
- Emitter (`output.rs`) writes a Rust crate with **per-function local registers** (`let mut x0: i64 = 0`) and a
`match block_label` string dispatch. Registers do not cross calls, so the output is semantically wrong for real ABI
code and slow.
- `recomp-gfx`: `CommandStream` + `GraphicsBackend` trait + `StubBackend` + FNV checksum. No NVN or Maxwell.
- `recomp-services`: a name → closure registry with allow-list and stub behaviours. No IPC or CMIF.
- `recomp-runtime`: flat memory regions, `mem_load/store_*`, syscall log, noop and panic.
- XCI intake shells out to hactool/hactoolnet. Fixtures are fake (`main.nso` is 5 bytes).
- Verdict: do not depend on it. LASRecomp (MIT) is a far better code base to learn from or borrow.
### Specs and docs: limited value
- Process and validation specs (provenance, A/V validation, run manifests, automation loop, agent pipeline security)
are coherent but generic and were written before any real title booted.
- `docs/llm-assisted-decompilation-long-tail.md` summarizes https://blog.chrislewis.au/the-long-tail-of-llm-assisted-decompilation/
(similarity-guided retrieval, lanes for gfx/math, hook-enforced guardrails, parallel worktrees). Worth reading if we
run an agent loop over functions.
- `docs/LEGAL-POLICY.md` and the provenance idea (sha256 of user inputs, inputs kept separate from outputs) are worth
copying in spirit.
### Skills: adapt one, rewrite the rest from better sources
| Skill | Content | Verdict |
|---|---|---|
| static-recomp-av-compare | SKILL.md + 3 real Python scripts (565 lines): ffmpeg normalize/align, SSIM, PSNR, optional VMAF, EBU R128 loudness, batch manifest, threshold check | **Adapt.** The only skill with working code. Port it to a Claude Code skill for "emulator reference capture vs our build" comparisons. Check the license first (no LICENSE file in the repo). |
| static-recomp-input-replay | Checklist | Concept is needed; the content is generic. Write our own around a frame-indexed input trace fed to both Ryujinx/Eden and our runtime. |
| static-recomp-regression-triage | Checklist mapping symptoms → loader / ISA / services / GPU / audio | Rewrite using nxnative `docs/agents/03-bringup.md` and `05-debugging.md`, which are concrete failure catalogues. |
| static-recomp-perf-profile | Checklist | Rewrite around a LASRecomp `perf_sweep.py`-style warp sweep plus a function profiler. |
| static-recomp-reference-capture | Checklist | Fold into av-compare. |
| static-recomp-scope-plan, static-recomp-batch-harness | Multi-title catalogue planning | Skip. We have one title. |
Better skill sources: nxnative `docs/agents/` (extract → survey → bring-up → graphics → debugging), LASRecomp
`tools/` (test_lift, test_simd, perf_sweep, shot, frames, explore_input), and ssbu-recomp's Unicorn differential tests
and deterministic state-hash mode.
## Takeaways for this project
1. The other TotK PC port (section A): no code, and its screenshots were taken down. Proof of feasibility, not a dependency.
Its stack (MSVC link + wgpu) is plausible but unconfirmed.
2. Base the lifter design on LASRecomp (MIT, AArch64→C, SIMD→C for x86) and validate it the way ssbu-recomp does
(Unicorn differential tests). Read suyu's `arm64_to_c.h` for its fallback strategy.
3. Base HLE and NVN behaviour for TotK on MacNativeGames (MIT): per-version hashes, memory layout constraints, NVN
texture layout fixes, frame-pacing patches, shader-variant list.
4. Shaders: Ryujinx `Ryujinx.Graphics.Shader` (MIT) → SPIR-V is the proven route. LASRecomp and ssbu-recomp wrote
their own Maxwell → GLSL translators.
5. SwitchRecomp: reuse the av-compare scripts and the legal/provenance policy only.
## D. Performance techniques in other Switch recomps (2026-10-21)
Clones read locally (not in `refs/`): HWDER (https://github.com/sonsegajp/HWDER, MIT), B2recomp
(https://github.com/K0NL/B2recomp, GPL-3.0; its runtime is a copy of ssbu-recomp), SuperMarioWonderRecomp
(https://github.com/BelmanteGu/SuperMarioWonderRecomp, MIT), ssbu-recomp (GPL-3.0). GPL code: read only.
1. FPCR: all four ignore it and run FP on host SSE in the default mode. HWDER stores FPCR and never applies it
(`tools/recomp/a64.py:371,394`); B2recomp and ssbu drop FPCR writes (`lifter/recomp.py:269-271`,
`tools/lifter/recomp.py:287-290`); Wonder documents FPCR as "not honoured" (`docs/RECOMP.md:31,71`). None handles
a game that sets FZ, as TotK does. Exactness work they keep: NaN sign, FRECPE/FRSQRTE (B2recomp `runtime/fpexact.h`).
2. Cores: HWDER and Wonder run guest threads free, one host thread each, no preemption model; GetCurrentProcessorNumber
returns the ideal core or the lowest mask bit (HWDER `svc.cpp:157-170`, Wonder `runtime/src/kernel.cpp:528-538`).
ssbu and B2recomp have an opt-in 3-core pinning model, shipped off because it costs frame rate
(`os_threads.c:143-156`). TotK crashes without our core gate (`TOTK_FREE_CORES=1`), so the gate stays.
3. Waits: ssbu sleeps with a high-resolution waitable timer (`os_threads.c:104-122`); Wonder removed its yield nap
(it cost the main thread about 18%, `kernel.cpp:81-96`).
4. Codegen: HWDER clang-cl /O2 /arch:AVX2 -mfma, one dense dispatch table per module with a musttail call
(`core.cpp:26-58,131-138`); Wonder -O2 with `-ffp-contract=off`; B2recomp `-ffp-contract=off`, `optnone` on
functions over 5000 statements, and a register-caching pass measured at no gain under -O2 (`regcache.py:1-30`);
ssbu stays at -O1 because huge lifted functions take over an hour at -O2 (`tools/lifter/build.py:55-68`). No LTO
or PGO anywhere.
5. Pipelines: HWDER compiles on worker threads and skips the draw until ready (`vk_renderer.cpp:1391-1430`); ssbu
waits by default, but its workers precompile predicted keys and replay `pipes.bin` plus a VkPipelineCache at start
(`nvn_vk_draw.c:4037-4062`); B2recomp prewarms from a saved list (`nvn_vk_m4.c:1702-1759`); Wonder's canonical
pipeline key removed 18% of compiles (`docs/nvn/android-perf/PIPELINES-1810.md:15-20`).
6. GPU to CPU data: ssbu copies idle compute buffers on the GPU into host memory and merges them into guest memory when
the submit completes, avoiding 19-20 ms reads of write-combined memory (`nvn_vk_draw.c:1940-1960`); Wonder splits
the submit after the last dispatch (`runtime/src/nvn/nvn_frames.cpp:11-19`).
7. Textures: ASTC on the CPU everywhere (Wonder multi-threaded, `runtime/src/nvn/nvn_astc.cpp`); HWDER detects dirty
textures by content hash with a 48 MB per-epoch budget (`texture_cache.cpp:322-334`).
8. Pacing: HWDER mailbox present plus its own 60 Hz limiter, swap interval 2 paced two slots ahead
(`display.cpp:160-163,317-334`); Wonder prefers MAILBOX; B2recomp patches 60-120 fps at lift time from a census of
frame-rate sites (`lifter/fps_ticks.txt`).
## E. SavageWindRecomp findings (2026-10-23, `refs/SavageWindRecomp` @ `575ad88`)
MIT (`LICENSE:1-4`), pinned in `tools/fetch_refs.sh`. 430 files, 8 commits, one week old. Its notes (`docs/SAVAGE-WIND.md`, Spanish, 904 lines, cited
as `SW.md:line` = first line of the entry) are a dated log of BOTW 1.8.2 bring-up fixes. Most are bugs TotK 1.2.1 hits too. Paths are relative to
`refs/SavageWindRecomp/`; our code is cited by file and function, and the Ours column was updated 2026-10-26. BOTW
addresses do not carry over to TotK; the mechanisms do.
| Area | Theirs | Ours | Differs |
|---|---|---|---|
| Cores | 4 tokens, ticket queue per core (`runtime/kernel.c:96-235`); mask used only if single-bit (`kernel.c:867-880`) | 3 cores, mask-aware grant, preempt flag, asynchronous preemption of a holder in game code (ours: `kernel.c` `core_take`, `kern_ap_stub`) | Yes |
| Preemption points | SVCs plus every indirect call (`runtime/dispatch.c:167`) | SVCs and every indirect call (ours: `svc.c`, `dispatch.c` `dispatch_preempt`), plus the waiter's interrupt | No |
| Exclusive monitor | Value CAS (`runtime/cpu.h:73-97`); pairs unsupported (`tools/lifter.py:391-392`) | Same macro plus 128-bit pair (ours: `cpu.h` `EXCL`) | No (ours is a superset) |
| Call/CopyCommands | Record `CMD_CALL`, executed recursively at submit (`runtime/nvn/cmdbuf.c:687-688`, `runtime/nvn/vulkan/exec.c:4689-4695`) | Inlined at record time (ours: `nvn_api_cmd.c` `inline_recording`) | Yes |
| UpdateUniformBuffer | Applied at execution into guest memory, skipped if bytes equal (`exec.c:4863-4871`) | Recorded, written at replay (ours: `nvn_api_cmd.c` `nvnCommandBufferUpdateUniformBuffer`) | Partly |
| Submit | Host queue thread, part-wise GPU submit (`runtime/nvn/objects.c:157-170`, `exec.c:1015-1017`) | Replay on the submitting thread, submitted in parts every 64 draws; a queued replay thread behind `TOTK_REPLAY_ASYNC=1` (ours: `nvn_api_core.c` `nvnQueueSubmitCommands`, `vk_backend.c`) | Partly |
| Frame pacing | Patches VFR and the vsync wait (`tools/patches.py:186-223`) | MacNativeGames' present-interval and real-time frame-step patches, back-edge present pacing (ours: `frame_rate.c`, `vk_backend.c`) | Yes |
### a. Threads
- Core gate like ours: a token per core, best priority then ticket order (`kernel.c:96-165`). Reason given: BOTW shares a sead heap without a lock between threads pinned to one core (`kernel.c:97-103`).
- Four tokens, indexed `core & 3` (`kernel.c:150`); a multi-bit affinity mask is not honoured, only a single-bit mask or the ideal core (`kernel.c:876`). `-1`/`-2` land on core 0 (`kernel.c:388`).
Ours takes any free allowed core and keeps the ideal core for -2/-3, closer to Horizon.
- Preemption at indirect calls: `rt_dispatch` compares `g_core_best_wait[core]` with the running priority and yields (`dispatch.c:167`, `kernel.c:120-130, 220-229`).
Reason: a prio-48 job thread ran about 5 ms of guest code without a syscall once a second while a prio-44 worker on the same core, which the main thread waited for, sat waiting (a frame 10 ms longer each second) (`SW.md:540`).
Measured p99 frame time 23.6-24.8 ms without, 17.4-20.6 ms with (`SW.md:553`). Ours has the same indirect-call check (`dispatch.c` `dispatch_preempt`).
- Host work releases the core: the submit and present path calls `kernel_host_work_begin/end` (`kernel.c:196-205`, `runtime/nvn/objects.c:132, 245-249, 292-294`). Reason: the translation took about 10 ms a frame on the core token (`SW.md:295`).
- Yield and `SleepThread(0)`: release then reacquire, so same-priority waiters go first (`kernel.c:231-236, 838`). `SignalProcessWideKey` and `ArbitrateUnlock` yield after unlocking, not before (`kernel.c:735-741`, `SW.md:256`).
- `svcWaitProcessWideKeyAtomic`: releasing the mutex before marking the condvar hung half the runs (`SW.md:224`); a timed-out wait returns without the mutex, and a timeout racing a wake does not count as expired (`SW.md:253`). Same family as port-lessons S6.
### b. Exclusive monitor
- Value CAS on the address recorded by LDXR, identical to ours (`runtime/cpu.h:73-97`). No ABA protection on either side. Nothing to adopt.
### c. NVN command buffers
- The game sees command memory accounted at an estimated NVN cost per command, `cmd_cost` (`cmdbuf.c:15-28`); records live in host "shadow" blocks keyed by guest memory.
Reason: BOTW records each display list once to measure it, then allocates exactly that from a heap sized by the ResourceSizeTable for real NVN (`SW.md:201`).
- A list dies only when a recording writes over its guest memory (`shadow_claim`), not when the game adds an overlapping window. The old rule lost the deferred-lighting material lists, so no light was composed (`SW.md:233`). Dead blocks are reused three frames later (`cmdbuf.c:25-26`).
- Textures in records are resolved at record time (`nvn_texture_host`, `cmdbuf.c:442-455`). The exposure chain (192x192 down to 3x3) re-inits one `NVNtexture` per step inside a command buffer; resolved at execution, every step drew into the last (`SW.md:242`).
- One Vulkan image per (memory, shape): `NVNtexture` objects on the same memory and shape share an image (copies, finalized originals still in the pool, re-created render targets) (`SW.md:220, 245`; `runtime/nvn/vulkan/textures.c`).
- `NVNsampler` objects are copied by value and the copy is registered after the original is finalized. Descriptions are shared per configuration and never freed; otherwise the Shrine corridor texture clamped instead of repeating (`SW.md:261`).
- Draws resolve at execution on the queue thread. UBO ranges are copied at the draw, shared per list, and `UpdateUniformBuffer` (about 450 a frame into about 40 ranges) invalidates only the copies over its range; identical rewrites (about 1 in 5) are skipped (`exec.c:471-503, 2415-2430, 4863-4871`; `SW.md:535, 565`).
- `BindTexturesDeferred` was a no-op and dropped the lighting composite's textures; the other `Bind*Deferred` entry points are still unimplemented there (`SW.md:240, 278`).
### d. Engine special cases
- Few guest patches: 17 in `tools/patches.py:209-223`, 15 for their own options screen. The two for us: the sead vsync wait `0x7000d811fc` (`patches.py:186-203`) and the `ksys::VFR::updateInterval` ratio load `0x70014092f4` (`patches.py:206-210`).
TotK equivalents are in macnativegames.md (present-interval controller, lines 438-446).
- Frame pacing (`runtime/hle/framepace.c`, `runtime/framepace.h`, `SW.md:430`): presents on a fixed grid; the game's clock is told the real frame length through `rt_vfr_update` (ratio = seconds x 30, at most 3 % change per frame, at most 1/20 s per step, debt paid back over later frames) (`SW.md:454`);
integer interval with hysteresis (1 above 43 fps, 2 below 35, 3 below 23); menus, events and movies pinned to 30 fps (`SW.md:443`); controller hold counters stepped by real time.
Havok, stamina, time of day and animation all read the ratio; a ratio that alternated made Link's camera judder.
- Dynamic resolution: BOTW picks 1600x900 or 1280x720 from GPU time in `ReportCounter`; their translation time dropped the open world to 720p. Fix: every counter in a frame gets the same timestamp (`exec.c:4890-4902`, `SW.md:414`).
- Render-thread layout: the main thread records about 30 of 62 lists per frame, two workers the rest; the sead render thread (`0x7000d81100`) submits one list a frame and presents (`SW.md:503`).
- FPCR.FZ is set at thread start (`0x7000b4de30`, `0x70000c4e3c`). Software flushing cost about 40 % of main-thread time; they mirror FZ into MXCSR (`runtime/simd.h:53-58, 239`; `SW.md:516`). Ours already mirrors it (ours: `cpu.h` `fpcr_host`).
- Depth clamp: BOTW enables it for the three sun-shadow cascades and the sun query. Ignored, geometry nearer than a cascade's near plane was clipped; tree shadows appeared, were cut and moved with the camera (`SW.md:593`). Ours records and applies it (ours: `nvn_api_cmd.c` `nvnCommandBufferSetDepthClamp`, `vk_pipeline.c` `depthClampEnable`).
`SAMPLES_PASSED` returns 0 and `SetRenderEnableConditional`/`SetRasterizerDiscard` are no-ops (`SW.md:593`).
- Same-pass feedback: the terrain fragment shader ORs bits into the G-buffer's own pixel, about 240 draws a frame. Copying the 1600x900 target before each draw cost 8.5 of 16 ms GPU. They use `VK_EXT_attachment_feedback_loop_layout` plus `VK_KHR_dynamic_rendering_local_read` with an in-pass barrier: copies 8.5 to 0.4 ms (`SW.md:325`).
Ours does the same since 2026-10-23 (`TOTK_FEEDBACK`, [maxwell.md](maxwell.md) check 10); TotK has 425-667 such draws.
- Submit in parts: a pass boundary about every 300 draws (`exec.c:1015-1017`, `SW.md:302`), except after a shader storing to guest memory (about 10 dispatches a frame). Ranges those storage buffers can write are tracked; a UBO inside them is read in place from the imported heap, and any other CPU copy of that memory waits for the GPU (`SW.md:524`). Holding the rest of the list caused device-lost in 1 of 7 loads when copying half-written data.
- Present on its own thread with a second queue; NVIDIA `vkQueuePresentKHR` blocked on the GPU for 13-18 ms in one frame of five (`runtime/nvn/vulkan/device.c:563-593`, `SW.md:306`).
- Shaders and pipelines: translation cached in memory and on disk (`cache/xlate/`, `SW.md:309`); `vk_pipelines.bin` with a GPU/driver/translator header plus `vk_pipelines.list`, replayed by worker threads at start-up, first world frame waits for it (`SW.md:337`, `device.c:310-384`).
Scripted walk, render-thread compile time 5898/865/362/449 ms to 16/1/32 ms, frames over 40 ms 56/51/9/11 to 24/4/5. The pipeline key held colour formats and blend bytes of unused targets, giving about 1,500 pipelines where about 870 were needed (`SW.md:364`).
- Synchronous shader compile by default: a draw skipped while its pipeline builds is lost for good if the game draws it once (save thumbnail) (`SW.md:657`).
- Targets are not cleared per frame (BOTW keeps data in them between frames); new images start at zero (`SW.md:409`).
### e. Known glitches
Issue list (`gh issue list -R Revivedx/SavageWindRecomp --state all`, 23 issues, 2 closed, none with a technical fix thread for graphics):
- #16 trees "vertex explosions" at some distances or angles, 4K, no upscaler. Open; a collaborator replies "almost fixed". No cause stated. Closest match to one-frame popping.
- #20 distant LODs not rendering fully. Open, no analysis. #26 and #5 "texture errors" and "graphic errors" (RX 570 and unknown GPU). Open, logs requested.
- #23 ROG Ally: 114-700 ms frames on transitions and shader generation, up to 250,000 draws reported per frame, VRAM use 0.4-1.4 GB. Open. Matches their own note that a first-use pipeline costs 40-100 ms (`SW.md:386`).
- #31 30 fps in menus, chests, cutscenes: by design (`framepace.h:15-17`). #30 asks to precompile all shaders; the pipeline list is their answer.
- Crashes with a stated cause: #2 voice banks swapped in are larger than the ResourceSizeTable entry, so a later heap allocation fails; #6 `movie.c` vtable slot 2 is a stub, crash at the first tower cutscene.
Fixed in `SW.md` and relevant to TotK: stretched texture from sampler copies (`SW.md:261`), blocky image from re-created render targets (`SW.md:220`), black cube arrays (`SW.md:248`), moving tree shadows from depth clamp (`SW.md:593`), missing lighting from dead command lists (`SW.md:233`).
Still open in their log: a differently tinted rectangle top-left from half-resolution fog passes (`SW.md:271`), a 32x32x32 BC4 3D texture sampled as 2D returning black (`SW.md:276`), an ambient palette that fades interior to exterior over 60 s (`SW.md:606`).
Nothing in the issues or the log describes objects teleporting for a frame, or a wrong-LOD mesh, with a cause.
### f. Linux platform layer
- `runtime/platform.h` is the split: `plat_reserve_guest`, `plat_commit`, `plat_reset_guest`, `plat_protect`, `plat_now_ns`, `plat_sleep_until_ns`, spawn, symbols, crash handler (`platform.h:16-65`). Implementations: `runtime/platform/{windows,linux,macos}.c`, with `win_compat.h` and `linux_compat.h` filling POSIX gaps.
- Memory: Linux maps the whole guest window once, `MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE|MAP_FIXED_NOREPLACE` (`platform/linux.c:29-33`). Windows reserves with `VirtualAlloc` and commits in chunks from a vectored handler (`platform/windows.c:20-27, 71-76`).
- Write watching is a debug tool: page `PAGE_NOACCESS` plus single-step (`windows.c:30-58`, `runtime/kernel.c:1150-1160`). The renderer does not track dirty pages; it copies guest memory when the draw runs.
- Threads are pthreads on both (winpthreads on mingw). Timers: `clock_nanosleep(TIMER_ABSTIME)` on Linux (`linux.c:54-60`); a high-resolution waitable timer plus `timeBeginPeriod(1)` on Windows (`windows.c:108-125, 205`).
- Linux needs a 4 KB or 16 KB page kernel (`docs/linux.md`).
### What to adopt, ranked by expected effect on glitches and stutter
Status 2026-10-26: 1 and 2 are in, 3 is in part (part-wise submit; the queue is off by default), the feedback-loop
layout is in.
1. Preemption at indirect calls in `rt_dispatch`, plus the host-work core release. Small change, removes a periodic 5-10 ms frame spike they measured (p99 23.6-24.8 to 17.4-20.6 ms). Needs TotK confirmation with our sched trace.
2. Persistent pipeline list pre-warmed at start-up, with a key that holds only the used targets. Removes the 40-100 ms first-use hitches that read as stutter (frames over 40 ms 56 to 5 on their walk).
3. Host submission queue with part-wise GPU submit (about 300 draws) and the guest-memory-write tracking for storing shaders. Their p99 went 38-43 ms to 29-34 ms with queue, MXCSR FZ and range tracking together (`SW.md:550-552`).
Next: feedback-loop layout instead of target copies, one image per (memory, shape), UBO copy sharing with per-range invalidation, equal-timestamp counters for dynamic resolution.
docs/research/totk-v121-inventory.md
-102
View File
@@ -1,102 +0,0 @@
# TotK 1.2.1 module inventory
Numbers measured on a retail 1.0.0 base plus the 1.2.1 update (`v393216`), extracted with `tools/extract.py` on 2026-10-20. Counts only; no game data.
## Input verification
| | Result |
|---|---|
| Base ExeFS (`main`, `sdk`, `subsdk0`, `rtld`) | match `expected-v1.0.0.json` |
| Base RomFS | 17,069,139,640 bytes, 304,207 files, SHA-256 matches; 1,041,818 level-5 IVFC blocks verified |
| 1.2.1 ExeFS including `main.npdm` | match `expected-v1.2.1.json`; ACID signature verifies |
| 1.2.1 RomFS (BKTR merge) | 17,052,047,360 bytes, 304,473 files, SHA-256 matches |
| MacNativeGames rewrite of `main`, `sdk`, `subsdk0` | output hashes equal `rewritten_sha256` in `expected-v1.2.1.json` |
The tested base dump's `main.npdm` differs from the reference only in bytes `0x80-0x27F` (ACID signature and ACID public key), so its ACID signature no longer verifies. A repacking tool that re-signs the NCA header does this. Code and RomFS are unaffected.
Timing on the development machine (NVMe): ExeFS under 1 s each, base RomFS 40 s, update merge with NCA copies 1 min 50 s.
## Modules
`image` is the loaded size (all segments plus `.bss`, page aligned).
| Module | image | `.text` | `.rodata` | `.data` | `.bss` | relocations | imports | exports |
|---|---|---|---|---|---|---|---|---|
| `main` | 74,625,024 | 45,193,808 | 23,658,396 | 5,006,256 | 765,008 | 510,904 | 675 | 29 |
| `sdk` | 14,520,320 | 6,161,120 | 7,150,872 | 429,712 | 774,512 | 35,032 | 13 | 27,747 |
| `subsdk0` | 6,983,680 | 3,424,448 | 2,202,004 | 199,416 | 1,152,264 | 19,397 | 667 | 10,954 |
`main` exceeds LAS Recomp's 64 MB module limit (see [lasrecomp.md](lasrecomp.md), "Scaling risks").
## Functions and instructions
LAS Recomp `tools/analyze.py --census` (seeds: `.eh_frame_hdr`, exported FUNC symbols, `R_AARCH64_RELATIVE` targets in `.text`, `BL` targets; each start is credited to the first source that finds it).
| Module | functions | eh | sym | ptr | bl | instructions | undecodable words | mnemonics | time |
|---|---|---|---|---|---|---|---|---|---|
| `main` | 247,495 | 246,909 | 0 | 3 | 583 | 11,298,063 | 389 | 259 | 15 s |
| `sdk` | 38,305 | 31,238 | 36 | 8 | 7,023 | 1,539,998 | 282 | 362 | 3 s |
| `subsdk0` | 13,690 | 10,061 | 15 | 2 | 3,612 | 855,798 | 314 | 178 | 1 s |
| total | 299,490 | | | | | 13,693,859 | | | |
About 2.5-3 times *Link's Awakening* (4-6 M instructions). `main` has unwind info for nearly every function, so `.eh_frame_hdr` alone gives a near-complete function list.
Instructions the lifter must cover, by module:
- Exclusive pairs (`LDXP`, `STXP`, `LDAXP`, `STLXP`): `sdk` only (26 / 38 / 50 / 63). `main` has none.
- LSE atomics (`CAS*`, `CASP*`, `SWP*`, `LDADD*`, `LDCLR*`, `LDEOR*`, `LDSET*`): `sdk` only, 1-2 of each variant, likely behind a CPU feature check.
- Crypto: `sdk` uses AES (`AESE`, `AESD`, `AESMC`, `AESIMC`), SHA-1 and SHA-256 instructions. CRC32: `main` 7,710, mostly `CRC32B`.
- Exclusive single (`LDXR`/`STXR`): `main` 15,821 / 16,308; acquire/release forms in all three.
- System: `SVC` 60 (`sdk`), `MRS` 865 / `MSR` 53 (`main`), `MRS` 181 / `MSR` 13 (`sdk`).
- Indirect branches: `BLR` 105,284 and `BR` 26,426 in `main`.
- SIMD: `FMLA` 24,520, `TBL` 2,327, `LD1`-`LD4`/`ST1`-`ST4` in `main`.
## Lifter coverage
`tests/diff/lift.py --coverage` lifts every distinct word of `.text` (`recomp/lifter.py`, portable FP/SIMD lowering). Every word is lifted except those that become an `rt_udf` trap:
| Module | distinct words | trapped words (distinct) | of which `UDF` | undecodable data (capstone `?`) | other |
|---|---|---|---|---|---|
| `main` | 1,393,284 | 28,266 (10) | 27,873 | 389 | 4: the module header at `main+0x0`-`0x20`, which capstone decodes as SVE |
| `sdk` | 313,813 | 22,494 (9) | 22,212 | 282 | 0 |
| `subsdk0` | 233,022 | 5,858 (8) | 5,545 | 313 | 0 |
Upstream LAS Recomp rejects 277 `sdk` words: the exclusive pairs and the LSE atomics. Both were added (`excl_pair`, `cas`, `casp`, `atomic` in the lifter; `ldxp128`/`stxp128` and `atomic_[su]{max,min}N` in `runtime/cpu.h`).
Differential test results (2026-10-20, seed 1):
| Test | Encodings | Cases | Failing |
|---|---|---|---|
| `lift.py` (all opcode signatures of the three modules, then random words; 20,000 per module) | 60,000 | 239,948 | 0 |
| `simd.py` (1,048,640 FP/SIMD encodings: lifter and Unicorn accept the same 270,299) | 20,000 | 159,976 | 0 |
| `atomic.py` (exclusive sequences, LSE, CAS, CASP; every variant) | 198 | 12,672 | 0 |
Unicorn 2.1.4 (QEMU 5.0) compares `LDSMAX`/`LDSMIN` on bytes, halfwords and words as unsigned. `atomic.py` checks those against a Python model of the ARM pseudocode instead. The game does not use them.
## Jump tables
`tests/jumptables_check.py` runs `recomp/jumptables.py` on every `br` and compares it with LAS Recomp's Unicorn method. Functions are `[seed, next seed)` over the module's seeds plus `BL` targets (2026-10-20, `.venv`, 36 s wall for all three modules):
| Module | `br` | computed | resolved | entries | oracle resolved | agree | oracle wrong | computed, unresolved | PLT stubs (not computed) |
|---|---|---|---|---|---|---|---|---|---|
| `main` | 26,426 | 2,242 | 2,219 | 16,657 | 1,771 | 1,758 | 12 + 1 | 22 | 642 |
| `sdk` | 11,457 | 707 | 694 | 5,330 | 586 | 585 | 0 | 12 (+1 only the oracle resolves) | 8,933 |
| `subsdk0` | 4,465 | 339 | 339 | 2,529 | 312 | 312 | 0 | 0 | 3,925 |
- "Oracle wrong" carries independent evidence: the table base the oracle replays is not the constant every reaching definition gives (8 sites; the register is reused for other values, e.g. `x3` at `main+0x7c6fa8`), or the table index comes from a register or stack slot the oracle does not set (5 sites, e.g. `and w9, w1, #3` at `main+0x3d358`, a stack reload at `main+0xce2af4`).
- Every hoisted table base (482 in `main`, 86 in `sdk`, 30 in `subsdk0`) is confirmed by a capstone reaching-definitions pass.
- Left unresolved, by pattern: index `(x & 7) - 1` with no check (an unreachable default; 17 sites in `main`, one templated function family), index from an argument, a field or a stack slot without a visible check (`main+0x832f0`, `+0xcd7b40`, `+0xce2f38`, `+0x1386f24`, `sdk+0x10ec9c`, `+0x10f7e4`, `+0x10fd90`, `+0x11f2f0`), index from a bit insert of an unknown 32-bit value (`main+0x17d61c`), pointer difference in the `memchr`-style tails (`sdk+0xff1d0` and five more), a 64-bit index reloaded from the stack and compared as 32-bit (`sdk+0x32a518`, `sdk+0x591a4`), and a thunk that jumps to the next function through a constant (`sdk+0x56f4f4`).
## OS touch-points (MacNativeGames `patch_nso.py`, `DM_RULESET=native`)
Site kinds from `sites.tsv` (`x18` = platform register uses, `W` = exclusive windows, `J` = jump tables, `C` = call window):
| Module | x18 | tpidr_r | tpidrro_r | tpidr_w | cntpct | svc | W | J | C |
|---|---|---|---|---|---|---|---|---|---|
| `main` | 30,577 | 813 | | | | | 84 | 19 | 1 |
| `sdk` | 12,731 | 2 | 123 | 3 | 40 | 60 | 5 | 6 | |
| `subsdk0` | 10,201 | | | | | | | 2 | |
Rewriter statistics for `main`: 29,716 plain, 610 conditional, 232 PC-relative, 19 `br`, 813 TLS reads, 84 exclusive windows, 1 call window; 31,475 slot references. Runtime: `main` 77 s, `sdk` 12 s, `subsdk0` 7 s.
`patch_nso.py` prints Chinese text at the end of the `main` run; on a non-UTF-8 Windows console run it with `python -X utf8`.
docs/research/windows-host.md
-83
View File
@@ -1,83 +0,0 @@
# Windows host behaviour
What the Windows host does differently from the console and from MacNativeGames' macOS host, measured on machine A
(Ryzen 9 7900X, RX 9070, NVMe) in 2026-10. Each entry: the behaviour, the numbers, where the runtime handles it, and
the rule for new code. Linux equivalents of the fixes: [backlog.md](../backlog.md) item 35.
## Rules
- The game threads (guest main, job workers) and the render thread never open, read, write or flush a file, and
never wait for a thread that does. Move file I/O to a background thread and hand results over in memory.
- Short waits spin; waits over 1 ms use the high-resolution timer.
- Every frame-time claim needs three runs per build: single runs of one recording differ by up to three times.
## Timers and time
- **Waitable timer overshoot.** A high-resolution waitable timer ends a wait 0.3-0.5 ms late at best: the SDK's
20 us polling sleeps (main thread and job workers, about 1.5 per frame each) took 460-515 us. `kernel.c`
`sleep_ns` spins on the performance counter up to `SLEEP_SPIN_NS` (1 ms), the virtual core already given up.
macOS `nanosleep` is accurate, so MacNativeGames never had this. Measure with `TOTK_TRACE=sched` ("sleep asked /
took" per thread).
- **A 1 ms timer thread does not help.** A run with a thread waking every 1 ms (the sampler of `TOTK_PROFILE_MAIN`)
had more slow frames than one without, not fewer: no evidence for core parking or C-state wake-up costs here.
- **Counter conversion.** `QueryPerformanceCounter` ticks to ns or to the guest's 19.2 MHz counter as
`(unsigned __int128)t * num / den` compiles to `__udivti3`, a library call; it took a tenth of the guest threads'
running time in a fight, as the scheduler and the guest's counter reads convert many thousands of times a second.
`platform.h` `PlatScale` multiplies by a precomputed integer and 64-bit fraction (branch `task/perf2`). A lazily
set two-field scale shared by threads must be published behind a flag stored after it.
## Files
- **Small-file opens stall behind other writers.** Opening a 20-byte file in a 139k-file cache folder takes 0.1 ms
alone (p99 0.3 ms) but took 11-40 ms in play, while the translator, the pipeline cache and the save wrote and
flushed on the same volume. Defender's real-time scan was off; the stall is the file system. `gfx_shader.c` reads
all `.use` masks on a background thread at start (45k files in 12 s).
- **Flushes are expensive.** A save Commit (copy or hard-link 82 files, `FlushFileBuffers` on each file and directory,
two `MoveFileExW` with `MOVEFILE_WRITE_THROUGH`, rotate backups) took 93-135 ms: compare 23-41, hard links 17-31,
copy 0-18, directory flush 22-24, oldest backup deletion 16. `fs_host.c` runs it on a thread; game writes wait for it.
- **Saving a cache holds its lock.** `vkGetPipelineCacheData` plus the file write took about 60 ms; done on the present
thread under the pipeline lock, every 20 s autosave froze the replay. `vk_pipeline.c` saves on its own thread
under a separate lock.
- **A crash leaves zero-filled files.** After bugchecks, thousands of `.spv` and `.res.json` files were zero-filled
(3647 of 21924 once). Validate cached files on load (`gfx_shader.c` checks the `.res.json` shape) instead of
trusting their presence.
- **Unicode paths.** Use the W APIs with UTF-8 to UTF-16 conversion (`fs_wide`); exes that take paths carry an
`activeCodePage` UTF-8 manifest ([installer.md](../installer.md), Constraints).
## Threads and sampling
- **Stopping a thread waits for user mode.** `SuspendThread` returns at once, but `GetThreadContext` waits until the
thread has left the kernel; a thread inside a long kernel call (a file open, a driver call) stalls the sampler for
that long. `prof_win.c` records such stops over 2 ms as `tstall`: they name the thread's kernel call.
- **Guard page.** A thread stopped between its `sub rsp` and the stack probe has Rsp inside its guard page; copying
the stack from Rsp raised `STATUS_GUARD_PAGE_VIOLATION` and ended the process. Check `PAGE_GUARD` with
`VirtualQuery` before reading another thread's stack.
- **Idle threads.** `QueryThreadCycleTime` tells without suspending whether a thread ran since the last look; the
sampler repeats an idle thread's last chain instead of stopping it.
- **Three virtual cores are the CPU budget.** TotK keys per-core data by `GetCurrentProcessorNumber` without locks,
so guest threads share three virtual cores whatever the host has. In a fight the guest main thread and five job
workers keep 2.6 of 3 busy and ready threads queue for a core: guest code speed decides the frame rate there.
## Vulkan and the driver (AMD, Windows)
- **New memory blocks.** VMA's `vkAllocateMemory` of a 256 MiB device-local block took 8-11 ms (memory type 3:
11-19 ms, 650 MiB: 23 ms) under VMA's block lock; texture creation on the render thread paid it four times in the
first minute of play while VRAM grew from 3.0 to 4.0 GB. Each allocation over 2 ms is logged by `vk_device.c`
`timed_allocate` (branch `task/perf2`).
- **Driver pipeline cache.** Keyed by the exe path and limited in size; preparing another build dir can evict this
one's libraries ([backlog.md](../backlog.md) item 12, [agent-playbook.md](../agent-playbook.md)).
- `win32u.dll` frames in a host stack are kernel graphics calls (WDDM), usually memory allocation or submission.
## Measuring
- **Run-to-run spread.** One recording replayed on near-identical builds gave 1% lows from 31.5 to 50 fps; three runs
of one build agreed within 46-50. Compare medians of three `--quiet` runs, never one run against another.
- **Replays need game time.** With the real-time frame step, input keyed by presents lands at other game times at
another frame rate; recordings key input by the world clock (`frame_rate.c`), and a load of any length no longer
shifts it. The fight still diverges after about 15 game seconds (enemy AI, physics).
- **Bit flips on machine A.** Its RAM flips bits (bugcheck 0x139 and others, several a day): a cached file, a
hard-linked object or a release can carry a flip without an error. Compare two independent builds; objects embed
the toolchain path of the folder they were built in, so compare their `.text`, `.data` and `.rdata`, not whole
files. A file that reads wrong only through the file cache: hash it with `FILE_FLAG_NO_BUFFERING` too.
- **Swapping exes.** `build_game.py` does not relink when only `totk.exe` was replaced by a copy; delete the exe
before rebuilding to restore a build dir.
docs/tester-guide.md
+10 -10
View File
@@ -13,16 +13,13 @@ from it may be shared.
## Build
1. Install the toolchain: [toolchain-windows.md](toolchain-windows.md) and
[toolchain-msys2-python.md](toolchain-msys2-python.md).
2. Follow steps 1-5 of "Building from source" in the [README](../README.md): Python environments, `config.local.toml`
with your own paths, extraction, the shader translator, then lifting and linking. The result is
1. Follow [building.md](building.md): the toolchain, then build steps 1-6 (Python environments, `config.local.toml`
with your own paths, extraction, the shader translator, lifting and linking, shader preparation). The result is
`build\v121\totk.exe`.
3. Check the build without the game: `build\v121\totk.exe --selftest` must print a pass.
4. To run the build on another of your machines, package a portable folder as in step 7 of "Building from source"
in the README, and run `totk.exe --prepare-shaders` once in the copied folder. The folder holds game code: for your
own machines only.
5. The installer ([installer.md](installer.md)) is the other route to a machine without the toolchain: it builds the
2. Check the build without the game: `build\v121\totk.exe --selftest` must print a pass.
3. To run the build on another of your machines, package a portable folder ([building.md](building.md), "Portable
folder"). The folder holds game code: for your own machines only.
4. The installer ([installer.md](installer.md)) is the other route to a machine without the toolchain: it builds the
game there from your own files. No installer release is published yet.
## Run and collect the log
@@ -37,7 +34,8 @@ build\v121\totk.exe 2> totk-log.txt
`TOTK_VK_VALIDATION=1` turns on the Vulkan validation layer (it comes with the Vulkan SDK). It makes the game slower;
do one run with it and one without (`set TOTK_VK_VALIDATION=`).
Keep the window in the foreground: an unfocused window stalls the game. The first launch spends minutes on shaders.
Keep the window in the foreground: an unfocused window stalls the game. Shaders the preparation did not cover compile on first use, so
the first minutes of play can stutter.
Play into the open field (load a save, walk around for a minute, fight something if you can), then close the window.
## Choosing the GPU
@@ -57,6 +55,8 @@ The line `vk: using gpu N: <name>` says which one ran.
3. What you saw: how far it got (menu, loading, field), roughly how smooth it ran, and screenshots of anything wrong
(black or missing areas, flickering, wrong colours, missing grass). Win+Shift+S is fine.
4. If it crashed or hung: the last 50 lines of the log and what you were doing.
5. For a bug you can trigger again: a recording of your input (`set TOTK_INPUT_RECORD=input.txt`, see
[controls.md](controls.md), "Scripted input") and the save you started from.
## Switches for the developer's questions
docs/toolchain-msys2-python.md
-115
View File
@@ -1,115 +0,0 @@
# Toolchain: MSYS2 UCRT64, Python, Switch content tools
Set up 2026-10-20. Package set follows `refs/LASRecomp/tools/setup_windows.ps1` plus cmake, ninja, gcc, git, unzip.
`setup_windows.ps1` itself was not run.
## MSYS2 UCRT64 (`C:\msys64`)
Update: `C:\msys64\usr\bin\bash.exe -lc "pacman -Syuu --noconfirm"` (twice: first pass updates the core and exits).
Install:
pacman -S --needed --noconfirm make mingw-w64-ucrt-x86_64-{clang,lld,sdl3,opus,libvpx,ffmpeg,vulkan-devel,shaderc,
spirv-tools,pkgconf,gdb,vulkan-memory-allocator,vulkan-validation-layers,nasm,cmake,ninja,gcc,make} base-devel git unzip zip p7zip
| Package | Version |
|---|---|
| clang / lld | 23.1.3 |
| gcc | 16.2.0 |
| make (msys) / mingw32-make | 4.4.1 |
| sdl3 | 3.4.18 |
| opus | 1.6.1 (`libopus-0.dll`, voice) |
| libvpx | 1.17.0 (`libvpx.a`, linked statically for the movies) |
| ffmpeg | 9.0.2 (never linked: the MSYS2 build is GPLv3) |
| vulkan-devel group (headers, loader, utility libs, spirv-headers, spirv-tools, validation layers) | 1.4.363 |
| shaderc (glslc) | 2026.4 |
| vulkan-memory-allocator | 3.4.0, header at `/ucrt64/include/vma/vk_mem_alloc.h` |
| pkgconf | 3.0.9 |
| cmake / ninja | 4.4.4 / 1.13.2 |
| nasm / gdb | 3.02 / 18.1 |
| git / unzip / zip / p7zip | 2.56.0 / 6.0 / 3.0 / 17.06 |
Use from Git Bash: `export PATH="/c/msys64/ucrt64/bin:/c/msys64/usr/bin:$PATH"`. `tools/build_game.py` copies
`SDL3.dll`, `libiconv-2.dll` and `libopus-0.dll` next to `totk.exe`, so the game runs without MSYS2 on `PATH`.
Verify:
clang --version; ld.lld --version
pkgconf --modversion sdl3 vulkan opus vpx
glslc --version; ffmpeg -version; spirv-val --version
### Vulkan back end
# all part of the install line above plus spirv-cross and glslang; --needed makes this a check
pacman -S --needed mingw-w64-ucrt-x86_64-{vulkan-devel,vulkan-memory-allocator,vulkan-validation-layers,shaderc,spirv-tools,spirv-cross,glslang}
Verify:
pkgconf --modversion vulkan # 1.4.363
spirv-val --version; spirv-dis --version
ls /ucrt64/include/vma/vk_mem_alloc.h /ucrt64/lib/libvulkan-1.dll.a
vulkaninfo --summary # Vulkan SDK; lists the discrete GPU, Vulkan 1.3 or newer
`tools/build_game.py` links `-lvulkan-1 -luser32 -lgdi32 -ldwmapi` and adds `/ucrt64/include/vma` to the runtime include
path (`#include "vk_mem_alloc.h"`). At run time `vulkan-1.dll` comes from `System32`.
## Python
uv 0.13.3 at `%USERPROFILE%\.local\bin\uv.exe` (standalone installer, not winget).
| venv | Python | Purpose | Pins |
|---|---|---|---|
| `.venv` | 3.12.15 (uv-managed) | lifter / generator (capstone, unicorn, lz4, keystone, lief, pyelftools), oead 1.3.0, evfl, quicktex, PyYAML, Pillow, numpy, cryptography, nsz, zstandard, pyinstaller | `tools/requirements.txt` |
| `.venv-py314` | 3.14.7 (system) | MacNativeGames TotK scripts: they use stdlib `compression.zstd` (3.14+) | `tools/requirements-py314.txt` |
Version pins taken from LASRecomp `Makefile` (capstone 5.0.9, unicorn 2.1.4, lz4 4.4.5, pyinstaller 6.22.3) and
`refs/MacNativeGames/games/totk/requirements*.txt` (cryptography 50.0.2, nsz 5.0.0, Pillow 12.3.0, brotli 1.2.0).
`capstone.__version__` reports `5.0.7` although the installed wheel is 5.0.9; the wheel's metadata is right.
Recreate:
uv venv --python 3.12 .venv
uv pip install --python .venv/Scripts/python.exe -r tools/requirements.txt
uv venv --python 3.14 .venv-py314
uv pip install --python .venv-py314/Scripts/python.exe -r tools/requirements-py314.txt
Verify:
.venv\Scripts\python -c "import capstone, unicorn, lz4; print(capstone.__version__, unicorn.__version__)"
.venv\Scripts\python -c "import capstone; md=capstone.Cs(capstone.CS_ARCH_ARM64, capstone.CS_MODE_ARM); print([i.mnemonic for i in md.disasm(bytes.fromhex('1f2003d5'),0)])"
# -> ['nop']
.venv-py314\Scripts\python -c "import compression.zstd, capstone, unicorn, lz4, cryptography, nsz"
## Switch content tools (`toolchain\`)
| Tool | Path | Status |
|---|---|---|
| hactool 1.4.0 @ `1d64a83450e025622f3468c28fc4164dad2c5ef6` (the MacNativeGames pin) | `toolchain\hactool\hactool.exe` | built, static, depends only on system DLLs |
| hactoolnet | - | not installed: `github.com/Thealexbarney/LibHac` returns 404 (repo, releases and assets gone) |
hactool source: `toolchain\src\hactool` (git checkout of the pinned commit). Build: UCRT64 gcc, `config.mk` =
`CC=gcc EXEEXT=.exe CFLAGS=-O2 ... LDFLAGS=... -static`, `make -j8`.
Applied the portable part of MacNativeGames `build_app.py:build_hactool()` (kept as
`toolchain\hactool\portable-patches.diff` in the local toolchain folder, not in the repository):
1. `MNG_TITLE_KEYS` env var selects the title-key file.
2. BKTR: allow 0x200-aligned padding after the subsection table (`nca.c`).
3. BKTR: accept the subsection total-size layout of newer cumulative updates (`nca.c`).
Ours (kept locally as `toolchain\hactool\unicode-patches.diff`): `hactool.manifest` with `<activeCodePage>UTF-8</activeCodePage>`,
embedded through `hactool.rc` (Makefile rule `hactool.res.o`, `windres`). hactool converts argv, `HOME` and
`USERPROFILE` from UTF-8 to UTF-16 (`filepath.c`); without the manifest they arrive in the ANSI code page, and a
non-ASCII user profile or path stops it with `Failed to convert ... to UTF-16!`. Needs Windows 10 1903 or later.
Rebuild: in an MSYS2 UCRT64 shell (`MSYSTEM=UCRT64 C:\msys64\usr\bin\bash.exe -lc ...`) in `toolchain\src\hactool`,
`make clean && make -j8`, then copy `hactool.exe` to `toolchain\hactool\`. gcc writes temporary files to `TMP`; where
it cannot (sandboxed shells), point `TMP` and `TEMP` at a writable folder.
Not applied: the `MNG_BASE_RANGE_OFFSET/SIZE` base-file hook. It uses `funopen` (BSD/macOS only); on Windows the
base NCA must be extracted to a file first.
Verify: `toolchain\hactool\hactool.exe` prints `hactool (c) SciresM.` and usage.
Keys: hactool reads `-k <prod.keys>`, else `$HOME\.switch\prod.keys` (falls back to `%USERPROFILE%` when `HOME` is
unset; MSYS shells set `HOME`, so pass `-k` explicitly). Use your own keys; no keys are in this repo.
docs/toolchain-windows.md
-51
View File
@@ -1,51 +0,0 @@
# Windows toolchain
Native Windows tools, installed with winget unless noted. MSYS2 packages and Python environments are in [toolchain-msys2-python.md](toolchain-msys2-python.md).
Versions are the ones verified on 2026-10-20. Newer patch releases are fine unless the row says otherwise.
| Tool | Version | Install | Verify |
|---|---|---|---|
| Visual Studio Build Tools 2026 (C++ workload, Windows SDK, VC.Llvm.Clang) | MSVC 19.51, SDK 10.0.26100 | `Microsoft.VisualStudio.BuildTools` with `--add Microsoft.VisualStudio.Workload.VCTools --includeRecommended --add Microsoft.VisualStudio.Component.VC.Llvm.Clang` | `cmd /c "call <VS>\VC\Auxiliary\Build\vcvars64.bat && cl"` |
| LLVM | 23.1.3 | `LLVM.LLVM` | `clang --version`, `lld-link --version`, `llvm-objdump --version` |
| CMake | 4.4.4 | `Kitware.CMake` | `cmake --version` |
| Ninja | 1.13.2 | `Ninja-build.Ninja` | `ninja --version` |
| Vulkan SDK | 1.4.363.0 | `KhronosGroup.VulkanSDK` | `glslc --version`, `spirv-cross --revision`, `vulkaninfo --summary` |
| .NET SDK | 10.0.402 | `Microsoft.DotNet.SDK.10` | `dotnet --list-sdks` |
| Rust | 1.99.1, target `x86_64-pc-windows-msvc` | `Rustlang.Rustup`, then `rustup default stable` | `rustc --version` |
| Zig | 0.17.0 | `zig.zig` | `zig version` |
| JDK | Temurin 21.0.12 | `EclipseAdoptium.Temurin.21.JDK` (sets `JAVA_HOME`) | `java -version` |
| Ghidra | 12.1.2 (pinned: SwitchLoader targets it; REA accepts 12.1.x) | Release zip from github.com/NationalSecurityAgency/ghidra into `toolchain/ghidra/` | `toolchain\ghidra\ghidra_12.1.2_PUBLIC\support\analyzeHeadless.bat` |
| Ghidra SwitchLoader | 1.6.1-c600d05 for Ghidra 12.1.2 | Release zip from github.com/Adubbz/Ghidra-Switch-Loader, extracted into `<ghidra>\Ghidra\Extensions\SwitchLoader` | Ghidra lists "Nintendo Switch Binary" as a loader |
| RenderDoc | 1.46 (`e4bd23b`) | Signed MSI from renderdoc.org, unpacked without installing: `msiexec /a <msi> /qn TARGETDIR=<tmp>`, then move `<tmp>\PFiles\RenderDoc\*` to `toolchain/renderdoc/` (drop `System`/`System64`, the VC++ runtime Windows already has). For Vulkan captures, once, as administrator: `renderdoccmd vulkanlayer --register --system` | `renderdoccmd version`; `renderdoccmd vulkanlayer --explain` reports the layer registered |
| REA | rea-agents 6.7.0 | Skill pinned in `.claude/skills/skills-lock.json`; MCP server registered per clone, see [AGENTS.md](../AGENTS.md) | `claude mcp get rea`; `npx -y rea-agents@6.7.0 doctor --json` shows the Ghidra provider available |
## PATH
The User PATH needs these, appended:
- `C:\Program Files (x86)\Microsoft Visual Studio\Installer`: `vswhere.exe`. Without it .NET NativeAOT publish fails at the link step (MSB3073).
- `C:\Program Files\LLVM\bin`: the LLVM installer does not add it.
A 32-bit winlibs MinGW on PATH can shadow `ninja`. Builds call Ninja from `C:\msys64\ucrt64\bin` by full path.
## Smoke tests
- **NativeAOT** (needed for the Ryujinx shader translator): `dotnet new console` then `dotnet publish -c Release -r win-x64 -p:PublishAot=true` produces a native exe that runs.
- **Generated-C dialect**: a C file using `__attribute__((musttail))`, `unsigned __int128` and `vector_size` types builds with MSYS2 UCRT64 `clang -O1 -fno-strict-aliasing -march=x86-64-v3 -fuse-ld=lld` and runs.
- **GPU**: `vulkaninfo --summary` lists the discrete GPU with Vulkan 1.3 or newer.
## Building the game code
`tools/build_game.py` compiles `generated/<version>/` (written by `recomp/gen.py`) and links it with the runtime into `build/<version>/totk.exe`:
```powershell
.venv\Scripts\python tools\build_game.py # generated/v121 -> build/v121/totk.exe
.venv\Scripts\python tools\build_game.py --modules subsdk0 # one module; --compile-only skips the link
build\v121\totk.exe --selftest # layout, guest window, dispatch table
```
- It writes `build/<version>/build.ninja` and runs `C:\msys64\ucrt64\bin\ninja.exe` with `C:\msys64\ucrt64\bin` first on `PATH`. Unchanged chunks are not recompiled.
- Compile jobs default to the smaller of the CPU count and a 50 GiB memory budget (80% of RAM on smaller machines) at `JOB_MEM` per job. `--jobs N` overrides it.
- Each phase prints its wall time and the peak committed memory of one process and of all processes together, then the exe's section sizes.
- 1.2.1 on 24 threads / 64 GB (2026-10-20): 351 files, 900 MB of C; clean compile 190 s at 24 jobs, under 600 MiB per job and 3.7 GiB all jobs together; link 1 s, 820 MiB working set; `totk.exe` 316 MB with 276 MiB of `.text`, well inside the 2 GiB reach of RIP-relative code.
skills-lock.json
-11
View File
@@ -1,11 +0,0 @@
{
"version": 1,
"skills": {
"reverse-engineer-anything": {
"source": "morluto/rea",
"sourceType": "github",
"skillPath": ".agents/skills/reverse-engineer-anything/SKILL.md",
"computedHash": "21d9abc08fd7903d80756abf60bba9895dbde9abc2848573a97a635c6cd70704"
}
}
}
tools/worktree.py
+5 -4
View File
@@ -4,8 +4,9 @@
worktree.py list worktrees with branch, ahead/behind main, dirty or not
worktree.py remove <tag> [--delete-branch] [--force]
A worktree holds tracked files only. build/, generated/, refs/, toolchain/, .venv/, .venv-py314/ and game/ of the main
checkout appear in it as directory junctions, config.local.toml as a hard link. Nothing is copied.
A worktree holds tracked files only. build/, generated/, refs/, toolchain/, .venv/, .venv-py314/, game/, and the local
agent setup .claude/ and notes/ of the main checkout appear in it as directory junctions, config.local.toml and
CLAUDE.local.md as hard links. Nothing is copied.
`remove` deletes the links first (os.rmdir / os.unlink never follow a junction), checks that none is left, and only
then lets git delete the worktree. Stdlib only.
"""
@@ -16,8 +17,8 @@ import subprocess
import sys
from pathlib import Path
LINK_DIRS = ['build', 'generated', 'refs', 'toolchain', '.venv', '.venv-py314', 'game']
LINK_FILES = ['config.local.toml']
LINK_DIRS = ['build', 'generated', 'refs', 'toolchain', '.venv', '.venv-py314', 'game', '.claude', 'notes']
LINK_FILES = ['config.local.toml', 'CLAUDE.local.md']
MAIN_BRANCH = 'main'
TAG_RE = re.compile(r'^[A-Za-z0-9][A-Za-z0-9._-]*$')