diff --git a/.claude/skills/run-game/SKILL.md b/.claude/skills/run-game/SKILL.md index 2efc4122..a216440e 100644 --- a/.claude/skills/run-game/SKILL.md +++ b/.claude/skills/run-game/SKILL.md @@ -22,7 +22,7 @@ prints `: rc=`. The game window opens behind the user's windows witho | `user-save-2026-10-21-monsters` | Moblins and Zonai constructs attack at once; use with `--script tools/input/fight.txt` | | `user-save-2026-10-21` | Hyrule field facing the castle, rainy; distant sky over the castle | | `user-save-2026-10-23-pillars` | Hyrule field by standing stone pillars, about 17:40, a Zonai construct beside Link; use with `--script tools/input/backrun.txt` (runs backward from 1550: phantom rocks, a one-frame glitch at about 1605) | -| `user-save-2026-10-24-brightbloom` | Cave, about 20:40: a Brightbloom plant at the lower left (its bulb floats beside the stem, `build/v121-bloom/NOTES.md`), teal Moblin and Bokoblin ahead or Zonai constructs (runs differ) | +| `user-save-2026-10-24-brightbloom` | Cave, about 20:40: a Brightbloom plant at the lower left (its bulb sits in its cup; it floated beside the stem before ddd0ce8), teal Moblin and Bokoblin ahead or Zonai constructs (runs differ) | | `user-save-2026-10-22-island` | Sky island edge; use with `--script tools/input/island.txt` (runs off, skydives, opens the paraglider with B at frame 1975; X does nothing in a dive) | `--frames` counts presents. Pick `1500 + the frames you need in-world`. diff --git a/AGENTS.md b/AGENTS.md index fb714a5b..24f78f25 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -46,7 +46,8 @@ Setup and verify commands: [docs/toolchain-windows.md](docs/toolchain-windows.md - Game code build (`.venv`): `recomp\gen.py --exefs build\extracted\v121\exefs --out generated\v121 --patches recomp\patches-v121.txt` writes the C (about 15 s). The patch table holds the 60 fps and real-time frame step code patches, switchable at run time (`recomp/patches.py`); `.venv-py314\Scripts\python tools\make_romfs_patches.py` writes their RomFS half to `build\v121\romfs-patches`. `tools\build_game.py` compiles and links `build\v121\totk.exe` with Ninja (about 200 s clean, incremental after). `recomp/layout.py` alone decides module bases. Jump tables are resolved by `recomp/jumptables.py`; after changing it, `tests\jumptables_check.py` must report 0 disagreements with its Unicorn oracle. - Shader census (`.venv-py314`): `tools\shader_census.py --version v121 [--translate]` extracts every Maxwell shader to `build/v121/shaders/` and runs it through `build\xlate\totk_xlate.exe` and `spirv-val`; failure groups go to `failures.txt`. `tools\xlate_smoke.py` checks the translator alone, without game files. - Shader preparation (`.venv-py314`, once per build dir and translator): `tools\prepare_shaders.py build\v121` translates the variants of the MacNativeGames shared shader list (read from `refs/` at run time) into the build's shader cache and builds their stage libraries into the driver cache (`totk.exe --prepare-shaders`, about 5 min, 1.3 GB); play then translates and compiles only variants the list lacks. -- The plan by stage and the proposed fixes with their evidence: [docs/backlog.md](docs/backlog.md). +- Open work with its evidence: [docs/backlog.md](docs/backlog.md). +- Installer (`installer/`, `tools/make_release.py`), which builds the game on a player's machine: [docs/installer.md](docs/installer.md). - Running, measuring or frame-dumping the game, save backups, task worktrees and their builds: [docs/agent-playbook.md](docs/agent-playbook.md). - Project skills (`.claude/skills/`): `private-build` before changing runtime/ or tools/ code and building totk.exe (one git worktree per task); `run-game` to run a build into gameplay from a save. - hactool: `toolchain/hactool/hactool.exe`. From an MSYS2 shell pass `-k ` explicitly; the shell's `HOME` hides the default key location. diff --git a/README.md b/README.md index 4023ff25..cf3640f7 100644 --- a/README.md +++ b/README.md @@ -16,15 +16,19 @@ The Switch code is lifted from ARM64 to C ahead of time, compiled with Clang, an ## Status -This is an early build. Version 1.2.1 starts, shows the title screen, loads saves and runs in the world at 60 fps, with keyboard or gamepad input and with sound. Graphics, stability and speed are still being worked on: visual bugs and crashes are expected. You build it from source; nothing is packaged for download. +This is an early build. Version 1.2.1 starts, shows the title screen, loads saves and runs in the world at 60 fps, with keyboard, mouse or gamepad input and with sound. Graphics, stability and speed are still being worked on: visual bugs and crashes are expected. No release has been published yet. You build the game from source, or assemble the installer described in [docs/installer.md](docs/installer.md), which builds it on the player's machine from the player's own files. -It is developed and measured on one machine (Ryzen 9 7900X, Radeon RX 9070). NVIDIA and older AMD GPUs are expected to work but are barely tested; [docs/tester-guide.md](docs/tester-guide.md) says what to send back. +It is developed on a Ryzen 9 7900X with a Radeon RX 9070 and tested on two more machines: a laptop with a GeForce GTX 1050 Ti and an 8th- to 10th-generation Core i7, and a Core i7 of the same generations with Intel UHD Graphics 630. An RTX 3090 Ti has also been measured. On the GTX 1050 Ti laptop the game is limited by the CPU and runs at 25 to 45 fps. Other GPUs are expected to work but are barely tested; [docs/tester-guide.md](docs/tester-guide.md) says what to send back. To report a problem, open an issue and follow [CONTRIBUTING.md](CONTRIBUTING.md). Include the GPU and driver, the log, and the steps that get there. Screenshots are fine. ### Known problems -- For one frame, a rock or pillar can appear in the wrong place, or nearby objects vanish, most often while running or falling from a sky island. The leading suspect is thread ordering the game expects from the Switch scheduler; see [docs/backlog.md](docs/backlog.md). +The full list, with the evidence for each item, is in [docs/backlog.md](docs/backlog.md). + +- For one frame, a rock or pillar can appear in the wrong place, or nearby objects vanish, most often while running or falling from a sky island. +- Yunobo fades out and disappears when his ability is used. +- On the GTX 1050 Ti, black patches appear for a moment, for example when the Purah Pad opens. - Without `tools\prepare_shaders.py` (build step 6) the first launch compiles shaders and pipelines and stutters until the caches fill. With it, a fight on a cleared driver cache had 1 slow frame instead of 37. - Newly streamed areas still have single slow frames while their textures upload. - Far terrain can turn flat gray or brown for a few frames after Link is knocked down. @@ -32,9 +36,9 @@ To report a problem, open an issue and follow [CONTRIBUTING.md](CONTRIBUTING.md) ### Not in yet - macOS and Linux -- Running 1.0.0, 1.1.2 or 1.4.3 (the extractor recognises those updates; the host is built for 1.2.1) -- Button remapping -- A package that builds the game code on a player's machine. Today you compile it yourself +- Running 1.0.0, 1.1.2 or 1.4.3. The extractor recognises those updates, but the host is built for 1.2.1. Work on 1.4.3 has started. +- An in-game settings menu. Settings are edited in `totk.ini`. +- A published installer release ## Features @@ -43,7 +47,8 @@ To report a problem, open an issue and follow [CONTRIBUTING.md](CONTRIBUTING.md) - Title screen, save loading and play, including the overworld, combat and the map. - 60 fps with the game running at real-time speed: MacNativeGames' 60 fps and frame-step patches are built in and can be switched off (`TOTK_FPS60=0`, `TOTK_REALTIME=0`). - F11 switches between the window and borderless fullscreen. -- Keyboard and the first SDL3 gamepad, merged into one controller. Buttons are taken by position, so the south face button is B. The key map is in [docs/controls.md](docs/controls.md). Keys count only while the game window is in front. +- Keyboard, mouse and the first SDL3 gamepad, merged into one controller. Buttons are taken by position, so the south face button is B. The mouse turns the camera (`mouse_sens` sets the speed) and Esc releases the cursor. Every button can be rebound in `totk.ini` (`bind_*` lines). The key map and the remapping syntax are in [docs/controls.md](docs/controls.md). Keys count only while the game window is in front. +- Rumble with adjustable strength and response curve, and motion controls on gamepads that have a gyro. - Sound through the game's own audio renderer, played with SDL3, including Opus voice. - Language from the Windows locale, or from `TOTK_LOCALE`. A locale the game does not ship becomes `en-US`. `TOTK_LANG` sets a Switch code directly (`en-US`, `ja`, `zh-Hant`, …). - Docked mode by default, which reports the boost performance configuration. `TOTK_OPERATION_MODE=handheld` reports the normal one. @@ -58,8 +63,8 @@ The design and the research it rests on are in [docs/architecture.md](docs/archi 1. **Input.** Your base game and the 1.2.1 update are decrypted and unpacked locally with your own `prod.keys`. File hashes are checked against known retail dumps. 2. **CPU.** `main`, `sdk` and `subsdk0` are lifted to C, one function per guest function. NEON and floating point are lowered to C that follows ARM rounding. Guest addresses are host addresses. Jump tables are resolved by a symbolic pass in the generator. -3. **Host.** Threads, memory, the RomFS, saves, time, account, language and the audio renderer are implemented on Windows. Every guest thread is a host thread, but only three run game code at once: the host models the Switch's three application cores and its priority preemption, because the game keeps per-core data without locks and orders some work by thread priority alone. Input and audio output go through SDL3. -4. **Graphics.** The NVN front end records what the game submits. A Vulkan 1.3 back end replays it, uploading the memory the CPU wrote and writing GPU results back when the CPU reads them. The Ryujinx Maxwell translator, built as `totk_xlate.exe`, turns shaders into SPIR-V. Pipelines are linked from precompiled stages on first use (`VK_EXT_graphics_pipeline_library`) and optimised in the background. +3. **Host.** Threads, memory, the RomFS, saves, time, account, language and the audio renderer are implemented on Windows. Every guest thread is a host thread, but only three run game code at once: the host models the Switch's three application cores and its priority preemption, because the game keeps per-core data without locks and orders some work by thread priority alone. A thread that waits for a core interrupts a lower-priority holder that is running game code, as MacNativeGames does. Input and audio output go through SDL3. +4. **Graphics.** The NVN front end records what the game submits. A Vulkan 1.3 back end replays it, uploading the memory the CPU wrote and writing GPU results back when the CPU reads them. On a discrete GPU, memory the game binds as buffers is mirrored in video memory. The Ryujinx Maxwell translator, built as `totk_xlate.exe`, turns shaders into SPIR-V. Pipelines are linked from precompiled stages on first use (`VK_EXT_graphics_pipeline_library`) and optimised in the background. ASTC textures are decoded by a compute shader, or uploaded as they are where the GPU samples ASTC. ## How this project is made @@ -70,7 +75,7 @@ Most of the code and documentation here is written by AI coding agents (Claude C - Players and testers: [controls](docs/controls.md), [tester guide](docs/tester-guide.md). - How it works: [architecture](docs/architecture.md), with research notes on the upstream projects, Maxwell GPU behaviour and what other ports learned in [docs/research/](docs/research/). - What is being worked on: [backlog](docs/backlog.md). -- Building: [toolchain-windows.md](docs/toolchain-windows.md), [toolchain-msys2-python.md](docs/toolchain-msys2-python.md). +- Building: [toolchain-windows.md](docs/toolchain-windows.md), [toolchain-msys2-python.md](docs/toolchain-msys2-python.md), and the [installer](docs/installer.md) that builds the game on a player's machine. - Coding agents: [AGENTS.md](AGENTS.md) and [docs/agent-playbook.md](docs/agent-playbook.md). ## Building from source @@ -171,6 +176,7 @@ This project stands on two MIT codebases and the projects they came from. Code t - **[hactool](https://github.com/SciresM/hactool)** by SciresM (ISC) extracts NCA and update containers. - **[SDL 3](https://libsdl.org)** (zlib) is the gamepad and the audio output. - **[libopus](https://opus-codec.org)** (BSD) decodes voice. +- **[libvpx](https://chromium.googlesource.com/webm/libvpx)** (BSD) decodes the cutscene movies. - **[Vulkan Memory Allocator](https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator)** (MIT) allocates GPU memory. - **[deko3d](https://github.com/devkitPro/deko3d)** (zlib) is the source of the block-linear image layout, via MacNativeGames, and a reference for Maxwell command semantics. - **[SavageWindRecomp](https://github.com/Revivedx/SavageWindRecomp)** (MIT), a static recompilation of *Breath of the Wild*, is the model for preemption at indirect calls and a reference for frame pacing and the platform split. @@ -195,6 +201,7 @@ Third-party code keeps its own license. The full list, with the commit each copy | SDL 3 | 3.4.18 | zlib | `SDL3.dll` next to `totk.exe` (gamepad, audio) | | libiconv | 1.19 | LGPL 2.1+ | `libiconv-2.dll`, unmodified, a dependency of SDL 3 | | libopus | 1.6.1 | BSD 3-clause | `libopus-0.dll` next to `totk.exe` (voice) | +| libvpx | 1.17.0 | BSD 3-clause, with the WebM patent grant | linked statically into `totk.exe` (movies) | | Vulkan Memory Allocator | 3.4.0 | MIT | compiled into `totk.exe` | | Ryujinx shader translator | 1.3.2 | MIT | `build\xlate\totk_xlate.exe`, a separate process | | GCC runtime (`libstdc++`) | 16.2.0 | GPL 3.0 with the GCC Runtime Library Exception | linked into `totk.exe` | diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 7ec88bda..49bc1da5 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -6,20 +6,20 @@ Code copied or adapted from another project is listed here with its source, the |---|---|---|---| | MacNativeGames extraction scripts and verification hashes | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11`, Copyright (c) 2026 MacNativeGames contributors | MIT | `tools/vendor/macnativegames/` (unmodified), `tools/data/expected-v*.json` | | LAS Recomp lifter, code generator, FP/SIMD lowering, CPU headers and differential tests | https://github.com/peiikai/LASRecomp @ `da35b83a9611d046a9a681aae668d7085954d37f`, Copyright (c) 2026 Peiikai | MIT | `recomp/lifter.py`, `recomp/simd_c.py`, `recomp/nso.py`, `recomp/gen.py`, `recomp/module.py`, `tests/jumptables_check.py`, `runtime/cpu.h`, `runtime/simd.h`, `runtime/simd_fp.h`, `tests/diff/` | -| LAS Recomp host runtime: NSO loader, host-function table, guest-space reservation and re-execution, guest thread start | https://github.com/peiikai/LASRecomp @ `da35b83a9611d046a9a681aae668d7085954d37f`, Copyright (c) 2026 Peiikai | MIT | `runtime/loader.c`, `runtime/hostfn.c`, `runtime/memory.c`; M3 tasks T2 and T6: `runtime/kernel.c`, `runtime/boot.c`; M4 T2: `runtime/vk_device.c`, `runtime/gfx_window.c`, M4 T5a: `runtime/tex_astc.c` (`runtime/nvn/astc.c`; its decode ported to GLSL in `runtime/shaders/astc_decode.comp`, compiled into `runtime/astc_decode_spv.h`, and its tables copied into `runtime/vk_astc.c`), `runtime/vk_formats.c` (format table of `runtime/nvn/vulkan/textures.c`), M4 T5b: `runtime/vk_memory.c` (page write generations of `runtime/nvn/vulkan/exec.c`, deferred destruction of `textures.c`), `runtime/vk_texture.c` (image creation, uploads and samplers of `runtime/nvn/vulkan/textures.c`), M4 T6a: `runtime/vk_sync.c` (timeline submissions and waits of `runtime/nvn/vulkan/exec.c`), M4 T6b: `runtime/vk_pipeline.c` (blend, compare and stencil mappings, vertex format table with its 3-component fallback and pipeline cache file of `runtime/nvn/vulkan/exec.c` and `device.c`), M4 T6c1: `runtime/vk_exec.c` (pass begin and end, scissor clip and attachment clears of `runtime/nvn/vulkan/exec.c`), M4 T6c3: `runtime/vk_exec_xfer.c` (copies of `runtime/nvn/vulkan/exec.c`), M4 T6c2: `runtime/vk_exec_draw.c` (uniform buffer spans, texture and sampler descriptors and dispatch flow of `runtime/nvn/vulkan/exec.c`) | -| MacNativeGames TotK host runtime: dynamic linking, identity check, module permissions, guest memory and SVCs, threads, IPC, audio renderer, system services, RomFS and save storage, boot sequence | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/runtime/`, `games/totk/scripts/prepare_runtime.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `runtime/loader.c`, `runtime/memory.c`; M3 tasks T2-T6: `runtime/kernel.c`, `runtime/svc.c`, `runtime/ipc.c`, `runtime/ipc.h`, `runtime/audren.c`, `runtime/audren.h`, `runtime/audio_mixer.c`, `runtime/audio_effects.c`, `runtime/audio_effects.h`, `runtime/audio_reverb.c`, `runtime/audio_reverb.h` (M5; SDL3 output replaces CoreAudio), `runtime/ov_system.c` (also `shared_font.c`: the BFTTF layout behind nn::pl), `runtime/ov_input.c` (also `hid_feedback.c`: vibration and six-axis handles and calls), `runtime/motion_fusion.c` (six-axis fusion of `motion_fusion.c`), `runtime/ov_media.c` (M5: hardware Opus decoder over libopus), `runtime/swkbd_win.c` (`swkbd_host.c`: ShowKeyboardArg offsets, UTF-8/UTF-16 conversion, disabled-key filter, result codes; Win32 window replaces the macOS text bar), `runtime/ov_gfx.c`, `runtime/romfs.c`, `runtime/romfs_path.h`, `runtime/fs_host.c`, `runtime/fs.h`, `runtime/boot.c`; M4 T2: `runtime/gfx_window.c` | +| LAS Recomp host runtime: NSO loader, host-function table, guest-space reservation and re-execution, guest thread start | https://github.com/peiikai/LASRecomp @ `da35b83a9611d046a9a681aae668d7085954d37f`, Copyright (c) 2026 Peiikai | MIT | `runtime/loader.c`, `runtime/hostfn.c`, `runtime/memory.c`; `runtime/kernel.c`, `runtime/boot.c`; `runtime/vk_device.c`, `runtime/gfx_window.c`, `runtime/tex_astc.c` (`runtime/nvn/astc.c`; its decode ported to GLSL in `runtime/shaders/astc_decode.comp`, compiled into `runtime/astc_decode_spv.h`, and its tables copied into `runtime/vk_astc.c`), `runtime/vk_formats.c` (format table of `runtime/nvn/vulkan/textures.c`), `runtime/vk_memory.c` (page write generations of `runtime/nvn/vulkan/exec.c`, deferred destruction of `textures.c`), `runtime/vk_texture.c` (image creation, uploads and samplers of `runtime/nvn/vulkan/textures.c`), `runtime/vk_sync.c` (timeline submissions and waits of `runtime/nvn/vulkan/exec.c`), `runtime/vk_pipeline.c` (blend, compare and stencil mappings, vertex format table with its 3-component fallback and pipeline cache file of `runtime/nvn/vulkan/exec.c` and `device.c`), `runtime/vk_exec.c` (pass begin and end, scissor clip and attachment clears of `runtime/nvn/vulkan/exec.c`), `runtime/vk_exec_xfer.c` (copies of `runtime/nvn/vulkan/exec.c`), `runtime/vk_exec_draw.c` (uniform buffer spans, texture and sampler descriptors and dispatch flow of `runtime/nvn/vulkan/exec.c`) | +| MacNativeGames TotK host runtime: dynamic linking, identity check, module permissions, guest memory and SVCs, threads, IPC, audio renderer, system services, RomFS and save storage, boot sequence | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/runtime/`, `games/totk/scripts/prepare_runtime.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `runtime/loader.c`, `runtime/memory.c`; `runtime/kernel.c`, `runtime/svc.c`, `runtime/ipc.c`, `runtime/ipc.h`, `runtime/audren.c`, `runtime/audren.h`, `runtime/audio_mixer.c`, `runtime/audio_effects.c`, `runtime/audio_effects.h`, `runtime/audio_reverb.c`, `runtime/audio_reverb.h` (SDL3 output replaces CoreAudio), `runtime/ov_system.c` (also `shared_font.c`: the BFTTF layout behind nn::pl), `runtime/ov_input.c` (also `hid_feedback.c`: vibration and six-axis handles and calls), `runtime/motion_fusion.c` (six-axis fusion of `motion_fusion.c`), `runtime/ov_media.c` (hardware Opus decoder over libopus), `runtime/swkbd_win.c` (`swkbd_host.c`: ShowKeyboardArg offsets, UTF-8/UTF-16 conversion, disabled-key filter, result codes; Win32 window replaces the macOS text bar), `runtime/ov_gfx.c`, `runtime/romfs.c`, `runtime/romfs_path.h`, `runtime/fs_host.c`, `runtime/fs.h`, `runtime/boot.c`; `runtime/gfx_window.c` | | MacNativeGames TotK frame-rate patches: 60 fps present-interval controller, real-time frame step and its present sampling, hair and cloth clock, trampoline slow-motion check (instruction words and 1.2.1 sites), player position chain; quick-menu capture background RomFS overlay | https://github.com/ly05010419/MacNativeGames @ `ebe621de99bfc48f3c42f2d2aa73e04fbcc6eae2` (`games/totk/runtime/game_patches.c`, `game_version.h`, `world_survey.c`, `games/totk/scripts/make_romfs_patches.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `runtime/frame_rate.c`, `recomp/patches-v121.txt`, `tools/make_romfs_patches.py` | | MacNativeGames TotK virtual-core preemption: asynchronous preemption of an owner in top-level game code, priority inheritance through guest mutexes, the SDK's preemption-disable protocol (TLR+0x100/+0x102, SVC 0x36), the core-wait diagnostics | https://github.com/ly05010419/MacNativeGames @ `33e8fa83052ea1ae09a3583137008e9797e12950` (`games/totk/runtime/threads.c`, commits 8bf0893 and 19730cd), Copyright (c) 2026 MacNativeGames contributors | MIT | `runtime/kernel.c` (SuspendThread and a register-saving stub replace SIGUSR2), `runtime/hostfn.c`, `runtime/svc.c` (host depth) | | MacNativeGames RomFS index builder (`TOTKRFS1` layout) | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/scripts/prepare_romfs_backend.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `tools/romfs_index.py` | -| d2r-mobile host thread environment (TLR, TLS, thread object), the origin of MacNativeGames' thread layer; d2r-mobile NVN host object layouts, command record format and NVN front end, as adapted for TotK in MacNativeGames' `d2r_gfx` copy | https://github.com/jhaizhou-ops/d2r-mobile @ `53e7c5bb9ca3ea53e2160005be8fbe920cd9cb53` (`dm/host/thread.c`, `dm/host/hle_thread.c`); the NVN layouts and front end via https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/third_party/d2r_gfx/gfx/nvn/nvn_obj.h`, the two types of `nvn_tex.h`, `hle_nvn.c`, `totk_nvn_ext.c`, `nvn_impl.h`, `nvn_hmap.h`, `nvn_pool_index.h`, the request checks and mapping semantics of `totk_vpool.c`, the generation stamps of `nvn_dirty.c`, texture mirroring and texel copies of `nvn_mtl.m`; licence in `games/totk/third_party/d2r_gfx/LICENSE`), Copyright (c) 2026 jhaizhou | MIT | M3 task T2: `runtime/kernel.c`; M4 task T1: `runtime/nvn_obj.h`; M4 task T3c: `runtime/nvn_api.h`, `runtime/nvn_api_core.c`, `runtime/nvn_hmap.h`, `runtime/nvn_pool_index.h`; M4 task T3d: `runtime/nvn_api_obj.c` (also `nvn_pace.h`); M4 task T3e: `runtime/nvn_api_cmd.c`, `runtime/nvn_ext.c`; M4 task T5b: `runtime/vk_memory.c`, `runtime/vk_texture.c`; M4 task T6a: `runtime/vk_sync.c` (side effects written after GPU completion, `hle_nvn.c`); M4 task T6b: `runtime/vk_pipeline.c` (pipeline key and state mapping of `nvn_mtl.m` `pso_key_fill` / `pso_desc`, its `vfmt` vertex formats and the missing-varying fix-up of `msl_varyings`); M4 task T6c1: `runtime/vk_exec.c`, `runtime/vk_exec.h` (op loop, pure-state ops and state defaults of `nvn_mtl.m` `mtl_exec` / `state_op` / `state_reset`); M4 task T6c3: `runtime/vk_exec_xfer.c` (copy, clear, event, counter and conditional-render semantics of `nvn_mtl.m` `copy_t2b` / `copy_b2t` / `copy_t2t` / `clear_tex` / `cond_eval`); M4 task T6c2: `runtime/vk_exec_draw.c` (bind ops, draw and dispatch flow, vertex streams of `nvn_mtl.m` `state_op` / `bind_vertex_streams` / `draw_inner` / `dispatch`) | -| nxnative NVN front end (d2r-mobile's successor): texture view depth-stencil mode bits, stencil-view rule, `Downsample` and indirect draw and dispatch records | https://github.com/seedds/nxnative @ `d5b380b3a80495609c2e751bb994911a638935ad` (`dm/gfx/nvn/hle_nvn.c`, `nvn_obj.h`, `nvn_mtl.m`), Copyright (c) 2026 jhaizhou | MIT | M5: `runtime/nvn_obj.h`, `runtime/nvn_ext.c`, `runtime/nvn_api_cmd.c`, `runtime/vk_exec_draw.c` | +| d2r-mobile host thread environment (TLR, TLS, thread object), the origin of MacNativeGames' thread layer; d2r-mobile NVN host object layouts, command record format and NVN front end, as adapted for TotK in MacNativeGames' `d2r_gfx` copy | https://github.com/jhaizhou-ops/d2r-mobile @ `53e7c5bb9ca3ea53e2160005be8fbe920cd9cb53` (`dm/host/thread.c`, `dm/host/hle_thread.c`); the NVN layouts and front end via https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/third_party/d2r_gfx/gfx/nvn/nvn_obj.h`, the two types of `nvn_tex.h`, `hle_nvn.c`, `totk_nvn_ext.c`, `nvn_impl.h`, `nvn_hmap.h`, `nvn_pool_index.h`, the request checks and mapping semantics of `totk_vpool.c`, the generation stamps of `nvn_dirty.c`, texture mirroring and texel copies of `nvn_mtl.m`; licence in `games/totk/third_party/d2r_gfx/LICENSE`), Copyright (c) 2026 jhaizhou | MIT | `runtime/kernel.c`; `runtime/nvn_obj.h`; `runtime/nvn_api.h`, `runtime/nvn_api_core.c`, `runtime/nvn_hmap.h`, `runtime/nvn_pool_index.h`; `runtime/nvn_api_obj.c` (also `nvn_pace.h`); `runtime/nvn_api_cmd.c`, `runtime/nvn_ext.c`; `runtime/vk_memory.c`, `runtime/vk_texture.c`; `runtime/vk_sync.c` (side effects written after GPU completion, `hle_nvn.c`); `runtime/vk_pipeline.c` (pipeline key and state mapping of `nvn_mtl.m` `pso_key_fill` / `pso_desc`, its `vfmt` vertex formats and the missing-varying fix-up of `msl_varyings`); `runtime/vk_exec.c`, `runtime/vk_exec.h` (op loop, pure-state ops and state defaults of `nvn_mtl.m` `mtl_exec` / `state_op` / `state_reset`); `runtime/vk_exec_xfer.c` (copy, clear, event, counter and conditional-render semantics of `nvn_mtl.m` `copy_t2b` / `copy_b2t` / `copy_t2t` / `clear_tex` / `cond_eval`); `runtime/vk_exec_draw.c` (bind ops, draw and dispatch flow, vertex streams of `nvn_mtl.m` `state_op` / `bind_vertex_streams` / `draw_inner` / `dispatch`) | +| nxnative NVN front end (d2r-mobile's successor): texture view depth-stencil mode bits, stencil-view rule, `Downsample` and indirect draw and dispatch records | https://github.com/seedds/nxnative @ `d5b380b3a80495609c2e751bb994911a638935ad` (`dm/gfx/nvn/hle_nvn.c`, `nvn_obj.h`, `nvn_mtl.m`), Copyright (c) 2026 jhaizhou | MIT | `runtime/nvn_obj.h`, `runtime/nvn_ext.c`, `runtime/nvn_api_cmd.c`, `runtime/vk_exec_draw.c` | | NVN type definitions reconstructed from Darksiders II DWARF information, with MacNativeGames' TotK size annotations | h1k421, via https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/third_party/d2r_gfx/gfx/nvn/include/nvn.h`), Copyright (c) 2020 h1k421 | MIT | `runtime/nvn.h` (unmodified) | -| Ryujinx.Audio voice state and effects, as ported in MacNativeGames' audio renderer | https://github.com/Ryujinx/Ryujinx, via MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/runtime/audio_renderer.c`, `audio_effects.c`, `audio_reverb.c`; licence text in `games/totk/third_party/licenses/Ryujinx-Audio.txt`), Copyright (c) Ryujinx Team and Contributors | MIT | M3 task T3: `runtime/audren.c` | -| REA agent skill `reverse-engineer-anything` | https://github.com/morluto/rea @ `799e8265b254e225dc1dabdd25e56f0ab7f86bd3`, Copyright (c) 2026 morluto | MIT | `.claude/skills/reverse-engineer-anything/`, `.agents/skills/reverse-engineer-anything/` (pinned in `skills-lock.json`) | -| Ryujinx shader translator 1.3.2 (`Ryujinx.Graphics.Shader`, `Spv.Generator`, `Ryujinx.Common`) with MacNativeGames' four optimizer patches and `totk_xlate` tool | https://github.com/Ryujinx/Ryujinx, via MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`vendor/ryujinx`, `games/totk/tools/totk_xlate/`), Copyright (c) Ryujinx Team and Contributors; tool Copyright (c) 2026 jhaizhou | MIT | `tools/vendor/ryujinx/` (unmodified), `tools/xlate/`, `runtime/gfx_shader.c` (job layout from `tools/shader_census.py`; M4 T6b: c2 handle ranges of `totk_xlate/Service.cs` for the reflection), `runtime/vk_pipeline.c` (M4 T6b: the translator's set/binding and support-buffer conventions) | -| deko3d block-linear image layout, as ported in MacNativeGames' `nvn_tex.c` (Apple parts dropped, layout math unchanged; licence text in `games/totk/third_party/d2r_gfx/LICENSE.deko3d`, reproduced in the file header) | https://github.com/devkitPro/deko3d (`source/dk_image.cpp`), via MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/third_party/d2r_gfx/gfx/nvn/nvn_tex.c`, `nvn_tex.h`), Copyright (C) 2018-2020 fincs | zlib | M4 task T3b: `runtime/nvn_tex.c`, `runtime/nvn_tex.h` | -| MacNativeGames texture auto-layout goldens (40 original-SDK size and block-height cases) | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/tests/texture_auto_layout/probe.c`), Copyright (c) 2026 jhaizhou | MIT | M4 task T3b: `runtime/nvn_tex_probe.c`; T3f: `runtime/nvn_probe.c` (four of the cases, through the procs) | -| MacNativeGames block-linear deswizzler (address function; copy loop split out as `tex_bl_copy`) | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/third_party/d2r_gfx/gfx/tex/nvn_deswizzle.c`), Copyright (c) 2026 jhaizhou | MIT | M4 task T5a: `runtime/tex_deswizzle.c`, `runtime/tex.h` | +| Ryujinx.Audio voice state and effects, as ported in MacNativeGames' audio renderer | https://github.com/Ryujinx/Ryujinx, via MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/runtime/audio_renderer.c`, `audio_effects.c`, `audio_reverb.c`; licence text in `games/totk/third_party/licenses/Ryujinx-Audio.txt`), Copyright (c) Ryujinx Team and Contributors | MIT | `runtime/audren.c` | +| REA agent skill `reverse-engineer-anything` | https://github.com/morluto/rea @ `799e8265b254e225dc1dabdd25e56f0ab7f86bd3`, Copyright (c) 2026 morluto | MIT | `.claude/skills/reverse-engineer-anything/`, `.agents/skills/reverse-engineer-anything/` (pinned in `.claude/skills/skills-lock.json`) | +| Ryujinx shader translator 1.3.2 (`Ryujinx.Graphics.Shader`, `Spv.Generator`, `Ryujinx.Common`) with MacNativeGames' four optimizer patches and `totk_xlate` tool | https://github.com/Ryujinx/Ryujinx, via MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`vendor/ryujinx`, `games/totk/tools/totk_xlate/`), Copyright (c) Ryujinx Team and Contributors; tool Copyright (c) 2026 jhaizhou | MIT | `tools/vendor/ryujinx/` (unmodified), `tools/xlate/`, `runtime/gfx_shader.c` (job layout from `tools/shader_census.py`; c2 handle ranges of `totk_xlate/Service.cs` for the reflection), `runtime/vk_pipeline.c` (the translator's set/binding and support-buffer conventions) | +| deko3d block-linear image layout, as ported in MacNativeGames' `nvn_tex.c` (Apple parts dropped, layout math unchanged; licence text in `games/totk/third_party/d2r_gfx/LICENSE.deko3d`, reproduced in the file header) | https://github.com/devkitPro/deko3d (`source/dk_image.cpp`), via MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/third_party/d2r_gfx/gfx/nvn/nvn_tex.c`, `nvn_tex.h`), Copyright (C) 2018-2020 fincs | zlib | `runtime/nvn_tex.c`, `runtime/nvn_tex.h` | +| MacNativeGames texture auto-layout goldens (40 original-SDK size and block-height cases) | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/tests/texture_auto_layout/probe.c`), Copyright (c) 2026 jhaizhou | MIT | `runtime/nvn_tex_probe.c`; `runtime/nvn_probe.c` (four of the cases, through the procs) | +| MacNativeGames block-linear deswizzler (address function; copy loop split out as `tex_bl_copy`) | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/third_party/d2r_gfx/gfx/tex/nvn_deswizzle.c`), Copyright (c) 2026 jhaizhou | MIT | `runtime/tex_deswizzle.c`, `runtime/tex.h` | | SDL 3.4.18, dynamic link, `SDL3.dll` copied next to `totk.exe`. Developer builds use the MSYS2 build (`mingw-w64-ucrt-x86_64-sdl3`, which imports `libiconv-2.dll`, LGPL-2.1, unmodified, copied beside it); the installer release ships the upstream release `SDL3-3.4.18-win32-x64.zip`, which imports no libiconv | https://libsdl.org, Copyright (c) 1997-2026 Sam Lantinga | zlib | `runtime/input_win.c` (gamepad input, rumble, gyro and accelerometer) | | libopus (MSYS2 `mingw-w64-ucrt-x86_64-opus`), dynamic link, `libopus-0.dll` copied next to `totk.exe` | https://opus-codec.org, Copyright (c) 2001-2023 Xiph.Org, Skype Limited, Octasic, Jean-Marc Valin, Timothy B. Terriberry, CSIRO, Gregory Maxwell, Mark Borgerding, Erik de Castro Lopo, Mozilla, Amazon | BSD-3-Clause | `runtime/ov_media.c` (nn::codec hardware Opus decoder) | | Vulkan Memory Allocator 3.4.0 (MSYS2 `mingw-w64-ucrt-x86_64-vulkan-memory-allocator`), header-only, compiled into `totk.exe` | https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator, Copyright (c) 2017-2026 Advanced Micro Devices, Inc. | MIT | `runtime/vk_vma.cpp` (the allocator behind `runtime/vk_memory.c`) | diff --git a/docs/agent-playbook.md b/docs/agent-playbook.md index f20cee66..aa468539 100644 --- a/docs/agent-playbook.md +++ b/docs/agent-playbook.md @@ -8,8 +8,7 @@ Read this before you run the game, take a frame dump or measure performance. Thi Start, build, hand back and clean up a task with the `private-build` skill (`.claude/skills/private-build/SKILL.md`): a git worktree on `task/` and a build dir `build/v121-`. The pipeline cache, `pipeline-keys.bin`, `shader-cache`, `shader-programs.bin` and `shader-answers.bin` live next to each `totk.exe`, so a task build starts cold: seed it by copying `build/v121`'s five before timing anything. A build seeded -from an older one stalls on pipeline and shader builds, and its slow frames say nothing about the change. (Until 2026-10-22 every task build wrote into -`build/v121`'s caches, and task builds broke each other's runs.) +from an older one stalls on pipeline and shader builds, and its slow frames say nothing about the change. ## Running @@ -39,7 +38,7 @@ other `totk.exe` or `ninja.exe` runs. The log's first and last lines record the - `TOTK_PROFILE= TOTK_PROFILE_FRAMES=lo-hi`: sampling profiler (`runtime/prof_win.c`), read with `tools/prof_report.py `. With `TOTK_PROFILE_RENDER=1` it samples the render thread only and keeps call chains: the report adds inclusive time per function and the exe callers of the driver and system DLLs. -- `tools/regress.py --base build/v121-m4 [--routes fight,rock,island,field] [--repeat 3] [--cold] [--validation] +- `tools/regress.py [--base ] [--routes fight,rock,island,field] [--repeat 3] [--cold] [--validation] [--out dir]`: runs the routes through `run_game.py --quiet`, scores logs and `TOTK_SHOTS` frames (`tools/regress_score.py`, tested by `tests/regress_score_test.py`) and prints median and worst per metric. Run the candidate alone and judge it against the targets: max fps, no slow frames (`perf: slow frame`), no black or flat frames, no new validation errors. @@ -52,7 +51,7 @@ other `totk.exe` or `ninja.exe` runs. The log's first and last lines record the - `fight.txt`: 59.1 fps mean, 60 median, replay 13.6 ms, 0 CPU-side slow frames; the slow frames left are pipeline and shader-translation waits. - `--validation` runs work in the world (1800 frames checked). The layer prints each message kind 10 times, so it - gives no totals; the known messages left are in the backlog correctness queue, item 8. + gives no totals; the known messages left are in the backlog, item 6. ## Frame dumps @@ -63,7 +62,7 @@ copies and clears. `TOTK_DUMP_BUF=addr:size[:off:stride:w:h]` dumps a guest buff - A draw's argument lines come before its pass header. - Pass numbers shift between runs. Find a pass by its draw count instead: the g-buffer pass has about 1990 draws. -RenderDoc cannot capture: the driver hides `VK_EXT_external_memory_host`. +RenderDoc cannot capture: it hides `VK_EXT_external_memory_host` ([controls.md](controls.md), "RenderDoc capture"). ## Other GPUs @@ -72,27 +71,29 @@ Vulkan code queries the feature, extension, format or limit it uses and keeps a it is missing (`vk_pipeline.c` provoking vertex and subgroup size pinning are examples). Never hard-code what this GPU reports: alignments, subgroup size, depth formats, `maxPushDescriptors`. -Numbers from this machine do not carry over. Measured on the user's other machines (2026-10-24/25, logs and analysis -in `build/mem-plan.md`): +Numbers from this machine do not carry over. The user's other machines (the test machines of +[backlog.md](backlog.md)) measured: - GTX 1050 Ti (4 GiB, 214 MiB BAR, no ReBAR; i7 8th-10th gen) and RTX 3090 Ti: GPU time 110-120 ms per frame in the world against 4.3 ms here, because the GPU reads imported host pools over PCIe. The VRAM mirror (cd6f490) brings - the 1050 Ti to 10-11 ms; that machine is now CPU-bound at 25-45 fps (replay 25-30 ms per frame, backlog 0b). + the 1050 Ti to 10-11 ms; that machine is now CPU-bound at 25-45 fps (replay 25-30 ms per frame, backlog item 8). - The same write-watch scan costs about 3x the CPU time it costs here. -- NVIDIA has no ASTC: first texture uploads decode on the CPU, 0.4-0.7 s load frames (backlog 5a.2). +- NVIDIA and AMD desktop GPUs have no ASTC: textures decode in a compute shader (`TOTK_ASTC`). Before that, CPU + decodes made 0.4-0.7 s load frames on the 1050 Ti. The UHD 630 reports ASTC, so it takes the native upload path + (not yet confirmed in a run). - `cannot alias its pool memory` fills its 16-line log limit on NVIDIA and on the RX 9070 alike (1920x1080, 1600x900, - 800x450 render targets); a suspect for the black patches of backlog 0a (`build/alias-plan.md`). + 800x450 render targets); a suspect for the black patches of backlog item 2. -A run on another machine goes through the user: copy `build/release/TotK-perf-log.vbs` next to the `totk.exe` you -hand over. It starts the game with `TOTK_TRACE=perf` and writes `logs\perf--