MacNativeGames now translates each shader's default bind state when it builds and keeps only other variants in its shared list, so the list at origin/HEAD has 134 rows of 1.4.3 shaders. prepare_shaders.py seeds the default variant of every shader it finds and reads the list through tools/shader_list.py, which adds the V rows older revisions held. On 1.4.3 that prepares 29,587 variants in 7 min with 0 translation failures, and every one of the 1,095 variants the field and intro runs met in play is among them. The installer's dev path and make_release.py read the list the same way. Also record that 1.2.1 loads saves 1.4.3 wrote. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6.9 KiB
TotK Recomp
Static recompilation of Tears of the Kingdom (Switch, ARM64) into a native Windows x86-64 exe with a Vulkan renderer. Plan: docs/architecture.md.
Game content boundary
This repository is public. Game data lives outside it and every derived file stays in ignored folders.
- The user's game and key paths are in
config.local.toml(ignored). Pass the keys file path to tools; the tools read it themselves. Key values stay out of your output, logs, commands and files. - Everything extracted or generated goes under
build/orgenerated/. - Every file the code generator emits carries the marker defined as
GENERATED_MARKERintools/check_content.py. - Run
python tools/check_content.py --stagedbefore proposing a commit. It is the same check CI runs.
Reusing code
refs/ holds upstream checkouts at the commits the research notes cite (tools/fetch_refs.sh recreates it). Read-only.
- Copy only MIT, BSD, ISC, zlib or Apache-2.0 code, keep its header, and add a row to
THIRD_PARTY_NOTICES.md. - yuzu, suyu, OpenSmash-Ultimate and other GPL code: read for behaviour, write our own.
Research notes
Read the note before working on its area; each cites file:line in refs/.
- Lifter, generator, Windows memory layer, differential tests: docs/research/lasrecomp.md
- TotK host behaviour, NVN, shaders, extraction, version tables, patches: docs/research/macnativegames.md
- Other ports and recompilers: docs/research/prior-art.md
- Sizes, function and instruction counts, OS touch-points of the 1.2.1 modules: docs/research/totk-v121-inventory.md
- Maxwell GPU semantics our Vulkan path must match, TotK bugs other ports and emulators fixed: docs/research/maxwell.md
- Symptoms other ports already solved (artifacts, crashes, pacing, scheduling) and whether we have the fix; search it before investigating any symptom: docs/research/port-lessons.md
Toolchain
Setup and verify commands: docs/toolchain-windows.md, docs/toolchain-msys2-python.md.
-
Build C with MSYS2 UCRT64 Clang (
C:\msys64\ucrt64\bin). Generated C uses GNU extensions, socl.exeis out. -
A 32-bit winlibs
cmake/ninjasits early on the user PATH. Call CMake and Ninja fromC:\msys64\ucrt64\binby full path or put that directory first. -
Two Python environments:
.venv(3.12) for the lifter and LAS Recomp tools,.venv-py314for MacNativeGames' extraction scripts, which needcompression.zstd. -
Game extraction:
.venv-py314\Scripts\python tools\extract.pywrites verified ExeFS and RomFS tobuild/extracted/v0(1.0.0) andbuild/extracted/v121(1.2.1, the primary target).tools/vendor/stays byte-identical to upstream; Windows changes go intools/extract.py. -
Lifter (
recomp/) and CPU headers (runtime/) start from LAS Recomp. Every lifter change passes the differential tests against Unicorn, all run with.venv:tests\diff\lift.py: instructions sampled from the extracted 1.2.1 modules (one per opcode signature, then random);--coveragelists every word the lifter rejects or turns into a trap.tests\diff\simd.py: the whole FP/SIMD encoding space; needs no game files.tests\diff\atomic.py: exclusive load/store sequences and ARMv8.1 atomics; needs no game files.tests\diff\func.py: whole leaf functions as the generator emits them, loops and jump tables included.tests\excl_mt.py: the exclusive monitor helpers ofcpu.hunder contention and ABA, on several threads; needs no game files.
-
Game code build (
.venv):recomp\gen.py --exefs build\extracted\v121\exefs --out generated\v121 --patches recomp\patches-v121.txtwrites the C (about 15 s). The patch table holds the 60 fps and real-time frame step code patches, switchable at run time (recomp/patches.py);.venv-py314\Scripts\python tools\make_romfs_patches.pywrites their RomFS half tobuild\v121\romfs-patches.tools\build_game.pycompiles and linksbuild\v121\totk.exewith Ninja (about 200 s clean, incremental after).recomp/layout.pyalone decides module bases. Jump tables are resolved byrecomp/jumptables.py; after changing it,tests\jumptables_check.pymust report 0 disagreements with its Unicorn oracle (on 1.4.3 too:--exefs build\extracted\v143\exefs). -
Game version 1.4.3: the same commands with
v143(--patches recomp\patches-v143.txt,build_game.py --version v143,make_romfs_patches.py --version v143). Its builds default tobuild\profile-v143;run_game.pyruns it on copies of the 1.2.1 save backups. -
Shader census (
.venv-py314):tools\shader_census.py --version v121 [--translate]extracts every Maxwell shader tobuild/v121/shaders/and runs it throughbuild\xlate\totk_xlate.exeandspirv-val; failure groups go tofailures.txt.tools\xlate_smoke.pychecks the translator alone, without game files. -
Shader preparation (
.venv-py314, once per build dir and translator):tools\prepare_shaders.py build\v121translates every game shader's default variant and the variants of the MacNativeGames shared shader list (read fromrefs/at run time, with the rows its history dropped:tools/shader_list.py) into the build's shader cache and builds their stage libraries into the driver cache (totk.exe --prepare-shaders; 1.4.3: 29,587 variants, 7 min on 24 threads, 1.7 GB); play then translates and compiles only variants neither covers. -
Open work with its evidence: docs/backlog.md.
-
Installer (
installer/,tools/make_release.py), which builds the game on a player's machine: docs/installer.md. -
Running, measuring or frame-dumping the game, save backups, task worktrees and their builds: docs/agent-playbook.md.
-
Project skills (
.claude/skills/):private-buildbefore changing runtime/ or tools/ code and building totk.exe (one git worktree per task);run-gameto run a build into gameplay from a save. -
hactool:
toolchain/hactool/hactool.exe. From an MSYS2 shell pass-k <keys>explicitly; the shell'sHOMEhides the default key location. -
Ghidra 12.1.2 with SwitchLoader:
toolchain/ghidra/ghidra_12.1.2_PUBLIC. Game modules (NSO) open here, headless or GUI. -
REA (
reverse-engineer-anythingskill + MCP server) analyses our own x86-64 builds. On Windows its Ghidra provider admits PE files only, so NSO analysis goes through Ghidra directly. Register the MCP server once per clone, at local scope:claude mcp add --scope local rea -e "GHIDRA_INSTALL_DIR=<repo>\toolchain\ghidra\ghidra_12.1.2_PUBLIC" -e "JAVA_HOME=<jdk-21>" -- cmd.exe /d /c npx -y rea-agents@6.7.0 mcp
Writing
Code, comments, docs and commit messages are in English. Commit messages are plain sentences without type prefixes.