The setup installs a build kit into the game folder, and the game's rebuild for code mods runs from it (#845)

The rebuild for ExeFS patches, code cheats and mod plugins ran the setup window from the setup folder, whose absolute
path build-cache/rebuild.txt held, with the generated C and objects in that folder's temp/. A moved or deleted setup
folder left code mods without a rebuild.

- A new Kit step after the check copies the release folder without TotK-Setup.exe into <game>/build-kit
  (installer/layout.py KIT_ITEMS), and every later step runs its tools from there. A newer setup replaces the kit,
  the kit's own rebuild keeps it.
- The release's temp folder is <game>/build-cache/work, so the lift cache and the objects stay with the game.
- rebuild.txt names the kit relative to the game folder; runtime/cheats.c and runtime/romfs.c resolve relative
  entries against the exe's folder.
- runtime_key drops the lines of clang --version that name its folder, so a rebuild from the kit or a moved game
  folder keeps the compile and prepare steps the setup recorded.
- An update over an install of an older setup moves that setup's temp folder into build-cache/work when both are on
  one disk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Valiantsin DzerakhandClaude Opus 5.5 committed 2026-11-17 10:54:17 +03:00
1 parent d33d8d2e2f
commit 501f0d3e6f
17 files changed
+410 -65

No files matched your search

+40 -20
View File
@@ -1,8 +1,9 @@
# Installer
A public release that turns the player's own game files into a playable TotK folder. The release carries no game
data and no code derived from the game: the installer builds the game on the player's machine. The installed folder
holds only the game; the installer and its tools stay outside it.
data and no code derived from the game: the installer builds the game on the player's machine. On Windows the
installed folder also holds the build kit, the installer's tools the game rebuilds itself with for code mods, so the
setup folder can be deleted after the install and the install folder moved.
## What the player does
@@ -16,7 +17,8 @@ holds only the game; the installer and its tools stay outside it.
needed for Chinese, Korean and Traditional Chinese text;
- the game language and the install folder.
3. Waits for the setup, one time. The window shows each step and its progress.
4. Plays `totk.exe` from the install folder. The installer and the source NSP or XCI (or NSZ, XCZ) can be deleted.
4. Plays `totk.exe` from the install folder. The setup folder and the source NSP or XCI (or NSZ, XCZ) can be deleted,
and the install folder can be moved: code mods rebuild the game with the build kit inside it.
The install folder (`main.c` `portable_paths()` reads this layout; it stays portable):
@@ -32,14 +34,22 @@ The install folder (`main.c` `portable_paths()` reads this layout; it stays port
logs/ totk.log of the last run, totk.1.log to totk.4.log of the runs before it
shader and pipeline caches prepared by the setup; shader-cache/shader-cache.pack holds the translations
totk-install.json what built this folder: release version, input hashes, finished steps
build-kit/ Windows: the release folder without TotK-Setup.exe (about 0.65 GB); the game's rebuild
for code mods runs the setup window from it
build-cache/ what the rebuild built in (cheats.status, exefs.status, plugins.status), rebuild.txt,
and work/: the generated C, the lift cache and the objects (about 1.6 GB, Windows)
```
## Updating the port
The player runs a newer installer on the same folder. It reads `totk-install.json`, keeps `romfs/`, `exefs/`,
`shared-fonts/`, `profile/` and `totk.ini` after checking their hashes, rebuilds `totk.exe` and re-prepares the
shaders. A newer release that changes nothing game-derived skips the rebuild. The window starts with the last install folder
and the base, update, keys and firmware paths it ran with (paths only, never the keys file's contents), kept in
`shared-fonts/`, `profile/` and `totk.ini` after checking their hashes, replaces `build-kit/` with its own files,
rebuilds `totk.exe` and re-prepares the shaders. The setup is an installer and updater: nothing after the install
needs its folder. An install made by a setup before the build kit kept its temp folder in that setup's folder
(`<setup>/temp/<install name>-<hash>`, the setup `build-cache/rebuild.txt` named); the first newer setup moves it into
`build-cache/work` when both are on one disk (on another it starts empty) and writes the new `rebuild.txt`. Until then
the old `rebuild.txt` keeps using the old setup folder. A newer release that changes nothing game-derived skips the
rebuild. The window starts with the last install folder and the base, update, keys and firmware paths it ran with (paths only, never the keys file's contents), kept in
`%LOCALAPPDATA%\TotK Recomp\setup.json` (`$XDG_STATE_HOME/totk-recomp/setup.json` elsewhere). Entries that no longer exist
stay empty. A reinstall or a switch to another game version starts filled in.
@@ -53,21 +63,27 @@ loads the saves 1.4.3 wrote.
`tools/extract.py`), the keys are new enough for the update, enough space is free on the target disk (about 36 GiB at the peak of a
first install, 20 GiB after it), the CPU is at least x86-64-v2 (SSE4.2; with AVX2/FMA the faster `v3` build is used,
otherwise the `v2` one), a Vulkan 1.3 GPU exists. Each failure is one clear message.
2. **Fonts** with Chinese, Korean or Traditional Chinese text and a firmware folder: extract the Switch system fonts
2. **Kit** (Windows): copy the release folder without `TotK-Setup.exe` into `build-kit/` in the install folder
(`installer/steps.py` `Kit`, `installer/layout.py` `KIT_ITEMS`); every step after it runs its tools from there, so
the commands the build records (the compiler's path in the Ninja file) are the ones the game's rebuild runs later.
A newer setup replaces the kit; the kit's own rebuild keeps it. The steps' keys fingerprint file contents, not
paths (`runtime_key` leaves out the lines of `clang --version` that name its folder), so a rebuild from the kit
or from a moved install folder keeps every step the setup recorded.
3. **Fonts** with Chinese, Korean or Traditional Chinese text and a firmware folder: extract the Switch system fonts
from the firmware into `shared-fonts/` (MacNativeGames' `extract_shared_fonts.py`, which verifies every NCA layer
and BFTTF header). Without a firmware folder the setup warns and goes on; the game then lacks those glyphs.
3. **Extract** ExeFS and RomFS into the install folder and verify the hashes.
4. **Lift** the ExeFS to C (`recomp/gen.py`, `recomp/patches-v121.txt` or `patches-v143.txt`) into a temp folder next
to the installer. The code cheats of the mods' cheat files and the mods' ExeFS patch files become patch sets first
4. **Extract** ExeFS and RomFS into the install folder and verify the hashes.
5. **Lift** the ExeFS to C (`recomp/gen.py`, `recomp/patches-v121.txt` or `patches-v143.txt`) into the temp folder,
`build-cache/work` in the install folder (`.setup-temp` on Linux, `--temp DIR` elsewhere). The code cheats of the mods' cheat files and the mods' ExeFS patch files become patch sets first
(`tools/cheats.py`, `tools/exefs_patches.py`, status in `build-cache/cheats.status` and `exefs.status`); a file the
tools refuse costs those cheats or patches, not the install. The patch files' writes into main's data go to
`build-cache/exefs.data`, which the game stores into main at start; after the lift they are checked against the
bytes the lifted code read as constants (`<gen>/main/folded.txt`, its jump tables).
5. **Compile** the generated C, `modules.c`, `romfs.c`, `patches.c` and `ov_zstd.c` (they bake the per-install layout, RomFS
6. **Compile** the generated C, `modules.c`, `romfs.c`, `patches.c` and `ov_zstd.c` (they bake the per-install layout, RomFS
hashes and the game version's patch page) and link
them with the prebuilt runtime library for the host's level (`libtotkrt.a` or `libtotkrt-v2.a`) into `totk.exe`
(`build_game.py --runtime-lib`; about 3 min on 24 cores).
6. **Prepare** (only when the runtime, the translator or the game version changed, not for a new lift) the RomFS
7. **Prepare** (only when the runtime, the translator or the game version changed, not for a new lift) the RomFS
patches and the shaders (`tools/make_romfs_patches.py`, `tools/prepare_shaders.py`,
`totk.exe --prepare-shaders`) in the install folder, then fold the shader cache's loose files (four per shader, about
117,000) into `shader-cache/shader-cache.pack` (`tools/pack_shader_cache.py`). totk.exe reads the pack first and
@@ -82,7 +98,7 @@ loads the saves 1.4.3 wrote.
When the graphics driver crashes while building the shaders, `prepare_shaders.py` runs that part again without the
variants it was building ([building.md](building.md), step 6), and the setup finishes with a warning that names how
many were left out.
7. **Shapes**: the UI laid out for each screen shape this player needs, `romfs-patches-<w>x<h>`
8. **Shapes**: the UI laid out for each screen shape this player needs, `romfs-patches-<w>x<h>`
(`--aspect`), and with Xbox or PC prompts in `totk.ini` that layout with those prompts too,
`romfs-patches-<style>-<w>x<h>` (`--prompts <style> --aspect --reuse romfs-patches-<style>`, which hard-links every
file but the UI archive; for `pc` with the same `--binds` as the PC prompts step, whose key the step follows). The shapes are those of every connected display (Windows: `EnumDisplayDevices`; Linux:
@@ -93,20 +109,20 @@ loads the saves 1.4.3 wrote.
again when the displays, the prompts or the patches change; at a game resolution whose shape has no folder the
game draws 16:9 with bars, and the settings menu marks such sizes and can write the shape's folders in the
background for the next start.
8. **DLSS** (the window's tick box, on by default, or `--dlss`; Windows with an NVIDIA RTX GPU): download
9. **DLSS** (the window's tick box, on by default, or `--dlss`; Windows with an NVIDIA RTX GPU): download
`nvngx_dlss.dll` from NVIDIA's own DLSS SDK repository (github.com/NVIDIA/DLSS, a pinned tag, NVIDIA RTX SDK
license) into the install folder and check its size and SHA-256 (`steps.py`, `Dlss`). The release does not ship
the library; NGX loads it from the folder of totk.exe for `dlaa` and the DLSS modes. A DLSS library already in the
folder (a newer one the player put there) is kept. Without an RTX GPU the step only logs and does nothing; a failed
download warns, the game runs without DLSS, and the next run of the setup tries again.
9. **Clean up** the temp folder, except the generated C and the objects (about 1.3 GB), and write
10. **Clean up** the temp folder, except the generated C and the objects (about 1.3 GB), and write
`build-cache/rebuild.txt` in the install folder.
A cancelled or failed setup resumes from the last finished step (`totk-install.json`).
## Rebuilding for the mods
The generated C and the objects stay in the temp folder. The next lift lifts again only the chunks whose inputs
The generated C and the objects stay in the temp folder, inside the install folder. The next lift lifts again only the chunks whose inputs
changed (the lift cache of `recomp/gen.py`: the patch sites inside a chunk, with their set indices) and rewrites only
those whose C changed, and Ninja recompiles only those. A new or edited code cheat costs a few seconds of lift
(discovery and a hash of the kept chunks; 7 s for one changed cheat word on 12 cores, against 102 s for the whole lift)
@@ -116,13 +132,16 @@ change to `recomp/*.py` lifts everything.
`build-cache/rebuild.txt` holds the command that opens the setup window in its rebuild mode on the install folder
(`installer/gui.py --rebuild <install folder>`, started through `python -c` so the release folder is on `sys.path`
whatever the working folder), one argument per line. When an enabled mod has code cheats or ExeFS patch files the build
whatever the working folder), one argument per line. On Windows it names the build kit relative to the install folder
(`build-kit\python\pythonw.exe`, `build-kit`, `--rebuild .`): `totk.exe` takes the program from its own folder and
starts the command there, so no path of the setup or of the install folder is stored. When an enabled mod has code cheats or ExeFS patch files the build
lacks, `totk.exe` runs it at start and exits (`runtime/cheats.c` `rebuild_if_needed`); the
window runs the setup on the folder at once and starts the game again when it succeeds. A failed or cancelled rebuild
offers to start the game with the code it has; `build-cache/rebuild.skip` then keeps the same files from starting it
again, until "Rebuild the game now" on the Cheats or Mods tab. A command that does not start at all leaves no
`rebuild.skip`, so the next start tries it again. `[mods] mods_rebuild = off` turns it off. The rebuild needs the
release folder the setup ran from: a moved or deleted release folder means running a setup again. On Linux the command
`rebuild.skip`, so the next start tries it again. `[mods] mods_rebuild = off` turns it off. A moved install folder
keeps working, but its first rebuild compiles everything once: Ninja's records hold the old absolute paths of the
generated C and of the compiler's own headers. On Linux the command
starts the AppImage file itself (`$APPIMAGE --rebuild <install folder>`), because the AppImage's mounted folder is gone
once the setup exits; a moved or deleted AppImage means running a setup again. Installs set up by an AppImage before
this change hold a command into that mount: running the new setup once on the install folder writes the new one.
@@ -170,7 +189,8 @@ python -m installer --base FILE --update FILE --keys FILE --lang CODE --dest DIR
The unpacked release; `tools/make_release.py` assembles it and `--zip` packs it:
```
TotK-Setup.exe starts python/pythonw.exe -m installer.gui (installer/launcher/launcher.c, no console)
TotK-Setup.exe starts python/pythonw.exe -m installer.gui (installer/launcher/launcher.c, no console); the
rest is what the Kit step copies into the install folder's build-kit/
installer/ the installer: steps, CLI, window, en/ru/zh-Hans text catalog
python/ Python 3.14.9 embeddable + numpy, lz4, cryptography, cffi, pycparser (+ tkinter for the window)
recomp/, tools/ the lifter and the build, extraction and shader scripts the steps run (copied from the repo)
+5 -3
View File
@@ -80,9 +80,11 @@ ahead of time, so these patches are built into `totk.exe`:
2. Until the rebuild succeeds, the whole mod stays off, its other files included.
3. After the rebuild the mod switches on and off from the Mods tab like any other, without another rebuild.
The rebuild needs the setup folder the game was installed from. When that folder was moved or deleted, run
`TotK-Setup.exe` again on the same install folder: it does the same job. `mods_rebuild = off` under `[mods]` in
`totk.ini` turns the automatic rebuild off.
The rebuild runs from `build-kit` in the game folder, which the setup put there: the setup folder can be deleted
and the game folder moved. The first rebuild after a move compiles the whole game once (several minutes). A game
installed by an older setup (no `build-kit` in its folder) still rebuilds from that setup folder until a newer
setup runs once on it.
`mods_rebuild = off` under `[mods]` in `totk.ini` turns the automatic rebuild off.
A code patch over one of the port's own patches (the 60 FPS patch, the resolution patch and the other entries under
**Built into the port**) turns that built-in patch off while the mod is on. The mod's row names it.