mirror of
https://github.com/valentderah/totk-pc-recomp.git
synced 2026-11-22 13:05:57 +00:00
The setup installs a build kit into the game folder, and the game's rebuild for code mods runs from it (#845)
The rebuild for ExeFS patches, code cheats and mod plugins ran the setup window from the setup folder, whose absolute path build-cache/rebuild.txt held, with the generated C and objects in that folder's temp/. A moved or deleted setup folder left code mods without a rebuild. - A new Kit step after the check copies the release folder without TotK-Setup.exe into <game>/build-kit (installer/layout.py KIT_ITEMS), and every later step runs its tools from there. A newer setup replaces the kit, the kit's own rebuild keeps it. - The release's temp folder is <game>/build-cache/work, so the lift cache and the objects stay with the game. - rebuild.txt names the kit relative to the game folder; runtime/cheats.c and runtime/romfs.c resolve relative entries against the exe's folder. - runtime_key drops the lines of clang --version that name its folder, so a rebuild from the kit or a moved game folder keeps the compile and prepare steps the setup recorded. - An update over an install of an older setup moves that setup's temp folder into build-cache/work when both are on one disk. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
d33d8d2e2f
commit
501f0d3e6f
17 files changed
+410
-65
No files matched your search
+40
-20
@@ -1,8 +1,9 @@
|
||||
# Installer
|
||||
|
||||
A public release that turns the player's own game files into a playable TotK folder. The release carries no game
|
||||
data and no code derived from the game: the installer builds the game on the player's machine. The installed folder
|
||||
holds only the game; the installer and its tools stay outside it.
|
||||
data and no code derived from the game: the installer builds the game on the player's machine. On Windows the
|
||||
installed folder also holds the build kit, the installer's tools the game rebuilds itself with for code mods, so the
|
||||
setup folder can be deleted after the install and the install folder moved.
|
||||
|
||||
## What the player does
|
||||
|
||||
@@ -16,7 +17,8 @@ holds only the game; the installer and its tools stay outside it.
|
||||
needed for Chinese, Korean and Traditional Chinese text;
|
||||
- the game language and the install folder.
|
||||
3. Waits for the setup, one time. The window shows each step and its progress.
|
||||
4. Plays `totk.exe` from the install folder. The installer and the source NSP or XCI (or NSZ, XCZ) can be deleted.
|
||||
4. Plays `totk.exe` from the install folder. The setup folder and the source NSP or XCI (or NSZ, XCZ) can be deleted,
|
||||
and the install folder can be moved: code mods rebuild the game with the build kit inside it.
|
||||
|
||||
The install folder (`main.c` `portable_paths()` reads this layout; it stays portable):
|
||||
|
||||
@@ -32,14 +34,22 @@ The install folder (`main.c` `portable_paths()` reads this layout; it stays port
|
||||
logs/ totk.log of the last run, totk.1.log to totk.4.log of the runs before it
|
||||
shader and pipeline caches prepared by the setup; shader-cache/shader-cache.pack holds the translations
|
||||
totk-install.json what built this folder: release version, input hashes, finished steps
|
||||
build-kit/ Windows: the release folder without TotK-Setup.exe (about 0.65 GB); the game's rebuild
|
||||
for code mods runs the setup window from it
|
||||
build-cache/ what the rebuild built in (cheats.status, exefs.status, plugins.status), rebuild.txt,
|
||||
and work/: the generated C, the lift cache and the objects (about 1.6 GB, Windows)
|
||||
```
|
||||
|
||||
## Updating the port
|
||||
|
||||
The player runs a newer installer on the same folder. It reads `totk-install.json`, keeps `romfs/`, `exefs/`,
|
||||
`shared-fonts/`, `profile/` and `totk.ini` after checking their hashes, rebuilds `totk.exe` and re-prepares the
|
||||
shaders. A newer release that changes nothing game-derived skips the rebuild. The window starts with the last install folder
|
||||
and the base, update, keys and firmware paths it ran with (paths only, never the keys file's contents), kept in
|
||||
`shared-fonts/`, `profile/` and `totk.ini` after checking their hashes, replaces `build-kit/` with its own files,
|
||||
rebuilds `totk.exe` and re-prepares the shaders. The setup is an installer and updater: nothing after the install
|
||||
needs its folder. An install made by a setup before the build kit kept its temp folder in that setup's folder
|
||||
(`<setup>/temp/<install name>-<hash>`, the setup `build-cache/rebuild.txt` named); the first newer setup moves it into
|
||||
`build-cache/work` when both are on one disk (on another it starts empty) and writes the new `rebuild.txt`. Until then
|
||||
the old `rebuild.txt` keeps using the old setup folder. A newer release that changes nothing game-derived skips the
|
||||
rebuild. The window starts with the last install folder and the base, update, keys and firmware paths it ran with (paths only, never the keys file's contents), kept in
|
||||
`%LOCALAPPDATA%\TotK Recomp\setup.json` (`$XDG_STATE_HOME/totk-recomp/setup.json` elsewhere). Entries that no longer exist
|
||||
stay empty. A reinstall or a switch to another game version starts filled in.
|
||||
|
||||
@@ -53,21 +63,27 @@ loads the saves 1.4.3 wrote.
|
||||
`tools/extract.py`), the keys are new enough for the update, enough space is free on the target disk (about 36 GiB at the peak of a
|
||||
first install, 20 GiB after it), the CPU is at least x86-64-v2 (SSE4.2; with AVX2/FMA the faster `v3` build is used,
|
||||
otherwise the `v2` one), a Vulkan 1.3 GPU exists. Each failure is one clear message.
|
||||
2. **Fonts** with Chinese, Korean or Traditional Chinese text and a firmware folder: extract the Switch system fonts
|
||||
2. **Kit** (Windows): copy the release folder without `TotK-Setup.exe` into `build-kit/` in the install folder
|
||||
(`installer/steps.py` `Kit`, `installer/layout.py` `KIT_ITEMS`); every step after it runs its tools from there, so
|
||||
the commands the build records (the compiler's path in the Ninja file) are the ones the game's rebuild runs later.
|
||||
A newer setup replaces the kit; the kit's own rebuild keeps it. The steps' keys fingerprint file contents, not
|
||||
paths (`runtime_key` leaves out the lines of `clang --version` that name its folder), so a rebuild from the kit
|
||||
or from a moved install folder keeps every step the setup recorded.
|
||||
3. **Fonts** with Chinese, Korean or Traditional Chinese text and a firmware folder: extract the Switch system fonts
|
||||
from the firmware into `shared-fonts/` (MacNativeGames' `extract_shared_fonts.py`, which verifies every NCA layer
|
||||
and BFTTF header). Without a firmware folder the setup warns and goes on; the game then lacks those glyphs.
|
||||
3. **Extract** ExeFS and RomFS into the install folder and verify the hashes.
|
||||
4. **Lift** the ExeFS to C (`recomp/gen.py`, `recomp/patches-v121.txt` or `patches-v143.txt`) into a temp folder next
|
||||
to the installer. The code cheats of the mods' cheat files and the mods' ExeFS patch files become patch sets first
|
||||
4. **Extract** ExeFS and RomFS into the install folder and verify the hashes.
|
||||
5. **Lift** the ExeFS to C (`recomp/gen.py`, `recomp/patches-v121.txt` or `patches-v143.txt`) into the temp folder,
|
||||
`build-cache/work` in the install folder (`.setup-temp` on Linux, `--temp DIR` elsewhere). The code cheats of the mods' cheat files and the mods' ExeFS patch files become patch sets first
|
||||
(`tools/cheats.py`, `tools/exefs_patches.py`, status in `build-cache/cheats.status` and `exefs.status`); a file the
|
||||
tools refuse costs those cheats or patches, not the install. The patch files' writes into main's data go to
|
||||
`build-cache/exefs.data`, which the game stores into main at start; after the lift they are checked against the
|
||||
bytes the lifted code read as constants (`<gen>/main/folded.txt`, its jump tables).
|
||||
5. **Compile** the generated C, `modules.c`, `romfs.c`, `patches.c` and `ov_zstd.c` (they bake the per-install layout, RomFS
|
||||
6. **Compile** the generated C, `modules.c`, `romfs.c`, `patches.c` and `ov_zstd.c` (they bake the per-install layout, RomFS
|
||||
hashes and the game version's patch page) and link
|
||||
them with the prebuilt runtime library for the host's level (`libtotkrt.a` or `libtotkrt-v2.a`) into `totk.exe`
|
||||
(`build_game.py --runtime-lib`; about 3 min on 24 cores).
|
||||
6. **Prepare** (only when the runtime, the translator or the game version changed, not for a new lift) the RomFS
|
||||
7. **Prepare** (only when the runtime, the translator or the game version changed, not for a new lift) the RomFS
|
||||
patches and the shaders (`tools/make_romfs_patches.py`, `tools/prepare_shaders.py`,
|
||||
`totk.exe --prepare-shaders`) in the install folder, then fold the shader cache's loose files (four per shader, about
|
||||
117,000) into `shader-cache/shader-cache.pack` (`tools/pack_shader_cache.py`). totk.exe reads the pack first and
|
||||
@@ -82,7 +98,7 @@ loads the saves 1.4.3 wrote.
|
||||
When the graphics driver crashes while building the shaders, `prepare_shaders.py` runs that part again without the
|
||||
variants it was building ([building.md](building.md), step 6), and the setup finishes with a warning that names how
|
||||
many were left out.
|
||||
7. **Shapes**: the UI laid out for each screen shape this player needs, `romfs-patches-<w>x<h>`
|
||||
8. **Shapes**: the UI laid out for each screen shape this player needs, `romfs-patches-<w>x<h>`
|
||||
(`--aspect`), and with Xbox or PC prompts in `totk.ini` that layout with those prompts too,
|
||||
`romfs-patches-<style>-<w>x<h>` (`--prompts <style> --aspect --reuse romfs-patches-<style>`, which hard-links every
|
||||
file but the UI archive; for `pc` with the same `--binds` as the PC prompts step, whose key the step follows). The shapes are those of every connected display (Windows: `EnumDisplayDevices`; Linux:
|
||||
@@ -93,20 +109,20 @@ loads the saves 1.4.3 wrote.
|
||||
again when the displays, the prompts or the patches change; at a game resolution whose shape has no folder the
|
||||
game draws 16:9 with bars, and the settings menu marks such sizes and can write the shape's folders in the
|
||||
background for the next start.
|
||||
8. **DLSS** (the window's tick box, on by default, or `--dlss`; Windows with an NVIDIA RTX GPU): download
|
||||
9. **DLSS** (the window's tick box, on by default, or `--dlss`; Windows with an NVIDIA RTX GPU): download
|
||||
`nvngx_dlss.dll` from NVIDIA's own DLSS SDK repository (github.com/NVIDIA/DLSS, a pinned tag, NVIDIA RTX SDK
|
||||
license) into the install folder and check its size and SHA-256 (`steps.py`, `Dlss`). The release does not ship
|
||||
the library; NGX loads it from the folder of totk.exe for `dlaa` and the DLSS modes. A DLSS library already in the
|
||||
folder (a newer one the player put there) is kept. Without an RTX GPU the step only logs and does nothing; a failed
|
||||
download warns, the game runs without DLSS, and the next run of the setup tries again.
|
||||
9. **Clean up** the temp folder, except the generated C and the objects (about 1.3 GB), and write
|
||||
10. **Clean up** the temp folder, except the generated C and the objects (about 1.3 GB), and write
|
||||
`build-cache/rebuild.txt` in the install folder.
|
||||
|
||||
A cancelled or failed setup resumes from the last finished step (`totk-install.json`).
|
||||
|
||||
## Rebuilding for the mods
|
||||
|
||||
The generated C and the objects stay in the temp folder. The next lift lifts again only the chunks whose inputs
|
||||
The generated C and the objects stay in the temp folder, inside the install folder. The next lift lifts again only the chunks whose inputs
|
||||
changed (the lift cache of `recomp/gen.py`: the patch sites inside a chunk, with their set indices) and rewrites only
|
||||
those whose C changed, and Ninja recompiles only those. A new or edited code cheat costs a few seconds of lift
|
||||
(discovery and a hash of the kept chunks; 7 s for one changed cheat word on 12 cores, against 102 s for the whole lift)
|
||||
@@ -116,13 +132,16 @@ change to `recomp/*.py` lifts everything.
|
||||
|
||||
`build-cache/rebuild.txt` holds the command that opens the setup window in its rebuild mode on the install folder
|
||||
(`installer/gui.py --rebuild <install folder>`, started through `python -c` so the release folder is on `sys.path`
|
||||
whatever the working folder), one argument per line. When an enabled mod has code cheats or ExeFS patch files the build
|
||||
whatever the working folder), one argument per line. On Windows it names the build kit relative to the install folder
|
||||
(`build-kit\python\pythonw.exe`, `build-kit`, `--rebuild .`): `totk.exe` takes the program from its own folder and
|
||||
starts the command there, so no path of the setup or of the install folder is stored. When an enabled mod has code cheats or ExeFS patch files the build
|
||||
lacks, `totk.exe` runs it at start and exits (`runtime/cheats.c` `rebuild_if_needed`); the
|
||||
window runs the setup on the folder at once and starts the game again when it succeeds. A failed or cancelled rebuild
|
||||
offers to start the game with the code it has; `build-cache/rebuild.skip` then keeps the same files from starting it
|
||||
again, until "Rebuild the game now" on the Cheats or Mods tab. A command that does not start at all leaves no
|
||||
`rebuild.skip`, so the next start tries it again. `[mods] mods_rebuild = off` turns it off. The rebuild needs the
|
||||
release folder the setup ran from: a moved or deleted release folder means running a setup again. On Linux the command
|
||||
`rebuild.skip`, so the next start tries it again. `[mods] mods_rebuild = off` turns it off. A moved install folder
|
||||
keeps working, but its first rebuild compiles everything once: Ninja's records hold the old absolute paths of the
|
||||
generated C and of the compiler's own headers. On Linux the command
|
||||
starts the AppImage file itself (`$APPIMAGE --rebuild <install folder>`), because the AppImage's mounted folder is gone
|
||||
once the setup exits; a moved or deleted AppImage means running a setup again. Installs set up by an AppImage before
|
||||
this change hold a command into that mount: running the new setup once on the install folder writes the new one.
|
||||
@@ -170,7 +189,8 @@ python -m installer --base FILE --update FILE --keys FILE --lang CODE --dest DIR
|
||||
The unpacked release; `tools/make_release.py` assembles it and `--zip` packs it:
|
||||
|
||||
```
|
||||
TotK-Setup.exe starts python/pythonw.exe -m installer.gui (installer/launcher/launcher.c, no console)
|
||||
TotK-Setup.exe starts python/pythonw.exe -m installer.gui (installer/launcher/launcher.c, no console); the
|
||||
rest is what the Kit step copies into the install folder's build-kit/
|
||||
installer/ the installer: steps, CLI, window, en/ru/zh-Hans text catalog
|
||||
python/ Python 3.14.9 embeddable + numpy, lz4, cryptography, cffi, pycparser (+ tkinter for the window)
|
||||
recomp/, tools/ the lifter and the build, extraction and shader scripts the steps run (copied from the repo)
|
||||
|
||||
+5
-3
@@ -80,9 +80,11 @@ ahead of time, so these patches are built into `totk.exe`:
|
||||
2. Until the rebuild succeeds, the whole mod stays off, its other files included.
|
||||
3. After the rebuild the mod switches on and off from the Mods tab like any other, without another rebuild.
|
||||
|
||||
The rebuild needs the setup folder the game was installed from. When that folder was moved or deleted, run
|
||||
`TotK-Setup.exe` again on the same install folder: it does the same job. `mods_rebuild = off` under `[mods]` in
|
||||
`totk.ini` turns the automatic rebuild off.
|
||||
The rebuild runs from `build-kit` in the game folder, which the setup put there: the setup folder can be deleted
|
||||
and the game folder moved. The first rebuild after a move compiles the whole game once (several minutes). A game
|
||||
installed by an older setup (no `build-kit` in its folder) still rebuilds from that setup folder until a newer
|
||||
setup runs once on it.
|
||||
`mods_rebuild = off` under `[mods]` in `totk.ini` turns the automatic rebuild off.
|
||||
|
||||
A code patch over one of the port's own patches (the 60 FPS patch, the resolution patch and the other entries under
|
||||
**Built into the port**) turns that built-in patch off while the mod is on. The mod's row names it.
|
||||
|
||||
Reference in new issue
Block a user