mirror of
https://github.com/valentderah/totk-pc-recomp.git
synced 2026-11-19 12:25:56 +00:00
The setup installs a build kit into the game folder, and the game's rebuild for code mods runs from it (#845)
The rebuild for ExeFS patches, code cheats and mod plugins ran the setup window from the setup folder, whose absolute path build-cache/rebuild.txt held, with the generated C and objects in that folder's temp/. A moved or deleted setup folder left code mods without a rebuild. - A new Kit step after the check copies the release folder without TotK-Setup.exe into <game>/build-kit (installer/layout.py KIT_ITEMS), and every later step runs its tools from there. A newer setup replaces the kit, the kit's own rebuild keeps it. - The release's temp folder is <game>/build-cache/work, so the lift cache and the objects stay with the game. - rebuild.txt names the kit relative to the game folder; runtime/cheats.c and runtime/romfs.c resolve relative entries against the exe's folder. - runtime_key drops the lines of clang --version that name its folder, so a rebuild from the kit or a moved game folder keeps the compile and prepare steps the setup recorded. - An update over an install of an older setup moves that setup's temp folder into build-cache/work when both are on one disk. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
d33d8d2e2f
commit
501f0d3e6f
17 files changed
+410
-65
No files matched your search
+40
-20
@@ -1,8 +1,9 @@
|
||||
# Installer
|
||||
|
||||
A public release that turns the player's own game files into a playable TotK folder. The release carries no game
|
||||
data and no code derived from the game: the installer builds the game on the player's machine. The installed folder
|
||||
holds only the game; the installer and its tools stay outside it.
|
||||
data and no code derived from the game: the installer builds the game on the player's machine. On Windows the
|
||||
installed folder also holds the build kit, the installer's tools the game rebuilds itself with for code mods, so the
|
||||
setup folder can be deleted after the install and the install folder moved.
|
||||
|
||||
## What the player does
|
||||
|
||||
@@ -16,7 +17,8 @@ holds only the game; the installer and its tools stay outside it.
|
||||
needed for Chinese, Korean and Traditional Chinese text;
|
||||
- the game language and the install folder.
|
||||
3. Waits for the setup, one time. The window shows each step and its progress.
|
||||
4. Plays `totk.exe` from the install folder. The installer and the source NSP or XCI (or NSZ, XCZ) can be deleted.
|
||||
4. Plays `totk.exe` from the install folder. The setup folder and the source NSP or XCI (or NSZ, XCZ) can be deleted,
|
||||
and the install folder can be moved: code mods rebuild the game with the build kit inside it.
|
||||
|
||||
The install folder (`main.c` `portable_paths()` reads this layout; it stays portable):
|
||||
|
||||
@@ -32,14 +34,22 @@ The install folder (`main.c` `portable_paths()` reads this layout; it stays port
|
||||
logs/ totk.log of the last run, totk.1.log to totk.4.log of the runs before it
|
||||
shader and pipeline caches prepared by the setup; shader-cache/shader-cache.pack holds the translations
|
||||
totk-install.json what built this folder: release version, input hashes, finished steps
|
||||
build-kit/ Windows: the release folder without TotK-Setup.exe (about 0.65 GB); the game's rebuild
|
||||
for code mods runs the setup window from it
|
||||
build-cache/ what the rebuild built in (cheats.status, exefs.status, plugins.status), rebuild.txt,
|
||||
and work/: the generated C, the lift cache and the objects (about 1.6 GB, Windows)
|
||||
```
|
||||
|
||||
## Updating the port
|
||||
|
||||
The player runs a newer installer on the same folder. It reads `totk-install.json`, keeps `romfs/`, `exefs/`,
|
||||
`shared-fonts/`, `profile/` and `totk.ini` after checking their hashes, rebuilds `totk.exe` and re-prepares the
|
||||
shaders. A newer release that changes nothing game-derived skips the rebuild. The window starts with the last install folder
|
||||
and the base, update, keys and firmware paths it ran with (paths only, never the keys file's contents), kept in
|
||||
`shared-fonts/`, `profile/` and `totk.ini` after checking their hashes, replaces `build-kit/` with its own files,
|
||||
rebuilds `totk.exe` and re-prepares the shaders. The setup is an installer and updater: nothing after the install
|
||||
needs its folder. An install made by a setup before the build kit kept its temp folder in that setup's folder
|
||||
(`<setup>/temp/<install name>-<hash>`, the setup `build-cache/rebuild.txt` named); the first newer setup moves it into
|
||||
`build-cache/work` when both are on one disk (on another it starts empty) and writes the new `rebuild.txt`. Until then
|
||||
the old `rebuild.txt` keeps using the old setup folder. A newer release that changes nothing game-derived skips the
|
||||
rebuild. The window starts with the last install folder and the base, update, keys and firmware paths it ran with (paths only, never the keys file's contents), kept in
|
||||
`%LOCALAPPDATA%\TotK Recomp\setup.json` (`$XDG_STATE_HOME/totk-recomp/setup.json` elsewhere). Entries that no longer exist
|
||||
stay empty. A reinstall or a switch to another game version starts filled in.
|
||||
|
||||
@@ -53,21 +63,27 @@ loads the saves 1.4.3 wrote.
|
||||
`tools/extract.py`), the keys are new enough for the update, enough space is free on the target disk (about 36 GiB at the peak of a
|
||||
first install, 20 GiB after it), the CPU is at least x86-64-v2 (SSE4.2; with AVX2/FMA the faster `v3` build is used,
|
||||
otherwise the `v2` one), a Vulkan 1.3 GPU exists. Each failure is one clear message.
|
||||
2. **Fonts** with Chinese, Korean or Traditional Chinese text and a firmware folder: extract the Switch system fonts
|
||||
2. **Kit** (Windows): copy the release folder without `TotK-Setup.exe` into `build-kit/` in the install folder
|
||||
(`installer/steps.py` `Kit`, `installer/layout.py` `KIT_ITEMS`); every step after it runs its tools from there, so
|
||||
the commands the build records (the compiler's path in the Ninja file) are the ones the game's rebuild runs later.
|
||||
A newer setup replaces the kit; the kit's own rebuild keeps it. The steps' keys fingerprint file contents, not
|
||||
paths (`runtime_key` leaves out the lines of `clang --version` that name its folder), so a rebuild from the kit
|
||||
or from a moved install folder keeps every step the setup recorded.
|
||||
3. **Fonts** with Chinese, Korean or Traditional Chinese text and a firmware folder: extract the Switch system fonts
|
||||
from the firmware into `shared-fonts/` (MacNativeGames' `extract_shared_fonts.py`, which verifies every NCA layer
|
||||
and BFTTF header). Without a firmware folder the setup warns and goes on; the game then lacks those glyphs.
|
||||
3. **Extract** ExeFS and RomFS into the install folder and verify the hashes.
|
||||
4. **Lift** the ExeFS to C (`recomp/gen.py`, `recomp/patches-v121.txt` or `patches-v143.txt`) into a temp folder next
|
||||
to the installer. The code cheats of the mods' cheat files and the mods' ExeFS patch files become patch sets first
|
||||
4. **Extract** ExeFS and RomFS into the install folder and verify the hashes.
|
||||
5. **Lift** the ExeFS to C (`recomp/gen.py`, `recomp/patches-v121.txt` or `patches-v143.txt`) into the temp folder,
|
||||
`build-cache/work` in the install folder (`.setup-temp` on Linux, `--temp DIR` elsewhere). The code cheats of the mods' cheat files and the mods' ExeFS patch files become patch sets first
|
||||
(`tools/cheats.py`, `tools/exefs_patches.py`, status in `build-cache/cheats.status` and `exefs.status`); a file the
|
||||
tools refuse costs those cheats or patches, not the install. The patch files' writes into main's data go to
|
||||
`build-cache/exefs.data`, which the game stores into main at start; after the lift they are checked against the
|
||||
bytes the lifted code read as constants (`<gen>/main/folded.txt`, its jump tables).
|
||||
5. **Compile** the generated C, `modules.c`, `romfs.c`, `patches.c` and `ov_zstd.c` (they bake the per-install layout, RomFS
|
||||
6. **Compile** the generated C, `modules.c`, `romfs.c`, `patches.c` and `ov_zstd.c` (they bake the per-install layout, RomFS
|
||||
hashes and the game version's patch page) and link
|
||||
them with the prebuilt runtime library for the host's level (`libtotkrt.a` or `libtotkrt-v2.a`) into `totk.exe`
|
||||
(`build_game.py --runtime-lib`; about 3 min on 24 cores).
|
||||
6. **Prepare** (only when the runtime, the translator or the game version changed, not for a new lift) the RomFS
|
||||
7. **Prepare** (only when the runtime, the translator or the game version changed, not for a new lift) the RomFS
|
||||
patches and the shaders (`tools/make_romfs_patches.py`, `tools/prepare_shaders.py`,
|
||||
`totk.exe --prepare-shaders`) in the install folder, then fold the shader cache's loose files (four per shader, about
|
||||
117,000) into `shader-cache/shader-cache.pack` (`tools/pack_shader_cache.py`). totk.exe reads the pack first and
|
||||
@@ -82,7 +98,7 @@ loads the saves 1.4.3 wrote.
|
||||
When the graphics driver crashes while building the shaders, `prepare_shaders.py` runs that part again without the
|
||||
variants it was building ([building.md](building.md), step 6), and the setup finishes with a warning that names how
|
||||
many were left out.
|
||||
7. **Shapes**: the UI laid out for each screen shape this player needs, `romfs-patches-<w>x<h>`
|
||||
8. **Shapes**: the UI laid out for each screen shape this player needs, `romfs-patches-<w>x<h>`
|
||||
(`--aspect`), and with Xbox or PC prompts in `totk.ini` that layout with those prompts too,
|
||||
`romfs-patches-<style>-<w>x<h>` (`--prompts <style> --aspect --reuse romfs-patches-<style>`, which hard-links every
|
||||
file but the UI archive; for `pc` with the same `--binds` as the PC prompts step, whose key the step follows). The shapes are those of every connected display (Windows: `EnumDisplayDevices`; Linux:
|
||||
@@ -93,20 +109,20 @@ loads the saves 1.4.3 wrote.
|
||||
again when the displays, the prompts or the patches change; at a game resolution whose shape has no folder the
|
||||
game draws 16:9 with bars, and the settings menu marks such sizes and can write the shape's folders in the
|
||||
background for the next start.
|
||||
8. **DLSS** (the window's tick box, on by default, or `--dlss`; Windows with an NVIDIA RTX GPU): download
|
||||
9. **DLSS** (the window's tick box, on by default, or `--dlss`; Windows with an NVIDIA RTX GPU): download
|
||||
`nvngx_dlss.dll` from NVIDIA's own DLSS SDK repository (github.com/NVIDIA/DLSS, a pinned tag, NVIDIA RTX SDK
|
||||
license) into the install folder and check its size and SHA-256 (`steps.py`, `Dlss`). The release does not ship
|
||||
the library; NGX loads it from the folder of totk.exe for `dlaa` and the DLSS modes. A DLSS library already in the
|
||||
folder (a newer one the player put there) is kept. Without an RTX GPU the step only logs and does nothing; a failed
|
||||
download warns, the game runs without DLSS, and the next run of the setup tries again.
|
||||
9. **Clean up** the temp folder, except the generated C and the objects (about 1.3 GB), and write
|
||||
10. **Clean up** the temp folder, except the generated C and the objects (about 1.3 GB), and write
|
||||
`build-cache/rebuild.txt` in the install folder.
|
||||
|
||||
A cancelled or failed setup resumes from the last finished step (`totk-install.json`).
|
||||
|
||||
## Rebuilding for the mods
|
||||
|
||||
The generated C and the objects stay in the temp folder. The next lift lifts again only the chunks whose inputs
|
||||
The generated C and the objects stay in the temp folder, inside the install folder. The next lift lifts again only the chunks whose inputs
|
||||
changed (the lift cache of `recomp/gen.py`: the patch sites inside a chunk, with their set indices) and rewrites only
|
||||
those whose C changed, and Ninja recompiles only those. A new or edited code cheat costs a few seconds of lift
|
||||
(discovery and a hash of the kept chunks; 7 s for one changed cheat word on 12 cores, against 102 s for the whole lift)
|
||||
@@ -116,13 +132,16 @@ change to `recomp/*.py` lifts everything.
|
||||
|
||||
`build-cache/rebuild.txt` holds the command that opens the setup window in its rebuild mode on the install folder
|
||||
(`installer/gui.py --rebuild <install folder>`, started through `python -c` so the release folder is on `sys.path`
|
||||
whatever the working folder), one argument per line. When an enabled mod has code cheats or ExeFS patch files the build
|
||||
whatever the working folder), one argument per line. On Windows it names the build kit relative to the install folder
|
||||
(`build-kit\python\pythonw.exe`, `build-kit`, `--rebuild .`): `totk.exe` takes the program from its own folder and
|
||||
starts the command there, so no path of the setup or of the install folder is stored. When an enabled mod has code cheats or ExeFS patch files the build
|
||||
lacks, `totk.exe` runs it at start and exits (`runtime/cheats.c` `rebuild_if_needed`); the
|
||||
window runs the setup on the folder at once and starts the game again when it succeeds. A failed or cancelled rebuild
|
||||
offers to start the game with the code it has; `build-cache/rebuild.skip` then keeps the same files from starting it
|
||||
again, until "Rebuild the game now" on the Cheats or Mods tab. A command that does not start at all leaves no
|
||||
`rebuild.skip`, so the next start tries it again. `[mods] mods_rebuild = off` turns it off. The rebuild needs the
|
||||
release folder the setup ran from: a moved or deleted release folder means running a setup again. On Linux the command
|
||||
`rebuild.skip`, so the next start tries it again. `[mods] mods_rebuild = off` turns it off. A moved install folder
|
||||
keeps working, but its first rebuild compiles everything once: Ninja's records hold the old absolute paths of the
|
||||
generated C and of the compiler's own headers. On Linux the command
|
||||
starts the AppImage file itself (`$APPIMAGE --rebuild <install folder>`), because the AppImage's mounted folder is gone
|
||||
once the setup exits; a moved or deleted AppImage means running a setup again. Installs set up by an AppImage before
|
||||
this change hold a command into that mount: running the new setup once on the install folder writes the new one.
|
||||
@@ -170,7 +189,8 @@ python -m installer --base FILE --update FILE --keys FILE --lang CODE --dest DIR
|
||||
The unpacked release; `tools/make_release.py` assembles it and `--zip` packs it:
|
||||
|
||||
```
|
||||
TotK-Setup.exe starts python/pythonw.exe -m installer.gui (installer/launcher/launcher.c, no console)
|
||||
TotK-Setup.exe starts python/pythonw.exe -m installer.gui (installer/launcher/launcher.c, no console); the
|
||||
rest is what the Kit step copies into the install folder's build-kit/
|
||||
installer/ the installer: steps, CLI, window, en/ru/zh-Hans text catalog
|
||||
python/ Python 3.14.9 embeddable + numpy, lz4, cryptography, cffi, pycparser (+ tkinter for the window)
|
||||
recomp/, tools/ the lifter and the build, extraction and shader scripts the steps run (copied from the repo)
|
||||
|
||||
+5
-3
@@ -80,9 +80,11 @@ ahead of time, so these patches are built into `totk.exe`:
|
||||
2. Until the rebuild succeeds, the whole mod stays off, its other files included.
|
||||
3. After the rebuild the mod switches on and off from the Mods tab like any other, without another rebuild.
|
||||
|
||||
The rebuild needs the setup folder the game was installed from. When that folder was moved or deleted, run
|
||||
`TotK-Setup.exe` again on the same install folder: it does the same job. `mods_rebuild = off` under `[mods]` in
|
||||
`totk.ini` turns the automatic rebuild off.
|
||||
The rebuild runs from `build-kit` in the game folder, which the setup put there: the setup folder can be deleted
|
||||
and the game folder moved. The first rebuild after a move compiles the whole game once (several minutes). A game
|
||||
installed by an older setup (no `build-kit` in its folder) still rebuilds from that setup folder until a newer
|
||||
setup runs once on it.
|
||||
`mods_rebuild = off` under `[mods]` in `totk.ini` turns the automatic rebuild off.
|
||||
|
||||
A code patch over one of the port's own patches (the 60 FPS patch, the resolution patch and the other entries under
|
||||
**Built into the port**) turns that built-in patch off while the mod is on. The mod's row names it.
|
||||
|
||||
+7
-17
@@ -12,7 +12,6 @@ line, 3 cancelled (Ctrl+C or Ctrl+Break).
|
||||
Windows with an NVIDIA RTX GPU (steps.py, Dlss).
|
||||
"""
|
||||
import argparse
|
||||
import hashlib
|
||||
import signal
|
||||
import sys
|
||||
import time
|
||||
@@ -22,7 +21,7 @@ from . import layout as layouts
|
||||
from . import state as statefile
|
||||
from .events import Reporter
|
||||
from .runner import Context, run_steps
|
||||
from .steps import all_steps, migrate_state
|
||||
from .steps import adopt_old_temp, all_steps, migrate_state
|
||||
|
||||
EXIT = {'ok': 0, 'failed': 1, 'cancelled': 3}
|
||||
|
||||
@@ -33,17 +32,6 @@ class Parser(argparse.ArgumentParser):
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
# The temp folder inside the install folder when the layout names none (the Linux release); the check step does not
|
||||
# count it when it requires an empty install folder
|
||||
TEMP_IN_DEST = '.setup-temp'
|
||||
|
||||
|
||||
def temp_folder(base, dest):
|
||||
"""One temp folder per install folder, so two installs never share generated files."""
|
||||
tag = hashlib.sha256(str(dest).lower().encode()).hexdigest()[:8]
|
||||
return base / f'{dest.name or "root"}-{tag}'
|
||||
|
||||
|
||||
def cancel(signum, frame):
|
||||
raise KeyboardInterrupt
|
||||
|
||||
@@ -58,8 +46,8 @@ def main(argv=None):
|
||||
ap.add_argument('--lang', help='game language, a Switch language code (en-US, en-GB, ja, de, ru, ...)')
|
||||
ap.add_argument('--dest', type=Path, required=True, help='install folder')
|
||||
ap.add_argument('--root', type=Path, help='release folder or repository (default: the folder above installer/)')
|
||||
ap.add_argument('--temp', type=Path, help='temp folder (default: temp/ of the release on Windows, .setup-temp in '
|
||||
'the install folder on Linux, build/install-temp in a repository)')
|
||||
ap.add_argument('--temp', type=Path, help='temp folder (default: build-cache/work in the install folder on Windows, '
|
||||
'.setup-temp there on Linux, build/install-temp in a repository)')
|
||||
ap.add_argument('--verify', action='store_true', help='hash every kept file, also those this release made')
|
||||
ap.add_argument('--dlss', action='store_true', help="download NVIDIA's nvngx_dlss.dll (NVIDIA RTX SDK license) next "
|
||||
'to the game, for dlaa and the DLSS modes')
|
||||
@@ -72,10 +60,12 @@ def main(argv=None):
|
||||
signal.signal(getattr(signal, name), cancel)
|
||||
layout = layouts.find(args.root)
|
||||
dest = args.dest.resolve()
|
||||
base = args.temp or layout.temp
|
||||
temp = temp_folder(base.resolve(), dest) if base else dest / TEMP_IN_DEST
|
||||
temp = layouts.temp_for(layout, dest, args.temp)
|
||||
adopted = adopt_old_temp(layout, dest, temp)
|
||||
log = temp / 'logs' / time.strftime('install-%Y%m%d-%H%M%S.log')
|
||||
rep = Reporter(log_path=log)
|
||||
if adopted:
|
||||
rep.log(adopted)
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
state = statefile.load(dest)
|
||||
|
||||
+3
-1
@@ -746,7 +746,9 @@ def main(argv=None):
|
||||
return selftest(layout, args.shots)
|
||||
fix_tcl_paths()
|
||||
root = tk.Tk()
|
||||
App(root, args.lang or windows_ui_language(), layout, rebuild=args.rebuild)
|
||||
# resolved now: the game starts the build kit's rebuild with the install folder as '.' and the working folder
|
||||
rebuild = args.rebuild.resolve() if args.rebuild else None
|
||||
App(root, args.lang or windows_ui_language(), layout, rebuild=rebuild)
|
||||
root.mainloop()
|
||||
return 0
|
||||
|
||||
|
||||
@@ -27,7 +27,7 @@ TEXT = Path(__file__).resolve().parent / 'text'
|
||||
UI_LANGUAGES = ('en', 'ru', 'pt-BR', 'zh-Hans')
|
||||
# what the language switch in the window's corner shows for each
|
||||
UI_LANGUAGE_LABELS = {'en': 'English', 'ru': 'Русский', 'pt-BR': 'Português (Brasil)', 'zh-Hans': '简体中文'}
|
||||
STEP_ORDER = ('check', 'fonts', 'extract', 'lift', 'compile', 'prepare', 'pc_prompts', 'shapes', 'dlss', 'cleanup')
|
||||
STEP_ORDER = ('check', 'kit', 'fonts', 'extract', 'lift', 'compile', 'prepare', 'pc_prompts', 'shapes', 'dlss', 'cleanup')
|
||||
# message parameter -> catalog id prefix of the names that replace its raw values
|
||||
PARAM_NAMES = {'lang': 'lang.', 'known': 'lang.', 'step': 'step.', 'input': 'param.input.'}
|
||||
LANG_RU = 0x19 # primary language id of Russian in a Windows LANGID
|
||||
|
||||
+45
-5
@@ -16,7 +16,13 @@ Release (docs/installer.md "Release layout"), root = the unpacked folder:
|
||||
without one the game is built at -O1
|
||||
totk.ini.example
|
||||
VERSION the release version, one line
|
||||
temp/ the default temp folder (created)
|
||||
|
||||
Build kit (Windows release): the setup copies the release folder without TotK-Setup.exe into <install>/build-kit
|
||||
(KIT_ITEMS, steps.py Kit) and runs every later step from it, so the game's rebuild for its mods
|
||||
(build-cache/rebuild.txt, relative to the install folder) needs neither the setup folder nor its path: the setup can be
|
||||
deleted and the install folder moved. The kit is a release folder itself (release_layout). The temp folder with the
|
||||
rebuild's working state (gen/, build/, the lift cache) is <install>/build-cache/work (TEMP_IN_DEST), so it moves
|
||||
with the install and survives a newer setup's kit.
|
||||
|
||||
Linux release (the AppImage's folder, docs/installer.md "Linux release"): python/bin/python3 (python-build-standalone,
|
||||
with tkinter), llvm/ with clang, lld and llvm-ar plus llvm/sysroot (the glibc headers and libraries the game links
|
||||
@@ -26,9 +32,11 @@ Dev mode, root = a repository checkout (no python/python.exe): build/py314-embed
|
||||
Python the release ships, with the same packages), else this Python; toolchain/llvm-mingw-*/ and toolchain/deps-mingw;
|
||||
runtime/ as the sources with build/install-rtlib/libtotkrt.a and libtotkrt-v2.a (tools/build_game.py --make-runtime-lib
|
||||
--out build/install-rtlib, once per --cpu); MSYS2 UCRT64 ninja.exe; toolchain/hactool; build/xlate/totk_xlate.exe; the shader list from
|
||||
refs/MacNativeGames; build/pgo/pgo-<v>.profdata when there is one; build/install-temp. The version is VERSION with -dev and the commit: 0.1.0-dev+973c626. On Linux,
|
||||
refs/MacNativeGames; build/pgo/pgo-<v>.profdata when there is one; build/install-temp/<install name>-<hash>. Dev mode and
|
||||
the Linux release install no build kit: the repository, or the AppImage file, is what the rebuild runs. The version is VERSION with -dev and the commit: 0.1.0-dev+973c626. On Linux,
|
||||
dev mode takes clang, lld and ninja from PATH and the system's libraries (no deps folder, no SDL3 copy).
|
||||
"""
|
||||
import hashlib
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
@@ -40,6 +48,14 @@ WIN = os.name == 'nt'
|
||||
EXE = '.exe' if WIN else ''
|
||||
UCRT = Path(r'C:\msys64\ucrt64\bin')
|
||||
SUPPORTED = ('1.2.1', '1.4.3', '1.1.0') # updates the installer builds; the check identifies the player's by hash
|
||||
KIT = 'build-kit' # the build kit's folder in the install folder
|
||||
# What the build kit holds of a release folder (tools/make_release.py writes them): all but TotK-Setup.exe. licenses/
|
||||
# and THIRD_PARTY_NOTICES.md go along with the binaries they cover.
|
||||
KIT_ITEMS = ('python', 'llvm', 'runtime', 'bin', 'recomp', 'tools', 'data', 'installer', 'licenses',
|
||||
'THIRD_PARTY_NOTICES.md', 'totk.ini.example', 'VERSION')
|
||||
# The temp folder inside the install folder, for a layout that names none (every release); the check step does not
|
||||
# count it when it requires an empty install folder. Linux keeps the name its installs already use.
|
||||
TEMP_IN_DEST = Path('build-cache', 'work') if WIN else Path('.setup-temp')
|
||||
|
||||
|
||||
def version_tag(version):
|
||||
@@ -139,9 +155,10 @@ def release_layout(root):
|
||||
dlls=(root / 'bin' / 'SDL3.dll', root / 'bin' / 'libopus-0.dll') if WIN else (root / 'bin' / 'libSDL3.so.0',),
|
||||
shader_list=root / 'data' / 'shader_list.tsv', ini_example=root / 'totk.ini.example',
|
||||
pgo_dir=root / 'data' if WIN else None,
|
||||
# Linux: None, the install folder's .setup-temp (installer/cli.py): the AppImage is mounted read-only, and on
|
||||
# a Steam Deck /home is often too full for the generated C while the install folder is on the SD card
|
||||
temp=root / 'temp' if WIN else None,
|
||||
# None: the install folder's TEMP_IN_DEST (temp_for). Windows: the rebuild's working state stays with the install
|
||||
# and its build kit. Linux: the AppImage is mounted read-only, and on a Steam Deck /home is often too full for
|
||||
# the generated C while the install folder is on the SD card
|
||||
temp=None,
|
||||
sysroot=None if WIN else root / 'llvm' / 'sysroot')
|
||||
|
||||
|
||||
@@ -191,3 +208,26 @@ def find(root=None):
|
||||
"""The layout of root (default: the folder above this package)."""
|
||||
root = Path(root or Path(__file__).resolve().parent.parent).resolve()
|
||||
return release_layout(root) if is_release(root) else dev_layout(root)
|
||||
|
||||
|
||||
def uses_kit(layout):
|
||||
"""True when the setup installs a build kit (Windows release); dev mode and the Linux AppImage run from where they
|
||||
are."""
|
||||
return WIN and not layout.dev
|
||||
|
||||
|
||||
def kit_folder(dest):
|
||||
return Path(dest) / KIT
|
||||
|
||||
|
||||
def temp_folder(base, dest):
|
||||
"""One temp folder per install folder below a shared base (dev mode, --temp), so two installs never share
|
||||
generated files."""
|
||||
tag = hashlib.sha256(str(dest).lower().encode()).hexdigest()[:8]
|
||||
return base / f'{dest.name or "root"}-{tag}'
|
||||
|
||||
|
||||
def temp_for(layout, dest, override=None):
|
||||
"""The temp folder of a run on dest: below --temp or the layout's, else TEMP_IN_DEST in the install folder."""
|
||||
base = override or layout.temp
|
||||
return temp_folder(Path(base).resolve(), dest) if base else dest / TEMP_IN_DEST
|
||||
@@ -49,6 +49,10 @@ class Step:
|
||||
"""Do the work. Returns fields to record with the step."""
|
||||
return {}
|
||||
|
||||
def settle(self, ctx):
|
||||
"""After the step ran or was skipped: a step whose output the later steps run from (the build kit) points the
|
||||
context at it."""
|
||||
|
||||
|
||||
class Context:
|
||||
def __init__(self, layout, dest, temp, reporter, options, state=None, verify=False):
|
||||
@@ -121,6 +125,7 @@ def run_steps(steps, ctx):
|
||||
for s in steps:
|
||||
if not ctx.runs[s.id]:
|
||||
rep.step(s.id, 'skipped', reason=ctx.reasons[s.id])
|
||||
s.settle(ctx)
|
||||
continue
|
||||
current = s.id
|
||||
rep.step(s.id, 'started')
|
||||
@@ -133,6 +138,7 @@ def run_steps(steps, ctx):
|
||||
ctx.save() # the first save, after the check, creates the install folder
|
||||
rep.step(s.id, 'done', seconds=seconds)
|
||||
current = None
|
||||
s.settle(ctx)
|
||||
ctx.state['release'] = ctx.layout.release
|
||||
ctx.save()
|
||||
return 'ok'
|
||||
|
||||
+133
-13
@@ -1,5 +1,5 @@
|
||||
"""The ten setup steps of docs/installer.md: check, fonts, extract, lift, compile, prepare, pc_prompts, shapes, dlss,
|
||||
cleanup.
|
||||
"""The eleven setup steps of docs/installer.md: check, kit, fonts, extract, lift, compile, prepare, pc_prompts, shapes,
|
||||
dlss, cleanup.
|
||||
|
||||
Extraction runs in this process through tools/extract.py (the package's sibling tools/, which ships with it); fonts
|
||||
runs the vendored extract_shared_fonts.py on the firmware folder; lift, compile and prepare run the bundled Python on
|
||||
@@ -11,8 +11,10 @@ Install folder: exefs/, romfs/ (romfs.bin, index.json), profile/, shared-fonts/
|
||||
and its DLLs, nvngx_dlss.dll (with --dlss), romfs-index.bin, romfs-patches/, romfs-patches-xbox/,
|
||||
romfs-patches-xbox-label/, romfs-patches-pc/, romfs-patches-<w>x<h>/ and romfs-patches-<style>-<w>x<h>/ (for the
|
||||
screen shapes and the prompts style this player needs, installer/screens.py), totk_xlate.exe, the
|
||||
shader state of totk.exe --prepare-shaders, totk.ini, totk-install.json.
|
||||
Temp folder (<temp>/<dest name>-<hash>): check/ (ExeFS of both inputs, the host check), unpacked/ (the NSP of an NSZ
|
||||
shader state of totk.exe --prepare-shaders, totk.ini, totk-install.json, and on Windows build-kit/ (Kit): the release
|
||||
folder the steps after the kit step run from, and the game's rebuild with them.
|
||||
Temp folder (build-cache/work in the install folder on Windows, .setup-temp there on Linux, <temp>/<dest name>-<hash>
|
||||
with --temp or in a repository; installer/layout.py temp_for): check/ (ExeFS of both inputs, the host check), unpacked/ (the NSP of an NSZ
|
||||
or XCZ input until the extraction is done), gen/ (C), build/ (objects), logs/. The cleanup keeps gen/, build/ and
|
||||
logs/: the next run lifts into the same gen/, where recomp/gen.py lifts again only the chunks whose inputs changed (its
|
||||
lift cache, gen/<mod>/lift_cache.json) and rewrites only those whose C changed, and Ninja in build/ recompiles only
|
||||
@@ -22,8 +24,8 @@ lift or compile.
|
||||
Install folder build-cache/: cheats.status (tools/cheats.py), exefs.status and exefs.data (tools/exefs_patches.py),
|
||||
plugins.status (tools/exefs_plugins.py; plugins.stale is the game's list of plugins whose start differed) and
|
||||
rebuild.txt, the command that starts this setup in its rebuild mode on the install folder (installer/gui.py --rebuild),
|
||||
one argument per line; totk.exe runs it at start when an enabled mod has code cheats the build lacks
|
||||
(runtime/cheats.c).
|
||||
one argument per line, relative to the install folder for the build kit; totk.exe runs it at start when an enabled mod
|
||||
has code cheats the build lacks (runtime/cheats.c).
|
||||
"""
|
||||
import hashlib
|
||||
import http.client
|
||||
@@ -40,6 +42,7 @@ import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
from . import ini, screens
|
||||
from . import layout as layouts
|
||||
from .layout import EXE, SUPPORTED, version_tag
|
||||
from .runner import InstallError, Step
|
||||
|
||||
@@ -53,6 +56,7 @@ from extract_exefs import (common_tickets, container_entries, load_keys, nca_ctr
|
||||
GiB = 1 << 30
|
||||
DEST_EXTRA = 4 * GiB # beside the RomFS: shader and pipeline caches (about 2 GB), exefs, totk.exe, seed files
|
||||
TEMP_NEED = 3 * GiB # generated C (about 0.9 GB) and objects
|
||||
KIT_NEED = 1 * GiB # the build kit (Kit), about 0.65 GB
|
||||
HOSTCHECK = Path(__file__).resolve().parent / 'hostcheck.c'
|
||||
STAMP = '.install-key'
|
||||
|
||||
@@ -598,6 +602,8 @@ class Check(Step):
|
||||
dest_need -= romfs.stat().st_size
|
||||
if ctx.will_run('fonts'):
|
||||
dest_need += FONTS_NEED
|
||||
if ctx.will_run('kit') and layouts.uses_kit(ctx.layout):
|
||||
dest_need += KIT_NEED
|
||||
if ctx.will_run('prepare') or ctx.will_run('extract'):
|
||||
dest_need += DEST_EXTRA
|
||||
if ctx.will_run('lift') or ctx.will_run('compile'):
|
||||
@@ -650,6 +656,110 @@ class Check(Step):
|
||||
ctx.state['version'] = found
|
||||
|
||||
|
||||
# ---- 1b. build kit
|
||||
|
||||
class Kit(Step):
|
||||
"""The build kit (installer/layout.py, Windows release): the release folder without TotK-Setup.exe, copied into
|
||||
<install>/build-kit. Every step after this one runs its tools from there (settle), and the game's rebuild for its
|
||||
mods starts the setup window from there, so the setup folder can go once the install is done and the install
|
||||
folder can move. A newer setup replaces the kit; the kit's own rebuild keeps it. The steps' keys fingerprint file
|
||||
contents, not paths, so the kit, a copy of the setup's files, keeps every step the setup recorded."""
|
||||
|
||||
id = 'kit'
|
||||
|
||||
def key(self, ctx):
|
||||
return fingerprint('kit-1', ctx.layout.release if layouts.uses_kit(ctx.layout) else 'none')
|
||||
|
||||
def current(self, ctx, record):
|
||||
if not layouts.uses_kit(ctx.layout):
|
||||
return True
|
||||
kit = layouts.kit_folder(ctx.dest)
|
||||
if not layouts.is_release(kit):
|
||||
return False
|
||||
found = layouts.release_layout(kit)
|
||||
return found.release == ctx.layout.release and not found.missing(getattr(ctx, 'cpu', 'v3'))
|
||||
|
||||
def run(self, ctx):
|
||||
layout, rep = ctx.layout, ctx.reporter
|
||||
if not layouts.uses_kit(layout):
|
||||
return {}
|
||||
kit = layouts.kit_folder(ctx.dest)
|
||||
if layout.root.resolve() == kit.resolve(): # the kit's own rebuild: it is in place
|
||||
return {}
|
||||
new, old = kit.with_name(layouts.KIT + '.new'), kit.with_name(layouts.KIT + '.old')
|
||||
try:
|
||||
remove(new) # what a cancelled or failed run left
|
||||
remove(old)
|
||||
except OSError as e:
|
||||
raise InstallError('kit.in_use', path=str(e.filename or old)) from None
|
||||
files = []
|
||||
for name in layouts.KIT_ITEMS:
|
||||
src = layout.root / name
|
||||
if src.is_dir():
|
||||
files += [f for f in sorted(src.rglob('*')) if f.is_file() and '__pycache__' not in f.parts]
|
||||
elif src.is_file():
|
||||
files.append(src)
|
||||
total, done = size_of(files), 0
|
||||
rep.progress('kit', 'copy', 0, total, 'bytes', force=True)
|
||||
try:
|
||||
for f in files:
|
||||
dst = new / f.relative_to(layout.root)
|
||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(f, dst)
|
||||
done += f.stat().st_size
|
||||
rep.progress('kit', 'copy', done, total, 'bytes')
|
||||
except OSError as e:
|
||||
raise InstallError('kit.copy_failed', path=str(e.filename or new), detail=str(e)) from None
|
||||
# An open rebuild window, or a layout generator the game's settings menu started, holds files of the old kit:
|
||||
# then nothing is replaced, and the message asks to close the game and run the setup again
|
||||
try:
|
||||
if kit.exists():
|
||||
os.replace(kit, old)
|
||||
os.replace(new, kit)
|
||||
except OSError:
|
||||
raise InstallError('kit.in_use', path=str(kit)) from None
|
||||
try:
|
||||
remove(old)
|
||||
except OSError as e:
|
||||
rep.log(f'kit: the old kit stays until the next run: {e}')
|
||||
rep.log(f'kit: {len(files)} files, {total / GiB:.2f} GiB in {kit}')
|
||||
return {'bytes': total}
|
||||
|
||||
def settle(self, ctx):
|
||||
if not layouts.uses_kit(ctx.layout):
|
||||
return
|
||||
kit = layouts.kit_folder(ctx.dest).resolve()
|
||||
if ctx.layout.root.resolve() != kit:
|
||||
ctx.layout = layouts.release_layout(kit)
|
||||
ctx.reporter.log(f'kit: the next steps run from {kit}')
|
||||
|
||||
|
||||
def adopt_old_temp(layout, dest, temp):
|
||||
"""An install by a setup before the build kit has its temp folder (generated C, lift cache, objects) in that
|
||||
setup's folder: <setup>/temp/<install name>-<hash>, the setup build-cache/rebuild.txt names. The first run of a
|
||||
setup with the build kit moves it into the install folder when both are on one disk, and the steps' keys decide
|
||||
what of it still holds. On another disk the new temp folder starts empty: an update recompiles everything anyway
|
||||
(the compiler's path is in every compile command), so a copy would save only part of the lift.
|
||||
Runs before the run's log opens in temp; returns the line for the log, None when there was nothing to move."""
|
||||
if not layouts.uses_kit(layout) or temp != dest / layouts.TEMP_IN_DEST or temp.exists():
|
||||
return None
|
||||
try:
|
||||
lines = (dest / 'build-cache' / 'rebuild.txt').read_text(encoding='utf-8').splitlines()
|
||||
except OSError:
|
||||
return None
|
||||
if len(lines) < 4 or lines[1] != '-c' or not os.path.isabs(lines[3]):
|
||||
return None
|
||||
old = layouts.temp_folder(Path(lines[3]) / 'temp', dest)
|
||||
if not (old / 'gen').is_dir():
|
||||
return None
|
||||
try:
|
||||
temp.parent.mkdir(parents=True, exist_ok=True)
|
||||
os.replace(old, temp)
|
||||
return f'temp: moved the temp folder of the setup in {lines[3]} to {temp}'
|
||||
except OSError as e:
|
||||
return f'temp: the temp folder {old} stays where it is ({e}); {temp} starts empty'
|
||||
|
||||
|
||||
# ---- 2. fonts
|
||||
|
||||
class Fonts(Step):
|
||||
@@ -1056,7 +1166,10 @@ def runtime_key(ctx):
|
||||
"""Fingerprint of what totk.exe is built from besides the lifted code: the runtime, the toolchain and the build
|
||||
tools. Shader preparation depends on these, not on the code cheats a lift adds."""
|
||||
layout, cpu = ctx.layout, getattr(ctx, 'cpu', 'v3')
|
||||
clang = subprocess.run([str(layout.clang), '--version'], capture_output=True, text=True).stdout
|
||||
# without the lines that name clang's folder: the build kit of a moved install folder is the same toolchain
|
||||
clang = '\n'.join(line for line in subprocess.run([str(layout.clang), '--version'], capture_output=True,
|
||||
text=True).stdout.splitlines()
|
||||
if not line.startswith(('InstalledDir:', 'Configuration file:')))
|
||||
tools = [layout.tools / n for n in ('build_game.py', 'romfs_index.py', 'check_content.py', 'check_isa.py')]
|
||||
return fingerprint('runtime-1', cpu, layout.runtime_lib_for(cpu), layout.runtime, layout.deps or 'system',
|
||||
layout.sysroot or 'host', *tools, clang, *[d.name for d in layout.dlls])
|
||||
@@ -1450,10 +1563,11 @@ class Cleanup(Step):
|
||||
return {}
|
||||
|
||||
|
||||
# Puts the release folder (the argument after -c) on sys.path and runs installer.gui with the rest. `-m installer.gui`
|
||||
# finds the package only through the release Python's ._pth: the Linux release's Python and the dev-mode embeddable one
|
||||
# lack that entry, and totk.exe starts the command in the install folder.
|
||||
REBUILD_BOOT = ('import runpy, sys; sys.path.insert(0, sys.argv.pop(1)); '
|
||||
# Puts the release folder or the build kit (the argument after -c, made absolute: the kit's is relative to the install
|
||||
# folder) on sys.path and runs installer.gui with the rest. `-m installer.gui` finds the package only through the
|
||||
# release Python's ._pth: the Linux release's Python and the dev-mode embeddable one lack that entry, and totk.exe
|
||||
# starts the command in the install folder.
|
||||
REBUILD_BOOT = ('import os, runpy, sys; sys.path.insert(0, os.path.abspath(sys.argv.pop(1))); '
|
||||
"runpy.run_module('installer.gui', run_name='__main__', alter_sys=True)")
|
||||
|
||||
|
||||
@@ -1472,7 +1586,13 @@ def rebuild_command(ctx):
|
||||
python = Path(sys.executable) if layout.dev else layout.python
|
||||
if os.name == 'nt' and python.with_name('pythonw.exe').is_file():
|
||||
python = python.with_name('pythonw.exe')
|
||||
return [str(python), '-c', REBUILD_BOOT, str(layout.root), '--root', str(layout.root), '--rebuild', str(ctx.dest)]
|
||||
root, dest = str(layout.root), str(ctx.dest)
|
||||
if layouts.uses_kit(layout) and layout.root.resolve() == layouts.kit_folder(ctx.dest).resolve():
|
||||
# the build kit: paths relative to the install folder, which totk.exe resolves against its own folder and
|
||||
# starts the command in, so a moved install folder keeps its rebuild
|
||||
python = Path(layouts.KIT) / python.relative_to(layout.root)
|
||||
root, dest = layouts.KIT, '.'
|
||||
return [str(python), '-c', REBUILD_BOOT, root, '--root', root, '--rebuild', dest]
|
||||
|
||||
|
||||
def write_rebuild_command(ctx):
|
||||
@@ -1500,4 +1620,4 @@ def romfs_problem(romfs, expected):
|
||||
|
||||
|
||||
def all_steps():
|
||||
return [Check(), Fonts(), Extract(), Lift(), Compile(), Prepare(), PcPrompts(), Shapes(), Dlss(), Cleanup()]
|
||||
return [Check(), Kit(), Fonts(), Extract(), Lift(), Compile(), Prepare(), PcPrompts(), Shapes(), Dlss(), Cleanup()]
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"step.check": "Checking the computer and the game files",
|
||||
"step.kit": "Copying the build kit into the game folder",
|
||||
"step.fonts": "Extracting the system fonts",
|
||||
"step.extract": "Extracting the game",
|
||||
"step.lift": "Translating the game code to C",
|
||||
@@ -17,6 +18,7 @@
|
||||
"phase.check.unpack_base": "Unpacking the compressed base game",
|
||||
"phase.check.unpack_update": "Unpacking the compressed update",
|
||||
"phase.check.versions": "Checking the game versions",
|
||||
"phase.kit.copy": "Copying the tools the game rebuilds itself with for code mods",
|
||||
"phase.fonts.extract": "Extracting the shared fonts from the firmware",
|
||||
"phase.fonts.copy": "Copying the shared fonts into the install folder",
|
||||
"phase.extract.hash_base": "Computing the checksum of the base game",
|
||||
@@ -95,6 +97,8 @@
|
||||
"lift.failed": "Translating the game code failed (exit code {code}). Details: {log}",
|
||||
"compile.failed": "Compiling {game_exe} failed (exit code {code}). Details: {log}",
|
||||
"compile.exe_in_use": "{path} is in use. Close the game and run the installer again.",
|
||||
"kit.in_use": "{path} is in use. Close the game and any open setup or rebuild window, then run the installer again.",
|
||||
"kit.copy_failed": "The build kit cannot be copied to {path}: {detail}",
|
||||
"compile.exe_locked": "{path} cannot be written: another program holds it, a game started from that folder or an antivirus that flagged the file. Close the game, add the setup's folder and the install folder to the antivirus exclusions, restart Windows, then run the installer again. Details: {log}",
|
||||
"prepare.xlate_in_use": "{path} is in use. Close the game, end any totk_xlate process left from an earlier run, then run the installer again.",
|
||||
"prepare.shader_list_missing": "The shader list is missing.",
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"step.check": "Verificando o computador e os arquivos do jogo",
|
||||
"step.kit": "Copiando o kit de compilação para a pasta do jogo",
|
||||
"step.fonts": "Extraindo as fontes do sistema",
|
||||
"step.extract": "Extraindo o jogo",
|
||||
"step.lift": "Traduzindo o código do jogo para C",
|
||||
@@ -17,6 +18,7 @@
|
||||
"phase.check.unpack_base": "Descompactando o jogo base compactado",
|
||||
"phase.check.unpack_update": "Descompactando a atualização compactada",
|
||||
"phase.check.versions": "Verificando as versões do jogo",
|
||||
"phase.kit.copy": "Copiando as ferramentas com que o jogo se recompila para mods de código",
|
||||
"phase.fonts.extract": "Extraindo as fontes compartilhadas do firmware",
|
||||
"phase.fonts.copy": "Copiando as fontes compartilhadas para a pasta de instalação",
|
||||
"phase.extract.hash_base": "Calculando o checksum do jogo base",
|
||||
@@ -95,6 +97,8 @@
|
||||
"lift.failed": "Falha ao traduzir o código do jogo (código de saída {code}). Detalhes: {log}",
|
||||
"compile.failed": "Falha ao compilar o {game_exe} (código de saída {code}). Detalhes: {log}",
|
||||
"compile.exe_in_use": "{path} está em uso. Feche o jogo e execute o instalador de novo.",
|
||||
"kit.in_use": "{path} está em uso. Feche o jogo e qualquer janela de instalação ou recompilação aberta, depois execute o instalador de novo.",
|
||||
"kit.copy_failed": "Não foi possível copiar o kit de compilação para {path}: {detail}",
|
||||
"compile.exe_locked": "Não é possível gravar {path}: outro programa o está usando, um jogo aberto dessa pasta ou um antivírus que marcou o arquivo. Feche o jogo, adicione a pasta do instalador e a pasta de instalação às exceções do antivírus, reinicie o Windows e execute o instalador de novo. Detalhes: {log}",
|
||||
"prepare.xlate_in_use": "{path} está em uso. Feche o jogo, encerre qualquer processo totk_xlate que tenha sobrado de uma execução anterior e execute o instalador de novo.",
|
||||
"prepare.shader_list_missing": "A lista de shaders está faltando.",
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"step.check": "Проверка компьютера и игровых файлов",
|
||||
"step.kit": "Копирование набора сборки в папку игры",
|
||||
"step.fonts": "Извлечение системных шрифтов",
|
||||
"step.extract": "Распаковка игры",
|
||||
"step.lift": "Перевод кода игры в C",
|
||||
@@ -17,6 +18,7 @@
|
||||
"phase.check.unpack_base": "Распаковка сжатой базовой игры",
|
||||
"phase.check.unpack_update": "Распаковка сжатого обновления",
|
||||
"phase.check.versions": "Проверка версий игры",
|
||||
"phase.kit.copy": "Копирование инструментов, которыми игра пересобирает себя для модов с кодом",
|
||||
"phase.fonts.extract": "Извлечение общих шрифтов из прошивки",
|
||||
"phase.fonts.copy": "Копирование общих шрифтов в папку установки",
|
||||
"phase.extract.hash_base": "Подсчёт контрольной суммы базовой игры",
|
||||
@@ -95,6 +97,8 @@
|
||||
"lift.failed": "Не удалось перевести код игры (код выхода {code}). Подробности: {log}",
|
||||
"compile.failed": "Не удалось собрать {game_exe} (код выхода {code}). Подробности: {log}",
|
||||
"compile.exe_in_use": "Файл {path} занят. Закройте игру и запустите установку снова.",
|
||||
"kit.in_use": "{path} занят другой программой. Закройте игру и все окна установки или пересборки, затем запустите установку снова.",
|
||||
"kit.copy_failed": "Не удалось скопировать набор сборки в {path}: {detail}",
|
||||
"compile.exe_locked": "Файл {path} не удаётся записать: его держит другая программа — игра, запущенная из этой папки, или антивирус, который пометил файл. Закройте игру, добавьте папку установщика и папку установки в исключения антивируса, перезагрузите Windows и запустите установку снова. Подробности: {log}",
|
||||
"prepare.xlate_in_use": "Файл {path} занят. Закройте игру, завершите процесс totk_xlate, оставшийся от прошлого запуска, и запустите установку снова.",
|
||||
"prepare.shader_list_missing": "Нет списка шейдеров.",
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"step.check": "检查电脑和游戏文件",
|
||||
"step.kit": "正在把构建工具包复制到游戏文件夹",
|
||||
"step.fonts": "正在提取系统字体",
|
||||
"step.extract": "解包游戏",
|
||||
"step.lift": "将游戏代码转换为 C 代码",
|
||||
@@ -17,6 +18,7 @@
|
||||
"phase.check.unpack_base": "正在解压压缩的游戏本体",
|
||||
"phase.check.unpack_update": "正在解压压缩的更新",
|
||||
"phase.check.versions": "检查游戏版本",
|
||||
"phase.kit.copy": "正在复制游戏为代码模组重新构建自身所用的工具",
|
||||
"phase.fonts.extract": "正在从固件中提取共享字体",
|
||||
"phase.fonts.copy": "正在将共享字体复制到安装文件夹",
|
||||
"phase.extract.hash_base": "计算游戏本体的校验和",
|
||||
@@ -95,6 +97,8 @@
|
||||
"lift.failed": "游戏代码转换失败(退出代码 {code})。详情:{log}",
|
||||
"compile.failed": "编译 {game_exe} 失败(退出代码 {code})。详情:{log}",
|
||||
"compile.exe_in_use": "{path} 正在使用中。请关闭游戏后重新运行安装程序。",
|
||||
"kit.in_use": "{path} 正在被占用。请关闭游戏以及所有打开的安装或重新构建窗口,再重新运行安装程序。",
|
||||
"kit.copy_failed": "无法把构建工具包复制到 {path}:{detail}",
|
||||
"compile.exe_locked": "无法写入 {path}:另一个程序正在占用它,可能是从该文件夹启动的游戏,或标记了该文件的杀毒软件。请关闭游戏,将安装程序文件夹和安装文件夹加入杀毒软件的排除项,重启 Windows,然后重新运行安装程序。详情:{log}",
|
||||
"prepare.xlate_in_use": "{path} 正在被占用。请关闭游戏,结束上次残留的 totk_xlate 进程,再重新运行安装程序。",
|
||||
"prepare.shader_list_missing": "缺少着色器列表。",
|
||||
|
||||
+9
-1
@@ -516,7 +516,10 @@ static void plain_name(char *s) {
|
||||
|
||||
// ---- rebuild: code cheats and ExeFS patches the build lacks
|
||||
// The setup writes build-cache\rebuild.txt (installer/steps.py write_rebuild_command), the command that starts its
|
||||
// window in rebuild mode on this install folder, one argument per line. When an enabled mod has code cheats this build
|
||||
// window in rebuild mode on this install folder, one argument per line. The build kit's command (installer/layout.py)
|
||||
// is relative to the install folder: its program is taken from this exe's folder, and the command runs with that
|
||||
// folder as its working folder, against which the setup resolves its other arguments. When an enabled mod has code
|
||||
// cheats this build
|
||||
// lacks (CHEAT_REBUILD) or ExeFS patch files it lacks (mods_exefs_pending), or the build has a mod plugin whose file is
|
||||
// gone (ldr_plugins_gone: its mod was removed), the game runs it and exits; the setup lifts and compiles them in (or the
|
||||
// plugin out) and starts the game again.
|
||||
@@ -555,6 +558,7 @@ static void rebuild_if_needed(const char *dir) {
|
||||
return;
|
||||
}
|
||||
const char *argv[32];
|
||||
char program[PLAT_PATH_MAX + 64];
|
||||
int argc = 0;
|
||||
for (char *line = cmd; line && *line && argc < 31;) {
|
||||
char *end = strchr(line, '\n'), *next = end ? end + 1 : NULL;
|
||||
@@ -565,6 +569,10 @@ static void rebuild_if_needed(const char *dir) {
|
||||
line = next;
|
||||
}
|
||||
argv[argc] = NULL;
|
||||
// CreateProcess resolves a relative program against this process's working folder, which a shortcut may set
|
||||
if (argc && !plat_path_absolute(argv[0]) &&
|
||||
snprintf(program, sizeof program, "%s%s", dir, argv[0]) < (int)sizeof program)
|
||||
argv[0] = program;
|
||||
if (argc < 2 || !plat_file_write(skip_path, want, strlen(want))) {
|
||||
rt_log("cheats: %d code cheats and %d ExeFS patch files or plugins differ from the build; cannot start the "
|
||||
"rebuild (%s)", need, patches, cmd_path);
|
||||
|
||||
@@ -379,6 +379,17 @@ static inline char *plat_last_sep(const char *path) {
|
||||
if (*p == '/' || *p == '\\') last = (char *)p;
|
||||
return last;
|
||||
}
|
||||
// 1 when path does not depend on the working directory: it starts with a separator (a root or UNC path) or, on
|
||||
// Windows, with a drive letter and a separator.
|
||||
static inline int plat_path_absolute(const char *path) {
|
||||
if (path[0] == '/' || path[0] == '\\') return 1;
|
||||
#ifdef _WIN32
|
||||
char c = path[0];
|
||||
return ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z')) && path[1] == ':' && (path[2] == '/' || path[2] == '\\');
|
||||
#else
|
||||
return 0;
|
||||
#endif
|
||||
}
|
||||
// Environment variable as UTF-8 into out; its length, or 0 when unset, empty or too long for cap.
|
||||
size_t plat_getenv(const char *name, char *out, size_t cap);
|
||||
// The user's UI language as a BCP-47 tag ("ru-RU", "en-US"); "" when the system gives none. Valid for the process.
|
||||
|
||||
+8
-3
@@ -1146,7 +1146,8 @@ int romfs_shaped_prompts_ready(const char *folder, unsigned w, unsigned h) { ret
|
||||
// The setup writes the shaped folders for the displays connected when it ran (installer/steps.py Shapes). After a
|
||||
// monitor change the menu can start the same generator for the shape of the game resolution: the setup's Python and
|
||||
// tools/make_romfs_patches.py, which build-cache/rebuild.txt names (installer/steps.py write_rebuild_command: line 1 the
|
||||
// Python, pythonw on Windows, so no console opens; line 4 the setup's folder), on the install folder's romfs/. Child
|
||||
// Python, pythonw on Windows, so no console opens; line 4 the setup's folder; both relative to the install folder for
|
||||
// the build kit, installer/layout.py), on the install folder's romfs/. Child
|
||||
// processes the game does not wait for: the generator writes manifest.tsv last, so a folder it has not finished is not
|
||||
// taken, and the next start loads the finished one.
|
||||
static int prepare_tools(char *python, size_t pcap, char *script, size_t scap, char *dir, size_t dcap) {
|
||||
@@ -1166,8 +1167,12 @@ static int prepare_tools(char *python, size_t pcap, char *script, size_t scap, c
|
||||
lines[k++] = p;
|
||||
p = e + 1;
|
||||
}
|
||||
int ok = k >= 4 && !strcmp(lines[1], "-c") && snprintf(python, pcap, "%s", lines[0]) < (int)pcap &&
|
||||
snprintf(script, scap, "%s/tools/make_romfs_patches.py", lines[3]) < (int)scap;
|
||||
int ok = k >= 4 && !strcmp(lines[1], "-c");
|
||||
if (ok) { // the build kit's lines are relative to the install folder
|
||||
const char *at0 = plat_path_absolute(lines[0]) ? "" : dir, *at3 = plat_path_absolute(lines[3]) ? "" : dir;
|
||||
ok = snprintf(python, pcap, "%s%s%s", at0, *at0 ? "/" : "", lines[0]) < (int)pcap &&
|
||||
snprintf(script, scap, "%s%s%s/tools/make_romfs_patches.py", at3, *at3 ? "/" : "", lines[3]) < (int)scap;
|
||||
}
|
||||
free(text);
|
||||
snprintf(path, sizeof path, "%s/romfs/romfs.bin", dir);
|
||||
return ok && plat_file_exists(python) && plat_file_exists(script) && plat_file_exists(path);
|
||||
|
||||
+125
-1
@@ -18,6 +18,7 @@ ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
from installer import ini, runner, screens, state, steps # noqa: E402
|
||||
from installer.events import Reporter # noqa: E402
|
||||
from installer import layout as layouts # noqa: E402
|
||||
from installer.layout import Layout # noqa: E402
|
||||
|
||||
|
||||
@@ -634,6 +635,128 @@ class CleanupTest(unittest.TestCase):
|
||||
self.assertEqual(self.run_cleanup(), 'cleanup.romfs_unreadable')
|
||||
|
||||
|
||||
def fake_release(root, version='1.0+abc'):
|
||||
"""A folder release_layout reads as a release: VERSION, the Python file and one file per other kit item."""
|
||||
for name in layouts.KIT_ITEMS:
|
||||
if '.' in name or name == 'VERSION':
|
||||
(root / name).parent.mkdir(parents=True, exist_ok=True)
|
||||
(root / name).write_text(version + '\n' if name == 'VERSION' else name)
|
||||
else:
|
||||
(root / name).mkdir(parents=True, exist_ok=True)
|
||||
(root / name / 'f.txt').write_text(name)
|
||||
layouts.release_python(root).parent.mkdir(parents=True, exist_ok=True)
|
||||
layouts.release_python(root).write_bytes(b'')
|
||||
(root / 'TotK-Setup.exe').write_bytes(b'launcher')
|
||||
(root / 'installer' / '__pycache__').mkdir()
|
||||
(root / 'installer' / '__pycache__' / 'x.pyc').write_bytes(b'')
|
||||
return layouts.release_layout(root)
|
||||
|
||||
|
||||
@unittest.skipUnless(steps.os.name == 'nt', 'the build kit is Windows-only')
|
||||
class KitTest(unittest.TestCase):
|
||||
"""The build kit: the setup's release folder copied into the install folder, the later steps and the rebuild
|
||||
command pointed at it, and an older setup's temp folder taken over."""
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
base = Path(self.tmp.name)
|
||||
self.setup, self.dest = base / 'TotK-Setup', base / 'game'
|
||||
self.dest.mkdir()
|
||||
self.layout = fake_release(self.setup)
|
||||
self.missing = mock.patch.object(Layout, 'missing', lambda self, cpu='v3': [])
|
||||
self.missing.start()
|
||||
|
||||
def tearDown(self):
|
||||
self.missing.stop()
|
||||
self.tmp.cleanup()
|
||||
|
||||
def ctx(self, layout):
|
||||
return SimpleNamespace(layout=layout, dest=self.dest, reporter=Reporter(out=io.StringIO()))
|
||||
|
||||
def install(self):
|
||||
ctx = self.ctx(self.layout)
|
||||
kit = steps.Kit()
|
||||
self.assertFalse(kit.current(ctx, {}))
|
||||
kit.run(ctx)
|
||||
kit.settle(ctx)
|
||||
return ctx, kit
|
||||
|
||||
def test_copies_the_release_and_the_next_steps_run_from_it(self):
|
||||
ctx, kit = self.install()
|
||||
folder = self.dest / 'build-kit'
|
||||
self.assertEqual(ctx.layout.root, folder.resolve())
|
||||
self.assertEqual(ctx.layout.release, '1.0+abc')
|
||||
self.assertEqual(sorted(p.name for p in folder.iterdir()), sorted(layouts.KIT_ITEMS))
|
||||
self.assertFalse((folder / 'installer' / '__pycache__').exists())
|
||||
self.assertFalse((self.dest / 'build-kit.new').exists() or (self.dest / 'build-kit.old').exists())
|
||||
self.assertTrue(kit.current(self.ctx(self.layout), {}))
|
||||
|
||||
def test_the_kits_own_rebuild_keeps_it(self):
|
||||
ctx, kit = self.install()
|
||||
(self.dest / 'build-kit' / 'tools' / 'mark').write_text('kept')
|
||||
self.assertTrue(kit.current(ctx, {}))
|
||||
kit.run(ctx)
|
||||
kit.settle(ctx)
|
||||
self.assertEqual((self.dest / 'build-kit' / 'tools' / 'mark').read_text(), 'kept')
|
||||
|
||||
def test_a_newer_setup_replaces_the_kit(self):
|
||||
self.install()
|
||||
(self.dest / 'build-kit' / 'tools' / 'old-only').write_text('old')
|
||||
self.layout = fake_release(Path(self.tmp.name) / 'TotK-Setup-2', '1.1+def')
|
||||
ctx, _ = self.install()
|
||||
self.assertEqual(ctx.layout.release, '1.1+def')
|
||||
self.assertFalse((self.dest / 'build-kit' / 'tools' / 'old-only').exists())
|
||||
|
||||
def test_rebuild_command_is_relative_to_the_install_folder(self):
|
||||
ctx, _ = self.install()
|
||||
cmd = steps.rebuild_command(ctx)
|
||||
self.assertEqual(cmd, [str(Path('build-kit', 'python', 'pythonw.exe' if (
|
||||
ctx.layout.python.with_name('pythonw.exe')).is_file() else 'python.exe')), '-c', steps.REBUILD_BOOT,
|
||||
'build-kit', '--root', 'build-kit', '--rebuild', '.'])
|
||||
self.assertFalse(any(str(self.dest) in a or str(self.setup) in a for a in cmd))
|
||||
|
||||
def test_dev_mode_has_no_kit(self):
|
||||
ctx = self.ctx(SimpleNamespace(dev=True, release='x', root=ROOT))
|
||||
kit = steps.Kit()
|
||||
self.assertTrue(kit.current(ctx, {}))
|
||||
self.assertEqual(kit.run(ctx), {})
|
||||
kit.settle(ctx)
|
||||
self.assertFalse((self.dest / 'build-kit').exists())
|
||||
|
||||
def test_temp_folder_of_an_older_setup_moves_in(self):
|
||||
old = layouts.temp_folder(self.setup / 'temp', self.dest)
|
||||
(old / 'gen').mkdir(parents=True)
|
||||
(old / 'gen' / '.install-key').write_text('k')
|
||||
(self.dest / 'build-cache').mkdir()
|
||||
(self.dest / 'build-cache' / 'rebuild.txt').write_text(
|
||||
'\n'.join(['py', '-c', 'boot', str(self.setup), '--root', str(self.setup), '--rebuild', str(self.dest)]))
|
||||
temp = layouts.temp_for(self.layout, self.dest)
|
||||
self.assertEqual(temp, self.dest / 'build-cache' / 'work')
|
||||
self.assertIn('moved', steps.adopt_old_temp(self.layout, self.dest, temp))
|
||||
self.assertEqual((temp / 'gen' / '.install-key').read_text(), 'k')
|
||||
self.assertFalse(old.exists())
|
||||
self.assertIsNone(steps.adopt_old_temp(self.layout, self.dest, temp)) # once: the new one exists now
|
||||
|
||||
def test_no_temp_folder_to_take_over(self):
|
||||
temp = layouts.temp_for(self.layout, self.dest)
|
||||
self.assertIsNone(steps.adopt_old_temp(self.layout, self.dest, temp))
|
||||
(self.dest / 'build-cache').mkdir()
|
||||
(self.dest / 'build-cache' / 'rebuild.txt').write_text('build-kit\\python\\pythonw.exe\n-c\nboot\nbuild-kit\n')
|
||||
self.assertIsNone(steps.adopt_old_temp(self.layout, self.dest, temp))
|
||||
|
||||
|
||||
class RuntimeKeyTest(unittest.TestCase):
|
||||
def test_clangs_folder_is_not_in_the_key(self):
|
||||
layout = SimpleNamespace(clang=Path('clang'), tools=Path('tools'), runtime=Path('rt'), deps=None, sysroot=None,
|
||||
dlls=(), runtime_lib_for=lambda cpu: None)
|
||||
keys = []
|
||||
for where in ('C:/setup/llvm/bin', 'D:/game/build-kit/llvm/bin'):
|
||||
out = f'clang version 23\nTarget: x86_64-w64-windows-gnu\nInstalledDir: {where}\nConfiguration file: {where}/x.cfg\n'
|
||||
with mock.patch.object(steps.subprocess, 'run', return_value=SimpleNamespace(stdout=out)):
|
||||
keys.append(steps.runtime_key(SimpleNamespace(layout=layout)))
|
||||
self.assertEqual(keys[0], keys[1])
|
||||
|
||||
|
||||
class FreeSpaceTest(unittest.TestCase):
|
||||
def test_missing_folders_use_the_nearest_existing_parent(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
@@ -806,7 +929,8 @@ class FontsTest(unittest.TestCase):
|
||||
|
||||
def test_all_steps_order(self):
|
||||
self.assertEqual([s.id for s in steps.all_steps()],
|
||||
['check', 'fonts', 'extract', 'lift', 'compile', 'prepare', 'pc_prompts', 'shapes', 'dlss', 'cleanup'])
|
||||
['check', 'kit', 'fonts', 'extract', 'lift', 'compile', 'prepare', 'pc_prompts', 'shapes', 'dlss',
|
||||
'cleanup'])
|
||||
|
||||
def test_extracts_into_the_install_folder(self):
|
||||
self.fake_tool()
|
||||
|
||||
@@ -19,6 +19,7 @@ cached in toolchain/release-dl/ and checked against the sha256 in PINS; a mismat
|
||||
install compiles that update's code and the runtime libraries with it
|
||||
VERSION the repository's VERSION plus the commit, 0.1.0+973c626; the installer's release string
|
||||
TotK-Setup.exe installer/launcher/launcher.c built with llvm/ (-mwindows): starts python/pythonw.exe -m installer.gui
|
||||
Everything else is the build kit the setup copies into the install folder (installer/layout.py KIT_ITEMS)
|
||||
|
||||
--prove compiles the generated C and links build/release-check/totk.exe with nothing but the folder's python/, llvm/,
|
||||
runtime/ and bin/ (PATH holds the Windows system directory and those), then runs the four probes there.
|
||||
|
||||
Reference in new issue
Block a user