Build TotK-Setup.exe, write VERSION, copy installer/ and totk.ini.example into the release and run a check-only installer pass and the window selftest on the bundle

clean_env compared the upper-cased os.environ keys with mixed-case names and dropped SystemRoot, which crashed Tk in the bundled python.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Valiantsin DzerakhandClaude Opus 5.5 committed 2026-10-24 09:19:02 +03:00
1 parent e15f037f65
commit 5c1626e7ac
2 files changed
+91 -8

No files matched your search

+10 -2
View File
@@ -59,7 +59,7 @@ python -m installer --base FILE --update FILE --keys FILE --lang CODE --dest DIR
- On a folder with `totk-install.json` only `--dest` is needed; the game files only when the extraction must run again.
- stdout: one JSON event per line, types `start`, `plan`, `step` (started, done, skipped), `progress` (step, phase,
done/total/unit), `warning`, `error`, `end` (`installer/events.py`). Messages are ids with parameters;
done/total/unit), `space` (bytes needed and free on the install disk and the temp disk), `warning`, `error`, `end` (`installer/events.py`). Messages are ids with parameters;
`installer/text/en.json` holds their English text. The human-readable log goes to `<temp>/logs/`.
- Exit codes: 0 done, 1 failed, 2 bad command line, 3 cancelled. The window cancels with Ctrl+Break to the
installer's process group.
@@ -67,6 +67,13 @@ python -m installer --base FILE --update FILE --keys FILE --lang CODE --dest DIR
resume and update are one rule; output kept from another release is re-hashed first.
- Peak space on the install disk is about 36 GiB, not 22: the extraction copies both Program NCAs (16.3 GiB) next to
the merged RomFS (15.9 GiB) before it deletes them. The check computes the exact figure.
- Window (`python -m installer.gui`, `installer/gui.py` draws, `installer/guilogic.py` decides): form, step list with
progress, space line, collapsible log, Cancel (Ctrl+Break to the CLI's process group, then it waits for the exit),
finish screen with Open folder. English or Russian, chosen from the Windows display language, switchable in the
window; every sentence is an id in `installer/text/<en|ru>.json`. A folder with `totk-install.json` needs only the
folder. `--selftest [--shots DIR]` builds every screen in both languages and closes itself.
- `tests/installer_gui_test.py`: event model, form, catalogs, cancel, the self-closing window and one end-to-end run
of the CLI in `--verify` mode (needs `build/install-test` and `build/extracted/v121`; see its docstring).
- `tests/installer_test.py`: bookkeeping with fake steps. `tests/installer_e2e.py`: a dev-mode run with the files of
`config.local.toml`.
@@ -75,7 +82,7 @@ python -m installer --base FILE --update FILE --keys FILE --lang CODE --dest DIR
The unpacked release; `tools/make_release.py` assembles it, CI zips it:
```
TotK-Setup.exe starts python/pythonw.exe -m installer (a small launcher)
TotK-Setup.exe starts python/pythonw.exe -m installer.gui (installer/launcher/launcher.c, no console)
installer/ the installer: steps, CLI, window, en/ru text catalog
python/ Python 3.14.9 embeddable + numpy, lz4, cryptography, cffi, pycparser (+ tkinter for the window)
recomp/, tools/ the lifter and the build, extraction and shader scripts the steps run (copied from the repo)
@@ -84,6 +91,7 @@ runtime/ libtotkrt.a and the headers the generated code and modules
bin/ totk_xlate.exe, hactool.exe, SDL3.dll, libopus-0.dll, ninja.exe
data/ shader_list.tsv (MacNativeGames, MIT)
THIRD_PARTY_NOTICES.md
totk.ini.example, VERSION the default ini; the release string (git describe of the build)
```
- SDL3 comes from the upstream release, not MSYS2: MSYS2's DLL imports the LGPL libiconv.
+81 -6
View File
@@ -13,16 +13,21 @@ cached in toolchain/release-dl/ and checked against the sha256 in PINS; a mismat
runtime/ libtotkrt.a (build_game.py --make-runtime-lib with llvm/), the runtime headers, romfs.c (compiled per
install), deps/ (headers and import libraries of SDL3, Opus, libvpx, Vulkan and VMA)
bin/ totk_xlate.exe (+ .id), hactool.exe, SDL3.dll (upstream), libopus-0.dll, ninja.exe
recomp/, tools/, data/, installer/ (if the checkout has one), licenses/, THIRD_PARTY_NOTICES.md
recomp/, tools/, data/, installer/, licenses/, THIRD_PARTY_NOTICES.md, totk.ini.example
VERSION git describe of the checkout (else the short hash); the installer's release string
TotK-Setup.exe installer/launcher/launcher.c built with llvm/ (-mwindows): starts python/pythonw.exe -m installer.gui
--prove compiles generated/v121 and links build/release-check/totk.exe with nothing but the folder's python/, llvm/,
runtime/ and bin/ (PATH holds the Windows system directory and those), then runs the four probes there. It needs a
generated/v121 and build/extracted/v121 from the usual game build (they are read, nothing is written to them).
The folder contains nothing derived from the game: check_content runs over it before the script finishes.
The folder contains nothing derived from the game: check_content runs over it before the script finishes. Then the
folder's own python runs a check-only installer pass (no game files: it must get through the release, toolchain and host
checks and stop at the missing game files) and the window's --selftest.
"""
import argparse
import hashlib
import json
import os
import shutil
import struct
@@ -154,8 +159,9 @@ def copy_tree(src, dst, skip=lambda rel: False):
def clean_env(*path_dirs):
"""A process environment without MSYS2 and the user's tools: the Windows system directory and path_dirs only."""
system = os.environ.get('SystemRoot', r'C:\Windows')
env = {k: v for k, v in os.environ.items() if k in ('SystemRoot', 'SYSTEMDRIVE', 'TEMP', 'TMP', 'USERPROFILE',
'LOCALAPPDATA', 'APPDATA', 'COMSPEC', 'PATHEXT', 'NUMBER_OF_PROCESSORS')}
keep = ('SYSTEMROOT', 'SYSTEMDRIVE', 'TEMP', 'TMP', 'USERPROFILE', 'LOCALAPPDATA', 'APPDATA', 'COMSPEC', 'PATHEXT',
'NUMBER_OF_PROCESSORS') # os.environ keys are upper case on Windows
env = {k: v for k, v in os.environ.items() if k.upper() in keep}
env['PATH'] = os.pathsep.join([*(str(Path(d)) for d in path_dirs), rf'{system}\System32', system])
return env
@@ -340,9 +346,55 @@ def build_sources(out):
text = subprocess.run(['git', '-C', MAIN_ROOT / 'refs' / 'MacNativeGames', 'show',
f'{MNG_COMMIT}:games/totk/assets/shader_list.tsv'], check=True, capture_output=True).stdout
(out / 'data' / 'shader_list.tsv').write_bytes(text)
if (ROOT / 'installer').is_dir():
copy_tree(ROOT / 'installer', out / 'installer', lambda r: '__pycache__' in r)
copy_tree(ROOT / 'installer', out / 'installer', lambda r: '__pycache__' in r or r.startswith('launcher/'))
shutil.copy2(ROOT / 'totk.ini.example', out / 'totk.ini.example')
shutil.copy2(ROOT / 'THIRD_PARTY_NOTICES.md', out / 'THIRD_PARTY_NOTICES.md')
(out / 'VERSION').write_text(version() + '\n', encoding='utf-8')
def version():
"""git describe of the checkout, else its short hash."""
for args in (['describe', '--tags', '--always'], ['rev-parse', '--short', 'HEAD']):
r = subprocess.run(['git', '-C', ROOT, *args], capture_output=True, text=True)
if r.returncode == 0 and r.stdout.strip():
return r.stdout.strip()
sys.exit('no git version: not a git checkout')
# ---- TotK-Setup.exe --------------------------------------------------------------------------------------------------
def c_wide(text):
"""A C wide string literal of text: ASCII as is, everything else as a universal character name."""
bs = chr(92)
out = []
for ch in text:
if ch in (bs, '"'):
out.append(bs + ch)
elif 32 <= ord(ch) < 127:
out.append(ch)
elif ord(ch) < 0x10000:
out.append(f'{bs}u{ord(ch):04x}')
else:
sys.exit(f'launcher text: {ch!r} is outside the BMP')
return 'L"' + ''.join(out) + '"'
def launcher_header(path):
"""launcher_text.h: the launcher's two messages in English and Russian, taken from the installer's catalog."""
lines = ['/* generated by tools/make_release.py from installer/text/{en,ru}.json */']
for lang in ('en', 'ru'):
texts = json.loads((ROOT / 'installer' / 'text' / f'{lang}.json').read_text(encoding='utf-8'))
for key in ('title', 'python_missing'):
lines.append(f'#define LAUNCHER_{key.upper()}_{lang.upper()} {c_wide(texts["launcher." + key])}')
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text('\n'.join(lines) + '\n', encoding='ascii')
def build_launcher(out, llvm):
work = MAIN_ROOT / 'build' / 'release-check' / 'launcher'
launcher_header(work / 'launcher_text.h')
run([llvm / 'bin' / 'clang.exe', '-O2', '-mwindows', '-Wall', '-Wl,-s', '-I', work,
ROOT / 'installer' / 'launcher' / 'launcher.c', '-o', out / 'TotK-Setup.exe'], env=clean_env(llvm / 'bin'))
# ---- checks ----------------------------------------------------------------------------------------------------------
@@ -358,6 +410,27 @@ def check_folder(out):
print(f'check_content: {sum(1 for p in out.rglob("*") if p.is_file())} files, no hit')
def check_bundle(out):
"""The folder's own python runs the installer without game files and the window's selftest."""
chk = MAIN_ROOT / 'build' / 'release-check' / 'bundle'
shutil.rmtree(chk, ignore_errors=True)
env = clean_env(out / 'python')
py = out / 'python' / 'python.exe'
r = subprocess.run([py, '-m', 'installer', '--lang', 'en-US', '--dest', chk / 'install', '--temp', chk / 'temp'],
cwd=out, env=env, capture_output=True, text=True, timeout=600)
events = [json.loads(line) for line in r.stdout.splitlines() if line.startswith('{')]
errors = [e['id'] for e in events if e['type'] == 'error']
started = any(e['type'] == 'start' and e.get('release') == (out / 'VERSION').read_text().strip() for e in events)
steps = [e['phase'] for e in events if e['type'] == 'progress']
if errors != ['check.inputs_needed'] or not started or 'inputs' not in steps:
sys.exit(f'bundle check: exit {r.returncode}, errors {errors}, release read {started}\n{r.stdout[-2000:]}\n{r.stderr[-2000:]}')
print(f'bundle check: the installer ran the host check from the folder and stopped at the game files (exit {r.returncode})')
r = subprocess.run([py, '-m', 'installer.gui', '--selftest'], cwd=out, env=env, capture_output=True, text=True, timeout=300)
if r.returncode:
sys.exit(f'bundle check: window selftest exit {r.returncode}\n{r.stdout[-2000:]}\n{r.stderr[-2000:]}')
print('bundle check: the window built every screen in both languages and closed')
def sizes(out):
rows = [(p.name if p.is_dir() else p.name, dir_size(p) if p.is_dir() else p.stat().st_size) for p in sorted(out.iterdir())]
for name, n in rows:
@@ -423,8 +496,10 @@ def main():
build_deps(out)
runtime_lib(out, llvm)
build_sources(out)
build_launcher(out, llvm)
collect_licenses(out)
check_folder(out)
check_bundle(out)
sizes(out)
if args.zip:
write_zip(out, args.zip.resolve())