Build the setup for Linux as an AppImage.

tools/make_release_linux.py assembles the Linux release folder and packs it with the static type-2 AppImage runtime:
python-build-standalone 3.14 with tkinter, LLVM 23.1.4 (clang, lld, llvm-ar) and the part of the Steam Runtime 3 sniper
SDK sysroot a game build reads, so the game, hostcheck, hactool and the translator need no newer glibc than 2.31 and run
on SteamOS. The game it builds needs only glibc, the system's Vulkan loader and the libSDL3.so.0 beside it; Opus, libvpx,
zstd and libstdc++ are linked in. --prove builds generated/v121 with the folder alone and runs the probes.

The installer runs on Linux: release and dev layouts, temp under the XDG cache (the AppImage mount is read-only), cancel
through the CLI's process group, language from LANG, xdg-open and a PNG icon, texts that name the right executable, and a
warning on kernels older than 6.7. hostcheck reads x86-64-v3 from CPUID, since clang's __builtin_cpu_supports reads it
from libgcc, which before GCC 12 has no level bits. build_game.py takes --sysroot, build_hactool.py --cc and
build_xlate.py --llvm/--sysroot; linux.c carries the Linux 6.7 write-watch definitions for older headers.

Also: shader_census.py and package_build.py use the Linux executable names, and probe-shader writes its shader path with
forward slashes.
This commit is contained in:
Wellington Dias committed 2026-10-29 18:00:26 -03:00
1 parent 5a84b19476
commit 29a14d69fe
18 files changed
+767 -46

No files matched your search

+21
View File
@@ -74,3 +74,24 @@ These tools run on the developer's or player's machine during the build. They ar
| hactool (SciresM @ `1d64a83450e025622f3468c28fc4164dad2c5ef6` plus `tools/hactool/portable.patch`, from MacNativeGames' `build_hactool()`: `MNG_TITLE_KEYS` and relaxed BKTR layout checks; our `tools/hactool/unicode.patch`: an embedded manifest that makes UTF-8 the process code page), built by `tools/build_hactool.py` with MSYS2 UCRT64 gcc; imports only KERNEL32 and the UCRT | https://github.com/SciresM/hactool | ISC | `bin/hactool.exe` |
| `totk_xlate.exe` (Ryujinx shader translator, NativeAOT build of `tools/xlate/`, see the Ryujinx row above) | built by `tools/build_xlate.py` | MIT | `bin/totk_xlate.exe` |
| MacNativeGames shader variant list | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/assets/shader_list.tsv`), Copyright (c) 2026 MacNativeGames contributors | MIT | `data/shader_list.tsv` |
## Distributed in the Linux release (AppImage)
`tools/make_release_linux.py` assembles these into the folder of `docs/installer.md` ("Linux release") and packs it as
`TotK-Setup-x86_64.AppImage`. Versions and sha256 of every download are pinned in that script; licence texts are in the
folder's `licenses/` and in each component's own folder.
| Component | Source | License | Where in the release |
|---|---|---|---|
| CPython 3.14.10, python-build-standalone 20261026 `install_only` build, with `tkinter` and the libraries it bundles (OpenSSL, libffi, SQLite, zlib, Tcl/Tk and others; their licences ship in the folder) | https://github.com/astral-sh/python-build-standalone/releases/tag/20261026 | PSF-2.0; bundled libraries under their own licences | `python/` |
| numpy 2.5.4 (with the libraries its manylinux wheel bundles), lz4 4.4.5, cryptography 50.0.2, cffi 2.1.1, pycparser 3.11 | PyPI binary wheels, pins in `tools/requirements-installer.txt` | as in the installer release row above | `python/lib/python3.14/site-packages/` |
| LLVM 23.1.4 official Linux X64 release: `clang`, `lld`, `llvm-ar`, `llvm-ranlib`, `llvm-objcopy`, `llvm-strip` and the clang resource headers; `llvm-LICENSE.txt` from the `llvmorg-23.1.4` source tag | https://github.com/llvm/llvm-project/releases/tag/llvmorg-23.1.4 | Apache-2.0 with LLVM exception | `llvm/` |
| Steam Runtime 3 "sniper" SDK sysroot 3.0.20261021.266025 (Debian 11 based), only the headers, libraries and objects a game build reads: glibc 2.31 (headers, `crt*.o`, `libc_nonshared.a`, link libraries), GCC 10 `libstdc++.a` and `libgcc`, SDL 3.4.18, libopus, libvpx 1.9 (static), the Vulkan loader's link library; their Debian `copyright` files go into `licenses/` | https://repo.steampowered.com/steamrt-images-sniper/snapshots/3.0.20261021.266025/ | glibc LGPL-2.1+; libstdc++ and libgcc GPL-3.0 with the GCC Runtime Library Exception; SDL zlib; libopus BSD-3-Clause; libvpx BSD-3-Clause with the WebM patent grant (`libvpx-PATENTS.txt` from the v1.17.0 source tag); Vulkan loader Apache-2.0 | `llvm/sysroot/`, `bin/libSDL3.so.0` (the sysroot's, copied next to the game by the setup), `licenses/` |
| Vulkan-Headers 1.4.363 (source tag) | https://github.com/KhronosGroup/Vulkan-Headers/tree/v1.4.363 | Apache-2.0 OR MIT | `runtime/deps/include/` |
| Vulkan Memory Allocator 3.4.0, header (the licence text is its header comment) | https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator/tree/v3.4.0 | MIT | `runtime/deps/include/vma/` |
| Zstandard 1.5.7, built from the release tarball into a static `libzstd.a` with the folder's clang and sysroot | https://github.com/facebook/zstd/releases/tag/v1.5.7 | BSD-3-Clause | `runtime/deps/` |
| Ninja 1.13.2 (`ninja-linux.zip`) | https://github.com/ninja-build/ninja/releases/tag/v1.13.2 | Apache-2.0 | `bin/ninja` |
| hactool (as in the installer release row above), built by `tools/build_hactool.py --cc` with the folder's clang and sysroot; includes its bundled mbedtls | https://github.com/SciresM/hactool | ISC; mbedtls Apache-2.0 | `bin/hactool` |
| `totk_xlate` (NativeAOT linux-x64 build of `tools/xlate/`, linked with the folder's clang, lld and sysroot) | built by `tools/build_xlate.py --llvm --sysroot` | MIT | `bin/totk_xlate` |
| AppImage type-2 runtime (static; includes libfuse and squashfuse), the AppImage's first part; appimagetool packs the folder and is not distributed | https://github.com/AppImage/type2-runtime | MIT; libfuse LGPL-2.1; squashfuse BSD-2-Clause | the AppImage header |
| MacNativeGames shader variant list | as in the installer release row above | MIT | `data/shader_list.tsv` |
+32 -2
View File
@@ -105,9 +105,39 @@ totk.ini.example, VERSION the default ini; the release string (the repository's
- The release contains nothing game-derived; `make_release.py` runs `tools/check_content.py` over the folder before it
finishes.
## Linux release
`tools/make_release_linux.py` builds the same setup for Linux and packs it as `TotK-Setup-x86_64.AppImage` (the static
type-2 runtime: no libfuse2 needed on the host). The player makes it executable and starts it; on a Steam Deck, from
Desktop Mode. The setup builds the game into the folder they pick, as on Windows, and the game is `totk` there with
`libSDL3.so.0` beside it. The folder inside the AppImage:
```
AppRun starts python/bin/python3 -m installer.gui (no bytecode written into the read-only mount)
totk-setup.desktop, totk-setup.png
installer/, recomp/, tools/, data/, licenses/, THIRD_PARTY_NOTICES.md, totk.ini.example, VERSION as on Windows
python/ python-build-standalone 3.14 (with tkinter) + the installer's packages; a .pth puts the folder,
tools/ and recomp/ on sys.path
llvm/ LLVM 23 (clang, lld, llvm-ar) and llvm/sysroot: the files of the Steam Runtime 3 "sniper" SDK the
game's compile and link read (about 130 files)
runtime/ libtotkrt.a, headers, the install-side sources, totk.png; deps/ with the Vulkan headers, VMA and a
static zstd
bin/ totk_xlate, hactool, ninja, libSDL3.so.0
```
- Everything that runs on the player's machine is built against the sniper sysroot (glibc 2.31): the game, hostcheck,
hactool and the translator need no newer glibc, so they run on SteamOS and current distributions. The game links
Opus, libvpx, zstd and libstdc++ statically and needs only glibc, the system's Vulkan loader and the SDL3 beside it.
- The setup's temp folder is `$XDG_CACHE_HOME/totk-setup` (default `~/.cache/totk-setup`): the AppImage is mounted
read-only.
- The window follows `LANG` for English or Russian, cancels through SIGTERM to the CLI's process group (the CLI handles
it as Ctrl+Break on Windows) and opens the finished folder with `xdg-open`.
- Requirements beyond Windows': Linux 6.7 or later (the write watch, `docs/building.md`), and enough space in the cache
folder's file system for the generated C (about 1.3 GB).
## Constraints
- The installer steps contain no Win32 calls: Linux follows with its own compiler and the same steps.
- The installer steps contain no Win32 calls: Linux runs the same steps with its own compiler ("Linux release").
- The keys file path is passed to the tools; key values never appear in the UI, logs or files.
- The window is a front end over the CLI (`python -m installer --base ... --update ... --keys ... --lang ... --dest
...`), which prints one JSON progress event per line. Tests and CI use the CLI.
@@ -123,4 +153,4 @@ totk.ini.example, VERSION the default ini; the release string (the repository's
## Later
- Re-warm the driver cache on start after a move, GPU or driver change.
- NSZ and XCZ input. Linux release.
- NSZ and XCZ input.
+5 -1
View File
@@ -15,6 +15,7 @@ Text is never the payload: ids name a step ("step.<id>"), a phase ("phase.<step>
and installer/text/en.json holds their English text with {param} placeholders. The log renders events with it.
"""
import json
import os
import sys
import threading
import time
@@ -28,11 +29,14 @@ def load_catalog(path=CATALOG):
return json.loads(path.read_text(encoding='utf-8'))
GAME_EXE = 'totk.exe' if os.name == 'nt' else 'totk' # {game_exe} in the catalogs
def render(catalog, id, params=None):
"""The English text of id, or the id itself when the catalog lacks it."""
text = catalog.get(id, id)
try:
return text.format(**(params or {}))
return text.format(**{'game_exe': GAME_EXE, **(params or {})})
except (KeyError, IndexError, ValueError):
return f'{text} {params}'
+10 -1
View File
@@ -10,6 +10,7 @@ this file only draws. The window text is in installer/text/<en|ru>.json under "u
import argparse
import json
import os
import subprocess
import sys
import tkinter as tk
from pathlib import Path
@@ -58,8 +59,12 @@ class App:
root.title(self.cat('ui.title'))
icon = layout.runtime / 'totk.ico'
png = layout.runtime / 'totk.png' # Tk loads no .ico off Windows; the Linux release converts it
if os.name == 'nt' and icon.is_file():
root.iconbitmap(default=str(icon))
elif png.is_file():
self.icon = tk.PhotoImage(file=str(png))
root.iconphoto(True, self.icon)
root.protocol('WM_DELETE_WINDOW', self.on_close)
style = ttk.Style(root)
style.configure('Title.TLabel', font=('Segoe UI', 15, 'bold'))
@@ -477,7 +482,11 @@ class App:
def open_folder(self):
dest = self.model.dest
if dest and Path(dest).is_dir():
os.startfile(dest)
if os.name == 'nt':
os.startfile(dest)
else:
subprocess.Popen(['xdg-open', dest], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
start_new_session=True)
# ---- self test ---------------------------------------------------------------------------------------------------
+19 -4
View File
@@ -11,6 +11,7 @@ import ctypes
import json
import os
import queue
import signal
import subprocess
import sys
import threading
@@ -75,7 +76,10 @@ def flat(value):
def windows_ui_language():
"""'ru' when the Windows display language is Russian, else 'en'."""
"""'ru' when the display language (Windows; elsewhere LC_ALL, LC_MESSAGES or LANG) is Russian, else 'en'."""
if os.name != 'nt':
loc = os.environ.get('LC_ALL') or os.environ.get('LC_MESSAGES') or os.environ.get('LANG') or ''
return 'ru' if loc.lower().startswith('ru') else 'en'
try:
langid = ctypes.windll.kernel32.GetUserDefaultUILanguage()
except (AttributeError, OSError):
@@ -316,7 +320,14 @@ else:
def send_break(proc, python=None):
"""Ctrl+Break to the process group of proc (started with CREATE_NEW_PROCESS_GROUP). The window itself may have no
console (pythonw), so a short helper process attaches to the child's console and sends it."""
console (pythonw), so a short helper process attaches to the child's console and sends it. Elsewhere SIGTERM to
proc's session (Process starts it in one of its own), which installer/cli.py handles as Ctrl+Break."""
if os.name != 'nt':
try:
os.killpg(proc.pid, signal.SIGTERM)
return True
except OSError:
return False
helper = [str(python or sys.executable), '-c', _BREAK, str(proc.pid)]
flags = CREATE_NO_WINDOW if os.name == 'nt' else 0
r = subprocess.run(helper, creationflags=flags, capture_output=True)
@@ -332,10 +343,14 @@ class Process:
self.command = command
self.helper_python = helper_python
self.lines = queue.Queue()
flags = (CREATE_NEW_PROCESS_GROUP | CREATE_NO_WINDOW) if creationflags is None else creationflags
env = dict(os.environ, PYTHONIOENCODING='utf-8', PYTHONUTF8='1', PYTHONUNBUFFERED='1')
if os.name == 'nt':
flags = (CREATE_NEW_PROCESS_GROUP | CREATE_NO_WINDOW) if creationflags is None else creationflags
group = {'creationflags': flags}
else:
group = {'start_new_session': True}
self.proc = subprocess.Popen(command, cwd=cwd, env=env, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, creationflags=flags)
stderr=subprocess.STDOUT, **group)
self.cancelled_at = None
self.reader = threading.Thread(target=self.read, daemon=True)
self.reader.start()
+20 -2
View File
@@ -1,6 +1,7 @@
// Host check of the installer's first step, compiled with the bundled clang at install time: does the CPU run
// -march=x86-64-v3 code, and does a Vulkan 1.3 device exist. Prints one JSON object on stdout.
// The Vulkan loader is opened at run time, so a machine without one gets "loader": false instead of a failed start.
#include <cpuid.h>
#include <stdio.h>
#include <string.h>
#include <vulkan/vulkan.h>
@@ -24,9 +25,26 @@ static void json_string(const char *s) {
putchar('"');
}
// The x86-64-v3 level from CPUID: AVX, AVX2, BMI1, BMI2, F16C, FMA, LZCNT and MOVBE, with the OS saving the YMM state.
// Not __builtin_cpu_supports("x86-64-v3"): clang reads that from the C runtime's CPU model, and libgcc before GCC 12
// (the Linux release's sysroot has GCC 10) has no level bits, so it answers false on every CPU.
static int x86_64_v3(void) {
unsigned a, b, c, d;
if (!__get_cpuid(1, &a, &b, &c, &d)) return 0;
const unsigned fma = 1u << 12, movbe = 1u << 22, osxsave = 1u << 27, avx = 1u << 28, f16c = 1u << 29;
if ((c & (fma | movbe | osxsave | avx | f16c)) != (fma | movbe | osxsave | avx | f16c)) return 0;
unsigned lo, hi;
__asm__("xgetbv" : "=a"(lo), "=d"(hi) : "c"(0));
if ((lo & 6) != 6) return 0; // XMM and YMM state enabled by the OS
if (!__get_cpuid_count(7, 0, &a, &b, &c, &d)) return 0;
const unsigned bmi1 = 1u << 3, avx2 = 1u << 5, bmi2 = 1u << 8;
if ((b & (bmi1 | avx2 | bmi2)) != (bmi1 | avx2 | bmi2)) return 0;
if (!__get_cpuid(0x80000001, &a, &b, &c, &d)) return 0;
return (c & (1u << 5)) != 0; // LZCNT (ABM)
}
int main(void) {
__builtin_cpu_init();
printf("{\"x86_64_v3\": %s, ", __builtin_cpu_supports("x86-64-v3") ? "true" : "false");
printf("{\"x86_64_v3\": %s, ", x86_64_v3() ? "true" : "false");
void *lib = open_lib();
PFN_vkGetInstanceProcAddr gipa = lib ? (PFN_vkGetInstanceProcAddr)lib_sym(lib, "vkGetInstanceProcAddr") : NULL;
if (!gipa) {
+51 -9
View File
@@ -15,19 +15,26 @@ Release (docs/installer.md "Release layout"), root = the unpacked folder:
VERSION the release version, one line
temp/ the default temp folder (created)
Linux release (the AppImage's folder, docs/installer.md "Linux release"): python/bin/python3 (python-build-standalone,
with tkinter), llvm/ with clang, lld and llvm-ar plus llvm/sysroot (the glibc headers and libraries the game links
against), runtime/deps/ with include/ and the static Opus, libvpx and zstd, bin/ without .exe and with libSDL3.so.0.
Dev mode, root = a repository checkout (no python/python.exe): build/py314-embed/py/python.exe (the embeddable
Python the release ships, with the same packages), else this Python; toolchain/llvm-mingw-*/ and toolchain/deps-mingw;
runtime/ as the sources with build/install-rtlib/libtotkrt.a (tools/build_game.py --make-runtime-lib --out
build/install-rtlib); MSYS2 UCRT64 ninja.exe; toolchain/hactool; build/xlate/totk_xlate.exe; the shader list from
refs/MacNativeGames; build/install-temp. The version is VERSION with -dev and the commit: 0.1.0-dev+973c626.
refs/MacNativeGames; build/install-temp. The version is VERSION with -dev and the commit: 0.1.0-dev+973c626. On Linux,
dev mode takes clang, lld and ninja from PATH and the system's libraries (no deps folder, no SDL3 copy).
"""
import os
import shutil
import subprocess
import sys
from dataclasses import dataclass, field
from pathlib import Path
EXE = '.exe' if os.name == 'nt' else ''
WIN = os.name == 'nt'
EXE = '.exe' if WIN else ''
UCRT = Path(r'C:\msys64\ucrt64\bin')
SUPPORTED = ('1.2.1', '1.4.3') # updates the installer builds; the check identifies the player's by hash
@@ -45,8 +52,8 @@ class Layout:
python: Path # runs the tools
tools: Path
recomp: Path
llvm: Path # llvm-mingw tree
deps: Path # include/ and lib/ for the link
llvm: Path # llvm-mingw tree; on Linux the tree with bin/clang (a release's, or the system's /usr)
deps: Path | None # include/ and lib/ for the link; None: the system's (Linux dev mode)
runtime: Path # runtime headers and install-side sources
runtime_lib: Path # libtotkrt.a
ninja: Path
@@ -56,6 +63,7 @@ class Layout:
shader_list: Path | None = None # None: dev, read from refs/MacNativeGames through git
ini_example: Path | None = None
temp: Path | None = None
sysroot: Path | None = None # Linux release: what the game links against, instead of the host's
@property
def clang(self):
@@ -73,15 +81,27 @@ class Layout:
"""Paths the steps need that do not exist."""
need = [self.python, self.tools / 'extract.py', self.tools / 'build_game.py', self.tools / 'prepare_shaders.py',
self.tools / 'make_romfs_patches.py', self.recomp / 'gen.py', self.clang,
self.deps / 'include', self.deps / 'lib', self.runtime, self.runtime_lib, self.ninja, self.hactool,
*((self.deps / 'include', self.deps / 'lib') if self.deps else ()), self.runtime, self.runtime_lib, self.ninja, self.hactool,
self.xlate, *self.dlls, *(f(v) for v in SUPPORTED for f in (self.patches, self.expected))]
if self.shader_list:
need.append(self.shader_list)
if self.sysroot:
need.append(self.sysroot)
return [str(p) for p in need if not p.exists()]
def cache_dir():
"""$XDG_CACHE_HOME, else ~/.cache: the Linux release runs from a read-only AppImage mount, so its temp goes here."""
xdg = os.environ.get('XDG_CACHE_HOME')
return Path(xdg) if xdg and Path(xdg).is_absolute() else Path.home() / '.cache'
def release_python(root):
return root / 'python' / ('python.exe' if WIN else 'bin/python3')
def is_release(root):
return (root / 'python' / ('python' + EXE)).is_file()
return release_python(root).is_file()
def release_layout(root):
@@ -89,17 +109,21 @@ def release_layout(root):
return Layout(
root=root, dev=False,
release=version.read_text(encoding='utf-8').strip() if version.is_file() else 'unknown',
python=root / 'python' / ('python' + EXE),
python=release_python(root),
tools=root / 'tools', recomp=root / 'recomp',
llvm=root / 'llvm', deps=root / 'runtime' / 'deps', runtime=root / 'runtime',
runtime_lib=root / 'runtime' / 'libtotkrt.a',
ninja=root / 'bin' / ('ninja' + EXE), hactool=root / 'bin' / ('hactool' + EXE),
xlate=root / 'bin' / ('totk_xlate' + EXE),
dlls=(root / 'bin' / 'SDL3.dll', root / 'bin' / 'libopus-0.dll'),
shader_list=root / 'data' / 'shader_list.tsv', ini_example=root / 'totk.ini.example', temp=root / 'temp')
dlls=(root / 'bin' / 'SDL3.dll', root / 'bin' / 'libopus-0.dll') if WIN else (root / 'bin' / 'libSDL3.so.0',),
shader_list=root / 'data' / 'shader_list.tsv', ini_example=root / 'totk.ini.example',
temp=root / 'temp' if WIN else cache_dir() / 'totk-setup',
sysroot=None if WIN else root / 'llvm' / 'sysroot')
def dev_layout(root):
if not WIN:
return dev_layout_posix(root)
commit = subprocess.run(['git', '-C', str(root), 'rev-parse', '--short', 'HEAD'], capture_output=True,
text=True).stdout.strip()
embed = root / 'build' / 'py314-embed' / 'py' / ('python' + EXE)
@@ -118,6 +142,24 @@ def dev_layout(root):
shader_list=None, ini_example=root / 'totk.ini.example', temp=root / 'build' / 'install-temp')
def dev_commit(root):
return subprocess.run(['git', '-C', str(root), 'rev-parse', '--short', 'HEAD'], capture_output=True,
text=True).stdout.strip()
def dev_layout_posix(root):
clang, ninja = shutil.which('clang'), shutil.which('ninja')
semver = (root / 'VERSION').read_text(encoding='utf-8').strip()
return Layout(
root=root, dev=True, release=f'{semver}-dev+{dev_commit(root) or "unknown"}',
python=Path(sys.executable), tools=root / 'tools', recomp=root / 'recomp',
llvm=Path(clang).parent.parent if clang else root / 'toolchain' / 'clang-missing', deps=None,
runtime=root / 'runtime', runtime_lib=root / 'build' / 'install-rtlib' / 'libtotkrt.a',
ninja=Path(ninja) if ninja else root / 'toolchain' / 'ninja-missing',
hactool=root / 'toolchain' / 'hactool' / 'hactool', xlate=root / 'build' / 'xlate' / 'totk_xlate',
shader_list=None, ini_example=root / 'totk.ini.example', temp=root / 'build' / 'install-temp')
def find(root=None):
"""The layout of root (default: the folder above this package)."""
root = Path(root or Path(__file__).resolve().parent.parent).resolve()
+29 -6
View File
@@ -15,6 +15,7 @@ import json
import os
import re
import shutil
import signal
import subprocess
import sys
import threading
@@ -94,10 +95,21 @@ def kill_tree(proc):
return
if os.name == 'nt':
subprocess.run(['taskkill', '/T', '/F', '/PID', str(proc.pid)], capture_output=True)
else:
return
try: # run_tool starts each tool in a session of its own: the group holds ninja, clang and the game
os.killpg(proc.pid, signal.SIGKILL)
except OSError:
proc.kill()
def linux_kernel():
"""(major, minor) of the running Linux kernel, None elsewhere."""
if not sys.platform.startswith('linux'):
return None
m = re.match(r'(\d+)\.(\d+)', os.uname().release)
return (int(m.group(1)), int(m.group(2))) if m else None
def run_tool(ctx, step, script, args, parse=None, cwd=None):
"""Run a tool script with the bundled Python. Its output goes to <temp>/logs/<step>-<script>.log and, line by line,
to parse(line). Returns (exit code, log path)."""
@@ -109,7 +121,8 @@ def run_tool(ctx, step, script, args, parse=None, cwd=None):
env = dict(os.environ, PYTHONUNBUFFERED='1', PYTHONIOENCODING='utf-8')
with open(log, 'w', encoding='utf-8', newline='\n') as out:
proc = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, cwd=cwd or ctx.layout.root,
env=env, text=True, encoding='utf-8', errors='replace', bufsize=1)
env=env, text=True, encoding='utf-8', errors='replace', bufsize=1,
start_new_session=os.name != 'nt')
try:
for line in proc.stdout:
out.write(line)
@@ -231,7 +244,11 @@ class Check(Step):
layout, work = ctx.layout, ctx.temp / 'check'
work.mkdir(parents=True, exist_ok=True)
exe = work / ('hostcheck' + EXE)
command = [str(layout.clang), '-O1', str(HOSTCHECK), '-I', str(layout.deps / 'include'), '-o', str(exe)]
command = [str(layout.clang), '-O1', str(HOSTCHECK), '-o', str(exe)]
if layout.deps:
command += ['-I', str(layout.deps / 'include')]
if layout.sysroot:
command += [f'--sysroot={layout.sysroot}', '-fuse-ld=lld']
if os.name != 'nt':
command.append('-ldl')
r = subprocess.run(command, capture_output=True, text=True, errors='replace')
@@ -254,6 +271,11 @@ class Check(Step):
if not any(tuple(d['api'][:2]) >= (1, 3) for d in devices):
raise InstallError('check.vulkan_too_old',
devices=', '.join(f'{d["name"]} ({d["api"][0]}.{d["api"][1]})' for d in devices))
kernel = linux_kernel()
if kernel and kernel < (6, 7):
# Without userfaultfd's asynchronous write protection nothing tracks the CPU's writes: the game copies whole
# pools to the GPU at each submit and its texture checks see every page as changed (docs/building.md, Linux)
ctx.reporter.warning('warn.kernel_old', version='.'.join(map(str, kernel)))
ctx.host = host
def inputs(self, ctx):
@@ -532,8 +554,8 @@ class Compile(Step):
layout = ctx.layout
clang = subprocess.run([str(layout.clang), '--version'], capture_output=True, text=True).stdout
tools = [layout.tools / n for n in ('build_game.py', 'romfs_index.py', 'check_content.py')]
return fingerprint('compile-1', ctx.keys['lift'], layout.runtime, layout.runtime_lib, layout.deps, *tools,
clang, *[d.name for d in layout.dlls])
return fingerprint('compile-1', ctx.keys['lift'], layout.runtime, layout.runtime_lib, layout.deps or 'system',
layout.sysroot or 'host', *tools, clang, *[d.name for d in layout.dlls])
def current(self, ctx, record):
exe = ctx.dest / ('totk' + EXE)
@@ -558,7 +580,8 @@ class Compile(Step):
rep.progress('compile', 'compile', force=True)
code, log = run_tool(ctx, 'compile', layout.tools / 'build_game.py', [
'--gen', ctx.temp / 'gen', '--out', build, '--toolchain', layout.llvm, '--deps', layout.deps,
'--gen', ctx.temp / 'gen', '--out', build, '--toolchain', layout.llvm,
*(['--deps', layout.deps] if layout.deps else []), *(['--sysroot', layout.sysroot] if layout.sysroot else []),
'--runtime', layout.runtime, '--runtime-lib', layout.runtime_lib, '--ninja', layout.ninja,
'--version', version_tag(game_version(ctx)), '--expected', layout.expected(game_version(ctx)),
'--exefs', dest / 'exefs', '--romfs', dest / 'romfs',
+5 -4
View File
@@ -2,7 +2,7 @@
"step.check": "Checking the computer and the game files",
"step.extract": "Extracting the game",
"step.lift": "Translating the game code to C",
"step.compile": "Compiling totk.exe",
"step.compile": "Compiling {game_exe}",
"step.prepare": "Preparing the shaders",
"step.cleanup": "Cleaning up",
@@ -21,7 +21,7 @@
"phase.lift.discover": "Finding the functions",
"phase.lift.emit": "Writing C code",
"phase.compile.compile": "Compiling",
"phase.compile.link": "Linking totk.exe",
"phase.compile.link": "Linking {game_exe}",
"phase.prepare.romfs_patches": "Writing the 60 fps game data patches",
"phase.prepare.seed": "Reading the shaders from the game data",
"phase.prepare.translate": "Translating the shaders",
@@ -70,7 +70,7 @@
"extract.romfs_merge_failed": "Applying the update to the game data failed: {detail}. Details: {log}",
"extract.romfs_mismatch": "The extracted game data does not match version {version} (size {size}, checksum {sha256}...).",
"lift.failed": "Translating the game code failed (exit code {code}). Details: {log}",
"compile.failed": "Compiling totk.exe failed (exit code {code}). Details: {log}",
"compile.failed": "Compiling {game_exe} failed (exit code {code}). Details: {log}",
"compile.exe_in_use": "{path} is in use. Close the game and run the installer again.",
"prepare.shader_list_missing": "The shader list is missing.",
"prepare.romfs_patches_failed": "Writing the game data patches failed (exit code {code}). Details: {log}",
@@ -79,6 +79,7 @@
"warn.lang_needs_fonts": "Text in {lang} needs the Switch system fonts, which the installer does not extract; some text may not show.",
"warn.npdm_differs": "main.npdm differs from the reference {version} dump; only the code modules are used.",
"warn.kernel_old": "Linux {version} lacks the write tracking the game relies on (6.7 or later); the game may run very slowly. Updating the system fixes this.",
"ui.title": "TotK Setup",
"ui.version": "version {version}",
@@ -132,7 +133,7 @@
"ui.result_ok": "The game is ready",
"ui.result_failed": "The setup failed",
"ui.result_cancelled": "The setup was cancelled",
"ui.done_text": "Start totk.exe from {dest}. The setup program and the game files you chose are no longer needed.",
"ui.done_text": "Start {game_exe} from {dest}. The setup program and the game files you chose are no longer needed.",
"ui.cancelled_text": "The finished steps are kept. Start the setup again with the same folder to continue.",
"ui.log_file": "Log file: {path}",
"ui.process_died": "The setup stopped unexpectedly (exit code {code}).",
+5 -4
View File
@@ -2,7 +2,7 @@
"step.check": "Проверка компьютера и игровых файлов",
"step.extract": "Распаковка игры",
"step.lift": "Перевод кода игры в C",
"step.compile": "Сборка totk.exe",
"step.compile": "Сборка {game_exe}",
"step.prepare": "Подготовка шейдеров",
"step.cleanup": "Уборка",
@@ -21,7 +21,7 @@
"phase.lift.discover": "Поиск функций",
"phase.lift.emit": "Запись кода на C",
"phase.compile.compile": "Компиляция",
"phase.compile.link": "Компоновка totk.exe",
"phase.compile.link": "Компоновка {game_exe}",
"phase.prepare.romfs_patches": "Запись патчей данных игры для 60 кадров/с",
"phase.prepare.seed": "Чтение шейдеров из данных игры",
"phase.prepare.translate": "Трансляция шейдеров",
@@ -70,7 +70,7 @@
"extract.romfs_merge_failed": "Не удалось применить обновление к данным игры: {detail}. Подробности: {log}",
"extract.romfs_mismatch": "Распакованные данные игры не соответствуют версии {version} (размер {size}, контрольная сумма {sha256}...).",
"lift.failed": "Не удалось перевести код игры (код выхода {code}). Подробности: {log}",
"compile.failed": "Не удалось собрать totk.exe (код выхода {code}). Подробности: {log}",
"compile.failed": "Не удалось собрать {game_exe} (код выхода {code}). Подробности: {log}",
"compile.exe_in_use": "Файл {path} занят. Закройте игру и запустите установку снова.",
"prepare.shader_list_missing": "Нет списка шейдеров.",
"prepare.romfs_patches_failed": "Не удалось записать патчи данных игры (код выхода {code}). Подробности: {log}",
@@ -79,6 +79,7 @@
"warn.lang_needs_fonts": "Для языка «{lang}» нужны системные шрифты Switch, которые установка не извлекает; часть текста может не отображаться.",
"warn.npdm_differs": "main.npdm отличается от эталонного дампа {version}; используются только модули с кодом.",
"warn.kernel_old": "В Linux {version} нет отслеживания записи, на которое опирается игра (нужна версия 6.7 или новее); игра может работать очень медленно. Обновление системы это исправит.",
"ui.title": "Установка TotK",
"ui.version": "версия {version}",
@@ -132,7 +133,7 @@
"ui.result_ok": "Игра готова",
"ui.result_failed": "Установка не удалась",
"ui.result_cancelled": "Установка отменена",
"ui.done_text": "Запустите totk.exe из папки {dest}. Программа установки и выбранные игровые файлы больше не нужны.",
"ui.done_text": "Запустите {game_exe} из папки {dest}. Программа установки и выбранные игровые файлы больше не нужны.",
"ui.cancelled_text": "Завершённые шаги сохранены. Чтобы продолжить, снова запустите установку с той же папкой.",
"ui.log_file": "Файл журнала: {path}",
"ui.process_died": "Установка неожиданно остановилась (код выхода {code}).",
+1 -1
View File
@@ -206,7 +206,7 @@ int probe_shader(void) {
*o = 0;
const Entry *e = &entries[0];
snprintf(line, sizeof line,
"{\"job\":{\"key\":\"%.64sa\",\"stage\":\"%s\",\"debug\":false,\"c1\":\"%.*s\",\"code\":\"%s\\\\build\\\\v121\\\\shaders\\\\code\\\\%.64s.code\"},\"out\":\"%s\"}",
"{\"job\":{\"key\":\"%.64sa\",\"stage\":\"%s\",\"debug\":false,\"c1\":\"%.*s\",\"code\":\"%s/build/v121/shaders/code/%.64s.code\"},\"out\":\"%s\"}",
e->key, e->stage == GFX_STAGE_VS ? "vs" : e->stage == GFX_STAGE_FS ? "ps" : "cs", (int)(e->c1_len * 2), e->c1,
esc_root, e->key, dir_json);
FAIL_IF(!gfx_shader_raw_request(line, reply, sizeof reply), "service did not answer a valid request after the malformed one");
+38
View File
@@ -33,6 +33,44 @@
#include "platform.h"
// The write-watch ABI of Linux 6.7 (uapi linux/fs.h and linux/userfaultfd.h), for headers older than it: the release
// builds against the Steam Runtime's 5.10 headers and runs on newer kernels. The kernel refuses what it lacks at run time.
#ifndef PAGEMAP_SCAN
struct page_region {
__u64 start;
__u64 end;
__u64 categories;
};
struct pm_scan_arg {
__u64 size;
__u64 flags;
__u64 start;
__u64 end;
__u64 walk_end;
__u64 vec;
__u64 vec_len;
__u64 max_pages;
__u64 category_inverted;
__u64 category_mask;
__u64 category_anyof_mask;
__u64 return_mask;
};
#define PAGEMAP_SCAN _IOWR('f', 16, struct pm_scan_arg)
#define PAGE_IS_WRITTEN (1 << 1)
#define PAGE_IS_PRESENT (1 << 3)
#define PAGE_IS_SWAPPED (1 << 4)
#define PAGE_IS_PFNZERO (1 << 5)
#define PM_SCAN_WP_MATCHING (1 << 0)
#define PM_SCAN_CHECK_WPASYNC (1 << 1)
#endif
#ifndef UFFD_FEATURE_WP_ASYNC
#define UFFD_FEATURE_WP_UNPOPULATED (1 << 13)
#define UFFD_FEATURE_WP_ASYNC (1 << 15)
#endif
#ifndef UFFD_USER_MODE_ONLY
#define UFFD_USER_MODE_ONLY 1
#endif
extern char **environ;
// ---- threads
+15 -2
View File
@@ -712,6 +712,8 @@ def main():
'bin/llvm-ar) instead of the clang on PATH')
ap.add_argument('--ninja', type=Path, help='ninja.exe to build with (default: MSYS2 UCRT64, else bin/ninja.exe of the '
'release folder)')
ap.add_argument('--sysroot', type=Path, help='Linux: the glibc headers and libraries to compile and link against '
'instead of the host\'s (the release\'s llvm/sysroot)')
ap.add_argument('--deps', type=Path, help=f'with --toolchain: include/, lib/ and bin/ of Vulkan, SDL3, Opus, libvpx '
f'and VMA (default {DEFAULT_DEPS.relative_to(ROOT).as_posix()})')
ap.add_argument('--make-runtime-lib', action='store_true',
@@ -738,8 +740,8 @@ def main():
global CLANG, AR, WINDRES, NINJA, TOOLCHAIN_BIN, DEPS
if args.ninja:
NINJA = args.ninja.resolve()
elif not NINJA.is_file() and (ROOT / 'bin' / 'ninja.exe').is_file():
NINJA = ROOT / 'bin' / 'ninja.exe' # the release folder (tools/make_release.py)
elif not NINJA.is_file() and (ROOT / 'bin' / ('ninja' + ('.exe' if WIN else ''))).is_file():
NINJA = ROOT / 'bin' / ('ninja' + ('.exe' if WIN else '')) # the release folder (tools/make_release.py)
dlls = SDL3_DLLS
suffix = '.exe' if WIN else ''
if args.toolchain:
@@ -750,6 +752,17 @@ def main():
if args.toolchain and WIN or args.deps and (args.toolchain or not WIN): # Linux: --deps only, never deps-mingw
DEPS = (args.deps or DEFAULT_DEPS).resolve()
dlls = sorted((DEPS / 'bin').glob('*.dll')) if WIN else []
if DEPS and not WIN:
# A release (--deps): Opus, libvpx and zstd go into the executable, so it needs no more than libc, Vulkan and
# the libSDL3.so.0 beside it
for lib in ('opus', 'vpx', 'zstd'):
LINK_LIBS[LINK_LIBS.index(f'-l{lib}')] = f'-l:lib{lib}.a'
if args.sysroot and not WIN:
# Every compile and the link: the game then needs no newer glibc than the sysroot's (the Steam Runtime's)
sysroot = ['--target=x86_64-linux-gnu', f'--sysroot={cmd_path(args.sysroot.resolve())}']
GEN_FLAGS[:0] = sysroot
RT_FLAGS[:0] = sysroot
LINK_FLAGS[:0] = sysroot
for tool in (CLANG, NINJA):
if not tool.is_file():
sys.exit(f'{tool} not found: install ' + ('MSYS2 UCRT64 clang, lld and ninja' if WIN else 'clang, lld and ninja')
+7 -3
View File
@@ -50,24 +50,28 @@ def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument('--src', type=Path, default=ROOT / 'toolchain' / 'src' / 'hactool', help='source checkout')
ap.add_argument('--out', type=Path, default=ROOT / 'toolchain' / 'hactool', help='where hactool(.exe) goes')
ap.add_argument('--cc', help='Linux: the C compiler command for hactool and its bundled mbedtls, for example '
'"clang --sysroot=DIR" (tools/make_release_linux.py); default cc')
args = ap.parse_args()
if WIN and not MSYS_BASH.is_file():
sys.exit(f'{MSYS_BASH} not found: install MSYS2 with the UCRT64 gcc and make (docs/building.md)')
if not WIN and not (shutil.which('make') and shutil.which('cc')):
if not WIN and not (shutil.which('make') and (args.cc or shutil.which('cc'))):
sys.exit('make and cc not found: install a C compiler and make (docs/building.md)')
src, out = args.src.resolve(), args.out.resolve()
fetch(src)
for p in PATCHES:
git(src, 'apply', '--whitespace=nowarn', str(p))
(src / 'config.mk').write_text(CONFIG_MK, encoding='utf-8', newline='\n')
config = CONFIG_MK.replace('CC = cc\n', f'CC = {args.cc}\n') if args.cc else CONFIG_MK
(src / 'config.mk').write_text(config, encoding='utf-8', newline='\n')
jobs = f'-j{os.cpu_count() or 4}'
if WIN:
env = dict(os.environ, MSYSTEM='UCRT64', CHERE_INVOKING='1')
r = subprocess.run([str(MSYS_BASH), '-lc', f'make {jobs}'], cwd=src, env=env, capture_output=True, text=True)
else:
r = subprocess.run(['make', jobs], cwd=src, capture_output=True, text=True)
r = subprocess.run(['make', jobs, *([f'CC={args.cc}'] if args.cc else [])], cwd=src, capture_output=True,
text=True) # CC= on the command line: the bundled mbedtls's own make reads it too
(src / 'build.log').write_text(r.stdout + r.stderr, encoding='utf-8')
if r.returncode:
sys.exit(f'make failed ({r.returncode}); log: {src / "build.log"}')
+8
View File
@@ -136,6 +136,9 @@ def main() -> None:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--id-only", action="store_true", help="write totk_xlate.id for the exe already built")
parser.add_argument("--out", type=Path, help="build directory instead of build/xlate")
parser.add_argument("--llvm", type=Path, help="Linux: an LLVM directory whose bin/clang and lld link the exe")
parser.add_argument("--sysroot", type=Path, help="Linux: link against this sysroot's glibc instead of the host's "
"(tools/make_release_linux.py)")
a = parser.parse_args()
if a.out:
global BUILD
@@ -166,6 +169,11 @@ def main() -> None:
dotnet, "publish", str(work / "totk_xlate.csproj"), "-c", "Release", "-r", "win-x64" if WIN else "linux-x64",
"-p:PublishAot=true", "-p:InvariantGlobalization=true", f"-p:RyuSrc={src}", "-o", str(out),
]
if a.llvm:
env["PATH"] = str(a.llvm.resolve() / "bin") + os.pathsep + env["PATH"]
cmd += ["-p:CppCompilerAndLinker=clang", "-p:LinkerFlavor=lld"]
if a.sysroot:
cmd.append(f"-p:SysRoot={a.sysroot.resolve()}")
if subprocess.run(cmd, env=env).returncode != 0:
sys.exit("dotnet publish failed")
exe = BUILD / EXE
+492
View File
@@ -0,0 +1,492 @@
"""Assemble the Linux release folder of docs/installer.md ("Linux release") and pack it as an AppImage.
.venv/bin/python tools/make_release_linux.py --out build/release/TotK-Setup-linux \\
[--appimage build/release/TotK-Setup-x86_64.AppImage]
.venv/bin/python tools/make_release_linux.py --out build/release/TotK-Setup-linux --prove
The same model as the Windows release (tools/make_release.py, whose download cache, unpacking and checks this reuses): the
folder carries the setup and a toolchain, never game code; the player's machine builds the game. Everything that runs on
the player's machine is built against the Steam Runtime 3 "sniper" SDK sysroot (glibc 2.31), so the AppImage and the
game it builds run on SteamOS and on any distribution at least as new. Needs network access on the first run, make,
the .NET 10 SDK and Python 3.12+ with Pillow (.venv). The folder holds:
python/ python-build-standalone 3.14 (with tkinter) + the packages of tools/requirements-installer.txt; a .pth
file puts the folder, tools/ and recomp/ on sys.path
llvm/ LLVM 23 (clang, lld, llvm-ar, the clang resource headers) and llvm/sysroot: the files of the sniper SDK
sysroot the game's compile and link read, nothing else (recorded while building the runtime library)
runtime/ libtotkrt.a (build_game.py --make-runtime-lib --sysroot), the runtime headers, romfs.c, patches.c and
ov_zstd.c (compiled per install), totk.png (the window icon), deps/ (Vulkan headers 1.4.363, VMA, zstd
built static; SDL3, Opus and libvpx come from the sysroot)
bin/ totk_xlate (+ .id), hactool, ninja, libSDL3.so.0 (the sysroot's; the game finds it beside itself)
recomp/, tools/, data/, installer/, licenses/, THIRD_PARTY_NOTICES.md, totk.ini.example, VERSION
AppRun, totk-setup.desktop, totk-setup.png the AppImage entry: python/bin/python3 -m installer.gui
--appimage packs the folder with appimagetool and the static type-2 runtime (no libfuse2 on the host). --prove compiles
generated/v121 and links build/release-linux-check/totk with the folder's tools only, then runs the probes beside it.
The folder contains nothing derived from the game: check_content runs over it before the script finishes, then the
folder's own python runs a check-only installer pass and the window's --selftest.
"""
import argparse
import os
import re
import shutil
import subprocess
import sys
import time
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import build_game # noqa: E402
import make_release as mr # noqa: E402
ROOT = mr.ROOT
MAIN_ROOT = mr.MAIN_ROOT
WORK = mr.WORK
CHECK = MAIN_ROOT / 'build' / 'release-linux-check'
LLVM_VERSION = '23.1.4'
LLVM_MAJOR = LLVM_VERSION.split('.')[0]
SNIPER = '3.0.20261021.266025'
SNIPER_URL = 'https://repo.steampowered.com/steamrt-images-sniper/snapshots'
PINS = {
'llvm-linux': (f'https://github.com/llvm/llvm-project/releases/download/llvmorg-{LLVM_VERSION}/'
f'LLVM-{LLVM_VERSION}-Linux-X64.tar.xz',
'25d814fda50d5fb0084b42be09605d8b1595d49d5e2e03a3345d7e112ff6f9f3'),
'llvm-license': (f'https://raw.githubusercontent.com/llvm/llvm-project/llvmorg-{LLVM_VERSION}/llvm/LICENSE.TXT',
'8d85c1057d742e597985c7d4e6320b015a9139385cff4cbae06ffc0ebe89afee'), # the binary release carries none
'sniper-sdk': (f'{SNIPER_URL}/{SNIPER}/com.valvesoftware.SteamRuntime.Sdk-amd64%2Ci386-sniper-sysroot.tar.gz',
'8f71947035f97925a311b2ed753f22efcf2f13d6f1631945b8e537f8903ce2cf'),
'python-linux': ('https://github.com/astral-sh/python-build-standalone/releases/download/20261026/'
'cpython-3.14.10%2B20261026-x86_64-unknown-linux-gnu-install_only.tar.gz',
'1b38024924eb7f4cf0cab45d7b871d1f1991354190fccd2625ac7529a43786e2'),
'ninja-linux': ('https://github.com/ninja-build/ninja/releases/download/v1.13.2/ninja-linux.zip',
'5749cbc4e668273514150a80e387a957f933c6ed3f5f11e03fb30955e2bbead6'),
'zstd-src': ('https://github.com/facebook/zstd/releases/download/v1.5.7/zstd-1.5.7.tar.gz',
'eb33e51f49a15e023950cd7825ca74a4a2b43db8354825ac24fc1b7ee09e6fa3'),
'vulkan-headers-src': ('https://github.com/KhronosGroup/Vulkan-Headers/archive/refs/tags/v1.4.363.tar.gz',
'cbaf687d3c59b9666fe080e7f8396b6b4f4d344a768ee6b337c59852f4526b68'),
'vma-header': build_game.VMA_FETCH,
# appimagetool and the runtime are published only under the moving "continuous" tag: pinned by hash, so a new
# upload stops the build until the pins are updated on purpose
'appimagetool': ('https://github.com/AppImage/appimagetool/releases/download/continuous/appimagetool-x86_64.AppImage',
'793fca0ce4bcc309dc65d3f66669f58e3755643e4dadbe03932e7b436128494f'),
'appimage-runtime': ('https://github.com/AppImage/type2-runtime/releases/download/continuous/runtime-x86_64',
'617aa77314d57c8b8e4fae9c3f0b35af7b30d5529cb69e98f64f92510ed98a46'),
}
mr.PINS.update(PINS) # one cache (toolchain/release-dl) and one fetch for both releases
# What clang, lld and the runtime build need from the LLVM release (the rest of its 8 GB stays packed).
LLVM_MEMBERS = [f'bin/clang-{LLVM_MAJOR}', 'bin/clang', 'bin/clang++', 'bin/lld', 'bin/ld.lld', 'bin/llvm-ar',
'bin/llvm-ranlib', 'bin/llvm-objcopy', 'bin/llvm-strip', f'lib/clang/{LLVM_MAJOR}/include']
# The parts of the sniper SDK a C/C++ compile and link for x86_64 can read.
SNIPER_MEMBERS = ['usr/include', 'usr/lib/x86_64-linux-gnu', 'usr/lib/gcc/x86_64-linux-gnu',
'lib/x86_64-linux-gnu', 'lib64', 'usr/share/doc/libsdl3-0', 'usr/share/doc/libopus0',
'usr/share/doc/libvpx6', 'usr/share/doc/libc6', 'usr/share/doc/gcc-10-base',
'usr/share/doc/libvulkan1']
# glibc's link libraries and their scripts (lib<name>.so, .a, _nonshared.a), and the GCC runtime pieces clang links.
GLIBC_LIBS = ['c', 'm', 'mvec', 'dl', 'pthread', 'rt', 'util']
GCC_LIBS = {'libgcc.a', 'libgcc_eh.a', 'libgcc_s.so', 'libstdc++.a', 'libsupc++.a'}
# The libraries of a release link (build_game.py LINK_LIBS with --deps: Opus, libvpx and zstd static).
RELEASE_LIBS = ['-lvulkan', '-lSDL3', '-l:libopus.a', '-l:libvpx.a', '-l:libzstd.a', '-l:libstdc++.a', '-lm',
'-lpthread', '-ldl']
def run(cmd, **kw):
return mr.run(cmd, **kw)
def tar_extract(name, dest, members, strip=0):
"""Extract only members (path prefixes inside the archive) of a pinned tarball with the system tar, once."""
stamp = dest / '.unpacked'
if stamp.is_file():
return dest
shutil.rmtree(dest, ignore_errors=True)
dest.mkdir(parents=True)
path = mr.fetch(name)
t0 = time.time()
run(['tar', '-xf', path, '-C', dest, f'--strip-components={strip}', '--wildcards', '--no-wildcards-match-slash',
*members])
print(f'{name}: unpacked in {time.time() - t0:.0f} s')
stamp.write_text('')
return dest
# ---- llvm/ and the sysroot -------------------------------------------------------------------------------------------
def build_llvm(out):
top = f'LLVM-{LLVM_VERSION}-Linux-X64'
src = tar_extract('llvm-linux', WORK / 'llvm-linux', [f'{top}/{m}' for m in LLVM_MEMBERS], strip=1)
dst = out / 'llvm'
shutil.copytree(src, dst, symlinks=True, ignore=shutil.ignore_patterns('.unpacked'))
strip(dst / 'bin', dst / 'bin' / 'llvm-strip')
r = subprocess.run([str(dst / 'bin' / 'clang'), '--version'], capture_output=True, text=True)
if r.returncode:
sys.exit(f'{dst / "bin" / "clang"} does not start:\n{r.stderr}')
print(r.stdout.splitlines()[0])
return dst
def strip(where, tool):
"""Drop the debug symbols of the ELF files under where (the LLVM and Python releases ship them: clang 270 MB)."""
for p in sorted(where.rglob('*')):
if p.is_file() and not p.is_symlink() and p.read_bytes()[:4] == b'\x7fELF':
subprocess.run([str(tool), '--strip-unneeded', str(p)], check=True)
def full_sysroot():
"""The sniper SDK's x86_64 compile and link files in toolchain/release-work/sniper, absolute symlinks made relative
(the SDK points some at /lib and /usr/lib, which outside its container are the host's)."""
root = WORK / 'sniper'
if (root / '.relinked').is_file():
return root
tar_extract('sniper-sdk', root, SNIPER_MEMBERS)
for p in root.rglob('*'):
if p.is_symlink() and os.readlink(p).startswith('/'):
target = root / os.readlink(p).lstrip('/')
p.unlink()
p.symlink_to(os.path.relpath(target, p.parent))
(root / '.relinked').write_text('')
return root
def sysroot_cc(llvm, sysroot):
return f'{llvm / "bin" / "clang"} --target=x86_64-linux-gnu --sysroot={sysroot} -fuse-ld=lld'
# ---- runtime/deps, bin/ --------------------------------------------------------------------------------------------
def build_deps(out, llvm, sysroot):
d = out / 'runtime' / 'deps'
(d / 'include' / 'vma').mkdir(parents=True, exist_ok=True)
(d / 'lib').mkdir(parents=True, exist_ok=True)
vk = tar_extract('vulkan-headers-src', WORK / 'vulkan-headers', ['Vulkan-Headers-1.4.363/include',
'Vulkan-Headers-1.4.363/LICENSE.md'], strip=1)
for sub in ('vulkan', 'vk_video'):
shutil.copytree(vk / 'include' / sub, d / 'include' / sub)
shutil.copy2(mr.fetch('vma-header'), d / 'include' / 'vma' / 'vk_mem_alloc.h')
zs = tar_extract('zstd-src', WORK / 'zstd', ['zstd-1.5.7'], strip=1)
lib = zs / 'lib'
if not (lib / 'libzstd.a').is_file():
run(['make', '-C', lib, f'-j{os.cpu_count() or 4}', 'libzstd.a', f'CC={sysroot_cc(llvm, sysroot)}',
f'AR={llvm / "bin" / "llvm-ar"}', 'ZSTD_LEGACY_SUPPORT=0', 'MOREFLAGS=-O2'], stdout=subprocess.DEVNULL)
shutil.copy2(lib / 'libzstd.a', d / 'lib' / 'libzstd.a')
for name in ('zstd.h', 'zstd_errors.h'):
shutil.copy2(lib / name, d / 'include' / name)
def build_bin(out, llvm, sysroot):
b = out / 'bin'
b.mkdir(parents=True, exist_ok=True)
zip_dir = mr.unpack('ninja-linux')
shutil.copy2(zip_dir / 'ninja', b / 'ninja')
(b / 'ninja').chmod(0o755)
hac = WORK / 'hactool-linux'
run([sys.executable, ROOT / 'tools' / 'build_hactool.py', '--src', WORK / 'hactool-src', '--out', hac,
'--cc', sysroot_cc(llvm, sysroot)])
shutil.copy2(hac / 'hactool', b / 'hactool')
xl = WORK / 'xlate-linux'
run([sys.executable, ROOT / 'tools' / 'build_xlate.py', '--out', xl, '--llvm', llvm, '--sysroot', sysroot])
shutil.copy2(xl / 'totk_xlate', b / 'totk_xlate')
shutil.copy2(xl / 'totk_xlate.id', b / 'totk_xlate.id')
sdl = (sysroot / 'usr' / 'lib' / 'x86_64-linux-gnu' / 'libSDL3.so.0').resolve()
shutil.copy2(sdl, b / 'libSDL3.so.0')
# ---- runtime/ ----------------------------------------------------------------------------------------------------------
def runtime_lib(out, llvm, sysroot):
build = CHECK / 'rtlib'
shutil.rmtree(build, ignore_errors=True)
run([sys.executable, ROOT / 'tools' / 'build_game.py', '--make-runtime-lib', '--toolchain', llvm, '--sysroot', sysroot,
'--deps', out / 'runtime' / 'deps', '--ninja', out / 'bin' / 'ninja', '--out', build])
rt = out / 'runtime'
shutil.copy2(build / 'libtotkrt.a', rt / 'libtotkrt.a')
for p in sorted((ROOT / 'runtime').rglob('*')):
if p.suffix in mr.RUNTIME_SUFFIXES and p.is_file():
(rt / p.relative_to(ROOT / 'runtime')).parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(p, rt / p.relative_to(ROOT / 'runtime'))
for name in build_game.INSTALL_SIDE:
shutil.copy2(ROOT / 'runtime' / name, rt / name)
from PIL import Image # Tk loads no .ico off Windows: the window and the AppImage take a PNG
with Image.open(ROOT / 'runtime' / 'totk.ico') as ico:
ico.size = max(ico.ico.sizes())
ico.convert('RGBA').save(rt / 'totk.png')
return build
def used_sysroot_files(out, llvm, sysroot, rtlib_build):
"""The sysroot files a game build reads: the headers of the runtime's compile (its .d files), of the install-side
sources and of hostcheck, and every library and object of a link (lld --trace) of the runtime library with the
flags and libraries build_game.py uses, its references to the generated code left unresolved."""
used = set()
for dep in rtlib_build.rglob('*.d'):
used.update(re.findall(r'\S+', dep.read_text(encoding='utf-8', errors='replace').replace('\\\n', ' ')))
probe = CHECK / 'sysroot-probe'
shutil.rmtree(probe, ignore_errors=True)
probe.mkdir(parents=True)
cc = [llvm / 'bin' / 'clang', '--target=x86_64-linux-gnu', f'--sysroot={sysroot}', '-I', out / 'runtime',
'-I', out / 'runtime' / 'deps' / 'include', '-I', CHECK / 'rtlib']
for i, src in enumerate([*(out / 'runtime' / n for n in sorted(build_game.INSTALL_SIDE)),
ROOT / 'installer' / 'hostcheck.c']):
# romfs.c needs the per-install romfs_config.h: whatever it reads up to there still lands in the .d file
subprocess.run([str(c) for c in [*cc, '-fsyntax-only', '-MD', '-MF', probe / f'{i}.d', src]],
capture_output=True)
if (probe / f'{i}.d').is_file():
used.update(re.findall(r'\S+', (probe / f'{i}.d').read_text().replace('\\\n', ' ')))
trace = subprocess.run([str(c) for c in [
llvm / 'bin' / 'clang', '--target=x86_64-linux-gnu', f'--sysroot={sysroot}', *build_game.LINK_FLAGS[:3],
'-Wl,--trace', '-Wl,--unresolved-symbols=ignore-all', '-Wl,--whole-archive', out / 'runtime' / 'libtotkrt.a',
'-Wl,--no-whole-archive', '-L', out / 'runtime' / 'deps' / 'lib',
*RELEASE_LIBS, '-o', probe / 'trace.out']],
capture_output=True, text=True)
if trace.returncode:
sys.exit(f'sysroot trace link failed:\n{trace.stderr[-3000:]}')
used.update(line.split('(')[0] for line in trace.stdout.splitlines())
root = str(sysroot.resolve())
files = set()
for p in used:
path = Path(p)
if not path.is_absolute():
continue
for q in (path, path.resolve()): # the name read and, through symlinks, the file
if str(q).startswith(root + '/') and q.exists():
files.add(q)
# The C library and kernel headers whole: which of them a compile reads depends on its flags (at -O1 glibc's inline
# variants pull in bits/stdio.h and others), and the setup compiles the generated code, hostcheck and the
# install-side sources with flags of its own
inc = sysroot / 'usr' / 'include'
arch = inc / 'x86_64-linux-gnu'
files.update(p for p in [*inc.glob('*.h'), *arch.glob('*.h')] if p.is_file())
for d in (arch / 'bits', arch / 'sys', arch / 'gnu', arch / 'asm', inc / 'linux', inc / 'asm-generic'):
files.update(p for p in d.rglob('*') if p.is_file() or p.is_symlink())
# The C runtime whole, for PIE links (hostcheck) as well as the game's: lld's trace names the libraries a link
# script loads but not the script (libc.so, libm.so, libgcc_s.so), and clang's GCC detection needs the crt objects
lib = sysroot / 'usr' / 'lib' / 'x86_64-linux-gnu'
for name in GLIBC_LIBS:
files.update(p for p in lib.glob(f'lib{name}[._]*') if p.suffix in ('.so', '.a') or '.so' in p.name)
files.update(p for p in (sysroot / 'lib' / 'x86_64-linux-gnu').glob(f'lib{name}.so*'))
files.update(lib.glob('*crt*.o'))
files.update((sysroot / 'lib64').glob('ld-linux*'))
files.update((sysroot / 'lib' / 'x86_64-linux-gnu').glob('ld-linux*'))
files.update((sysroot / 'lib' / 'x86_64-linux-gnu').glob('libgcc_s.so*'))
gcc = next((sysroot / 'usr' / 'lib' / 'gcc' / 'x86_64-linux-gnu').iterdir())
files.update(p for p in gcc.iterdir() if p.name.startswith('crt') or p.name in GCC_LIBS)
return files
def link_chain(path):
"""path and every link on the way to its file: libvulkan.so -> libvulkan.so.1 -> libvulkan.so.1.4.309."""
hops = [path]
while hops[-1].is_symlink() and len(hops) < 16:
hops.append(hops[-1].parent / os.readlink(hops[-1]))
return hops
def build_sysroot(out, sysroot, files):
dst = out / 'llvm' / 'sysroot'
files = {Path(os.path.normpath(h)) for f in files for h in link_chain(f)}
for f in sorted(files):
rel = f.relative_to(sysroot)
(dst / rel).parent.mkdir(parents=True, exist_ok=True)
if f.is_symlink():
if not (dst / rel).is_symlink():
(dst / rel).symlink_to(os.readlink(f))
elif not (dst / rel).exists():
shutil.copy2(f, dst / rel)
# a symlink copied above may name a file the build reads only through it
for link in [p for p in dst.rglob('*') if p.is_symlink()]:
target = (sysroot / link.relative_to(dst)).resolve()
mirrored = dst / target.relative_to(sysroot.resolve())
if not mirrored.exists():
mirrored.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(target, mirrored)
print(f'sysroot: {sum(1 for p in dst.rglob("*") if p.is_file())} files, {mr.mib(mr.dir_size(dst))}')
# ---- python/ ---------------------------------------------------------------------------------------------------------
def build_python(out):
src = mr.unpack('python-linux') / 'python'
py = out / 'python'
shutil.copytree(src, py, symlinks=True,
ignore=shutil.ignore_patterns('test', 'idlelib', 'turtledemo', '__pycache__', '*.a'))
exe = py / 'bin' / 'python3'
env = dict(os.environ, PIP_DISABLE_PIP_VERSION_CHECK='1')
run([exe, '-m', 'pip', 'install', '--no-compile', '--only-binary', ':all:', '-r',
ROOT / 'tools' / 'requirements-installer.txt'], env=env, stdout=subprocess.DEVNULL)
run([exe, '-m', 'pip', 'uninstall', '-y', 'pip'], stdout=subprocess.DEVNULL) # the setup installs nothing
strip(py, out / 'llvm' / 'bin' / 'llvm-strip')
site = next(py.glob('lib/python3.*/site-packages'))
# The release folder, tools/ and recomp/ on sys.path, as the Windows python's ._pth does: `python -m installer` and
# the tools importing their siblings work from any directory
(site / 'totk-release.pth').write_text('../../../..\n../../../../tools\n../../../../recomp\n', encoding='utf-8')
check = subprocess.run([str(exe), '-c', 'import tkinter, numpy, lz4.block, cryptography, cffi, pycparser; '
'print("python ok: Tcl", tkinter.Tcl().eval("info patchlevel"), "numpy", numpy.__version__)'],
cwd='/', capture_output=True, text=True)
if check.returncode:
sys.exit('bundled python check failed:\n' + check.stderr)
print(check.stdout.strip())
# ---- licences, AppImage entry ------------------------------------------------------------------------------------------
def collect_licenses(out, sysroot):
lic = out / 'licenses'
lic.mkdir(parents=True, exist_ok=True)
doc = sysroot / 'usr' / 'share' / 'doc'
pairs = [
(mr.fetch('llvm-license'), 'llvm-LICENSE.txt'),
(doc / 'libsdl3-0' / 'copyright', 'SDL3-copyright.txt'),
(doc / 'libopus0' / 'copyright', 'opus-copyright.txt'),
(doc / 'libvpx6' / 'copyright', 'libvpx-copyright.txt'),
(doc / 'libc6' / 'copyright', 'glibc-copyright.txt'),
(doc / 'gcc-10-base' / 'copyright', 'libstdc++-copyright.txt'),
(doc / 'libvulkan1' / 'copyright', 'vulkan-loader-copyright.txt'),
(WORK / 'vulkan-headers' / 'LICENSE.md', 'vulkan-headers-LICENSE.txt'),
(WORK / 'zstd' / 'LICENSE', 'zstd-LICENSE.txt'),
(out / 'python' / 'lib' / f'python{sys_version(out)}' / 'LICENSE.txt', 'python-LICENSE.txt'),
(ROOT / 'runtime' / 'shaders' / 'ffx' / 'LICENSE.txt', 'FidelityFX-FSR-LICENSE.txt'),
(WORK / 'hactool-linux' / 'LICENSE', 'hactool-LICENSE.txt'),
(MAIN_ROOT / 'refs' / 'MacNativeGames' / 'LICENSE', 'MacNativeGames-LICENSE.txt'),
]
for src, name in pairs:
shutil.copy2(src, lic / name)
import tarfile # Debian's libvpx copyright carries no patent grant: the WebM PATENTS text from the pinned source tag
with tarfile.open(mr.fetch('libvpx-src')) as t:
(lic / 'libvpx-PATENTS.txt').write_bytes(t.extractfile('libvpx-1.17.0/PATENTS').read())
head = (out / 'runtime' / 'deps' / 'include' / 'vma' / 'vk_mem_alloc.h').read_text(encoding='utf-8').split('*/')[0]
(lic / 'vulkan-memory-allocator-LICENSE.txt').write_text(head.lstrip('/*') + '\n', encoding='utf-8')
def sys_version(out):
return next((out / 'python' / 'lib').glob('python3.*')).name.removeprefix('python')
APPRUN = '''#!/bin/sh
# The AppImage entry: the setup window, run by the bundled Python from the read-only mount (no bytecode written there).
HERE="$(dirname "$(readlink -f "$0")")"
export PYTHONDONTWRITEBYTECODE=1
exec "$HERE/python/bin/python3" -m installer.gui "$@"
'''
DESKTOP = '''[Desktop Entry]
Type=Application
Name=TotK Recomp Setup
Comment=Builds Tears of the Kingdom for this PC from your own game files
Exec=AppRun
Icon=totk-setup
Categories=Game;
Terminal=false
'''
def build_entry(out):
(out / 'AppRun').write_text(APPRUN, encoding='utf-8')
(out / 'AppRun').chmod(0o755)
(out / 'totk-setup.desktop').write_text(DESKTOP, encoding='utf-8')
shutil.copy2(out / 'runtime' / 'totk.png', out / 'totk-setup.png')
(out / '.DirIcon').symlink_to('totk-setup.png')
def write_appimage(out, path):
tool = mr.fetch('appimagetool')
tool.chmod(0o755)
runtime = mr.fetch('appimage-runtime')
path.parent.mkdir(parents=True, exist_ok=True)
env = dict(os.environ, ARCH='x86_64', APPIMAGE_EXTRACT_AND_RUN='1')
run([tool, '--no-appstream', '--runtime-file', runtime, out, path], env=env)
print(f'{path}: {mr.mib(path.stat().st_size)}')
# ---- checks -------------------------------------------------------------------------------------------------------------
def check_bundle(out):
"""The folder's own python runs the installer without game files and the window's selftest."""
import json
chk = CHECK / 'bundle'
shutil.rmtree(chk, ignore_errors=True)
py = out / 'python' / 'bin' / 'python3'
env = dict(os.environ, PYTHONDONTWRITEBYTECODE='1')
r = subprocess.run([str(py), '-m', 'installer', '--lang', 'en-US', '--dest', str(chk / 'install'), '--temp',
str(chk / 'temp'), '--root', str(out)], cwd='/', env=env, capture_output=True, text=True,
timeout=600)
events = [json.loads(line) for line in r.stdout.splitlines() if line.startswith('{')]
errors = [e['id'] for e in events if e['type'] == 'error']
started = any(e['type'] == 'start' and e.get('release') == (out / 'VERSION').read_text().strip() for e in events)
steps = [e['phase'] for e in events if e['type'] == 'progress']
if errors != ['check.inputs_needed'] or not started or 'inputs' not in steps:
sys.exit(f'bundle check: exit {r.returncode}, errors {errors}, release read {started}\n{r.stdout[-2000:]}\n'
f'{r.stderr[-2000:]}')
print(f'bundle check: the installer ran the host check from the folder and stopped at the game files '
f'(exit {r.returncode})')
r = subprocess.run([str(py), '-m', 'installer.gui', '--selftest', '--root', str(out)], cwd='/', env=env,
capture_output=True, text=True, timeout=300)
if r.returncode:
sys.exit(f'bundle check: window selftest exit {r.returncode}\n{r.stdout[-2000:]}\n{r.stderr[-2000:]}')
print('bundle check: the window built every screen in both languages and closed')
def prove(out):
"""Compile generated/v121 and link totk with the folder's tools only, then run the probes beside it."""
out = out.resolve()
gen = MAIN_ROOT / 'generated' / 'v121'
ext = MAIN_ROOT / 'build' / 'extracted' / 'v121'
chk = CHECK / 'game'
env = dict(os.environ, PATH=os.pathsep.join([str(out / 'llvm' / 'bin'), str(out / 'bin'), '/usr/bin', '/bin']))
t0 = time.time()
run([out / 'python' / 'bin' / 'python3', out / 'tools' / 'build_game.py', '--toolchain', out / 'llvm',
'--sysroot', out / 'llvm' / 'sysroot', '--deps', out / 'runtime' / 'deps', '--runtime', out / 'runtime',
'--runtime-lib', out / 'runtime' / 'libtotkrt.a', '--ninja', out / 'bin' / 'ninja', '--gen', gen, '--out', chk,
'--exefs', ext / 'exefs', '--romfs', ext / 'romfs', '--profile', chk / 'profile'], env=env)
print(f'prove: build {time.time() - t0:.0f} s')
shutil.copy2(out / 'bin' / 'libSDL3.so.0', chk / 'libSDL3.so.0')
needed = subprocess.run(['readelf', '-d', str(chk / 'totk')], capture_output=True, text=True).stdout
print('prove: needs', ', '.join(re.findall(r'\[(lib[^\]]+)\]', needed)))
glibc = sorted(set(re.findall(r'GLIBC_(\d+\.\d+)', subprocess.run(['readelf', '-V', str(chk / 'totk')],
capture_output=True, text=True).stdout)),
key=lambda v: tuple(map(int, v.split('.'))))
print(f'prove: newest glibc symbol version {glibc[-1] if glibc else "none"}')
failed = []
for probe in ['--selftest', *mr.PROBES]:
r = subprocess.run([str(chk / 'totk'), probe], cwd=chk, capture_output=True, text=True, timeout=900)
tail = (r.stdout.strip().splitlines() or [''])[-1]
print(f'{probe}: exit {r.returncode}: {tail}')
if r.returncode:
failed.append(probe)
if failed:
sys.exit(f'probes failed: {failed}')
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument('--out', type=Path, required=True, help='release folder (replaced)')
ap.add_argument('--appimage', type=Path, help='also pack the folder as this AppImage')
ap.add_argument('--prove', action='store_true', help='only run the compile-and-probe check on an existing folder')
args = ap.parse_args()
out = args.out.resolve()
if args.prove:
prove(out)
return
if out.exists():
shutil.rmtree(out)
out.mkdir(parents=True)
llvm = build_llvm(out)
sysroot = full_sysroot()
build_deps(out, llvm, sysroot)
build_bin(out, llvm, sysroot)
rtlib = runtime_lib(out, llvm, sysroot)
build_sysroot(out, sysroot, used_sysroot_files(out, llvm, sysroot, rtlib))
build_python(out)
mr.build_sources(out)
collect_licenses(out, sysroot)
build_entry(out)
mr.check_folder(out)
check_bundle(out)
mr.sizes(out)
if args.appimage:
write_appimage(out, args.appimage.resolve())
if __name__ == '__main__':
main()
+8 -6
View File
@@ -20,7 +20,9 @@ from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
DLLS = ('SDL3.dll', 'libopus-0.dll', 'libiconv-2.dll')
EXE = '.exe' if os.name == 'nt' else ''
# A Linux source build links the system's SDL3, Opus and libvpx: the other machine needs them installed
DLLS = ('SDL3.dll', 'libopus-0.dll', 'libiconv-2.dll') if os.name == 'nt' else ()
# Prepared state next to the exe: translator answers, programs, the prepared list, pipeline cache and its keys, and the
# variant lists `totk.exe --prepare-shaders` reads (the driver keys its library cache by exe path, so each copy runs it).
STATE = ('romfs-index.bin', 'pipeline-cache.bin', 'pipeline-keys.bin', 'shader-answers.bin', 'shader-programs.bin',
@@ -51,7 +53,7 @@ def main():
ap.add_argument('--version', default='v121', help='game version (v121 or v143): the defaults below follow it')
ap.add_argument('--build', type=Path, help='build dir with the prepared state (default build/<version>)')
ap.add_argument('--exe', type=Path, help='totk.exe to package (default: <build>/totk.exe); its DLLs are taken from --build')
ap.add_argument('--xlate', type=Path, default=ROOT / 'build' / 'xlate' / 'totk_xlate.exe')
ap.add_argument('--xlate', type=Path, default=ROOT / 'build' / 'xlate' / f'totk_xlate{EXE}')
ap.add_argument('--extracted', type=Path, help='default build/extracted/<version>')
ap.add_argument('--out', type=Path, help='default build/<version>-portable')
ap.add_argument('--with-romfs', action='store_true', help='also copy romfs.bin (about 16 GB)')
@@ -62,7 +64,7 @@ def main():
a.out = a.out or ROOT / 'build' / f'{a.version}-portable'
build, out = a.build.resolve(), a.out.resolve()
exe = (a.exe or build / 'totk.exe').resolve()
exe = (a.exe or build / f'totk{EXE}').resolve()
fonts = a.extracted.resolve().parent / 'firmware' / 'shared-fonts'
exefs = a.extracted.resolve() / 'exefs'
romfs = a.extracted.resolve() / 'romfs' / 'romfs.bin'
@@ -78,10 +80,10 @@ def main():
print('package_build: warning: the folder holds game code; keep it out of the repository')
out.mkdir(parents=True, exist_ok=True)
copy(exe, out / 'totk.exe')
copy(exe, out / f'totk{EXE}')
for d in DLLS:
copy(build / d, out / d)
copy(a.xlate, out / 'totk_xlate.exe')
copy(a.xlate, out / f'totk_xlate{EXE}')
idf = a.xlate.with_suffix('.id')
if idf.exists():
copy(idf, out / 'totk_xlate.id')
@@ -101,7 +103,7 @@ def main():
print('package_build: copying romfs.bin (16 GB) ...')
shutil.copyfile(romfs, out / 'romfs' / 'romfs.bin')
print(f'package_build: {out}: {n} files in folders, {size_of(out) / 2**30:.2f} GiB'
+ ('' if a.with_romfs else '; no RomFS: put romfs.bin into romfs\\ or set TOTK_ROMFS'))
+ ('' if a.with_romfs else f'; no RomFS: put romfs.bin into romfs{os.sep} or set TOTK_ROMFS'))
if __name__ == '__main__':
+1 -1
View File
@@ -34,7 +34,7 @@ from pathlib import Path
import lz4.block
ROOT = Path(__file__).resolve().parent.parent
XLATE = ROOT / 'build' / 'xlate' / 'totk_xlate.exe'
XLATE = ROOT / 'build' / 'xlate' / ('totk_xlate.exe' if os.name == 'nt' else 'totk_xlate')
SPIRV_VAL = Path(r'C:\msys64\ucrt64\bin\spirv-val.exe') if os.name == 'nt' else Path(shutil.which('spirv-val') or '/usr/bin/spirv-val')
ZSTD = b'\x28\xb5\x2f\xfd'
# NVN program control section magic, u32 0x98761234 (d2r-mobile dm/gfx/tools/shader_map.py)