Keep the Linux setup's temp files and Mesa's shader cache with the game, and stop the setup with one message on a glibc older than 2.34.

A Steam Deck tester had 1.2 GB free in /home, where the setup put its 1.3 GB of generated C and objects
(~/.cache/totk-setup) and RADV its shader cache (up to 1 GB, about 35 MB a minute at the title screen), while the game
goes to the SD card. The setup's temp is now .setup-temp in the install folder (--temp changes it; the empty-folder
check leaves it out), and totk points MESA_SHADER_CACHE_DIR at mesa-cache beside itself unless it is set. A scan of
every ELF file in the release folder showed the bundled LLVM and the cryptography wheel need glibc 2.34, not the 2.31
the docs gave: make_release_linux.py now checks that floor (check_glibc), the setup's check stops below it
(check.glibc_old) instead of failing on a compiler that cannot start, and the docs give both floors. The Linux notes
also record the testers' results, that the Deck has the host-import extension too, and its software ASTC decoding.
This commit is contained in:
Wellington Dias committed 2026-10-30 14:56:54 -03:00
1 parent 48d1a6d3a0
commit b7baf43332
9 files changed
+101 -21

No files matched your search

+8 -3
View File
@@ -122,8 +122,9 @@ folder. The folder holds code and data derived from your game: for your own mach
## Linux (in progress)
The runtime builds and runs on x86-64 Linux with a window, input and menu on SDL3 (`*_sdl.c`) in place of the Win32
files (`*_win.c`); `runtime/platform/linux.c` implements `platform.h`. Status: the selftest and the probes pass, and
1.2.1 plays at 60 fps on RADV (RX 7800 XT); 1.4.3 and long play sessions are not verified yet.
files (`*_win.c`); `runtime/platform/linux.c` implements `platform.h`. Status: the selftest and the probes pass; 1.2.1
plays at 60 fps on RADV (RX 7800 XT) and NVIDIA (RTX 3060); 1.4.3 builds, probes and plays on RADV (RX 6600, and the
Vega 7 of a Ryzen 5 4600G at about 34 fps). Long play sessions and the Steam Deck itself are not verified yet.
Requirements:
@@ -154,7 +155,11 @@ re-protecting imported memory invalidates the driver's userptr buffer on every s
amdgpu), and importing pins the pages, which drops their write protection. Each CPU-visible pool therefore gets a
host-visible copy (shadow storage, `runtime/vk_memory.c`): the pages the CPU wrote go in before each submit, and the GPU's
writes come back at its fence. In play that costs about 0.2 ms a frame and doubles the memory of those pools (about
1.1 GB). `TOTK_SHADOW=1` turns it on under Windows too.
1.1 GB). `TOTK_SHADOW=1` turns it on under Windows too. The Steam Deck's RADV has `VK_EXT_external_memory_host` as
well: the shadow copies are chosen for the measured cost above, not for a missing extension.
The Deck's GPU (Van Gogh) has no native ASTC, so its textures go through the compute decoder (`runtime/vk_astc.c`,
`TOTK_ASTC`) as on desktop AMD GPUs without it; frame times measured there include it.
## Checks
+14 -7
View File
@@ -125,15 +125,22 @@ runtime/ libtotkrt.a, headers, the install-side sources, totk.png;
bin/ totk_xlate, hactool, ninja, libSDL3.so.0
```
- Everything that runs on the player's machine is built against the sniper sysroot (glibc 2.31): the game, hostcheck,
hactool and the translator need no newer glibc, so they run on SteamOS and current distributions. The game links
Opus, libvpx, zstd and libstdc++ statically and needs only glibc, the system's Vulkan loader and the SDL3 beside it.
- The setup's temp folder is `$XDG_CACHE_HOME/totk-setup` (default `~/.cache/totk-setup`): the AppImage is mounted
read-only.
- What we compile is built against the sniper sysroot (glibc 2.31): the game, hostcheck, hactool and the translator
need no newer glibc. The game links Opus, libvpx, zstd and libstdc++ statically and needs only glibc, the system's
Vulkan loader and the SDL3 beside it. The setup itself needs glibc 2.34 (Ubuntu 22.04, Debian 12, Fedora 35,
SteamOS 3): the official LLVM binaries and the `cryptography` wheel are built against it. `make_release_linux.py`
checks every ELF file in the folder against that floor (`check_glibc`), and the setup's check step stops with one
message on an older system.
- The setup's temp folder is `.setup-temp` inside the install folder (`--temp` changes it): the AppImage is mounted
read-only, and a Steam Deck's `/home` is often too full for the generated C while the game goes to the SD card.
- The game keeps Mesa's shader cache in `mesa-cache` beside `totk` unless `MESA_SHADER_CACHE_DIR` is set, for the
same reason: RADV fills its default folder in `~/.cache` up to 1 GB.
- The window follows `LANG` for English or Russian, cancels through SIGTERM to the CLI's process group (the CLI handles
it as Ctrl+Break on Windows) and opens the finished folder with `xdg-open`.
- Requirements beyond Windows': Linux 6.7 or later (the write watch, `docs/building.md`), and enough space in the cache
folder's file system for the generated C (about 1.3 GB).
- Requirements beyond Windows': Linux 6.7 or later (the write watch, `docs/building.md`) and glibc 2.34.
- Building the release needs one game version extracted and lifted (`tools/extract.py`, `recomp/gen.py`, either
version): the runtime library includes the generated `layout.h`, whose shared part (`MOD_COUNT`, `MOD_MAIN_BASE`)
is the same for both. `tools/make_release.py` takes `generated/v121`, else `generated/v143`.
## Constraints
+9 -3
View File
@@ -28,6 +28,11 @@ class Parser(argparse.ArgumentParser):
sys.exit(2)
# The temp folder inside the install folder when the layout names none (the Linux release); the check step does not
# count it when it requires an empty install folder
TEMP_IN_DEST = '.setup-temp'
def temp_folder(base, dest):
"""One temp folder per install folder, so two installs never share generated files."""
tag = hashlib.sha256(str(dest).lower().encode()).hexdigest()[:8]
@@ -46,8 +51,8 @@ def main(argv=None):
ap.add_argument('--lang', help='game language, a Switch language code (en-US, en-GB, ja, de, ru, ...)')
ap.add_argument('--dest', type=Path, required=True, help='install folder')
ap.add_argument('--root', type=Path, help='release folder or repository (default: the folder above installer/)')
ap.add_argument('--temp', type=Path, help='temp folder (default: temp/ of the release, build/install-temp in a '
'repository)')
ap.add_argument('--temp', type=Path, help='temp folder (default: temp/ of the release on Windows, .setup-temp in '
'the install folder on Linux, build/install-temp in a repository)')
ap.add_argument('--verify', action='store_true', help='hash every kept file, also those this release made')
args = ap.parse_args(argv)
@@ -58,7 +63,8 @@ def main(argv=None):
signal.signal(getattr(signal, name), cancel)
layout = layouts.find(args.root)
dest = args.dest.resolve()
temp = temp_folder((args.temp or layout.temp).resolve(), dest)
base = args.temp or layout.temp
temp = temp_folder(base.resolve(), dest) if base else dest / TEMP_IN_DEST
log = temp / 'logs' / time.strftime('install-%Y%m%d-%H%M%S.log')
rep = Reporter(log_path=log)
t0 = time.monotonic()
+3 -7
View File
@@ -90,12 +90,6 @@ class Layout:
return [str(p) for p in need if not p.exists()]
def cache_dir():
"""$XDG_CACHE_HOME, else ~/.cache: the Linux release runs from a read-only AppImage mount, so its temp goes here."""
xdg = os.environ.get('XDG_CACHE_HOME')
return Path(xdg) if xdg and Path(xdg).is_absolute() else Path.home() / '.cache'
def release_python(root):
return root / 'python' / ('python.exe' if WIN else 'bin/python3')
@@ -117,7 +111,9 @@ def release_layout(root):
xlate=root / 'bin' / ('totk_xlate' + EXE),
dlls=(root / 'bin' / 'SDL3.dll', root / 'bin' / 'libopus-0.dll') if WIN else (root / 'bin' / 'libSDL3.so.0',),
shader_list=root / 'data' / 'shader_list.tsv', ini_example=root / 'totk.ini.example',
temp=root / 'temp' if WIN else cache_dir() / 'totk-setup',
# Linux: None, the install folder's .setup-temp (installer/cli.py): the AppImage is mounted read-only, and on
# a Steam Deck /home is often too full for the generated C while the install folder is on the SD card
temp=root / 'temp' if WIN else None,
sysroot=None if WIN else root / 'llvm' / 'sysroot')
+21 -1
View File
@@ -102,6 +102,21 @@ def kill_tree(proc):
proc.kill()
def linux_glibc():
"""(major, minor) of the C library this Linux process runs on, None elsewhere or when it is not glibc."""
try:
name = os.confstr('CS_GNU_LIBC_VERSION') if sys.platform.startswith('linux') else None
except (ValueError, OSError):
return None
m = re.match(r'glibc (\d+)\.(\d+)', name or '')
return (int(m.group(1)), int(m.group(2))) if m else None
# The bundled clang, lld and the cryptography wheel are built against glibc 2.34 (tools/make_release_linux.py
# GLIBC_FLOOR); on an older one they cannot start
GLIBC_NEEDED = (2, 34)
def linux_kernel():
"""(major, minor) of the running Linux kernel, None elsewhere."""
if not sys.platform.startswith('linux'):
@@ -214,7 +229,8 @@ class Check(Step):
raise InstallError('check.release_incomplete', paths=missing[:5])
if dest.exists() and not dest.is_dir():
raise InstallError('check.dest_not_folder', path=str(dest))
if ctx.mode == 'install' and dest.is_dir() and any(dest.iterdir()):
own = (ctx.temp, *ctx.temp.parents) # a temp folder inside the install folder (installer/cli.py) is ours
if ctx.mode == 'install' and dest.is_dir() and any(p not in own for p in dest.iterdir()):
raise InstallError('check.dest_not_empty', path=str(dest))
lang = ctx.options.lang or ctx.state.get('lang')
if not lang:
@@ -242,6 +258,10 @@ class Check(Step):
def host(self, ctx):
layout, work = ctx.layout, ctx.temp / 'check'
glibc = linux_glibc()
if glibc and glibc < GLIBC_NEEDED and not layout.dev:
raise InstallError('check.glibc_old', version='.'.join(map(str, glibc)),
need='.'.join(map(str, GLIBC_NEEDED)))
work.mkdir(parents=True, exist_ok=True)
exe = work / ('hostcheck' + EXE)
command = [str(layout.clang), '-O1', str(HOSTCHECK), '-o', str(exe)]
+1
View File
@@ -41,6 +41,7 @@
"check.lang_missing": "Choose the game language.",
"check.lang_unknown": "Unknown language '{lang}'. Known: {known}.",
"check.toolchain_broken": "The bundled compiler does not work: {detail}",
"check.glibc_old": "This system's C library is glibc {version}; the setup's compiler needs {need} or later (Ubuntu 22.04, Debian 12, Fedora 35, SteamOS 3 and newer have it). Updating the system fixes this.",
"check.host_probe_failed": "The computer check did not run (exit code {code}).",
"check.cpu_no_avx2": "This processor lacks AVX2 (x86-64-v3), which the game needs. Intel Haswell (2013) or AMD Excavator / Zen and newer have it.",
"check.vulkan_missing": "No Vulkan driver is installed. Install the latest driver for your graphics card.",
+1
View File
@@ -41,6 +41,7 @@
"check.lang_missing": "Выберите язык игры.",
"check.lang_unknown": "Неизвестный язык «{lang}». Доступные: {known}.",
"check.toolchain_broken": "Встроенный компилятор не работает: {detail}",
"check.glibc_old": "Библиотека C этой системы — glibc {version}; компилятору установщика нужна {need} или новее (она есть в Ubuntu 22.04, Debian 12, Fedora 35, SteamOS 3 и новее). Обновление системы это исправит.",
"check.host_probe_failed": "Проверка компьютера не выполнилась (код выхода {code}).",
"check.cpu_no_avx2": "Этому процессору не хватает AVX2 (x86-64-v3), который нужен игре. AVX2 есть у Intel Haswell (2013) и новее, у AMD Excavator и Zen и новее.",
"check.vulkan_missing": "Драйвер Vulkan не установлен. Установите свежий драйвер видеокарты.",
+14
View File
@@ -41,7 +41,21 @@ static int layer_present(const char *name) {
return found;
}
// Mesa's drivers (RADV on Linux and the Steam Deck) keep a shader cache of their own beside our pipeline cache, by
// default in ~/.cache and up to 1 GB: about 35 MB in a minute of the title screen. A Deck's /home is small and the game
// usually sits on the SD card, so unless MESA_SHADER_CACHE_DIR names a folder, it goes to mesa-cache beside the exe.
static void mesa_cache_beside_exe(void) {
char dir[PLAT_PATH_MAX * 2];
if (plat_getenv("MESA_SHADER_CACHE_DIR", dir, sizeof dir)) return;
int len = plat_exe_path(dir, sizeof dir - 16);
char *cut = len ? plat_last_sep(dir) : NULL;
if (!cut) return;
strcpy(cut + 1, "mesa-cache");
plat_setenv("MESA_SHADER_CACHE_DIR", dir);
}
static int create_instance(void) {
mesa_cache_beside_exe();
const char *ext[8];
uint32_t n_ext = (uint32_t)gfx_window_vk_extensions(ext, 7);
const char *layers[] = {"VK_LAYER_KHRONOS_validation"};
+30
View File
@@ -404,6 +404,35 @@ def write_appimage(out, path):
# ---- checks -------------------------------------------------------------------------------------------------------------
# The newest glibc any program or library in the folder may need: the official LLVM binaries and the cryptography
# wheel are built against 2.34 (Ubuntu 22.04, Debian 12, SteamOS 3 have it); the game itself links the sysroot's 2.31.
GLIBC_FLOOR = (2, 34)
def check_glibc(out):
"""Every ELF file in the folder (llvm/sysroot aside: glibc itself, read by the link) needs glibc GLIBC_FLOOR or
older; prints which files set the floor."""
newest, by = (0, 0), []
for p in sorted(out.rglob('*')):
if not p.is_file() or p.is_symlink() or out / 'llvm' / 'sysroot' in p.parents:
continue
with open(p, 'rb') as f:
if f.read(4) != b'\x7fELF':
continue
text = subprocess.run(['readelf', '-V', str(p)], capture_output=True, text=True).stdout
need = max((tuple(map(int, v.split('.'))) for v in re.findall(r'GLIBC_(\d+\.\d+(?:\.\d+)?)', text)),
default=(0, 0))
if need > newest:
newest, by = need, []
if need == newest:
by.append(str(p.relative_to(out)))
floor = '.'.join(map(str, GLIBC_FLOOR))
if newest > GLIBC_FLOOR:
sys.exit(f'glibc check: {", ".join(by[:8])} need glibc {".".join(map(str, newest))}, above {floor}')
print(f'glibc check: the folder needs glibc {".".join(map(str, newest))} (floor {floor}), set by '
f'{", ".join(by[:6])}{" and others" if len(by) > 6 else ""}')
def check_bundle(out):
"""The folder's own python runs the installer without game files and the window's selftest."""
import json
@@ -487,6 +516,7 @@ def main():
collect_licenses(out, sysroot)
build_entry(out)
mr.check_folder(out)
check_glibc(out)
check_bundle(out)
mr.sizes(out)
if args.appimage: