Merge task/prwave1: eleven community pull requests that merge into dev cleanly

#272 #273 (texture writes and Wine), #247 #274 #284 (docs), #275 #278 (build and shader preparation), #267 (sun
shadow map size, the game's own by default), #239 (GPU idle split in the perf line), #197 (walk key), #186 (Brazilian
Portuguese installer catalog, with the three strings it lacked).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Valiantsin DzerakhandClaude Opus 5.5 committed 2026-11-01 11:07:22 +03:00
commit e1693cee8e
37 files changed
+802 -91

No files matched your search

+16 -6
View File
@@ -1,22 +1,22 @@
# TotK Recomp
**TotK Recomp** is a static recompilation of *The Legend of Zelda: Tears of the Kingdom* (Nintendo Switch, 2023) into a native Windows x86-64 application ([docs/architecture.md](docs/architecture.md)).
**TotK Recomp** is a static recompilation of *The Legend of Zelda: Tears of the Kingdom* (Nintendo Switch, 2023) into a native x86-64 application for Windows and Linux ([docs/architecture.md](docs/architecture.md)).
> [!IMPORTANT]
> **No game files are included.** You need your own legally obtained copy of *Tears of the Kingdom*, dumped from your own console: the base game and the 1.2.1 or 1.4.3 update. Instructions for obtaining those files will not be provided.
> **No game files are included.** You need your own legally obtained copy of *Tears of the Kingdom*, dumped from your own console: the base game and the 1.1.1, 1.2.1 or 1.4.3 update. Instructions for obtaining those files will not be provided.
>
> The game is built from your files on your machine, so the result contains the game. It is for your own use. Do not share it.
## Status
Early development. The game is playable on Windows; visual bugs and crashes are still expected.
Early development. The game is playable on Windows and Linux; visual bugs and crashes are still expected.
Report problems as issues, following [CONTRIBUTING.md](CONTRIBUTING.md).
## Features
- Native code, not emulation
- Game updates 1.2.1 and 1.4.3, with saves that carry over between them
- Game updates 1.1.1, 1.2.1 and 1.4.3; 1.2.1 and 1.4.3 saves carry over between them
- 60 fps by default; up to 144 fps or the monitor's refresh rate (experimental)
- Custom resolutions (1080p, 2k...), supersampling and AMD FSR 1 upscaling
- 16:10 (Steam Deck, handhelds), 21:9 and 32:9 (ultrawide) game resolutions, with the camera and the interface in that shape
@@ -27,12 +27,13 @@ Report problems as issues, following [CONTRIBUTING.md](CONTRIBUTING.md).
## Planned
- Linux and the Steam Deck
- Mods made for emulators: unpack a mod into a `mods` folder
- VRR
## Installing
### Windows
Requirements: Windows 10 or later, an x86-64 CPU with AVX2, a GPU with Vulkan 1.3 and `VK_EXT_external_memory_host`, about 36 GiB free during setup (20 GiB after).
1. Download `TotK-Setup-<version>-windows.zip` from [Releases](../../releases/latest) and unpack it.
@@ -40,7 +41,16 @@ Requirements: Windows 10 or later, an x86-64 CPU with AVX2, a GPU with Vulkan 1.
3. Pick your game files, the language and an install folder, and press *Start*. The setup builds the game once.
4. Play `totk.exe` from the install folder.
To update, run a newer `TotK-Setup.exe` on the same folder; saves and settings are kept. Settings are in the in-game menu and in `totk.ini` ([totk.ini.example](totk.ini.example) lists every key). Controls: [docs/controls.md](docs/controls.md).
### Linux
Requirements: as on Windows, plus Linux 6.7 or later and glibc 2.34 or later (Ubuntu 22.04, Debian 12, Fedora 35, SteamOS 3).
1. Download `TotK-Setup-x86_64.AppImage` from [Releases](../../releases/latest), or build it ([docs/installer.md](docs/installer.md), "Linux release").
2. Make it executable (`chmod +x TotK-Setup-x86_64.AppImage`) and start it. On a Steam Deck, do this in Desktop Mode.
3. Pick your game files, the language and an install folder, and press *Start*.
4. Play `totk` from the install folder.
To update, run a newer setup on the same folder; saves and settings are kept. Settings are in the in-game menu and in `totk.ini` ([totk.ini.example](totk.ini.example) lists every key). Controls: [docs/controls.md](docs/controls.md).
## Credits
+7 -1
View File
@@ -69,7 +69,8 @@ of them.
keys file. `config.local.toml` is ignored by git. The tools read the keys file themselves.
3. Extract and verify. This writes `build\extracted\v0` (1.0.0) and `build\extracted\<ver>` for each update in
`updates`, identified by its hash; a hash mismatch stops the run:
`updates`, identified by its hash; a hash mismatch stops the run. With another `[work] dir` in `config.local.toml`,
`extracted` goes there instead, and `build_game.py` reads the ExeFS and RomFS from there:
```powershell
.venv-py314\Scripts\python tools\extract.py
@@ -115,6 +116,11 @@ of them.
Without this step the first launch compiles shaders and pipelines during play and stutters until the caches fill.
The AMD driver keeps its pipeline cache per exe path and holds a limited amount: preparing another build folder can
evict this one's. `totk.exe --warm-shaders` rebuilds it from the shader cache.
When the driver's shader compiler crashes on a variant (NVIDIA 615.78.08 on an RTX 5080 does, issue #30),
`prepare_shaders.py` runs the driver step again with `--warm-shaders`, up to three times, without the variants the
driver was building at the crash. `shader-prepare-skip.txt` next to the exe lists them with the GPU and driver
version; they build at their first draw instead, another driver version builds them again, and so does deleting the
file.
7. Run. The game keeps running when its window loses the focus (`assume_focus = 1` in `totk.ini`, the default; the
Game tab switches it). With 0 it pauses, as it does on the Switch when Home is pressed.
+12 -7
View File
@@ -9,7 +9,7 @@ holds only the game; the installer and its tools stay outside it.
1. Downloads `TotK-Setup-<version>-windows.zip` from GitHub Releases (none is published yet), unpacks it anywhere and
starts `TotK-Setup.exe`.
2. Picks:
- the base game (XCI or NSP) and the 1.2.1 or 1.4.3 update (NSP);
- the base game (XCI or NSP) and the 1.1.1, 1.2.1 or 1.4.3 update (NSP);
- their `prod.keys`;
- the game language and the install folder.
3. Waits for the setup, one time. The window shows each step and its progress.
@@ -34,12 +34,13 @@ The player runs a newer installer on the same folder. It reads `totk-install.jso
`profile/` and `totk.ini` after checking their hashes, rebuilds `totk.exe` and re-prepares the shaders. A newer
release that changes nothing game-derived skips the rebuild.
Given an update file other than the recorded one (1.2.1, then 1.4.3), the installer extracts and builds again from it and
keeps `profile/`. Saves carry over both ways: 1.4.3 converts a 1.2.1 save on load, and 1.2.1 loads the saves 1.4.3 wrote.
Given an update file other than the recorded one (1.1.1, then 1.2.1, then 1.4.3), the installer extracts and builds again from it
and keeps `profile/`. Saves carry over both ways between 1.2.1 and 1.4.3: 1.4.3 converts a 1.2.1 save on load, and 1.2.1
loads the saves 1.4.3 wrote.
## Setup steps
1. **Check** before writing anything: the files open with the keys, the update is 1.2.1 or 1.4.3 (by hash, as
1. **Check** before writing anything: the files open with the keys, the update is 1.1.1, 1.2.1 or 1.4.3 (by hash, as
`tools/extract.py`), the keys are new enough for the update, enough space is free on the target disk (about 36 GiB at the peak of a
first install, 20 GiB after it), the CPU has
AVX2 (`-march=x86-64-v3`), a Vulkan 1.3 GPU exists. Each failure is one clear message.
@@ -55,6 +56,9 @@ keeps `profile/`. Saves carry over both ways: 1.4.3 converts a 1.2.1 save on loa
writes the translations it makes in play as loose files; running the tool again with the game closed folds those in.
The patches are written twice: `romfs-patches` and `romfs-patches-xbox` (`--xbox-prompts`, the A/B and X/Y prompts
swapped for `pad_layout = xbox_prompts`).
When the graphics driver crashes while building the shaders, `prepare_shaders.py` runs that part again without the
variants it was building ([building.md](building.md), step 6), and the setup finishes with a warning that names how
many were left out.
6. **Clean up** the temp folder (generated C and objects, about 1.3 GB).
A cancelled or failed setup resumes from the last finished step (`totk-install.json`).
@@ -79,9 +83,10 @@ python -m installer --base FILE --update FILE --keys FILE --lang CODE --dest DIR
the merged RomFS (15.9 GiB) before it deletes them. The check computes the exact figure.
- Window (`python -m installer.gui`, `installer/gui.py` draws, `installer/guilogic.py` decides): form, step list with
progress, space line, collapsible log, Cancel (Ctrl+Break to the CLI's process group, then it waits for the exit),
finish screen with Open folder. English or Russian, chosen from the Windows display language, switchable in the
window; every sentence is an id in `installer/text/<en|ru>.json`. A folder with `totk-install.json` needs only the
folder. `--selftest [--shots DIR]` builds every screen in both languages and closes itself.
finish screen with Open folder. English, Russian or Brazilian Portuguese, chosen from the Windows display language,
switchable in the window; every sentence is an id in `installer/text/<en|ru|pt-BR>.json`. A folder with
`totk-install.json` needs only the folder. `--selftest [--shots DIR]` builds every screen in every language and
closes itself.
- `tests/installer_gui_test.py`: event model, form, catalogs, cancel, the self-closing window and one end-to-end run
of the CLI in `--verify` mode (needs `build/install-test` and `build/extracted/v121`; see its docstring).
- `tests/installer_test.py`: bookkeeping with fake steps. `tests/installer_e2e.py`: a dev-mode run with the files of
+1
View File
@@ -142,6 +142,7 @@ Source shorthands:
| TotK | "detective mode" Link on AMD | AMD EDS3 colour-blend driver regression | YZ-2306 PR 10946 |
| BotW | shadow artifacts at object edges (AMD) | gather rounding | yuzu commit 780240e (2.10) |
| TotK | dashed dark lines along wall and floor edges, changing with the camera angle (AMD, RADV) | gather rounding | gather bias (2.10) |
| TotK | blocky foliage shadows that jump instead of sliding | not a GPU difference: the engine sizes its sun shadow map to the area it covers, down to about 400 near the ground, and each change of size moves the shadow texels | fixed size, `shadow_res` (the shadow set of `recomp/patches-v121.txt`; the community mods force 1024 the same way) |
| BotW | black textures on GCN | HTILE (depth compression) | `AMD_DEBUG=nohyperz` wiki (1.9) |
| BotW | 3D scene black | deferred texture handles | NXN |
| BotW | albedo scattered squares | duplicate colour attachments | NXN |
+1 -1
View File
@@ -8,7 +8,7 @@ from it may be shared.
- Windows 10 or later, an x86-64 CPU with AVX2, about 60 GB free.
- A GPU with Vulkan 1.3 and `VK_EXT_external_memory_host`, with a current driver.
- *Tears of the Kingdom* base game (1.0.0) and the 1.2.1 or 1.4.3 update, dumped from your own Switch as NSP or XCI.
- *Tears of the Kingdom* base game (1.0.0) and the 1.1.1, 1.2.1 or 1.4.3 update, dumped from your own Switch as NSP or XCI.
- `prod.keys` dumped from your own Switch. Never send it to anyone, and never paste it into a log or an issue.
## Build
+4 -4
View File
@@ -2,10 +2,10 @@
A front end over the CLI (docs/installer.md): it collects the game files, the keys file path, the game language and the
install folder, runs `python -m installer ...` as a subprocess and shows its JSON events. Decisions live in guilogic.py;
this file only draws. The window text is in installer/text/<en|ru>.json under "ui.*" and "launcher.*" ids.
this file only draws. The window text is in installer/text/<en|ru|pt-BR>.json under "ui.*" and "launcher.*" ids.
--selftest [--shots DIR] build every screen in both languages and close by itself (PNG per screen with --shots)
--lang en|ru window language (default: the Windows display language)
--selftest [--shots DIR] build every screen in every language and close by itself (PNG per screen with --shots)
--lang en|ru|pt-BR window language (default: the Windows display language)
"""
import argparse
import json
@@ -534,7 +534,7 @@ def grab(root, path):
def selftest(layout, shots=None):
"""Every screen in both languages; the window closes itself. Returns 0 when nothing raised."""
"""Every screen in every language; the window closes itself. Returns 0 when nothing raised."""
fix_tcl_paths()
root = tk.Tk()
root.geometry('+60+40')
+7 -5
View File
@@ -24,11 +24,12 @@ from . import state as statefile
from .events import render
TEXT = Path(__file__).resolve().parent / 'text'
UI_LANGUAGES = ('en', 'ru')
UI_LANGUAGES = ('en', 'ru', 'pt-BR')
STEP_ORDER = ('check', 'extract', 'lift', 'compile', 'prepare', 'cleanup')
# message parameter -> catalog id prefix of the names that replace its raw values
PARAM_NAMES = {'lang': 'lang.', 'known': 'lang.', 'step': 'step.', 'input': 'param.input.'}
LANG_RU = 0x19 # primary language id of Russian in a Windows LANGID
LANG_PT_BR = 0x0416 # Portuguese (Brazil), the whole LANGID: Portugal's Portuguese stays English
def load_texts(lang):
@@ -76,15 +77,16 @@ def flat(value):
def windows_ui_language():
"""'ru' when the display language (Windows; elsewhere LC_ALL, LC_MESSAGES or LANG) is Russian, else 'en'."""
"""'ru' when the display language (Windows; elsewhere LC_ALL, LC_MESSAGES or LANG) is Russian, 'pt-BR' when it is
Brazilian Portuguese (LANGID 0x0416, or a pt_BR locale), else 'en'."""
if os.name != 'nt':
loc = os.environ.get('LC_ALL') or os.environ.get('LC_MESSAGES') or os.environ.get('LANG') or ''
return 'ru' if loc.lower().startswith('ru') else 'en'
loc = (os.environ.get('LC_ALL') or os.environ.get('LC_MESSAGES') or os.environ.get('LANG') or '').lower()
return 'ru' if loc.startswith('ru') else 'pt-BR' if loc.startswith(('pt_br', 'pt-br')) else 'en'
try:
langid = ctypes.windll.kernel32.GetUserDefaultUILanguage()
except (AttributeError, OSError):
return 'en'
return 'ru' if langid & 0x3ff == LANG_RU else 'en'
return 'ru' if langid & 0x3ff == LANG_RU else 'pt-BR' if langid == LANG_PT_BR else 'en'
def default_game_language(ui):
+9 -4
View File
@@ -9,14 +9,19 @@
#include "launcher_text.h"
#define LANGID_RUSSIAN 0x19
#define LANGID_RUSSIAN 0x19 /* primary language id */
#define LANGID_PT_BR 0x0416 /* the whole LANGID: Portugal's Portuguese stays English */
#define PATH_CHARS (MAX_PATH * 2)
static void fail(void)
{
int russian = (GetUserDefaultUILanguage() & 0x3ff) == LANGID_RUSSIAN;
MessageBoxW(NULL, russian ? LAUNCHER_PYTHON_MISSING_RU : LAUNCHER_PYTHON_MISSING_EN,
russian ? LAUNCHER_TITLE_RU : LAUNCHER_TITLE_EN, MB_OK | MB_ICONERROR);
LANGID id = GetUserDefaultUILanguage();
int russian = (id & 0x3ff) == LANGID_RUSSIAN, brazilian = id == LANGID_PT_BR;
MessageBoxW(NULL,
russian ? LAUNCHER_PYTHON_MISSING_RU : brazilian ? LAUNCHER_PYTHON_MISSING_PT_BR
: LAUNCHER_PYTHON_MISSING_EN,
russian ? LAUNCHER_TITLE_RU : brazilian ? LAUNCHER_TITLE_PT_BR : LAUNCHER_TITLE_EN,
MB_OK | MB_ICONERROR);
}
int WINAPI WinMain(HINSTANCE instance, HINSTANCE previous, LPSTR command_line, int show)
+9
View File
@@ -644,6 +644,9 @@ class Compile(Step):
# ---- 5. prepare
TRANSLATED = re.compile(r'gfx_shader: prepare: (\d+) of (\d+) pairs')
# runtime/vk_pipeline.c lb_warm_prepared: variants the driver crashed on in an earlier run (tools/prepare_shaders.py
# runs the driver step again after such a crash)
SKIPPED = re.compile(r'prepare-shaders: (\d+) variants skipped: the graphics driver crashed')
# The game resolutions of another shape than 16:9 that runtime/ui/menu.cpp row_output_res lists: 21:9 and 32:9 monitors,
@@ -715,10 +718,14 @@ class Prepare(Step):
shader_list = ctx.temp / 'shader_list.tsv'
shader_list.write_text(shader_list_text(layout), encoding='utf-8', newline='\n')
skipped = [0]
def parse(line):
m = TRANSLATED.search(line)
if m:
rep.progress('prepare', 'translate', int(m.group(1)), int(m.group(2)), 'shaders')
elif SKIPPED.search(line):
skipped[0] = int(SKIPPED.search(line).group(1))
elif 'listed variants' in line:
rep.progress('prepare', 'translate', force=True)
elif 'gfx_shader: prepare:' in line and 'translated' in line and 'programs' in line:
@@ -730,6 +737,8 @@ class Prepare(Step):
dest / 'exefs', '--list', shader_list], parse)
if code or not (dest / 'shader-prepared.bin').is_file():
raise InstallError('prepare.shaders_failed', code=code, log=str(log))
if skipped[0]:
rep.warning('warn.shaders_skipped', count=skipped[0], log=str(log))
# One shader-cache.pack instead of four files per shader (~117,000): totk.exe reads the pack first.
rep.progress('prepare', 'pack', force=True)
+1
View File
@@ -86,6 +86,7 @@
"warn.lang_needs_fonts": "Text in {lang} needs the Switch system fonts, which the installer does not extract; some text may not show.",
"warn.npdm_differs": "main.npdm differs from the reference {version} dump; only the code modules are used.",
"warn.xbox_prompts_failed": "The Xbox button prompts could not be prepared (exit code {code}); the game keeps the Switch prompts. Details: {log}",
"warn.shaders_skipped": "The graphics driver crashed while building some shaders, so {count} were left out of the preparation. The game builds them when it first needs them, and the driver may crash there too; a driver update may fix it. Details: {log}",
"warn.kernel_old": "Linux {version} lacks the write tracking the game relies on (6.7 or later); the game may run very slowly. Updating the system fixes this.",
"ui.title": "TotK Setup",
+180
View File
@@ -0,0 +1,180 @@
{
"step.check": "Verificando o computador e os arquivos do jogo",
"step.extract": "Extraindo o jogo",
"step.lift": "Traduzindo o código do jogo para C",
"step.compile": "Compilando o {game_exe}",
"step.prepare": "Preparando os shaders",
"step.cleanup": "Finalizando",
"phase.check.host": "Verificando o processador e a placa de vídeo",
"phase.check.verify": "Verificando os arquivos já instalados",
"phase.check.verify_romfs": "Verificando os dados do jogo instalados",
"phase.check.inputs": "Abrindo os arquivos do jogo com as chaves",
"phase.check.versions": "Verificando as versões do jogo",
"phase.extract.hash_base": "Calculando o checksum do jogo base",
"phase.extract.hash_update": "Calculando o checksum da atualização",
"phase.extract.exefs": "Extraindo o código do jogo",
"phase.extract.copy": "Copiando os dados do jogo",
"phase.extract.merge": "Aplicando a atualização aos dados do jogo",
"phase.extract.diagnose": "Procurando o arquivo do jogo danificado",
"phase.extract.index": "Verificando os dados do jogo",
"phase.lift.discover": "Encontrando as funções",
"phase.lift.emit": "Gerando o código C",
"phase.compile.compile": "Compilando",
"phase.compile.link": "Ligando o {game_exe}",
"phase.prepare.romfs_patches": "Gravando os patches de 60 fps nos dados do jogo",
"phase.prepare.seed": "Lendo os shaders dos dados do jogo",
"phase.prepare.translate": "Traduzindo os shaders",
"phase.prepare.libraries": "Montando os shaders para o driver de vídeo",
"phase.prepare.pack": "Juntando o cache de shaders num único arquivo",
"phase.cleanup.ini": "Gravando o totk.ini",
"phase.cleanup.temp": "Removendo os arquivos temporários",
"cli.usage": "Linha de comando inválida: {detail}",
"internal": "Erro inesperado ({exception}): {detail}",
"state.unreadable": "Não foi possível ler {path}. Escolha outra pasta ou apague esse arquivo.",
"state.newer_format": "{path} foi gravado por um instalador mais novo. Use esse instalador ou outra pasta.",
"state.output_changed": "Os arquivos instalados da etapa '{step}' mudaram desde que foram criados; essa etapa será executada de novo.",
"check.release_incomplete": "A pasta do instalador está incompleta; faltam: {paths}. Extraia o pacote de novo.",
"check.dest_not_folder": "O local de instalação {path} é um arquivo, não uma pasta.",
"check.dest_not_empty": "A pasta de instalação {path} não está vazia e não tem uma instalação anterior. Escolha uma pasta vazia.",
"check.lang_missing": "Escolha o idioma do jogo.",
"check.lang_unknown": "Idioma desconhecido '{lang}'. Conhecidos: {known}.",
"check.toolchain_broken": "O compilador incluído não funciona: {detail}",
"check.glibc_old": "A biblioteca C deste sistema é a glibc {version}; o compilador da instalação precisa da {need} ou mais nova (Ubuntu 22.04, Debian 12, Fedora 35, SteamOS 3 e mais novos têm). Atualizar o sistema resolve.",
"check.host_probe_failed": "A verificação do computador não rodou (código de saída {code}).",
"check.cpu_no_avx2": "Este processador não tem AVX2 (x86-64-v3), que o jogo exige. Intel Haswell (2013) ou AMD Excavator / Zen e mais novos têm.",
"check.vulkan_missing": "Nenhum driver Vulkan está instalado. Instale o driver mais recente da sua placa de vídeo.",
"check.vulkan_no_device": "O Vulkan não encontra nenhuma placa de vídeo. Instale o driver mais recente da sua placa de vídeo.",
"check.vulkan_too_old": "O jogo precisa de uma placa de vídeo com Vulkan 1.3; encontrado: {devices}. Atualize o driver de vídeo.",
"check.inputs_needed": "Os arquivos do jogo são necessários: escolha o jogo base, a atualização 1.2.1 ou 1.4.3 e o prod.keys.",
"check.input_missing": "Arquivo não encontrado ({input}): {path}",
"check.keys_unreadable": "Não foi possível ler o arquivo de chaves. Escolha um arquivo prod.keys.",
"check.keys_no_header_key": "O arquivo de chaves não tem header_key. Extraia o prod.keys do seu console de novo.",
"check.keys_wrong": "As chaves não abrem o arquivo de {input}. Use o prod.keys do seu console.",
"check.keys_too_old": "O arquivo de chaves é antigo demais para o arquivo de {input} (precisa da geração de chaves {generation}). Extraia o prod.keys de um console com firmware mais novo.",
"check.not_game_file": "O arquivo de {input} não é um arquivo de jogo XCI ou NSP.",
"check.file_unreadable": "Não foi possível ler o arquivo de {input}: {detail}",
"check.ticket_personalized": "O arquivo de {input} tem um ticket personalizado; é preciso um ticket comum.",
"check.ticket_missing": "O arquivo de {input} não tem o ticket de que o conteúdo precisa.",
"check.base_not_totk": "O arquivo do jogo base não é The Legend of Zelda: Tears of the Kingdom ({detail}).",
"check.base_is_update": "O arquivo do jogo base é uma atualização. Escolha aqui o NSP ou XCI do jogo base (versão 1.0.0) e este arquivo como a atualização.",
"check.base_combined": "O arquivo do jogo base contém vários pacotes (jogo base, atualização ou DLC em um só arquivo). Escolha um dump do jogo base que contenha só o jogo.",
"check.update_not_nsp": "A atualização precisa ser um arquivo NSP.",
"check.update_is_base": "O arquivo de atualização contém um jogo base. Escolha o NSP da atualização 1.2.1 ou 1.4.3.",
"check.update_not_totk": "O arquivo de atualização não é uma atualização de Tears of the Kingdom ({detail}).",
"check.exefs_failed": "O código do jogo no arquivo de {input} falhou nas verificações; veja {log}.",
"check.input_damaged": "O arquivo de {input} está danificado ou incompleto: {path} ({detail}). Copie esse arquivo de novo da origem, ou extraia-o de novo, e então execute a instalação outra vez.",
"check.base_wrong_version": "O jogo base não é a versão 1.0.0 ({modules} diferem).",
"check.update_wrong_version": "A atualização é a versão {found}; esta versão do instalador aceita {wanted}.",
"check.disk_space": "Espaço livre insuficiente em {path}: são necessários {need_gb} GB, há {free_gb} GB livres.",
"extract.exefs_mismatch": "O código do jogo extraído não corresponde à versão {version} ({modules}).",
"extract.romfs_merge_failed": "Falha ao aplicar a atualização aos dados do jogo: {detail}. Detalhes: {log}",
"extract.romfs_mismatch": "Os dados do jogo extraídos não correspondem à versão {version} (tamanho {size}, checksum {sha256}...).",
"lift.failed": "Falha ao traduzir o código do jogo (código de saída {code}). Detalhes: {log}",
"compile.failed": "Falha ao compilar o {game_exe} (código de saída {code}). Detalhes: {log}",
"compile.exe_in_use": "{path} está em uso. Feche o jogo e execute o instalador de novo.",
"prepare.shader_list_missing": "A lista de shaders está faltando.",
"prepare.romfs_patches_failed": "Falha ao gravar os patches nos dados do jogo (código de saída {code}). Detalhes: {log}",
"prepare.shaders_failed": "Falha ao preparar os shaders (código de saída {code}). Detalhes: {log}",
"cleanup.romfs_unreadable": "Os dados do jogo em {path} estão faltando ou não podem ser lidos depois da instalação. Reinicie o Windows e execute o instalador de novo na mesma pasta: ele extrai os dados do jogo outra vez e mantém o resto.",
"prepare.pack_failed": "Falha ao juntar o cache de shaders (código de saída {code}). Detalhes: {log}",
"cleanup.ini_failed": "Não foi possível gravar o totk.ini: {detail}",
"warn.lang_needs_fonts": "Textos em {lang} precisam das fontes de sistema do Switch, que o instalador não extrai; parte do texto pode não aparecer.",
"warn.npdm_differs": "O main.npdm difere do dump de referência da versão {version}; só os módulos de código são usados.",
"warn.xbox_prompts_failed": "Não foi possível preparar os ícones de botão do Xbox (código de saída {code}); o jogo mantém os ícones do Switch. Detalhes: {log}",
"warn.shaders_skipped": "O driver gráfico travou ao compilar alguns shaders, então {count} ficaram fora da preparação. O jogo os compila quando precisar deles pela primeira vez, e o driver pode travar ali também; uma atualização do driver pode resolver. Detalhes: {log}",
"warn.kernel_old": "O Linux {version} não tem o rastreamento de escrita de que o jogo depende (6.7 ou mais novo); o jogo pode rodar muito devagar. Atualizar o sistema resolve.",
"ui.title": "Instalação do TotK",
"ui.version": "versão {version}",
"ui.intro": "Monta uma pasta jogável de Tears of the Kingdom a partir dos seus próprios arquivos do jogo. Isso demora um pouco e só precisa ser feito uma vez.",
"ui.notice": "A pasta que esta instalação monta contém o jogo. Ela é só para seu uso pessoal; não compartilhe nem envie para a internet.",
"ui.keys_private": "O arquivo de chaves é lido no lugar. O conteúdo dele nunca é mostrado, copiado nem guardado.",
"ui.base": "Jogo base (NSP ou XCI)",
"ui.update": "Atualização 1.2.1 ou 1.4.3 (NSP)",
"ui.keys": "prod.keys",
"ui.game_language": "Idioma do jogo",
"ui.dest": "Pasta de instalação",
"ui.browse": "Procurar...",
"ui.pick_game": "Escolha o arquivo do jogo base",
"ui.pick_update": "Escolha o arquivo da atualização 1.2.1 ou 1.4.3",
"ui.pick_keys": "Escolha o prod.keys",
"ui.pick_dest": "Escolha a pasta de instalação",
"ui.type_game": "Arquivos de jogo (NSP, XCI)",
"ui.type_nsp": "Arquivos NSP",
"ui.type_keys": "Arquivos de chaves (*.keys)",
"ui.type_all": "Todos os arquivos",
"ui.start": "Iniciar",
"ui.cancel": "Cancelar",
"ui.cancelling": "Cancelando...",
"ui.show_log": "Mostrar log",
"ui.hide_log": "Ocultar log",
"ui.close": "Fechar",
"ui.back": "Voltar",
"ui.open_folder": "Abrir pasta",
"ui.update_mode": "Esta pasta tem uma instalação anterior. A instalação vai continuar ou atualizá-la; os arquivos do jogo só são pedidos se forem necessários de novo.",
"ui.need_dest": "Escolha a pasta de instalação.",
"ui.need_base": "Escolha o arquivo do jogo base.",
"ui.need_update": "Escolha o arquivo da atualização 1.2.1 ou 1.4.3.",
"ui.need_keys": "Escolha o arquivo prod.keys.",
"ui.need_lang": "Escolha o idioma do jogo.",
"ui.missing_base": "O arquivo do jogo base não existe.",
"ui.missing_update": "O arquivo de atualização não existe.",
"ui.missing_keys": "O arquivo de chaves não existe.",
"ui.running_install": "Instalando",
"ui.running_resume": "Continuando a instalação anterior",
"ui.running_update": "Atualizando a instalação",
"ui.state_started": "iniciada",
"ui.state_done": "concluída",
"ui.state_skipped": "pulada",
"ui.state_pending": "aguardando",
"ui.state_running": "em andamento",
"ui.state_failed": "falhou",
"ui.log_warning": "Aviso",
"ui.log_error": "Erro",
"ui.log_space": "Espaço necessário: {need} GB, livre: {free} GB",
"ui.log_end": "Concluído",
"ui.result_ok": "O jogo está pronto",
"ui.result_failed": "A instalação falhou",
"ui.result_cancelled": "A instalação foi cancelada",
"ui.done_text": "Abra o {game_exe} em {dest}. O programa de instalação e os arquivos do jogo que você escolheu não são mais necessários.",
"ui.cancelled_text": "As etapas concluídas são mantidas. Inicie a instalação de novo com a mesma pasta para continuar.",
"ui.log_file": "Arquivo de log: {path}",
"ui.process_died": "A instalação parou de forma inesperada (código de saída {code}).",
"ui.confirm_cancel": "A instalação está em andamento. Cancelar e fechar a janela?",
"ui.space_one": "Espaço livre necessário em {path}: {need} GB (livre: {free} GB).",
"ui.space_two": "Espaço livre necessário em {path}: {need} GB (livre: {free} GB); em {temp}: {temp_need} GB (livre: {temp_free} GB).",
"ui.amount_bytes": "{done} / {total} GB",
"ui.amount_chunks": "{done} / {total} partes",
"ui.amount_objects": "{done} / {total} arquivos",
"ui.amount_shaders": "{done} / {total} shaders",
"ui.amount_other": "{done} / {total} {unit}",
"ui.seconds": "{n} s",
"param.input.base": "jogo base",
"param.input.update": "atualização",
"param.input.keys": "chaves",
"launcher.title": "Instalação do TotK",
"launcher.python_missing": "Os arquivos de instalação estão incompletos: a pasta python ao lado do TotK-Setup.exe não existe. Extraia o arquivo compactado inteiro de novo e abra o TotK-Setup.exe a partir da pasta extraída.",
"lang.ja": "Japonês",
"lang.en-US": "Inglês (EUA)",
"lang.en-GB": "Inglês (Reino Unido)",
"lang.fr": "Francês",
"lang.fr-CA": "Francês (Canadá)",
"lang.de": "Alemão",
"lang.it": "Italiano",
"lang.es": "Espanhol",
"lang.es-419": "Espanhol (América Latina)",
"lang.nl": "Holandês",
"lang.pt": "Português",
"lang.pt-BR": "Português (Brasil)",
"lang.ru": "Russo",
"lang.ko": "Coreano",
"lang.zh-Hans": "Chinês (Simplificado)",
"lang.zh-Hant": "Chinês (Tradicional)"
}
+1
View File
@@ -86,6 +86,7 @@
"warn.lang_needs_fonts": "Для языка «{lang}» нужны системные шрифты Switch, которые установка не извлекает; часть текста может не отображаться.",
"warn.npdm_differs": "main.npdm отличается от эталонного дампа {version}; используются только модули с кодом.",
"warn.xbox_prompts_failed": "Не удалось подготовить подсказки кнопок Xbox (код выхода {code}); игра оставит подсказки Switch. Подробности: {log}",
"warn.shaders_skipped": "Графический драйвер аварийно завершился при сборке части шейдеров, поэтому {count} из них пропущены при подготовке. Игра соберёт их, когда они впервые понадобятся, и драйвер может упасть и там; обновление драйвера может это исправить. Подробности: {log}",
"warn.kernel_old": "В Linux {version} нет отслеживания записи, на которое опирается игра (нужна версия 6.7 или новее); игра может работать очень медленно. Обновление системы это исправит.",
"ui.title": "Установка TotK",
+10
View File
@@ -9,6 +9,7 @@ set cloth TOTK_CLOTH_REALTIME 1
set bounce TOTK_HIGHFPS_FIX 1
set output TOTK_PATCH_OUTPUT 0
set scene TOTK_PATCH_SCENE 0
set shadow TOTK_PATCH_SHADOW 0
# 60 fps: the adaptive present-interval controller (main+0x8f26c8, 1.0.0 0x8f6bec) gets the minimum interval 1.
site fps60 main 8f273c 6b08015f 7100055f # clamp: cmp w10, #1 (was cmp w10, w8 = minimum)
@@ -947,3 +948,12 @@ spin sdk 23a590 23a5d8 # 19 instructions; loads
spin sdk 23a658 23a6a0 # 19 instructions; loads
spin sdk 23a7fc 23a918 # 19 instructions; loads
spin sdk 3d3250 3d3270 # 9 instructions; loads
# Sun shadow map size (runtime/resolution.c shadow_init switches the set on). Each frame the engine sizes its shadow
# cascades (a D16 4-layer array) in main+0xcebdf4: a base size (960 in the field), interpolated down by how close the
# cover is, rounded up to 8 at main+0xcec22c and stored as the width (+0x3d8 of the cascade set) and, through the packed
# store at +0xe4, the height (+0x3f8). Near the ground it falls to about 400, which draws foliage shadows blocky, and
# every change of size moves the shadow texels, so leaf shadows jump instead of sliding. The page's fixed size (+92)
# goes in just before the rounding, as the community's shadow resolution mods do (1.1.0: mov w11, #1024 over the
# computation). Found by a write watch on the size field (the writer was this function, and no other).
site shadow main cec22c 11001d6b @page:x11 b9405d6b 11001d6b # add w11, w11, #7 -> adrp x11, page; ldr w11, [x11, #92]; add w11, w11, #7
+6
View File
@@ -10,6 +10,7 @@ set cloth TOTK_CLOTH_REALTIME 1
set bounce TOTK_HIGHFPS_FIX 1
set output TOTK_PATCH_OUTPUT 0
set scene TOTK_PATCH_SCENE 0
set shadow TOTK_PATCH_SHADOW 0
# 60 fps: the adaptive present-interval controller (main+0x2db174, 1.2.1 0x8f26c8) gets the minimum interval 1.
site fps60 main 2db204 6b08015f 7100055f # clamp: cmp w10, #1 (was cmp w10, w8 = minimum)
@@ -976,3 +977,8 @@ spin subsdk0 1c4e54 1c4e60 # 4 instructions; loads
spin sdk 2d3654 2d3668 # 6 instructions; loads
spin sdk 443864 443884 # 9 instructions; loads
spin sdk 541e54 541f98 # 32 instructions; loads
# Sun shadow map size (patches-v121.txt explains the set and the page). 1.4.3 interpolates the size between a minimum
# and the base and rounds it up to 8 at main+0x683604; the rounded value is the width (+0x3d8) and, through the packed
# store at +0xf4, the height (+0x3f8). The page's size goes in just before the rounding.
site shadow main 683604 11001d6b @page:x11 b9405d6b 11001d6b # add w11, w11, #7 -> adrp x11, page; ldr w11, [x11, #92]; add w11, w11, #7
+2 -2
View File
@@ -14,8 +14,8 @@ encoded for the site's address. The patch page is the first 4 KiB page after mai
of the 2 MiB-aligned module slot free); the runtime commits it and owns its contents. Words in use, by byte offset:
+0..15 the frame-rate step (runtime/frame_rate.c), +16..55 the output, swapchain and scene sizes
(runtime/resolution.c res_init), +56..59 the camera's aspect ratio (runtime/resolution.c res_init, set aspect), +60..63
the field-of-view factor (runtime/resolution.c res_fov_live, set fov). A new
set takes words outside these.
the field-of-view factor (runtime/resolution.c res_fov_live, set fov), +92..95 the sun shadow map size
(runtime/resolution.c shadow_init, set shadow). A new set takes words outside these.
The replacement words are lifted at the site's address in place of the original instruction, and the generated C reads
`if (rt_patch_on[<set>]) { replacement } else { original }`: one byte load and a branch per executed site. Replacement
+8 -4
View File
@@ -47,10 +47,11 @@ static const char *const known[] = {
"RENDER_SCALE", "REPLAY_ASYNC",
"REPLAY_CHECK", "RESOLVE_CHECK", "ROMFS", "ROMFS_INDEX", "ROMFS_MMAP", "ROMFS_MODS", "ROMFS_PATCHES",
"ROMFS_PROOF_DIR", "ROMFS_VERIFY", "RUMBLE", "RUMBLE_CURVE", "RUMBLE_STRENGTH", "RUMBLE_TRIGGERS", "SHADER_CACHE",
"SHADER_MEMO", "SHARED_FONTS", "SHARPNESS", "SHOTS", "SHOTS_DIR", "SHOTS_FULL", "SPEED_PROBE", "SPIN", "SPIN_HANDOFF_US", "SPLIT_DRAWS", "SPLIT_US", "STICK_DEADZONE", "SWKBD_ANSWER",
"SHADER_MEMO", "SHADOW_RES", "SHARED_FONTS", "SHARPNESS", "SHOTS", "SHOTS_DIR", "SHOTS_FULL", "SPEED_PROBE", "SPIN", "SPIN_HANDOFF_US", "SPLIT_DRAWS", "SPLIT_US", "STICK_DEADZONE", "SWKBD_ANSWER",
"SWKBD_TEST", "TEST_PAD", "TEST_PREPARE_SHADERS", "TEX_ALIAS", "TEX_MEMO", "TEX_VRAM_PRESSURE", "TIMER_1MS", "TRACE", "UBO_SNAPSHOT", "UPSCALER", "VK_DYN_A2C", "VK_DYN_VERTEX", "VK_NO_FLOAT64",
"VK_PRESENT", "VK_RECORDER", "VK_SSBO_ALIGN", "VK_SUBGROUP", "VK_SUBGROUP_PIN", "VK_VALIDATION", "VOLUME", "VRAM_HEADROOM", "VS_OUTS",
"WB_EXACT", "WB_FENCE_LIMIT", "WINDOW_SIZE", "XLATE", "XLATE_ID", "XLATE_PROCS", "ZSTD_CHECK", "ZSTD_NATIVE",
"WALK_KEY", "WALK_SCALE", "WB_EXACT", "WB_FENCE_LIMIT", "WINDOW_SIZE", "XLATE", "XLATE_ID", "XLATE_PROCS",
"ZSTD_CHECK", "ZSTD_NATIVE",
};
// The totk.ini section a key cfg_set may write goes to when the file lacks it (totk.ini.example's sections).
@@ -61,12 +62,13 @@ static const struct { const char *key, *section; } sections[] = {
{"GPU_INDEX", "display"}, {"OUTPUT_RES", "display"}, {"RENDER_SCALE", "display"}, {"UPSCALER", "display"},
{"SHARPNESS", "display"}, {"ASPECT", "display"}, {"FOV", "display"}, {"COLOR_PRESET", "display"},
{"COLOR_BRIGHTNESS", "display"}, {"COLOR_CONTRAST", "display"}, {"COLOR_SATURATION", "display"},
{"COLOR_GAMMA", "display"},
{"COLOR_GAMMA", "display"}, {"SHADOW_RES", "display"},
{"DEVICE_PROFILE", "display"}, {"ASSUME_FOCUS", "display"},
{"MENU_LANG", "language"}, {"LANG", "language"}, {"OPERATION_MODE", "language"},
{"VOLUME", "audio"},
{"MOTION", "input"}, {"RUMBLE_STRENGTH", "input"}, {"RUMBLE_CURVE", "input"}, {"RUMBLE_TRIGGERS", "input"},
{"PAD_LAYOUT", "input"}, {"STICK_DEADZONE", "input"}, {"MOUSE_SENS", "input"}, {"INPUT_RECORD", "input"},
{"PAD_LAYOUT", "input"}, {"STICK_DEADZONE", "input"}, {"MOUSE_SENS", "input"},
{"WALK_KEY", "input"}, {"WALK_SCALE", "input"}, {"INPUT_RECORD", "input"},
{"BIND_A", "input"}, {"BIND_B", "input"}, {"BIND_X", "input"}, {"BIND_Y", "input"}, {"BIND_LCLICK", "input"},
{"BIND_RCLICK", "input"}, {"BIND_L", "input"}, {"BIND_R", "input"}, {"BIND_ZL", "input"}, {"BIND_ZR", "input"},
{"BIND_PLUS", "input"}, {"BIND_MINUS", "input"}, {"BIND_LEFT", "input"}, {"BIND_UP", "input"},
@@ -103,6 +105,7 @@ static const CfgMeta metas[] = {
{"COLOR_CONTRAST", "100", "70..150", LIVE}, {"COLOR_SATURATION", "100", "50..200", LIVE},
{"COLOR_GAMMA", "1", "0.8..1.2", LIVE},
{"DEVICE_PROFILE", "auto", "auto|desktop|deck", RESTART},
{"SHADOW_RES", "game", "game|512|768|1024|1536|2048", RESTART},
// Game
{"FPS_LIMIT", "60", "30|40|45|60|90|120|144|display", LIVE}, {"FPS60", "1", "0|1", LIVE},
{"REALTIME", "1", "0|1", RESTART}, {"CLOTH_REALTIME", "1", "0|1", RESTART},
@@ -115,6 +118,7 @@ static const CfgMeta metas[] = {
{"MOTION", "1", "0|1", LIVE}, {"RUMBLE_STRENGTH", "130", "0..300", LIVE}, {"RUMBLE_CURVE", "0.6", "0.2..1.5", LIVE},
{"RUMBLE_TRIGGERS", "0", "0|1", LIVE}, {"PAD_LAYOUT", "auto", "auto|nintendo|xbox|xbox_prompts", LIVE},
{"STICK_DEADZONE", "8", "0..50", LIVE}, {"MOUSE_SENS", "1", "0..10", LIVE},
{"WALK_KEY", "none", NULL, LIVE}, {"WALK_SCALE", "0.51", "0.2..0.9", LIVE},
{"MENU_LANG", "", "en|ru", LIVE},
// Advanced: scheduling
{"ASYNC_PREEMPT", "1", "0|1|2", RESTART}, {"REPLAY_ASYNC", "0", "0|1", RESTART},
+3
View File
@@ -57,6 +57,9 @@ typedef struct {
int have_gpu, have_vram, vram_exact;
uint64_t gpu_ns, gpu_parts, gpu_missed; // busy time, submission parts timed, parts left untimed
uint64_t vram_used, vram_budget;
int have_idle; // calibrated timestamps: the idle split below is filled
uint64_t idle_ns, starved_ns, queued_ns; // GPU idle between timed parts; of it, before the part was submitted; submit to start
uint64_t idle_gaps, max_gap_ns; // gaps over 0.5 ms; the longest
} PerfGpu;
extern void (*perf_gpu_stats)(PerfGpu *out);
// The performance overlay (perfmon.c): mode 0 off, 1 fps, 2 full. While it is on, perf_present hands every frame to
+59 -4
View File
@@ -236,6 +236,52 @@ static float mouse_sens_get(void) {
}
// ---- walk key: the keyboard's stick samples scaled down while a key is held (TOTK_WALK_KEY, TOTK_WALK_SCALE)
// A keyboard key is all or nothing, so every keyboard-driven stick sample is +/-32767 and the game only ever sees a
// full tilt. Holding this key scales those two samples, which gives the half tilt a gamepad gets from its analog range,
// so the keyboard can walk as well as run. yuzu and eden do the same with their modifier and modifier_scale settings.
// Only the keyboard is scaled: the gamepad's axes and the mouse camera sample their own values.
static int walk_key = KEY_NONE; // TOTK_WALK_KEY as a KEY_*, KEY_NONE = off; live (cfg_version)
static float walk_scale = 0.51f; // TOTK_WALK_SCALE, 0.2 to 0.9; live (cfg_version)
static unsigned walk_seen = ~0u;
// The ini takes the key names the Controls tab writes; a hand-written line may spell them in any case.
static int name_matches(const char *name, const char *key) {
for (; *name && *key; name++, key++) {
unsigned char a = (unsigned char)*name, b = (unsigned char)*key;
if (a >= 'A' && a <= 'Z') a = (unsigned char)(a - 'A' + 'a');
if (b >= 'A' && b <= 'Z') b = (unsigned char)(b - 'A' + 'a');
if (a != b) return 0;
}
return *name == *key;
}
static void walk_get(void) {
unsigned ver = cfg_version();
if (ver == walk_seen) return;
walk_seen = ver;
const char *name = cfg_value("TOTK_WALK_KEY"); // "none" (the default) turns the key off
int key = KEY_NONE;
if (*name && !name_matches(name, "none")) {
for (int k = KEY_A; k < KEY_COUNT && key == KEY_NONE; k++)
if (input_key_name(k) && name_matches(name, input_key_name(k))) key = k;
if (key == KEY_NONE) rt_log("input: walk_key: unknown key '%s', the keyboard sticks keep their full tilt", name);
}
// 0.51 rather than 0.5: a keyboard sample is 32767, and 32767 * 0.5 = 16383 falls one short of stick_bits'
// STICK_DIR (16384), which would leave the game's stick-direction bits clear while the walk key is held. The value
// in the settings table carries that default; this code only rejects what the table's range does not allow.
const char *sv = cfg_value("TOTK_WALK_SCALE");
char *end;
double d = strtod(sv, &end);
double s = end == sv || *end || !(d >= 0.2 && d <= 0.9) ? strtod(cfg_meta("TOTK_WALK_SCALE")->def, NULL) : d;
if (key != walk_key || (float)s != walk_scale) {
walk_key = key;
walk_scale = (float)s;
rt_log("input: walk key %s, keyboard sticks at %.0f%% of their tilt (mouse and gamepad unchanged)",
key != KEY_NONE ? input_key_name(key) : "off", s * 100);
}
}
// The mouse camera as a right-stick sample, held for the whole presented frame. Under `lock`.
static void mouse_camera(int32_t out[2]) {
static uint64_t frame = ~0ull, t0;
@@ -456,9 +502,13 @@ static void poll_merged(uint64_t *buttons, int32_t *lx, int32_t *ly, int32_t *rx
was_captured = cap;
b |= input_map_buttons(down);
int32_t k[2];
walk_get();
int walk = walk_key != KEY_NONE && down[walk_key];
input_map_stick(down, 0, k);
if (walk) k[0] = (int32_t)(k[0] * walk_scale), k[1] = (int32_t)(k[1] * walk_scale);
pick(&sl[0], &sl[1], k[0], k[1]);
input_map_stick(down, 1, k);
if (walk) k[0] = (int32_t)(k[0] * walk_scale), k[1] = (int32_t)(k[1] * walk_scale);
pick(&sr[0], &sr[1], k[0], k[1]);
mouse_camera(k);
pick(&sr[0], &sr[1], k[0], k[1]);
@@ -505,15 +555,20 @@ void input_menu(int open) {
ikb_cursor_update();
}
// 1 while flag (g_block, g_menu) blocks the game's input; state 2 ends at the first poll with nothing held.
static int blocked(volatile int32_t *flag, uint64_t buttons) {
// 1 while flag (g_block, g_menu) blocks the game's input; state 2 ends at the first poll with nothing held. A stick
// axis counts as held as well: with the walk key the keyboard's sticks move part way without setting a direction bit, so
// a gate that watched the buttons alone would let that movement through the moment the menu or the text entry closes.
static int blocked(volatile int32_t *flag, uint64_t buttons, const int32_t *axes) {
int32_t k = *flag;
return k && !(k == 2 && !buttons && plat_cas32(flag, 0, 2) == 2);
int held = buttons != 0;
for (int i = 0; i < 4; i++) held |= axes[i] != 0;
return k && !(k == 2 && !held && plat_cas32(flag, 0, 2) == 2);
}
void input_poll(uint64_t *buttons, int32_t *lx, int32_t *ly, int32_t *rx, int32_t *ry) {
poll_merged(buttons, lx, ly, rx, ry);
int text = blocked(&g_block, *buttons), menu = blocked(&g_menu, *buttons);
const int32_t axes[4] = {*lx, *ly, *rx, *ry};
int text = blocked(&g_block, *buttons, axes), menu = blocked(&g_menu, *buttons, axes);
if (!text && !menu) return;
*buttons = 0; *lx = *ly = *rx = *ry = 0;
}
+5 -2
View File
@@ -323,8 +323,11 @@ NVN_API(nvnTextureReadTexelsStrided) { read_texels(c, 1); }
NVN_API(nvnTextureWriteTexels) { write_texels(c, 0); }
NVN_API(nvnTextureWriteTexelsStrided) { write_texels(c, 1); }
// void nvnTexture{Flush,Invalidate}Texels(const NVNtexture *t, const NVNtextureView *v, const NVNcopyRegion *r): texel
// access is coherent
NVN_API(nvnTextureFlushTexels) { (void)c; }
// access is coherent; flush marks the storage as written for re-upload
NVN_API(nvnTextureFlushTexels) {
const tex_t *t = arg_p(c, 0);
if (t && t->mem && !nvn_gpu_only((uintptr_t)t->mem)) nvn_guest_written((uintptr_t)t->mem, t->L.size);
}
NVN_API(nvnTextureInvalidateTexels) { (void)c; }
// ---------- Samplers
+8
View File
@@ -183,6 +183,9 @@ static uint64_t ft_frame(uint64_t frame, uint64_t now, uint64_t d) {
// process memory, and which side holds the frame back. pace: the frame waited for pacing (vsync, the frame limiter) at
// least 15% of the window; gpu: the GPU was busy for 80% of the frame interval or more (without timestamps: threads
// waited for fences for half the interval or more); cpu otherwise.
// With calibrated timestamps a third line splits the GPU's idle time between timed parts: the part was submitted only
// after the previous one had ended (the GPU waited for the CPU's replay or the game), or it was already queued (untimed
// work such as uploads and presents ran there, or a wait held it).
static void perf_gpu_line(uint64_t frame, double interval_ms, double pace_share, uint64_t n) {
PerfGpu g;
memset(&g, 0, sizeof g);
@@ -205,6 +208,11 @@ static void perf_gpu_line(uint64_t frame, double interval_ms, double pace_share,
rt_log("perf: frame %llu: %s; fence wait per frame: render thread %.2f ms, other threads %.2f ms; %s; ram working set "
"%.0f MiB, private %.0f MiB; bound: %s",
(unsigned long long)frame, gs, rw, ow, vs, ws, priv, bound);
if (g.have_gpu && g.have_idle)
rt_log("perf: frame %llu: gpu idle per frame %.2f ms between timed parts: %.2f ms waiting for the next submission, "
"%.2f ms with it queued (untimed work or waits); %.1f gaps over 0.5 ms, longest %.2f ms; submit to start avg %.2f ms",
(unsigned long long)frame, g.idle_ns / 1e6 / n, g.starved_ns / 1e6 / n, (g.idle_ns - g.starved_ns) / 1e6 / n,
(double)g.idle_gaps / n, g.max_gap_ns / 1e6, g.gpu_parts ? g.queued_ns / 1e6 / g.gpu_parts : 0.0);
}
int rt_ab(void) {
+2
View File
@@ -1247,6 +1247,8 @@ int plat_os_version(char *out, size_t cap) {
return snprintf(out, cap, "%s (Linux %s)", name, u.release) < (int)cap;
}
int plat_is_wine(void) { return 0; }
int plat_stderr_path(char *out, size_t cap) {
struct stat st;
if (fstat(2, &st) || !S_ISREG(st.st_mode)) return 0;
+1
View File
@@ -367,6 +367,7 @@ uint32_t plat_thread_id(void); // id of the calling t
int plat_font_path(const char *lang, char *out, size_t cap);
// The operating system and its build ("Windows 10.0.28000") into out; 0 when unknown.
int plat_os_version(char *out, size_t cap);
int plat_is_wine(void); // 1 when the Windows build runs under Wine; 0 elsewhere
// The file the process's stderr (the log) goes to, as a UTF-8 path; 0 when stderr is not a file (a console, a pipe).
int plat_stderr_path(char *out, size_t cap);
+5
View File
@@ -1191,6 +1191,11 @@ int plat_os_version(char *out, size_t cap) {
return 1;
}
int plat_is_wine(void) {
HMODULE nt = GetModuleHandleW(L"ntdll.dll");
return nt && GetProcAddress(nt, "wine_get_version") != NULL;
}
int plat_stderr_path(char *out, size_t cap) {
HANDLE h = GetStdHandle(STD_ERROR_HANDLE);
wchar_t w[WPATH];
+38 -3
View File
@@ -43,6 +43,7 @@
#define SCENE_H 900
#define DTA_KIB 113664u // DynamicTextureAllocatorMem, shipped
#define SCENE_MAX 4096 // per side, as MacNativeGames' TOTK_RENDER_RES
#define SHADOW_BASE 960 // the sun shadow map's base size (its cascades: a D16 array of 4 layers)
static struct {
int decided;
@@ -50,6 +51,7 @@ static struct {
int scene_set; // the scene differs from the game's 5/6 of the output
int heapdef_ok; // romfs.c serves the grown HeapDef
int on; // res_init switched the patch sets on
unsigned shadow; // fixed sun shadow map size (TOTK_SHADOW_RES), 0: the game's own
} R;
static void decide(void) {
@@ -83,19 +85,31 @@ static void decide(void) {
if (fit < 1.0) R.scene_set = 1;
R.sw = (unsigned)(sw * fit + 0.5);
R.sh = (unsigned)(sh * fit + 0.5);
const char *sr = cfg_value("TOTK_SHADOW_RES");
if (strcmp(sr, "game")) {
unsigned n = (unsigned)atoi(sr);
if (n >= 256 && n <= 4096 && !(n & 7)) R.shadow = n;
else rt_log("resolution: TOTK_SHADOW_RES=%s is not game or a size 256..4096 (a multiple of 8); game", sr);
}
}
// Texture heap for a sun shadow map above the base size: the four layers of 2 bytes, three maps in flight (the engine
// makes the array anew each frame).
static uint64_t shadow_kib(void) {
return R.shadow > SHADOW_BASE ? ((uint64_t)R.shadow * R.shadow - SHADOW_BASE * SHADOW_BASE) * 8 * 3 / 1024 : 0;
}
static int bigger(void) {
decide();
return R.ow != BASE_W || R.oh != BASE_H || R.scene_set;
return R.ow != BASE_W || R.oh != BASE_H || R.scene_set || shadow_kib();
}
uint64_t res_heap_extra_kib(void) {
if (!bigger()) return 0;
double grow = fmax(0.0, (double)R.ow * R.oh / (BASE_W * BASE_H) - 1.0) +
fmax(0.0, (double)R.sw * R.sh / (SCENE_W * SCENE_H) - 1.0);
if (grow <= 0) return 0;
return (uint64_t)ceil(DTA_KIB * (1.0 + grow) / 1024.0) * 1024 - DTA_KIB;
uint64_t kib = grow > 0 ? (uint64_t)ceil(DTA_KIB * (1.0 + grow) / 1024.0) * 1024 - DTA_KIB : 0;
return kib + ((shadow_kib() + 1023) & ~1023ull); // whole MiB
}
uint64_t res_heap_extra_bytes(void) { return ((res_heap_extra_kib() << 10) + 0x1fffff) & ~0x1fffffull; }
@@ -190,12 +204,32 @@ int res_heapdef_build(const void *in, size_t n, void **out, size_t *out_n) {
void res_heapdef_served(void) { R.heapdef_ok = 1; }
// ---- sun shadow map size: each frame the engine sizes its shadow cascades to the area they cover, down to about 400 near
// the ground, which draws foliage shadows blocky, and each change of size moves the shadow texels, so leaf shadows jump.
// A fixed size (the shadow set, recomp/patches-v121.txt) keeps them still and sharper. Page +92.
static void shadow_init(uint8_t *page) {
if (!R.shadow) {
rt_log("resolution: sun shadow map sized by the game (TOTK_SHADOW_RES=game)");
return;
}
int set = rt_patch_set("shadow");
if (set < 0) {
rt_log("resolution: the generated C has no shadow patch set: the game sizes its shadow map");
return;
}
int32_t n = (int32_t)R.shadow;
memcpy(page + 92, &n, 4);
rt_patch_switch(set, 1);
rt_log("resolution: sun shadow map %ux%u (TOTK_SHADOW_RES)", R.shadow, R.shadow);
}
// ---- start: the patch page and the sets
static uint64_t pack(unsigned w, unsigned h) { return (uint64_t)h << 32 | w; }
void res_init(void) {
if (!bigger()) {
rt_log("resolution: the game's own (output 1920x1080, scene 1600x900)");
shadow_init(rt_patch_page());
return;
}
int output = rt_patch_set("output"), scene = rt_patch_set("scene");
@@ -238,6 +272,7 @@ void res_init(void) {
} else {
rt_patch_switch(aspect, 0); // the page holds no ratio
}
shadow_init(page);
rt_log("resolution: output %ux%u, scene %ux%u%s; texture heap +%llu KiB", R.ow, R.oh, R.sw, R.sh,
scene_on ? "" : " (the game's own 5/6 of the output)", (unsigned long long)res_heap_extra_kib());
}
+47
View File
@@ -505,6 +505,15 @@ static void row_render_scale(void) {
opt_choice_list("TOTK_RENDER_SCALE", UI_MENU_RENDER_SCALE, vals, names, N, UI_MENU_RENDER_SCALE_DESC);
}
static void row_shadow_res(void) {
static const char *const vals[] = {"game", "512", "768", "1024", "1536", "2048"};
enum { N = sizeof vals / sizeof *vals };
const char *names[N];
names[0] = menu_text(UI_MENU_SHADOW_RES_GAME);
for (int i = 1; i < N; i++) names[i] = vals[i];
opt_choice_list("TOTK_SHADOW_RES", UI_MENU_SHADOW_RES, vals, names, N, UI_MENU_SHADOW_RES_DESC);
}
static void row_gpu(void) {
const char *vals[MAX_GPUS + 1], *names[MAX_GPUS + 1];
char idx[MAX_GPUS][4];
@@ -623,6 +632,7 @@ static void tab_screen(float scale) {
opt_group(UI_MENU_GRAPHICS);
row_output_res();
row_render_scale();
row_shadow_res();
static const UiTextId up[] = {UI_MENU_UPSCALER_FSR, UI_MENU_UPSCALER_BILINEAR};
opt_choice("TOTK_UPSCALER", UI_MENU_UPSCALER, up, 2, UI_MENU_UPSCALER_DESC);
opt_slider("TOTK_SHARPNESS", UI_MENU_SHARPNESS, 0, 100, UI_MENU_SHARPNESS_DESC, 5, "%.0f %%");
@@ -668,6 +678,41 @@ static const char *pad_layout_note(void) {
return note;
}
// The action a key presses, or -1: the walk key is a key of its own, but Shift is B in the default bindings, and a
// walk key that also presses something walks and does that something at the same time. The bindings come from a
// snapshot, and every key of an action is looked at: binds_refresh rebuilds the live table on the polling thread while
// the menu draws, and an action may hold more than the two keys its cells show.
static int key_action(const InputMap *m, int key) {
for (int a = 0; a < ACTION_COUNT; a++) {
uint8_t keys[INPUT_BINDING_MAX];
int n = input_map_keys_of(m, a, keys, INPUT_BINDING_MAX);
for (int i = 0; i < n; i++)
if (keys[i] == key) return a;
}
return -1;
}
// The walk key as the handful of keys a player is likely to pick. "none" rather than an empty value closes it: an
// unset key falls back to the default in the settings table, the same as a fresh installation. totk.ini takes any name
// input_map.c knows (walks of the keyboard sticks, not of the gamepad). A key the current bindings also use says which
// action it presses.
static void row_walk_key(void) {
static const char *const keys[] = {"none", "lshift", "lalt", "lctrl", "tab", "capslock"};
const char *names[sizeof keys / sizeof *keys] = {menu_text(UI_MENU_OFF), "Shift", "Alt", "Ctrl", "Tab", "Caps Lock"};
const char *cur = cfg_value("TOTK_WALK_KEY");
static char note[64];
const char *n = nullptr;
InputMap m;
input_map_snapshot(&m);
for (int k = KEY_A; k < KEY_COUNT && !n; k++) {
if (!input_key_name(k) || _stricmp(input_key_name(k), cur)) continue;
int a = key_action(&m, k);
if (a >= 0 && input_action_name(a)) snprintf(note, sizeof note, menu_text(UI_MENU_WALK_KEY_TAKEN), input_action_name(a)), n = note;
}
opt_choice_list("TOTK_WALK_KEY", UI_MENU_WALK_KEY, keys, names, sizeof keys / sizeof *keys, UI_MENU_WALK_KEY_DESC,
-1, n);
}
static void tab_controls(float scale) {
opt_begin("##controls", scale);
static const UiTextId layout[] = {UI_MENU_PAD_LAYOUT_AUTO, UI_MENU_PAD_LAYOUT_NINTENDO, UI_MENU_PAD_LAYOUT_XBOX,
@@ -679,6 +724,8 @@ static void tab_controls(float scale) {
opt_toggle("TOTK_RUMBLE_TRIGGERS", UI_MENU_RUMBLE_TRIGGERS, UI_MENU_RUMBLE_TRIGGERS_DESC);
opt_slider("TOTK_STICK_DEADZONE", UI_MENU_STICK_DEADZONE, 0, 50, UI_MENU_STICK_DEADZONE_DESC, 1, "%.0f %%");
opt_slider("TOTK_MOUSE_SENS", UI_MENU_MOUSE_SENS, 0, 10, UI_MENU_MOUSE_SENS_DESC, 0.1f, "%.1f");
row_walk_key();
opt_slider("TOTK_WALK_SCALE", UI_MENU_WALK_SCALE, 0.2f, 0.9f, UI_MENU_WALK_SCALE_DESC, 0.01f, "%.2f");
binds_rows(scale);
opt_end();
binds_after_page();
+24
View File
@@ -69,6 +69,11 @@ static const char *const text_en[UI_TEXT_COUNT] = {
[UI_MENU_ASPECT_FIT] = "Black bars", [UI_MENU_ASPECT_STRETCH] = "Stretch",
[UI_MENU_FOV] = "Field of view",
[UI_MENU_FOV_DESC] = "How much of the world the camera shows, in percent of the game's own. Game version 1.4.3 only.",
[UI_MENU_SHADOW_RES] = "Shadow map size",
[UI_MENU_SHADOW_RES_DESC] = "Size of the sun's shadow map. The game's own size changes with the view, down to about 400 "
"near the ground, so leaf shadows look blocky and jump; a fixed size keeps them still and "
"sharper. Above 1024 the shadows cost more GPU time and video memory.",
[UI_MENU_SHADOW_RES_GAME] = "Game (changes)",
[UI_MENU_SHARPNESS] = "FSR sharpness",
[UI_MENU_SHARPNESS_DESC] = "Sharpening after FSR scaling; 0 turns it off.",
[UI_MENU_COLOR] = "Colour",
@@ -136,6 +141,13 @@ static const char *const text_en[UI_TEXT_COUNT] = {
[UI_MENU_STICK_DEADZONE_DESC] = "How far a gamepad stick moves before the game sees it. Raise it if the character or the camera moves on its own.",
[UI_MENU_MOUSE_SENS] = "Mouse camera sensitivity",
[UI_MENU_MOUSE_SENS_DESC] = "How fast the mouse turns the camera.",
[UI_MENU_WALK_KEY] = "Walk key",
[UI_MENU_WALK_KEY_DESC] = "Hold it to tilt the keyboard's sticks part of the way, so the keyboard walks instead of "
"runs. The walk scale sets how far. A controller and the mouse are not affected.",
[UI_MENU_WALK_KEY_TAKEN] = "also %s",
[UI_MENU_WALK_SCALE] = "Walk scale",
[UI_MENU_WALK_SCALE_DESC] = "The tilt the keyboard's sticks get while the walk key is held. Half is a walk, the "
"full tilt a run.",
[UI_BINDS_HELP] = "Select a cell and press A or Enter, then the key, mouse button or controller button. Delete or "
"X clears a cell. A key in use moves here; a controller button swaps with the one it replaces.",
@@ -330,6 +342,11 @@ static const char *const text_ru[UI_TEXT_COUNT] = {
[UI_MENU_ASPECT_FIT] = "Чёрные полосы", [UI_MENU_ASPECT_STRETCH] = "Растянуть",
[UI_MENU_FOV] = "Поле зрения",
[UI_MENU_FOV_DESC] = "Сколько мира показывает камера, в процентах от игрового. Только для версии игры 1.4.3.",
[UI_MENU_SHADOW_RES] = "Размер карты теней",
[UI_MENU_SHADOW_RES_DESC] = "Размер карты теней от солнца. Собственный размер игры меняется вместе с видом, у земли до "
"примерно 400, поэтому тени листвы выглядят квадратными и дёргаются; постоянный размер делает "
"их ровными и чётче. Выше 1024 тени требуют больше времени GPU и видеопамяти.",
[UI_MENU_SHADOW_RES_GAME] = "Как в игре (меняется)",
[UI_MENU_SHARPNESS] = "Резкость FSR",
[UI_MENU_SHARPNESS_DESC] = "Повышение резкости после масштабирования FSR; 0 выключает его.",
[UI_MENU_COLOR] = "Цвет",
@@ -398,6 +415,13 @@ static const char *const text_ru[UI_TEXT_COUNT] = {
[UI_MENU_STICK_DEADZONE_DESC] = "Насколько нужно отклонить стик геймпада, чтобы игра это заметила. Увеличьте, если персонаж или камера двигаются сами.",
[UI_MENU_MOUSE_SENS] = "Чувствительность мыши",
[UI_MENU_MOUSE_SENS_DESC] = "Как быстро мышь поворачивает камеру.",
[UI_MENU_WALK_KEY] = "Клавиша шага",
[UI_MENU_WALK_KEY_DESC] = "Удерживайте: стики клавиатуры отклоняются частично, и персонаж идёт, а не бежит. "
"Насколько — задаёт сила шага. Контроллер и мышь не затрагиваются.",
[UI_MENU_WALK_KEY_TAKEN] = "также %s",
[UI_MENU_WALK_SCALE] = "Сила шага",
[UI_MENU_WALK_SCALE_DESC] = "Отклонение стиков клавиатуры при удержании клавиши шага. Половина — шаг, полное "
"отклонение — бег.",
[UI_BINDS_HELP] = "Выберите ячейку и нажмите A или Enter, затем клавишу, кнопку мыши или кнопку контроллера. "
"Delete или X очищает ячейку. Занятая клавиша переходит сюда; кнопка контроллера меняется "
+2
View File
@@ -37,6 +37,7 @@ typedef enum {
UI_MENU_UPSCALER, UI_MENU_UPSCALER_DESC, UI_MENU_UPSCALER_FSR, UI_MENU_UPSCALER_BILINEAR,
UI_MENU_ASPECT, UI_MENU_ASPECT_DESC, UI_MENU_ASPECT_FIT, UI_MENU_ASPECT_STRETCH,
UI_MENU_FOV, UI_MENU_FOV_DESC,
UI_MENU_SHADOW_RES, UI_MENU_SHADOW_RES_DESC, UI_MENU_SHADOW_RES_GAME,
UI_MENU_SHARPNESS, UI_MENU_SHARPNESS_DESC,
UI_MENU_COLOR, // group heading over the colour rows
UI_MENU_COLOR_PRESET, UI_MENU_COLOR_PRESET_DESC, UI_MENU_COLOR_DEFAULT, UI_MENU_COLOR_VIVID, UI_MENU_COLOR_CLARITY,
@@ -68,6 +69,7 @@ typedef enum {
UI_MENU_PAD_LAYOUT_PROMPTS_MISSING, UI_MENU_PAD_LAYOUT_PROMPTS_RESTART,
UI_MENU_STICK_DEADZONE, UI_MENU_STICK_DEADZONE_DESC,
UI_MENU_MOUSE_SENS, UI_MENU_MOUSE_SENS_DESC,
UI_MENU_WALK_KEY, UI_MENU_WALK_KEY_DESC, UI_MENU_WALK_KEY_TAKEN, UI_MENU_WALK_SCALE, UI_MENU_WALK_SCALE_DESC,
// Controls tab: the remapping rows (ui/binds_page.cpp)
UI_BINDS_HELP, UI_BINDS_COL_KEY1, UI_BINDS_COL_KEY2, UI_BINDS_COL_PAD,
+2 -1
View File
@@ -33,7 +33,8 @@ typedef struct VkCtx {
int has_custom_border_color, has_eds, has_eds2, has_eds3, has_vertex_input_dynamic_state,
has_conditional_rendering, has_transform_feedback, has_provoking_vertex, has_device_fault, has_memory_budget,
has_robustness2, has_buffer_device_address, has_fragment_shader_interlock, has_buffer_marker,
has_astc_ldr; // textureCompressionASTC_LDR (enabled when reported)
has_astc_ldr, // textureCompressionASTC_LDR (enabled when reported)
has_calibrated; // VK_KHR/EXT_calibrated_timestamps (GPU idle split in the perf line)
// Feedback loops (maxwell.md 1.9, vk_exec_draw.c feedback_prepare): a draw samples a target of its open pass that
// an earlier draw of the pass wrote. TOTK_FEEDBACK=0: VK_FEEDBACK_OFF (sampled in GENERAL without a barrier, the
// old behaviour); VK_FEEDBACK_LAYOUT with VK_EXT_attachment_feedback_loop_layout and dynamicRenderingLocalRead
+3
View File
@@ -247,6 +247,9 @@ static int create_device(void) {
CHAIN(lr, VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_DYNAMIC_RENDERING_LOCAL_READ_FEATURES_KHR);
}
if (has_ext(ep, ne, VK_EXT_MEMORY_BUDGET_EXTENSION_NAME)) { g_vk.has_memory_budget = 1; ext[n_ext++] = VK_EXT_MEMORY_BUDGET_EXTENSION_NAME; }
// GPU timestamps on the CPU clock (vk_sync.c): the perf line splits GPU idle time into waiting for a submission or not.
if (has_ext(ep, ne, VK_KHR_CALIBRATED_TIMESTAMPS_EXTENSION_NAME)) { g_vk.has_calibrated = 2; ext[n_ext++] = VK_KHR_CALIBRATED_TIMESTAMPS_EXTENSION_NAME; }
else if (has_ext(ep, ne, VK_EXT_CALIBRATED_TIMESTAMPS_EXTENSION_NAME)) { g_vk.has_calibrated = 1; ext[n_ext++] = VK_EXT_CALIBRATED_TIMESTAMPS_EXTENSION_NAME; }
// GPU breadcrumbs for a lost device (vk_crumbs.c): AMD's drivers and RADV report it.
if (has_ext(ep, ne, VK_AMD_BUFFER_MARKER_EXTENSION_NAME)) { g_vk.has_buffer_marker = 1; ext[n_ext++] = VK_AMD_BUFFER_MARKER_EXTENSION_NAME; }
// Graphics pipeline library: chained last so it can be dropped again when vk_caps_gpl declines it.
+2 -2
View File
@@ -827,7 +827,7 @@ static int query_stamp(uint64_t lo, uint64_t hi, uint32_t g) {
// - GPU -> guest: the replay notes each GPU write into CPU-visible memory at record time (vk_mem_gpu_pending) and
// reports it at the fence (vk_gpu_written), which copies those bytes back into guest memory. Until then the CPU
// copies skip them: a page the CPU and the GPU both write keeps the GPU's bytes.
// TOTK_SHADOW=1/0 forces the mode on or off; the default is on for Linux.
// TOTK_SHADOW=1/0 forces the mode on or off; the default is on for Linux and Wine.
typedef struct { uint64_t lo, hi; } SRange;
static PlatMutex shadow_lock = PLAT_MUTEX_INIT; // after pool_lock; before gen_lock
static PoolRec **stouch; // shadow_lock: pools resolved since the last sync
@@ -1187,7 +1187,7 @@ static void mirror_mode(int discrete) {
if (freeze_idle < 0) freeze_idle = 0;
const char *sh = cfg_get("TOTK_SHADOW");
#ifdef _WIN32
shadow_on = sh && *sh && strcmp(sh, "0");
shadow_on = sh && *sh ? strcmp(sh, "0") != 0 : plat_is_wine();
#else
shadow_on = !sh || !*sh || strcmp(sh, "0");
#endif
+174 -23
View File
@@ -2569,7 +2569,8 @@ static struct {
uint8_t clamp, poly, minus_w, samples; // library state (GKey fields)
double t0, ms[3]; // ms: build time per kind (vertex, fragment, compute)
// warmed: driver cache only; nolib: vertex and fragment variants skipped with the graphics pipeline library off
uint64_t built[3], have[3], fail[3], warmed[3], noshader, nolib, done, cs_draw;
// skipped: variants of the skip list (SK below)
uint64_t built[3], have[3], fail[3], warmed[3], noshader, nolib, skipped, done, cs_draw;
double warm_ms[3];
volatile int32_t kept; // libraries entered in libs[] so far
int32_t keep_n;
@@ -2577,6 +2578,134 @@ static struct {
PlatMutex stat;
} LB = {.stat = PLAT_MUTEX_INIT};
// ---- preparation that outlives a driver crash. A bug in the driver's shader compiler can kill the process inside
// vkCreate*Pipelines: NVIDIA 615.78.08 on an RTX 5080 crashed on the same vertex library at the same point of every
// preparation, natively and under Wine, so preparation never finished (issue #30 and a report on PR #60).
// While lb_warm_prepared builds, slot k of <dir>/shader-prepare-journal.bin holds the variant its worker k has handed to
// the driver (input id and stage; zeros between variants). The writes go straight to the OS, so they outlast the
// process, and a run that finishes deletes the file. The next preparation or boot (skip_init) moves the variants a
// leftover journal names into <dir>/shader-prepare-skip.txt with this GPU and driver version, and preparation and the
// boot prebuild (lb_worker) skip what that file lists for them. The variants the other workers were building at the
// crash are skipped along with the culprit; like it, they build at their first draw, where the driver can still crash.
// Another driver version builds them again, and so does deleting the file. tools/prepare_shaders.py runs the build
// again (--warm-shaders) after a run that left a journal.
#define JR_SLOT 64 // bytes per journal slot: input id, stage, zeros
#define JR_SLOTS 64 // lb_warm_prepared's thread cap
static struct {
PlatFile *journal; // while lb_warm_prepared builds
uint8_t (*iid)[32]; // the skip list's variants for this GPU and driver
size_t n;
int loaded;
} SK;
static int cache_dir(char *dir, size_t n) { // the folder of pipeline-cache.bin and shader-prepared.bin
snprintf(dir, n, "%s", P.cache_path);
char *cut = plat_last_sep(dir);
if (!cut) return 0;
*cut = 0;
return 1;
}
static void gpu_key(char *out, size_t n) { // vendor:device:driver version, as the skip list names a GPU and driver
const VkPhysicalDeviceProperties *p = &vk_ctx()->props;
snprintf(out, n, "%04x:%04x:%08x", p->vendorID, p->deviceID, p->driverVersion);
}
static const char *stage_name(int st) {
return st == GFX_STAGE_VS ? "vertex" : st == GFX_STAGE_FS ? "fragment" : st == GFX_STAGE_CS ? "compute" : "other";
}
static int hex_digit(int c) {
return c >= '0' && c <= '9' ? c - '0' : c >= 'a' && c <= 'f' ? c - 'a' + 10 : c >= 'A' && c <= 'F' ? c - 'A' + 10 : -1;
}
// A journal a crashed run left into the skip list, then the skip list's variants for this GPU and driver into SK.
static void skip_init(const char *dir) {
if (SK.loaded) return;
SK.loaded = 1;
char jp[PLAT_PATH_MAX * 2 + 48], sp[PLAT_PATH_MAX * 2 + 48], key[32];
snprintf(jp, sizeof jp, "%s/shader-prepare-journal.bin", dir);
snprintf(sp, sizeof sp, "%s/shader-prepare-skip.txt", dir);
gpu_key(key, sizeof key);
uint64_t jn = 0, sn = 0;
uint8_t *j = plat_file_read(jp, &jn);
char *s = (char *)plat_file_read(sp, &sn);
int keep_journal = 0;
if (j) {
static const char head[] =
"# Shader variants the graphics driver was building when the process died during preparation\n"
"# (runtime/vk_pipeline.c skip_init): input id, stage, GPU vendor:device:driver version. Preparation skips\n"
"# them on that GPU and driver. Delete this file to build them again.\n";
size_t cap = (s ? sn : sizeof head) + (jn / JR_SLOT + 1) * 128, len = 0;
char *out = malloc(cap);
int added = 0;
if (out) {
len = s ? sn : sizeof head - 1;
memcpy(out, s ? s : head, len);
if (len && out[len - 1] != '\n') out[len++] = '\n';
for (uint64_t o = 0; o + 33 <= jn; o += JR_SLOT) {
int set = 0;
for (int b = 0; b < 32; b++) set |= j[o + b];
if (!set) continue;
char hex[65];
for (int b = 0; b < 32; b++) snprintf(hex + 2 * b, 3, "%02x", j[o + b]);
len += (size_t)snprintf(out + len, cap - len, "%s %s %s\n", hex, stage_name(j[o + 32]), key);
added++;
}
out[len] = 0; // parsed below as the list's text
}
if (added && plat_file_write_atomic(sp, out, len)) {
rt_log("vk_pipeline: the last shader preparation died while the driver built %d variants; %s now lists them "
"and preparation skips them on this GPU and driver (%s)", added, sp, key);
free(s), s = out, sn = len, out = NULL;
} else if (added) {
rt_log("vk_pipeline: cannot write %s; %s stays for the next run", sp, jp);
keep_journal = 1;
}
free(out);
free(j);
}
if (!keep_journal) plat_file_delete(jp);
for (char *line = s, *next; line && *line; line = next) {
next = strchr(line, '\n');
if (next) *next++ = 0;
char hex[65], st[16], k[32];
uint8_t iid[32];
int ok = line[0] != '#' && sscanf(line, "%64s %15s %31s", hex, st, k) == 3 && strlen(hex) == 64 &&
!strcmp(k, key);
for (int b = 0; ok && b < 32; b++) {
int hi = hex_digit(hex[2 * b]), lo = hex_digit(hex[2 * b + 1]);
if (hi < 0 || lo < 0) ok = 0;
else iid[b] = (uint8_t)(hi << 4 | lo);
}
uint8_t (*grown)[32] = ok ? realloc(SK.iid, (SK.n + 1) * sizeof *SK.iid) : NULL;
if (grown) memcpy(grown[SK.n++], iid, 32), SK.iid = grown;
}
free(s);
if (SK.n) rt_log("vk_pipeline: %zu shader variants skipped ahead of play on this GPU and driver (%s)", SK.n, sp);
}
static int skip_listed(const uint8_t iid[32]) {
for (size_t i = 0; i < SK.n; i++)
if (!memcmp(SK.iid[i], iid, 32)) return 1;
return 0;
}
static void skip_count(void) {
plat_mutex_lock(&LB.stat);
LB.skipped++;
plat_mutex_unlock(&LB.stat);
}
// Journal slot `slot`: the variant e its worker hands to the driver next, or zeros (e NULL) when the build returned.
static void journal_mark(int slot, const GfxWarmEnt *e) {
if (!SK.journal || slot < 0 || slot >= JR_SLOTS) return;
uint8_t rec[JR_SLOT] = {0};
if (e) memcpy(rec, e->iid, 32), rec[32] = e->stage;
plat_file_pwrite(SK.journal, rec, sizeof rec, (uint64_t)slot * JR_SLOT);
}
VkPipeline vk_pipeline_compute(const GfxShader *cs, const VkPipeInfo **info) {
if (info) *info = NULL;
if (!cs || !ensure_init()) return VK_NULL_HANDLE;
@@ -2749,7 +2878,8 @@ static void lb_report(const char *how) {
rt_log("vk_pipeline: stage library prebuild %s: %llu of %zu variants in %.0f ms; kept: vertex %llu built (avg %.1f ms), "
"%llu there; fragment %llu built (avg %.1f ms), %llu there; compute %llu built (avg %.1f ms), %llu there; driver "
"cache only: vertex %llu (avg %.1f ms), fragment %llu (avg %.1f ms), compute %llu (avg %.1f ms); %llu failed, %llu not in the shader cache, "
"%llu vertex/fragment skipped (graphics pipeline library off); process working set %.0f MB", how, (unsigned long long)LB.done, LB.n,
"%llu vertex/fragment skipped (graphics pipeline library off), %llu skipped (the driver crashed on them before, "
"shader-prepare-skip.txt); process working set %.0f MB", how, (unsigned long long)LB.done, LB.n,
now_ms() - LB.t0, (unsigned long long)LB.built[0], LB.built[0] ? LB.ms[0] / LB.built[0] : 0.0,
(unsigned long long)LB.have[0], (unsigned long long)LB.built[1], LB.built[1] ? LB.ms[1] / LB.built[1] : 0.0,
(unsigned long long)LB.have[1], (unsigned long long)LB.built[2], LB.built[2] ? LB.ms[2] / LB.built[2] : 0.0,
@@ -2757,7 +2887,7 @@ static void lb_report(const char *how) {
(unsigned long long)LB.warmed[1], LB.warmed[1] ? LB.warm_ms[1] / LB.warmed[1] : 0.0,
(unsigned long long)LB.warmed[2], LB.warmed[2] ? LB.warm_ms[2] / LB.warmed[2] : 0.0,
(unsigned long long)(LB.fail[0] + LB.fail[1] + LB.fail[2]),
(unsigned long long)LB.noshader, (unsigned long long)LB.nolib, (double)mem_proc);
(unsigned long long)LB.noshader, (unsigned long long)LB.nolib, (unsigned long long)LB.skipped, (double)mem_proc);
plat_mutex_unlock_shared(&LB.stat);
}
@@ -2768,6 +2898,13 @@ static void lb_worker(void *arg) {
for (;;) {
size_t i = (size_t)plat_inc64(&LB.next) - 1;
if (i >= LB.n || LB.stop) break;
if (skip_listed(LB.e[i].iid)) {
skip_count();
plat_mutex_lock(&LB.stat);
LB.done++;
plat_mutex_unlock(&LB.stat);
continue;
}
plat_thread_priority(PLAT_PRIO_NORMAL); // gfx_shader's cache lock, disk reads
const GfxShader *sh = gfx_shader_load_input(LB.e[i].iid, 0);
plat_thread_priority(PLAT_PRIO_IDLE);
@@ -2805,6 +2942,8 @@ static void lb_start(const uint32_t pr_hist[16], const uint32_t fs_hist[4]) {
const char *e = cfg_value("TOTK_LIB_PREBUILD"), *ke = cfg_get("TOTK_LIB_PREBUILD_KEEP");
const char *te = cfg_get("TOTK_LIB_PREBUILD_THREADS"); // no fixed default: a quarter of the CPUs when unset
if (!A.on || !cfg_bool("TOTK_LIB_PREBUILD")) return;
char dir[PLAT_PATH_MAX * 2];
if (cache_dir(dir, sizeof dir)) skip_init(dir);
GfxWarmEnt *all = NULL;
size_t total = gfx_shader_warm_list(&all), n = 0, n_cs = 0;
LB.keep_n = ke && *ke ? atoi(ke) : 0;
@@ -2847,16 +2986,20 @@ static void lb_stop(void) {
LB.threads = 0;
}
// Preparation: one prepared variant at a time into the driver's disk cache (private shader copy, nothing kept).
// Preparation: one prepared variant at a time into the driver's disk cache (private shader copy, nothing kept). arg is
// the worker's journal slot.
static void lb_warm_worker(void *arg) {
(void)arg;
int slot = (int)(intptr_t)arg;
for (;;) {
size_t i = (size_t)plat_inc64(&LB.next) - 1;
if (i >= LB.n) return;
int owned = 0;
const GfxShader *sh = gfx_shader_load_private(LB.e[i].iid, &owned);
int owned = 0, skip = skip_listed(LB.e[i].iid);
const GfxShader *sh = skip ? NULL : gfx_shader_load_private(LB.e[i].iid, &owned);
int st = sh ? gfx_shader_stage(sh) : -1;
if (!sh) {
if (sh) journal_mark(slot, &LB.e[i]); // from here to its clearing below, a crash names this variant
if (skip) {
skip_count();
} else if (!sh) {
plat_mutex_lock(&LB.stat);
LB.noshader++;
plat_mutex_unlock(&LB.stat);
@@ -2874,6 +3017,7 @@ static void lb_warm_worker(void *arg) {
if (P.gpl) lb_library(sh, st == GFX_STAGE_VS, 0);
else lb_skip_lib();
}
if (sh) journal_mark(slot, NULL);
if (owned) gfx_shader_release_private(sh);
plat_mutex_lock(&LB.stat);
uint64_t d = ++LB.done;
@@ -2894,23 +3038,36 @@ static void lb_warm_prepared(const char *dir) {
memcpy(LB.e[i].iid, f + 16 + 36 * i + 4, 32), LB.e[i].stage = f[16 + 36 * i], LB.e[i].prio = f[16 + 36 * i + 1];
free(f);
if (!LB.e) return;
skip_init(dir);
char jp[PLAT_PATH_MAX * 2 + 48];
snprintf(jp, sizeof jp, "%s/shader-prepare-journal.bin", dir);
SK.journal = plat_fs_create(jp, NULL); // skip_init removed a leftover one
if (!SK.journal) rt_log("vk_pipeline: cannot create %s: a driver crash in this run will not be recorded", jp);
LB.n = n, LB.idle = 0, LB.t0 = now_ms();
static const uint32_t no_pr[16], no_fs[4];
lb_state(no_pr, no_fs);
int cpus = (int)plat_cpu_usable(), k = 0;
PlatThread *t[64];
for (int i = 0; i < cpus && i < 64; i++)
if ((t[k] = plat_thread_new(lb_warm_worker, NULL, "lib warm"))) k++;
if (!k) lb_warm_worker(NULL);
PlatThread *t[JR_SLOTS];
for (int i = 0; i < cpus && i < JR_SLOTS; i++)
if ((t[k] = plat_thread_new(lb_warm_worker, (void *)(intptr_t)k, "lib warm"))) k++;
if (!k) lb_warm_worker((void *)(intptr_t)0);
for (int i = 0; i < k; i++) plat_thread_join(t[i]);
if (SK.journal) plat_file_close(SK.journal), SK.journal = NULL, plat_file_delete(jp); // every build returned
lb_report("preparation done");
// with gpl off only compute variants are built: say so, rather than count the skipped ones as built
unsigned long long skipped = LB.skipped;
size_t tried = n - (size_t)skipped;
if (LB.nolib)
printf("prepare-shaders: %zu variants on %d threads in %.0f ms, %llu vertex/fragment variants skipped (graphics "
"pipeline library off, they build at their first draw)\n", n, k, now_ms() - LB.t0, (unsigned long long)LB.nolib);
"pipeline library off, they build at their first draw)\n", tried, k, now_ms() - LB.t0,
(unsigned long long)LB.nolib);
else
printf("prepare-shaders: %zu variants built into the driver cache on %d threads in %.0f ms\n", n, k, now_ms() - LB.t0);
free(LB.e), LB.e = NULL, LB.n = 0, LB.next = 0, LB.done = 0;
printf("prepare-shaders: %zu variants built into the driver cache on %d threads in %.0f ms\n", tried, k,
now_ms() - LB.t0);
if (skipped) // installer/steps.py Prepare reads this line
printf("prepare-shaders: %llu variants skipped: the graphics driver crashed while building them in an earlier "
"run (shader-prepare-skip.txt); they build at their first draw\n", skipped);
free(LB.e), LB.e = NULL, LB.n = 0, LB.next = 0, LB.done = 0, LB.skipped = 0;
}
// totk.exe --prepare-shaders (tools/prepare_shaders.py): the device sets the translator's host answers (they are in every
@@ -2922,10 +3079,7 @@ int vk_pipeline_prepare_shaders(void) {
return 1;
}
char dir[PLAT_PATH_MAX * 2];
snprintf(dir, sizeof dir, "%s", P.cache_path);
char *cut = plat_last_sep(dir);
if (!cut) return 1;
*cut = 0;
if (!cache_dir(dir, sizeof dir)) return 1;
int cpus = (int)plat_cpu_usable();
const char *pe = cfg_get("TOTK_PREWARM_XLATE"); // no fixed default: half the CPUs, at most 16, when unset
int procs = pe && *pe ? atoi(pe) : cpus / 2 > 16 ? 16 : cpus / 2;
@@ -2944,10 +3098,7 @@ int vk_pipeline_warm_shaders(void) {
return 1;
}
char dir[PLAT_PATH_MAX * 2];
snprintf(dir, sizeof dir, "%s", P.cache_path);
char *cut = plat_last_sep(dir);
if (!cut) return 1;
*cut = 0;
if (!cache_dir(dir, sizeof dir)) return 1;
char prp[PLAT_PATH_MAX * 2 + 32];
snprintf(prp, sizeof prp, "%s/shader-prepared.bin", dir);
if (!plat_file_exists(prp)) {
+73
View File
@@ -61,6 +61,14 @@ static volatile uint8_t ts_busy[TS_PAIRS]; // written until the completion threa
static int ts_cur = -1; // the pair of the part being recorded, -1 untimed
static uint64_t ts_last_end; // completion thread: end of the latest part read (ticks)
static volatile int64_t ts_busy_ns, ts_parts, ts_missed;
// GPU idle (with calibrated timestamps): the gap before each timed part, split by whether the part was submitted after
// the previous one had ended (the GPU waited for the CPU: starved) or before (it waited for untimed work or a wait).
static uint64_t ts_sub_ns[TS_PAIRS]; // CPU time the part was handed to vkQueueSubmit2
static int ts_unsent[4], ts_n_unsent; // pairs closed and not yet submitted (sub_lock)
static uint64_t ts_last_end_ns; // completion thread: ts_last_end on the CPU clock
static volatile int64_t ts_idle_ns, ts_starved_ns, ts_gaps, ts_max_gap_ns, ts_queued_ns;
static int64_t cal_tick, cal_ns, cal_at; // completion thread: a calibrated (GPU tick, CPU ns) pair and when it was taken
static PFN_vkGetCalibratedTimestampsKHR cal_fn;
// On the first submission that wants timing (perf_gpu_timing: TOTK_TRACE=perf or the full performance overlay).
static void ts_setup(VkCtx *c) {
@@ -91,6 +99,49 @@ static void ts_setup(VkCtx *c) {
typedef struct { uint32_t pair; } TsRead;
// Completion thread: refresh the GPU tick / CPU clock pair every 250 ms; 0 when there is no calibration. The host
// domain is the clock plat_now_ns reads (QPC on Windows, CLOCK_MONOTONIC elsewhere), plat_counter_hz ticks a second.
static int cal_refresh(VkCtx *c) {
if (!c->has_calibrated) return 0;
if (!cal_fn) {
cal_fn = (PFN_vkGetCalibratedTimestampsKHR)vkGetDeviceProcAddr(c->device, c->has_calibrated == 2 ? "vkGetCalibratedTimestampsKHR"
: "vkGetCalibratedTimestampsEXT");
if (!cal_fn) { c->has_calibrated = 0; return 0; }
}
int64_t now = (int64_t)plat_now_ns();
if (cal_at && now - cal_at < 250000000) return 1;
VkCalibratedTimestampInfoKHR ci[2] = {{.sType = VK_STRUCTURE_TYPE_CALIBRATED_TIMESTAMP_INFO_KHR, .timeDomain = VK_TIME_DOMAIN_DEVICE_KHR},
#ifdef _WIN32
{.sType = VK_STRUCTURE_TYPE_CALIBRATED_TIMESTAMP_INFO_KHR, .timeDomain = VK_TIME_DOMAIN_QUERY_PERFORMANCE_COUNTER_KHR}};
#else
{.sType = VK_STRUCTURE_TYPE_CALIBRATED_TIMESTAMP_INFO_KHR, .timeDomain = VK_TIME_DOMAIN_CLOCK_MONOTONIC_KHR}};
#endif
uint64_t v[2], dev;
if (cal_fn(c->device, 2, ci, v, &dev) != VK_SUCCESS) {
if (!cal_at) { // the domain pair is not supported: no split
rt_log("vk: calibrated timestamps failed: the perf line shows no GPU idle split");
c->has_calibrated = 0;
}
return cal_at != 0;
}
cal_tick = (int64_t)(v[0] & ts_mask), cal_ns = (int64_t)((double)v[1] * 1e9 / (double)plat_counter_hz()), cal_at = now;
return 1;
}
static int64_t tick_ns(uint64_t t) { return cal_ns + (int64_t)((double)((int64_t)t - cal_tick) * ts_period); }
static void max64(volatile int64_t *m, int64_t v) {
int64_t o = __atomic_load_n(m, __ATOMIC_RELAXED);
while (v > o && !__atomic_compare_exchange_n(m, &o, v, 0, __ATOMIC_RELAXED, __ATOMIC_RELAXED)) {}
}
// Just before vkQueueSubmit2 (sub_lock held): the parts closed since the last submit leave the CPU now.
static void ts_submitted(void) {
uint64_t now = plat_now_ns();
for (int i = 0; i < ts_n_unsent; i++) ts_sub_ns[ts_unsent[i]] = now;
ts_n_unsent = 0;
}
static void ts_read_cb(void *arg) { // completion thread, in submission order
TsRead *t = arg;
VkCtx *c = vk_ctx();
@@ -98,6 +149,18 @@ static void ts_read_cb(void *arg) { // completion thread, in submission order
if (vkGetQueryPoolResults(c->device, ts_pool, 2 * t->pair, 2, sizeof v, v, sizeof v[0], VK_QUERY_RESULT_64_BIT) == VK_SUCCESS) {
uint64_t s = v[0] & ts_mask, e = v[1] & ts_mask;
if (e >= s) {
if (cal_refresh(c)) {
int64_t sn = tick_ns(s), en = tick_ns(e), sub = (int64_t)ts_sub_ns[t->pair];
if (ts_last_end_ns && sn > ts_last_end_ns) {
int64_t gap = sn - ts_last_end_ns, late = sub - ts_last_end_ns;
plat_add64(&ts_idle_ns, gap);
if (late > 0) plat_add64(&ts_starved_ns, late < gap ? late : gap);
if (gap > 500000) plat_inc64(&ts_gaps);
max64(&ts_max_gap_ns, gap);
}
if (sub && sn > sub) plat_add64(&ts_queued_ns, sn - sub);
if (en > ts_last_end_ns) ts_last_end_ns = en;
}
uint64_t from = s > ts_last_end ? s : ts_last_end;
if (e > from) {
int64_t ns = (int64_t)((double)(e - from) * ts_period);
@@ -132,6 +195,8 @@ static int ts_open(VkCommandBuffer cb) {
static void ts_close(VkCommandBuffer cb, int pair, uint64_t fence) {
if (pair < 0) return;
vkCmdWriteTimestamp(cb, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, ts_pool, 2 * (uint32_t)pair + 1);
ts_sub_ns[pair] = 0;
if (ts_n_unsent < 4) ts_unsent[ts_n_unsent++] = pair;
TsRead *t = malloc(sizeof *t);
if (!t) rt_fatal("vk: out of memory");
t->pair = (uint32_t)pair;
@@ -159,6 +224,12 @@ static void gpu_stats(PerfGpu *g) {
g->gpu_parts = (uint64_t)plat_xchg64(&ts_parts, 0);
g->gpu_missed = (uint64_t)plat_xchg64(&ts_missed, 0);
g->have_vram = vk_mem_vram(&g->vram_used, &g->vram_budget, &g->vram_exact);
g->have_idle = vk_ctx()->has_calibrated != 0 && cal_at != 0;
g->idle_ns = (uint64_t)plat_xchg64(&ts_idle_ns, 0);
g->starved_ns = (uint64_t)plat_xchg64(&ts_starved_ns, 0);
g->queued_ns = (uint64_t)plat_xchg64(&ts_queued_ns, 0);
g->idle_gaps = (uint64_t)plat_xchg64(&ts_gaps, 0);
g->max_gap_ns = (uint64_t)plat_xchg64(&ts_max_gap_ns, 0);
}
// Weak stand-in for vk_exec.c's replay: only empty recordings ({OP_BEGIN, OP_END}) can be submitted.
@@ -303,6 +374,7 @@ VkCommandBuffer vk_sync_split(void) {
{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO, .commandBuffer = cb}};
VkSubmitInfo2 si = {.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO_2, .commandBufferInfoCount = pro ? 2 : 1,
.pCommandBufferInfos = pro ? cbi : cbi + 1};
ts_submitted();
vk_queue_lock();
VkResult r = vkQueueSubmit2(c->gfx_queue, 1, &si, VK_NULL_HANDLE);
vk_queue_unlock();
@@ -366,6 +438,7 @@ uint64_t vk_sync_submit(const uint64_t *handles, int n) {
.commandBufferInfoCount = (pro ? 1u : 0u) + (recorded ? 1u : 0u),
.pCommandBufferInfos = pro ? cbi : cbi + 1, .signalSemaphoreInfoCount = 1,
.pSignalSemaphoreInfos = &sig};
ts_submitted();
vk_queue_lock();
VkResult r = vkQueueSubmit2(c->gfx_queue, 1, &si, VK_NULL_HANDLE);
vk_queue_unlock();
+15 -11
View File
@@ -31,8 +31,8 @@ from installer import state as statefile # noqa: E402
from installer.events import Reporter # noqa: E402
from installer.runner import Context # noqa: E402
from installer.steps import all_steps # noqa: E402
from installer.guilogic import (DONE, FAILED, PENDING, RUNNING, SKIPPED, Catalog, Form, Model, Process, # noqa: E402
amount, describe_event, parse_line, windows_ui_language)
from installer.guilogic import (DONE, FAILED, PENDING, RUNNING, SKIPPED, UI_LANGUAGES, Catalog, Form, Model, # noqa: E402
Process, amount, describe_event, parse_line, windows_ui_language)
INSTALLED = ROOT / 'build' / 'install-test'
FIXTURE = ROOT / 'build' / 'inst-gui' / 'e2e-install'
@@ -233,12 +233,16 @@ class FormTest(unittest.TestCase):
class CatalogTest(unittest.TestCase):
def test_russian_covers_english_with_the_same_placeholders(self):
en, ru = Catalog('en').english, Catalog('ru').texts
self.assertEqual(sorted(set(en) - set(ru)), [])
self.assertEqual(sorted(set(ru) - set(en)), [])
holes = [k for k in en if sorted(re.findall(r'\{\w+\}', en[k])) != sorted(re.findall(r'\{\w+\}', ru[k]))]
self.assertEqual(holes, [])
def test_every_language_covers_english_with_the_same_placeholders(self):
en = Catalog('en').english
for lang in UI_LANGUAGES[1:]:
with self.subTest(lang=lang):
other = Catalog(lang).texts
self.assertEqual(sorted(set(en) - set(other)), [])
self.assertEqual(sorted(set(other) - set(en)), [])
holes = [k for k in en
if sorted(re.findall(r'\{\w+\}', en[k])) != sorted(re.findall(r'\{\w+\}', other[k]))]
self.assertEqual(holes, [])
def test_every_id_the_window_uses_exists(self):
cat = Catalog('en')
@@ -246,12 +250,12 @@ class CatalogTest(unittest.TestCase):
'process_died', 'result_ok', 'result_failed', 'result_cancelled')]
ids += [f'step.{s}' for s in ('check', 'extract', 'lift', 'compile', 'prepare', 'cleanup')]
ids += ['ui.state_' + n for n in ('started', 'done', 'skipped', 'pending', 'running', 'failed')]
for lang in ('en', 'ru'):
for lang in UI_LANGUAGES:
cat.set_language(lang)
self.assertEqual([i for i in ids if not cat.has(i)], [])
def test_codes_become_names(self):
for lang, korean in (('en', 'Korean'), ('ru', 'Корейский')):
for lang, korean in (('en', 'Korean'), ('ru', 'Корейский'), ('pt-BR', 'Coreano')):
cat = Catalog(lang)
self.assertIn(korean, cat('warn.lang_needs_fonts', lang='ko'))
self.assertNotIn(' ko ', cat('warn.lang_needs_fonts', lang='ko'))
@@ -280,7 +284,7 @@ class CatalogTest(unittest.TestCase):
self.assertEqual(describe_event(cat, {'type': 'start'}), '')
def test_windows_language_is_a_known_value(self):
self.assertIn(windows_ui_language(), ('en', 'ru'))
self.assertIn(windows_ui_language(), UI_LANGUAGES)
BREAK_CHILD = 'import time\nprint("ready", flush=True)\ntime.sleep(60)\n'
+27 -7
View File
@@ -17,9 +17,11 @@ first (rpath $ORIGIN). docs/building.md lists the packages.
Every runtime/*.c is compiled. build/<version>/modules.c carries the layout, the linked function tables, the
expected SHA-256 of each ExeFS file (tools/data/expected-<display version>.json) and the default game paths
(--exefs, --romfs, --profile), which totk.exe uses unless its own command line names others. tools/romfs_index.py
writes build/<version>/romfs-index.bin and romfs_config.h from the RomFS directory's index.json (runtime/romfs.c reads
the index next to totk.exe).
(--exefs, --romfs, --profile), which totk.exe uses unless its own command line names others. --exefs and --romfs
default to extracted/<version> in the work folder: the [work] dir of config.local.toml, where tools/extract.py writes
them, else build/. tools/romfs_index.py writes build/<version>/romfs-index.bin and romfs_config.h from the RomFS
directory's index.json (runtime/romfs.c reads the index next to totk.exe). Without that index.json the build stops:
the exe would stop at start. --no-romfs builds it anyway.
Private builds reuse the main build's objects. When --out differs from build/<version>, the main build's generated
objects are up to date (checked with a dry run of its build.ninja) and every header they include from runtime/ is
@@ -241,6 +243,16 @@ def copy_romfs_patches(version, out):
print(f'romfs patches: copied {name} from {src}')
def work_dir():
"""The work folder: the [work] dir of config.local.toml (relative to the checkout), where tools/extract.py writes
extracted/; build/ without the file or the key."""
cfg = ROOT / 'config.local.toml'
if not cfg.is_file():
return ROOT / 'build'
import tomllib
return ROOT / tomllib.loads(cfg.read_text(encoding='utf-8')).get('work', {}).get('dir', 'build')
def c_string(s):
return '"' + s.replace('\\', '\\\\').replace('"', '\\"') + '"'
@@ -710,9 +722,12 @@ def main():
help='comma-separated runtime/*.c names to leave out (the link needs another definition of what they '
'define)')
ap.add_argument('--exefs', type=Path,
help="totk.exe's default ExeFS directory (default build/extracted/<version>/exefs)")
help="totk.exe's default ExeFS directory (default <work dir>/extracted/<version>/exefs, the work "
"dir being the [work] dir of config.local.toml, else build)")
ap.add_argument('--romfs', type=Path, help="totk.exe's default RomFS directory with romfs.bin and index.json "
'(default build/extracted/<version>/romfs)')
'(default <work dir>/extracted/<version>/romfs)')
ap.add_argument('--no-romfs', action='store_true',
help='build even when the RomFS directory has no index.json (the exe then stops at start)')
ap.add_argument('--profile', type=Path, help="totk.exe's default profile directory (default build/profile for v121, "
"build/profile-<version> otherwise, so trying another version leaves "
"the main profile alone; saves load on 1.2.1 and 1.4.3 alike)")
@@ -735,11 +750,16 @@ def main():
gen = (args.gen or ROOT / 'generated' / args.version).resolve()
out = (args.out or ROOT / 'build' / args.version).resolve()
extracted = ROOT / 'build' / 'extracted' / args.version
extracted = None if args.exefs and args.romfs else work_dir() / 'extracted' / args.version
paths = {'exefs': (args.exefs or extracted / 'exefs').resolve(),
'romfs': (args.romfs or extracted / 'romfs').resolve(),
'profile': (args.profile or ROOT / 'build' / ('profile' if args.version == 'v121'
else f'profile-{args.version}')).resolve()}
has_romfs = (paths['romfs'] / 'index.json').is_file()
if not (has_romfs or args.no_romfs or args.make_runtime_lib or args.compile_only):
sys.exit(f'{paths["romfs"] / "index.json"} not found: without the RomFS the game stops at start. Run '
'tools/extract.py (it writes extracted/<version> into the [work] dir of config.local.toml), pass '
'--romfs DIR, or --no-romfs to build without it')
expected = args.expected
if not expected:
if args.version not in DISPLAY_VERSIONS:
@@ -795,7 +815,7 @@ def main():
if missing:
sys.exit(f'{expected}: no exefs_sha256 for {missing}')
write_if_changed(out / 'modules.c', module_glue(layout, linked, hashes, paths, args.version))
if (paths['romfs'] / 'index.json').is_file():
if has_romfs:
count, size, dt = romfs_index.write(paths['romfs'], out)
print(f'romfs index: {count} files, {mib(size)}, {dt:.1f} s')
else:
+6 -4
View File
@@ -424,12 +424,14 @@ def c_wide(text):
def launcher_header(path):
"""launcher_text.h: the launcher's two messages in English and Russian, taken from the installer's catalog."""
lines = ['/* generated by tools/make_release.py from installer/text/{en,ru}.json */']
for lang in ('en', 'ru'):
"""launcher_text.h: the launcher's two messages in English, Russian and Brazilian Portuguese, taken from the
installer's catalog (LAUNCHER_TITLE_PT_BR for pt-BR.json)."""
lines = ['/* generated by tools/make_release.py from installer/text/{en,ru,pt-BR}.json */']
for lang in ('en', 'ru', 'pt-BR'):
texts = json.loads((ROOT / 'installer' / 'text' / f'{lang}.json').read_text(encoding='utf-8'))
for key in ('title', 'python_missing'):
lines.append(f'#define LAUNCHER_{key.upper()}_{lang.upper()} {c_wide(texts["launcher." + key])}')
macro = f'LAUNCHER_{key.upper()}_{lang.upper().replace("-", "_")}'
lines.append(f'#define {macro} {c_wide(texts["launcher." + key])}')
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text('\n'.join(lines) + '\n', encoding='ascii')
+20
View File
@@ -18,6 +18,11 @@ shader-prepared.bin (one record per variant, in priority order) and builds every
pipeline once, so the GPU driver's own disk cache holds them (vk_pipeline.c "stage library prebuild"). The seed files
can be deleted afterwards.
A driver whose shader compiler crashes on a variant crashes on it in every run (NVIDIA 615.78.08 on an RTX 5080, issue
#30). The translations and shader-prepared.bin are written by then, and the journal the run leaves names the variants
the driver was building (vk_pipeline.c skip_init), so only the driver step runs again (`totk.exe --warm-shaders`), up to
RETRIES times, and skips them.
Game-derived output stays in the build directory (ignored). The list is read at preparation time from tools/data/shader_list.tsv when that file exists (the installer archive
bundles it; it stays out of the repository), else from refs/ through git with the rows its history dropped
(tools/shader_list.py); nothing of it is copied into the repository. Needs Python 3.14 (compression.zstd), so run it in .venv-py314 or the embeddable 3.14:
@@ -54,6 +59,8 @@ TYPES = {'float': 0, 'sint': 1, 'uint': 2}
DEFAULT = {'tex': {str(slot): '2d' for slot in range(32)}} # answers outside a shader's use mask are dropped
CONTROL = 0x880
ANS_REC = 4 + 32 + 32 + 147 + 1
JOURNAL = 'shader-prepare-journal.bin' # vk_pipeline.c skip_init: a 64-byte slot per worker, input id first
RETRIES = 3
BUNDLED_LIST = ROOT / 'tools' / 'data' / 'shader_list.tsv' # the installer archive carries a copy (no git, no refs/)
@@ -128,6 +135,13 @@ def code_hash(stage, control, code, length, c1):
struct.pack('<I', len(c1)) + c1).digest()
def driver_crashed(out):
"""totk died while the graphics driver built variants: the journal it left names at least one."""
p = out / JOURNAL
data = p.read_bytes() if p.is_file() else b''
return any(any(data[i:i + 32]) for i in range(0, len(data), 64))
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument('build', type=Path, help='build directory holding totk.exe (build\\v121)')
@@ -193,6 +207,12 @@ def main():
return 0
exe = out / ('totk.exe' if os.name == 'nt' else 'totk')
r = subprocess.run([str(exe), '--prepare-shaders'], cwd=str(out))
for _ in range(RETRIES):
if not r.returncode or not driver_crashed(out):
break
print(f'prepare_shaders: the graphics driver crashed while building shaders (rc={r.returncode}); building them '
'again without the variants it was building (totk.exe --warm-shaders)', flush=True)
r = subprocess.run([str(exe), '--warm-shaders'], cwd=str(out))
print(f'totk.exe --prepare-shaders: rc={r.returncode}; total {time.monotonic() - t0:.1f} s')
return r.returncode
+12
View File
@@ -155,6 +155,10 @@
# color_contrast = 100
# color_saturation = 100
# color_gamma = 1
# Sun shadow map size: game (the game's own, which changes with the view and drops to about 400 near the ground, so
# leaf shadows look blocky and jump) or a fixed 512, 768, 1024, 1536 or 2048. Above 960 the texture heap grows with it.
# Read at start.
# shadow_res = game
[language]
# Language: a Windows locale such as de-DE, or a Switch code (en-US, ja, zh-Hant, ...). Default: the Windows locale.
@@ -194,6 +198,14 @@
# stick_deadzone = 8
# Mouse camera sensitivity (0 to 10; 0 = no mouse camera and no cursor capture).
# mouse_sens = 1
# The keyboard's sticks are all or nothing, so the game only sees a full tilt from them; while this key is held the
# keyboard-driven stick values shrink and the keyboard can walk instead of run. Key names as in the bind_ lines below,
# none turns it off. 0.2 to 0.9, the default holds for anything else. Only the keyboard is scaled: the mouse camera and
# the gamepad are unchanged. The Controls tab sets both. The default stops just above half because a keyboard sample of
# 32767 times 0.5 is one short of the 16384 the game's stick-direction bits need, and below that the keyboard stick
# reads as a partial tilt only.
# walk_key = lshift
# walk_scale = 0.51
# Write the input of this run to a file that replays as a script (docs/controls.md, "Scripted input"). Default: off.
# input_record = input.txt
# Remap buttons: the Controls tab of the settings menu (F1) writes these lines. One line per Switch button or stick