diff --git a/.github/ISSUE_TEMPLATE/1-crash.yml b/.github/ISSUE_TEMPLATE/1-crash.yml new file mode 100644 index 00000000..adb68726 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/1-crash.yml @@ -0,0 +1,65 @@ +name: Crash +description: The game closes, freezes or shows a FATAL message. +labels: [bug, crash] +body: + - type: markdown + attributes: + value: | + Check the pinned issues at the top of the issue list first: they cover the known crashes and their workarounds. + + Do not attach or link game files, keys, firmware or builds. Issues that do are closed. + - type: checkboxes + id: duplicates + attributes: + label: Existing issues + options: + - label: I read the pinned issues and [searched open and closed issues](https://github.com/valentderah/totk-pc-recomp/issues?q=is%3Aissue) for this crash. + required: true + - type: input + id: version + attributes: + label: Version + description: The title line of the settings menu (F1), for example "TotK Recomp 0.1.0, game 1.4.3". + validations: + required: true + - type: input + id: system + attributes: + label: GPU, driver and Windows version + description: Copy system info in the menu's Advanced tab puts them on the clipboard. + validations: + required: true + - type: dropdown + id: when + attributes: + label: When does it crash? + options: + - At startup, before the Nintendo logo + - Between the Nintendo logo and the title screen + - While loading a save + - In gameplay + - In a cutscene + validations: + required: true + - type: dropdown + id: blocker + attributes: + label: Can you keep playing? + options: + - "Yes, it does not happen again after a restart, or I can avoid it" + - "No, it happens every time at the same point" + validations: + required: true + - type: textarea + id: steps + attributes: + label: What you did before the crash + validations: + required: true + - type: textarea + id: log + attributes: + label: Log + description: Attach logs\totk.log from the folder of totk.exe. Copy it before you start the game again, because the next start overwrites it. + validations: + required: true diff --git a/.github/ISSUE_TEMPLATE/2-graphics.yml b/.github/ISSUE_TEMPLATE/2-graphics.yml new file mode 100644 index 00000000..bf297032 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/2-graphics.yml @@ -0,0 +1,59 @@ +name: Graphics glitch +description: Artifacts, missing or broken textures, flicker, wrong colours or shadows. +labels: [bug, graphics] +body: + - type: markdown + attributes: + value: | + Check the pinned issues at the top of the issue list first: they cover the known glitches on each GPU. + + Do not attach or link game files, keys, firmware or builds. Issues that do are closed. + - type: checkboxes + id: duplicates + attributes: + label: Existing issues + options: + - label: I read the pinned issues and [searched open and closed issues](https://github.com/valentderah/totk-pc-recomp/issues?q=is%3Aissue) for this glitch. + required: true + - type: input + id: version + attributes: + label: Version + description: The title line of the settings menu (F1), for example "TotK Recomp 0.1.0, game 1.4.3". + validations: + required: true + - type: input + id: system + attributes: + label: GPU, driver and Windows version + description: Copy system info in the menu's Advanced tab puts them on the clipboard. + validations: + required: true + - type: textarea + id: screenshot + attributes: + label: Screenshot or video + description: Drag the file into this field. Say where in the game it was taken. + validations: + required: true + - type: textarea + id: details + attributes: + label: What looks wrong and when + description: Does it show all the time, in one place, or after some time in play? Which settings did you change from the defaults? + validations: + required: true + - type: dropdown + id: blocker + attributes: + label: Can you keep playing? + options: + - "Yes" + - "No, I cannot see enough to play" + validations: + required: true + - type: textarea + id: log + attributes: + label: Log + description: Attach logs\totk.log from the folder of totk.exe. diff --git a/.github/ISSUE_TEMPLATE/3-bug.yml b/.github/ISSUE_TEMPLATE/3-bug.yml new file mode 100644 index 00000000..20d7087f --- /dev/null +++ b/.github/ISSUE_TEMPLATE/3-bug.yml @@ -0,0 +1,65 @@ +name: Other bug +description: Sound, controls, performance, the settings menu, the installer, or game logic. +labels: [bug] +body: + - type: markdown + attributes: + value: | + Check the pinned issues at the top of the issue list first. + + Do not attach or link game files, keys, firmware or builds. Issues that do are closed. + - type: checkboxes + id: duplicates + attributes: + label: Existing issues + options: + - label: I read the pinned issues and [searched open and closed issues](https://github.com/valentderah/totk-pc-recomp/issues?q=is%3Aissue) for this bug. + required: true + - type: dropdown + id: area + attributes: + label: Area + options: + - Audio + - Controls and input + - Performance and stutter + - Settings menu and overlay + - Installer + - Game logic and physics + - Something else + validations: + required: true + - type: input + id: version + attributes: + label: Version + description: The title line of the settings menu (F1), for example "TotK Recomp 0.1.0, game 1.4.3". + validations: + required: true + - type: input + id: system + attributes: + label: GPU, driver and Windows version + description: Copy system info in the menu's Advanced tab puts them on the clipboard. + validations: + required: true + - type: dropdown + id: blocker + attributes: + label: Can you keep playing? + options: + - "Yes" + - "No, it stops the game or a quest" + validations: + required: true + - type: textarea + id: steps + attributes: + label: What happened and how to reproduce it + validations: + required: true + - type: textarea + id: log + attributes: + label: Log + description: Attach logs\totk.log from the folder of totk.exe. diff --git a/.github/ISSUE_TEMPLATE/4-feature.yml b/.github/ISSUE_TEMPLATE/4-feature.yml new file mode 100644 index 00000000..7d2f4ffe --- /dev/null +++ b/.github/ISSUE_TEMPLATE/4-feature.yml @@ -0,0 +1,17 @@ +name: Feature request +description: A setting, option or support for something the port does not do yet. +labels: [enhancement] +body: + - type: checkboxes + id: duplicates + attributes: + label: Existing issues + options: + - label: I [searched open and closed issues](https://github.com/valentderah/totk-pc-recomp/issues?q=is%3Aissue) and the Planned section of the README for this request. + required: true + - type: textarea + id: request + attributes: + label: What you want and why + validations: + required: true diff --git a/.github/ISSUE_TEMPLATE/bug.yml b/.github/ISSUE_TEMPLATE/bug.yml deleted file mode 100644 index eb22ae3b..00000000 --- a/.github/ISSUE_TEMPLATE/bug.yml +++ /dev/null @@ -1,32 +0,0 @@ -name: Bug report -description: Something breaks, crashes or looks wrong. -body: - - type: markdown - attributes: - value: | - Do not attach or link game files, keys, firmware or builds. Issues that do are closed. - - type: input - id: version - attributes: - label: Version - description: The title line of the settings menu (F1), for example "TotK Recomp 0.1.0, game 1.4.3". - validations: - required: true - - type: input - id: system - attributes: - label: GPU, driver and Windows version - description: Copy system info in the menu's Advanced tab puts them on the clipboard. - validations: - required: true - - type: textarea - id: steps - attributes: - label: What happened and how to reproduce it - validations: - required: true - - type: textarea - id: log - attributes: - label: Log - description: Attach logs\totk.log from the folder of totk.exe. After a crash, copy it before you start the game again. diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index 3ba13e0c..7e047e82 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -1 +1,5 @@ blank_issues_enabled: false +contact_links: + - name: Question or help with setup + url: https://github.com/valentderah/totk-pc-recomp/discussions + about: Installing, updating and configuring the port. Ask in Discussions. diff --git a/.gitignore b/.gitignore index d2fdab74..e365e074 100644 --- a/.gitignore +++ b/.gitignore @@ -42,6 +42,9 @@ __pycache__/ *.o *.lib *.a +*.so +*.so.* +*.AppImage # .NET build output of the shader translator (tools/build_xlate.py) /tools/xlate/bin/ diff --git a/README.md b/README.md index 13e75988..f3a41b6b 100644 --- a/README.md +++ b/README.md @@ -19,15 +19,17 @@ Report problems as issues, following [CONTRIBUTING.md](CONTRIBUTING.md). - Game updates 1.2.1 and 1.4.3, with saves that carry over between them - 60 fps by default; up to 144 fps or the monitor's refresh rate (experimental) - Custom resolutions (1080p, 2k...), supersampling and AMD FSR 1 upscaling +- 16:10 (Steam Deck, handhelds), 21:9 and 32:9 (ultrawide) game resolutions on 1.4.3, with the camera and the interface in that shape - In-game settings menu (F1) and performance overlay (F10) - Keyboard, mouse and gamepad with remapping, rumble and gyro +- Xbox button prompts: a layout that keeps the Switch button positions and shows the Xbox labels - Shaders prepared at install instead of during play ## Planned - Linux and the Steam Deck - Mods made for emulators: unpack a mod into a `mods` folder -- Custom aspect ratios, including ultrawide +- 16:10 and ultrawide on 1.2.1 - VRR ## Installing diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 0f165636..86d96747 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -9,6 +9,9 @@ Code copied or adapted from another project is listed here with its source, the | LAS Recomp host runtime: NSO loader, host-function table, guest-space reservation and re-execution, guest thread start | https://github.com/peiikai/LASRecomp @ `da35b83a9611d046a9a681aae668d7085954d37f`, Copyright (c) 2026 Peiikai | MIT | `runtime/loader.c`, `runtime/hostfn.c`, `runtime/memory.c`; `runtime/kernel.c`, `runtime/boot.c`; `runtime/vk_device.c`, `runtime/gfx_window.c`, `runtime/tex_astc.c` (`runtime/nvn/astc.c`; its decode ported to GLSL in `runtime/shaders/astc_decode.comp`, compiled into `runtime/astc_decode_spv.h`, and its tables copied into `runtime/vk_astc.c`), `runtime/vk_formats.c` (format table of `runtime/nvn/vulkan/textures.c`), `runtime/vk_memory.c` (page write generations of `runtime/nvn/vulkan/exec.c`, deferred destruction of `textures.c`), `runtime/vk_texture.c` (image creation, uploads and samplers of `runtime/nvn/vulkan/textures.c`), `runtime/vk_sync.c` (timeline submissions and waits of `runtime/nvn/vulkan/exec.c`), `runtime/vk_pipeline.c` (blend, compare and stencil mappings, vertex format table with its 3-component fallback and pipeline cache file of `runtime/nvn/vulkan/exec.c` and `device.c`), `runtime/vk_exec.c` (pass begin and end, scissor clip and attachment clears of `runtime/nvn/vulkan/exec.c`), `runtime/vk_exec_xfer.c` (copies of `runtime/nvn/vulkan/exec.c`), `runtime/vk_exec_draw.c` (uniform buffer spans, texture and sampler descriptors and dispatch flow of `runtime/nvn/vulkan/exec.c`); in-game menu: `runtime/ui/menu.cpp` (Dear ImGui Vulkan set-up of `runtime/nvn/vulkan/imgui_glue.cpp`) | | MacNativeGames TotK host runtime: dynamic linking, identity check, module permissions, guest memory and SVCs, threads, IPC, audio renderer, system services, RomFS and save storage, boot sequence, the NVN proc name list | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/runtime/`, `games/totk/scripts/prepare_runtime.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `runtime/loader.c`, `runtime/memory.c`; `runtime/kernel.c`, `runtime/svc.c`, `runtime/ipc.c`, `runtime/ipc.h`, `runtime/audren.c`, `runtime/audren.h`, `runtime/audio_mixer.c`, `runtime/audio_effects.c`, `runtime/audio_effects.h`, `runtime/audio_reverb.c`, `runtime/audio_reverb.h` (SDL3 output replaces CoreAudio), `runtime/ov_system.c` (also `shared_font.c`: the BFTTF layout behind nn::pl), `runtime/ov_input.c` (also `hid_feedback.c`: vibration and six-axis handles and calls), `runtime/motion_fusion.c` (six-axis fusion of `motion_fusion.c`), `runtime/ov_media.c` (hardware Opus decoder over libopus), `runtime/swkbd_win.c` (`swkbd_host.c`: ShowKeyboardArg offsets, UTF-8/UTF-16 conversion, disabled-key filter, result codes; Win32 window replaces the macOS text bar), `runtime/ov_gfx.c`, `runtime/romfs.c`, `runtime/romfs_path.h`, `runtime/fs_host.c`, `runtime/fs.h`, `runtime/boot.c`; `runtime/gfx_window.c`; `tools/gen_nvn_procs.py`, `runtime/nvn_procs.inc` (`NVN/names.txt` and the `NVN_EXTRA` names of `totk_nvn_ext.c`) | | MacNativeGames TotK frame-rate patches: 60 fps present-interval controller, real-time frame step and its present sampling, hair and cloth clock, trampoline slow-motion check (instruction words and 1.2.1 sites), player position chain; quick-menu capture background RomFS overlay | https://github.com/ly05010419/MacNativeGames @ `ebe621de99bfc48f3c42f2d2aa73e04fbcc6eae2` (`games/totk/runtime/game_patches.c`, `game_version.h`, `world_survey.c`, `games/totk/scripts/make_romfs_patches.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `runtime/frame_rate.c`, `recomp/patches-v121.txt`, `tools/make_romfs_patches.py` | +| MacNativeGames TotK 1.4.3 fade fix: the four distance / LOD and pre-actor fade sites, their instruction words and the floor of the fade step at 1 | https://github.com/ly05010419/MacNativeGames @ `3e9e7f67fdfdd3404134dcfa0f24969a9425a33f` (`games/totk/runtime/game_patches_v143.c` patch_fade_increment), Copyright (c) 2026 MacNativeGames contributors | MIT | `recomp/patches-v143.txt` (set fade; the floor without flags, clz >> 5, as in pull request #78) | +| MacNativeGames TotK aspect ratio: the camera and viewport sites that hand the perspective projection its aspect ratio, and the UI layout edits for a canvas wider than 16:9 | https://github.com/ly05010419/MacNativeGames @ `933b6a0d1438de160208a0a13eb5e8d7a2cb8094` (`games/totk/runtime/aspect_patch.h`, `aspect_math.h`, `game_patches_v143.c`, `games/totk/scripts/ultrawide_layout.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `recomp/patches-v143.txt` (set aspect), `tools/ultrawide_layout.py`, `tools/make_romfs_patches.py` (--aspect) | +| TOTK-Ultrawide, the community mod for emulators: the list of UI panes the ultrawide layout edits and the distances they move (data only, no code copied; the edit code is our own, and MacNativeGames' `ultrawide_layout.py` in the row above already carries the same values). The mod's repository URL and licence version are not recorded in this repository | TOTK-Ultrawide (community mod; URL not recorded here) | GPL (version not recorded here) | `tools/ultrawide_layout.py` (`KEEP_ROOT`, `edits`, `anim_edits`) | | MacNativeGames TotK virtual-core preemption: asynchronous preemption of an owner in top-level game code, priority inheritance through guest mutexes, the SDK's preemption-disable protocol (TLR+0x100/+0x102, SVC 0x36), the core-wait diagnostics | https://github.com/ly05010419/MacNativeGames @ `33e8fa83052ea1ae09a3583137008e9797e12950` (`games/totk/runtime/threads.c`, commits 8bf0893 and 19730cd), Copyright (c) 2026 MacNativeGames contributors | MIT | `runtime/kernel.c` (SuspendThread and a register-saving stub replace SIGUSR2), `runtime/hostfn.c`, `runtime/svc.c` (host depth) | | MacNativeGames RomFS index builder (`TOTKRFS1` layout) | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/scripts/prepare_romfs_backend.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `tools/romfs_index.py` | | d2r-mobile host thread environment (TLR, TLS, thread object), the origin of MacNativeGames' thread layer; d2r-mobile NVN host object layouts, command record format and NVN front end, as adapted for TotK in MacNativeGames' `d2r_gfx` copy | https://github.com/jhaizhou-ops/d2r-mobile @ `53e7c5bb9ca3ea53e2160005be8fbe920cd9cb53` (`dm/host/thread.c`, `dm/host/hle_thread.c`); the NVN layouts and front end via https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/third_party/d2r_gfx/gfx/nvn/nvn_obj.h`, the two types of `nvn_tex.h`, `hle_nvn.c`, `totk_nvn_ext.c`, `nvn_impl.h`, `nvn_hmap.h`, `nvn_pool_index.h`, the request checks and mapping semantics of `totk_vpool.c`, the generation stamps of `nvn_dirty.c`, texture mirroring and texel copies of `nvn_mtl.m`; licence in `games/totk/third_party/d2r_gfx/LICENSE`), Copyright (c) 2026 jhaizhou | MIT | `runtime/kernel.c`; `runtime/nvn_obj.h`; `runtime/nvn_api.h`, `runtime/nvn_api_core.c`, `runtime/nvn_hmap.h`, `runtime/nvn_pool_index.h`; `runtime/nvn_api_obj.c` (also `nvn_pace.h`); `runtime/nvn_api_cmd.c`, `runtime/nvn_ext.c`; `runtime/vk_memory.c`, `runtime/vk_texture.c`; `runtime/vk_sync.c` (side effects written after GPU completion, `hle_nvn.c`); `runtime/vk_pipeline.c` (pipeline key and state mapping of `nvn_mtl.m` `pso_key_fill` / `pso_desc`, its `vfmt` vertex formats and the missing-varying fix-up of `msl_varyings`); `runtime/vk_exec.c`, `runtime/vk_exec.h` (op loop, pure-state ops and state defaults of `nvn_mtl.m` `mtl_exec` / `state_op` / `state_reset`); `runtime/vk_exec_xfer.c` (copy, clear, event, counter and conditional-render semantics of `nvn_mtl.m` `copy_t2b` / `copy_b2t` / `copy_t2t` / `clear_tex` / `cond_eval`); `runtime/vk_exec_draw.c` (bind ops, draw and dispatch flow, vertex streams of `nvn_mtl.m` `state_op` / `bind_vertex_streams` / `draw_inner` / `dispatch`) | @@ -71,6 +74,27 @@ These tools run on the developer's or player's machine during the build. They ar | Vulkan-Headers and Vulkan-Loader 1.4.363.0 (MSYS2 packages; headers and `libvulkan-1.dll.a` only, no loader DLL) | https://github.com/KhronosGroup | Apache-2.0 OR MIT | `runtime/deps/` | | Vulkan Memory Allocator 3.4.0 (MSYS2 package), header | https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator | MIT | `runtime/deps/` | | Ninja 1.13.2 (`ninja-win.zip`) | https://github.com/ninja-build/ninja/releases/tag/v1.13.2 | Apache-2.0 | `bin/ninja.exe` | -| hactool (SciresM @ `1d64a83450e025622f3468c28fc4164dad2c5ef6` plus `tools/hactool/portable.patch`, from MacNativeGames' `build_hactool()`: `MNG_TITLE_KEYS` and relaxed BKTR layout checks; our `tools/hactool/unicode.patch`: an embedded manifest that makes UTF-8 the process code page), built by `tools/build_hactool.py` with MSYS2 UCRT64 gcc; imports only KERNEL32 and the UCRT | https://github.com/SciresM/hactool | ISC | `bin/hactool.exe` | +| hactool (SciresM @ `1d64a83450e025622f3468c28fc4164dad2c5ef6` plus `tools/hactool/portable.patch`, from MacNativeGames' `build_hactool()`: `MNG_TITLE_KEYS` and relaxed BKTR layout checks; our `tools/hactool/unicode.patch`: an embedded manifest that makes UTF-8 the process code page; our `tools/hactool/mbedtls-asm-clobbers.patch`: the clobbers the bundled mbedtls's x86-64 multiply asm lacks), built by `tools/build_hactool.py` with MSYS2 UCRT64 gcc; imports only KERNEL32 and the UCRT | https://github.com/SciresM/hactool | ISC | `bin/hactool.exe` | | `totk_xlate.exe` (Ryujinx shader translator, NativeAOT build of `tools/xlate/`, see the Ryujinx row above) | built by `tools/build_xlate.py` | MIT | `bin/totk_xlate.exe` | | MacNativeGames shader variant list | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/assets/shader_list.tsv`), Copyright (c) 2026 MacNativeGames contributors | MIT | `data/shader_list.tsv` | + +## Distributed in the Linux release (AppImage) + +`tools/make_release_linux.py` assembles these into the folder of `docs/installer.md` ("Linux release") and packs it as +`TotK-Setup-x86_64.AppImage`. Versions and sha256 of every download are pinned in that script; licence texts are in the +folder's `licenses/` and in each component's own folder. + +| Component | Source | License | Where in the release | +|---|---|---|---| +| CPython 3.14.10, python-build-standalone 20261026 `install_only` build, with `tkinter` and the libraries it bundles (OpenSSL, libffi, SQLite, zlib, Tcl/Tk and others; their licences ship in the folder) | https://github.com/astral-sh/python-build-standalone/releases/tag/20261026 | PSF-2.0; bundled libraries under their own licences | `python/` | +| numpy 2.5.4 (with the libraries its manylinux wheel bundles), lz4 4.4.5, cryptography 50.0.2, cffi 2.1.1, pycparser 3.11 | PyPI binary wheels, pins in `tools/requirements-installer.txt` | as in the installer release row above | `python/lib/python3.14/site-packages/` | +| LLVM 23.1.4 official Linux X64 release: `clang`, `lld`, `llvm-ar`, `llvm-ranlib`, `llvm-objcopy`, `llvm-strip` and the clang resource headers; `llvm-LICENSE.txt` from the `llvmorg-23.1.4` source tag | https://github.com/llvm/llvm-project/releases/tag/llvmorg-23.1.4 | Apache-2.0 with LLVM exception | `llvm/` | +| Steam Runtime 3 "sniper" SDK sysroot 3.0.20261021.266025 (Debian 11 based), only the headers, libraries and objects a game build reads: glibc 2.31 (headers, `crt*.o`, `libc_nonshared.a`, link libraries), GCC 10 `libstdc++.a` and `libgcc`, SDL 3.4.18, libopus, libvpx 1.9 (static), the Vulkan loader's link library; their Debian `copyright` files go into `licenses/` | https://repo.steampowered.com/steamrt-images-sniper/snapshots/3.0.20261021.266025/ | glibc LGPL-2.1+; libstdc++ and libgcc GPL-3.0 with the GCC Runtime Library Exception; SDL zlib; libopus BSD-3-Clause; libvpx BSD-3-Clause with the WebM patent grant (`libvpx-PATENTS.txt` from the v1.17.0 source tag); Vulkan loader Apache-2.0 | `llvm/sysroot/`, `bin/libSDL3.so.0` (the sysroot's, copied next to the game by the setup), `licenses/` | +| Vulkan-Headers 1.4.363 (source tag) | https://github.com/KhronosGroup/Vulkan-Headers/tree/v1.4.363 | Apache-2.0 OR MIT | `runtime/deps/include/` | +| Vulkan Memory Allocator 3.4.0, header (the licence text is its header comment) | https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator/tree/v3.4.0 | MIT | `runtime/deps/include/vma/` | +| Zstandard 1.5.7, built from the release tarball into a static `libzstd.a` with the folder's clang and sysroot | https://github.com/facebook/zstd/releases/tag/v1.5.7 | BSD-3-Clause | `runtime/deps/` | +| Ninja 1.13.2 (`ninja-linux.zip`) | https://github.com/ninja-build/ninja/releases/tag/v1.13.2 | Apache-2.0 | `bin/ninja` | +| hactool (as in the installer release row above), built by `tools/build_hactool.py --cc` with the folder's clang and sysroot; includes its bundled mbedtls | https://github.com/SciresM/hactool | ISC; mbedtls Apache-2.0 | `bin/hactool` | +| `totk_xlate` (NativeAOT linux-x64 build of `tools/xlate/`, linked with the folder's clang, lld and sysroot) | built by `tools/build_xlate.py --llvm --sysroot` | MIT | `bin/totk_xlate` | +| AppImage type-2 runtime (static; includes libfuse and squashfuse), the AppImage's first part; appimagetool packs the folder and is not distributed | https://github.com/AppImage/type2-runtime | MIT; libfuse LGPL-2.1; squashfuse BSD-2-Clause | the AppImage header | +| MacNativeGames shader variant list | as in the installer release row above | MIT | `data/shader_list.tsv` | diff --git a/VERSION b/VERSION index c946ee61..1a030947 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.1.6 +0.1.9 diff --git a/docs/architecture.md b/docs/architecture.md index af1c6605..2b65a3e1 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -82,7 +82,7 @@ Every guest thread is a host thread. TotK assumes Horizon's scheduler: three app - Vulkan 1.3 back end in place of MacNativeGames' Metal one. At `QueueSubmitCommands` the recorded NVN commands are replayed into Vulkan command buffers (`vk_exec*.c`). - Memory pools. Apple's unified memory let MacNativeGames use the game's pools zero-copy. The back end imports the pools as host memory (`VK_EXT_external_memory_host`). On a discrete GPU, pools bound as buffers are mirrored in device-local memory and the pages the CPU wrote (write watch, above) are uploaded before a replay; idle pool regions are made read-only so the write-watch queries skip them (`vk_memory.c`). GPU-rendered textures are written back when the CPU reads them: on a pool invalidate, on a CPU copy, and for small targets once the CPU waits for a fence (`vk_texture.c`). - Replay thread. `TOTK_REPLAY_ASYNC=1` moves the replay off the submitting thread onto a queue at most one frame behind the game; reads of GPU results wait for the newest writer instead of draining the queue (`vk_backend.c`). It is off by default because no timing has shown a gain yet. -- Recorder thread. The Vulkan calls the replay makes into its command buffer are made by a thread of their own (`vk_rec.c`, after Savage Wind Recomp and LAS Recomp): each `vkCmd*` copies its arguments into a packet of a ring and the recorder makes the calls in order; calls into other command buffers are made at once, and the submit waits until the ring is empty. In the town on an RX 9070 the replay takes 17% less time on the submitting thread. `TOTK_VK_RECORDER=0` makes every call on the submitting thread. +- Recorder thread. The Vulkan calls the replay makes into its command buffer are made by a thread of their own (`vk_rec.c`, after Savage Wind Recomp and LAS Recomp): each `vkCmd*` copies its arguments into a packet of a ring and the recorder makes the calls in order; calls into other command buffers are made at once; the recorder also ends the command buffer, and the submit waits until it has. In the town on an RX 9070 the replay takes 17% less time on the submitting thread. `TOTK_VK_RECORDER=0` makes every call on the submitting thread. - Uniform snapshot. The game rewrites per-object uniform blocks on other threads soon after it submits the frame that reads them, while the replay still records it. At the start of a submission's replay every uniform range it binds from CPU-written pools is copied, and the draws bind the copy (`vk_usnap.c`, after MacNativeGames' @@ -93,7 +93,15 @@ Every guest thread is a host thread. TotK assumes Horizon's scheduler: three app - Resolution. Patch sets make the game read its output, swapchain and scene sizes from the patch page (MacNativeGames' sites); `resolution.c` sets them from `output_res` and `render_scale` and grows the texture heap and the application pool to match. When the window is larger than the output, `vk_fsr.c` runs AMD FSR 1 (EASU, - RCAS) on the present. + RCAS) on the present. An `output_res` that is not 16:9 (more than 1% off, `res_is_16x9`) also switches on the + `aspect` patch set (1.4.3 only), which hands the output's ratio to the camera, and `romfs.c` `overlay_init` loads the + `romfs-patches-x` overlay for it when it exists (`tools/make_romfs_patches.py --aspect`, laid out by + `tools/ultrawide_layout.py`; the installer builds ten sizes: 21:9 2560x1080, 3440x1440, 3840x1600, 5120x2160, 32:9 + 3840x1080, 5120x1440 and 16:10 1280x800, 1680x1050, 1920x1200, 2560x1600). A wider layout moves the HUD to the + screen's corners, a narrower one to its top and bottom edges. Other sizes get the camera of their shape and the 16:9 + HUD stretched over it. 1.2.1 has no camera patch: a size of another shape there keeps the 16:9 view. The `aspect` setting (`fit`, the default; `stretch`) decides what the present does + when the window's shape differs from the output's: `fit` letterboxes, where a 16:9 game in a wider window used to be + stretched. - Occlusion queries never block the CPU; a result that is not ready counts as visible. - ASTC textures are uploaded as they are where the device reports `textureCompressionASTC_LDR`, and decoded by a compute shader everywhere else (`vk_astc.c`, `TOTK_ASTC`). The CPU decoder (`tex_astc*.c`) is the fallback and the test oracle. - Draw state. A draw whose program, bound state and textures repeat reuses the previous draw's shaders and skips rebuilding its descriptor sets; descriptor sets are written through update templates (`vk_exec_draw.c`, `vk_pipeline.c`). diff --git a/docs/building.md b/docs/building.md index 30ebfd21..e7d48480 100644 --- a/docs/building.md +++ b/docs/building.md @@ -37,13 +37,15 @@ PATH: ### hactool -`tools/build_hactool.py` fetches SciresM/hactool at `1d64a83` into `toolchain\src\hactool`, applies the two patches in +`tools/build_hactool.py` fetches SciresM/hactool at `1d64a83` into `toolchain\src\hactool`, applies the three patches in `tools/hactool/` and builds a static `toolchain\hactool\hactool.exe` with MSYS2 UCRT64 gcc (a few seconds): - `portable.patch`: the parts of MacNativeGames' `build_hactool()` that run on Windows, the `MNG_TITLE_KEYS` variable and the two BKTR fixes in `nca.c` for the subsection table padding and the subsection layout of newer cumulative updates; - `unicode.patch`: a manifest with `UTF-8`, so paths under a non-ASCII user profile - reach it intact. + reach it intact; +- `mbedtls-asm-clobbers.patch`: the `memory` and `cc` clobbers the bundled mbedtls 2.6.1's x86-64 multiply asm lacks. + Without them a clang build (the Linux release) fails every RSA signature check and the update merge stops. hactool reads `-k `; MSYS2 shells set `HOME`, which hides its default key location, so pass `-k` there. @@ -78,7 +80,8 @@ From the repository root, in PowerShell. Every step that depends on the game ver .venv\Scripts\python tools\build_xlate.py ``` -5. Lift the modules to C, write the RomFS half of the frame-rate patches, then compile and link `build\\totk.exe`: +5. Lift the modules to C, write the RomFS overlay (the frame-rate fix), then compile and link + `build\\totk.exe`: ```powershell .venv\Scripts\python recomp\gen.py --exefs build\extracted\\exefs --out generated\ --patches recomp\patches-.txt @@ -87,8 +90,19 @@ From the repository root, in PowerShell. Every step that depends on the game ver ``` Lifting takes about 15 s, a clean compile about 200 s on 24 threads; later builds recompile only what changed. + A compiler cache in front of clang shortens it further: with `SCCACHE` set to an sccache executable, `build_game.py` + writes it into the ninja file ahead of `cc`, and every lift rewrites the generated chunks with the same contents, so + a second build of the same version hits all of their objects: + + ```powershell + $env:SCCACHE = "path\to\sccache.exe" + .venv\Scripts\python tools\build_game.py --version + ``` `--patches` builds in the frame-rate, resolution and other patch sets, switchable at run time. The link copies `SDL3.dll`, `libiconv-2.dll` and `libopus-0.dll` next to the exe. + For Xbox button prompts, run `make_romfs_patches.py` once more with `--xbox-prompts`: it writes + `build\\romfs-patches-xbox` ([controls.md](controls.md), "Xbox button prompts"); `build_game.py` copies it next + to `totk.exe` like `romfs-patches`. 6. Prepare the shaders (once per build folder and translator). It needs `refs\MacNativeGames` (`tools\fetch_refs.sh`) for the shared shader list: @@ -101,7 +115,8 @@ From the repository root, in PowerShell. Every step that depends on the game ver The AMD driver keeps its pipeline cache per exe path and holds a limited amount: preparing another build folder can evict this one's. `totk.exe --warm-shaders` rebuilds it from the shader cache. -7. Run. Keep the window in the foreground; an unfocused window stalls the game. +7. Run. The game keeps running when its window loses the focus (`assume_focus = 1` in `totk.ini`, the default; the + Game tab switches it). With 0 it pauses, as it does on the Switch when Home is pressed. ```powershell build\\totk.exe @@ -117,6 +132,48 @@ your machines without MSYS2, Python or .NET. `romfs.bin` stays out unless you pa `romfs\` next to the exe or point `TOTK_ROMFS` at its folder. Run `totk.exe --prepare-shaders` once in the copied folder. The folder holds code and data derived from your game: for your own machines only. +## Linux (in progress) + +The runtime builds and runs on x86-64 Linux with a window, input and menu on SDL3 (`*_sdl.c`) in place of the Win32 +files (`*_win.c`); `runtime/platform/linux.c` implements `platform.h`. Status: the selftest and the probes pass; 1.2.1 +plays at 60 fps on RADV (RX 7800 XT) and NVIDIA (RTX 3060); 1.4.3 builds, probes and plays on RADV (RX 6600, and the +Vega 7 of a Ryzen 5 4600G at about 34 fps). Long play sessions and the Steam Deck itself are not verified yet. + +Requirements: + +- Linux 6.7 or later: the write watch uses userfaultfd write protection with `PAGEMAP_SCAN`. SteamOS 3.8 qualifies. +- A Vulkan 1.3 driver with `VK_EXT_external_memory_host` (RADV, NVIDIA). +- Packages on Arch: + + ```sh + pacman -S --needed clang lld ninja make sdl3 opus libvpx zstd vulkan-headers vulkan-icd-loader \ + vulkan-validation-layers shaderc spirv-tools dotnet-sdk uv git + ``` + + Vulkan Memory Allocator is a single header (AUR `vulkan-memory-allocator`). Without it, `build_game.py` fetches the + version the Windows release pins (3.4.0) into `toolchain/vma/` on first use and checks its hash. + +The steps are the Windows ones with POSIX paths: `.venv/bin/python` in place of `.venv\Scripts\python`, and `totk` in +place of `totk.exe`. `build_game.py` takes clang, lld and Ninja from PATH (or the ones in `/bin` with +`--toolchain `) and links against the system libraries, with `libstdc++` static and an `$ORIGIN` rpath; +`build_hactool.py` builds `hactool` with make. `tools/prepare_shaders.py` reads the shader list from +`refs/MacNativeGames` at `origin/HEAD`, which must be the pinned commit with its history (the list format changed later). + +The probes open a window, so from an SSH session they need the desktop session's display, or they stop at `gpu 0 +skipped: no graphics+compute+present queue family`: on Wayland `XDG_RUNTIME_DIR=/run/user/$(id -u) +WAYLAND_DISPLAY=wayland-1 SDL_VIDEODRIVER=wayland` (the socket name is in `$XDG_RUNTIME_DIR`), on X11 `DISPLAY=:0`. + +Memory: on Windows the GPU imports guest memory and `GetWriteWatch` tracks the CPU's writes to it. On Linux, +re-protecting imported memory invalidates the driver's userptr buffer on every submit (about 1.3 ms per 16 MiB on +amdgpu), and importing pins the pages, which drops their write protection. Each CPU-visible pool therefore gets a +host-visible copy (shadow storage, `runtime/vk_memory.c`): the pages the CPU wrote go in before each submit, and the GPU's +writes come back at its fence. In play that costs about 0.2 ms a frame and doubles the memory of those pools (about +1.1 GB). `TOTK_SHADOW=1` turns it on under Windows too. The Steam Deck's RADV has `VK_EXT_external_memory_host` as +well: the shadow copies are chosen for the measured cost above, not for a missing extension. + +The Deck's GPU (Van Gogh) has no native ASTC, so its textures go through the compute decoder (`runtime/vk_astc.c`, +`TOTK_ASTC`) as on desktop AMD GPUs without it; frame times measured there include it. + ## Checks Before a commit: @@ -139,7 +196,9 @@ change passes these, all run with `.venv`: | `tests\jumptables_check.py` | `recomp/jumptables.py` against a Unicorn oracle; must report 0 disagreements, on 1.4.3 too | yes | Runtime probes, each a mode of `totk.exe` (the list is at the top of `runtime/main.c`): `--probe-kernel`, -`--probe-vkexec`, `--probe-nvn-tex` and `--probe-shader` cover most runtime changes. +`--probe-vkexec`, `--probe-nvn-tex` and `--probe-shader` cover most runtime changes. `--probe-shader` and the draw +part of `--probe-vkexec` translate shaders of the census below: they look for `build//shaders/jobs.json` of the +exe's own version first, then of the other one. Shaders: `.venv-py314\Scripts\python tools\shader_census.py --version --translate` extracts every Maxwell shader to `build\\shaders\` and runs it through the translator and `spirv-val`; failure groups go to `failures.txt`. diff --git a/docs/controls.md b/docs/controls.md index 00f5a5d1..3cd4082c 100644 --- a/docs/controls.md +++ b/docs/controls.md @@ -7,8 +7,9 @@ Input comes from the first SDL3 gamepad, the keyboard and the mouse, merged into - `nintendo`: by position, as on a Switch (the table below). - `xbox`: south A, east B, west X, north Y, as an Xbox pad labels them; on a PlayStation pad cross is A, circle B, square X, triangle Y. - `auto` (default): `nintendo` for a pad whose east button SDL reports labelled A (Switch Pro Controller, Nintendo-layout pads), `xbox` for the others. +- `xbox_prompts`: the buttons as `nintendo`, and the game's prompts name the Xbox labels (see Xbox button prompts below). -The log names the layout in effect (`input: gamepad layout ...`); the menu shows it under *Automatic*. The game's button prompts stay Switch letters either way. +The log names the layout in effect (`input: gamepad layout ...`); the menu shows it under *Automatic*. With `nintendo`, `xbox` and `auto` the game's button prompts stay Switch letters; `xbox_prompts` changes them. | Switch | Gamepad (by position, Nintendo layout) | Keyboard and mouse | Action in the world | |---|---|---|---| @@ -30,12 +31,25 @@ The log names the layout in effect (`input: gamepad layout ...`); the menu shows Esc is unbound in the game; it opens the settings menu and releases the mouse cursor (see Mouse camera). Z and X duplicate ZL and ZR so menu pages turn with two neighbouring keys; the menu tabs (L, R) are Q and R. The mouse wheel is unbound. The bindings are the tables of `input_map` in `runtime/input_map.c`: the keys (platform-neutral key ids) and one gamepad table per layout (gamepad inputs by position); `runtime/input_win.c` translates Win32 virtual keys and SDL buttons. +## Xbox button prompts + +`pad_layout = xbox_prompts` (*Xbox prompts (A right)* on the Controls tab) is for an Xbox pad played by position. The buttons work as with `nintendo`: the right button, labelled B, does Switch A (interact, confirm), and the bottom one, labelled A, does Switch B (run, cancel). The letters the game shows are swapped to match the labels, A with B and X with Y, so each prompt names the button to press. The other layouts leave the prompts alone: with `xbox` the Switch letters already match the labels. + +The swapped letters are part of the game's textures, fonts and messages, so they follow the layout and not the bindings. A button remapped with a `bind_` line or on the Controls tab keeps the letter the game draws for it, and the prompt then names the wrong button. Use `xbox_prompts` with the default bindings. + +The buttons change at once, but the game reads its UI archive and fonts once at boot, so the prompts change at the next start (the menu says *prompts change after restart*). `xbox_prompts` loads `romfs-patches-xbox` next to `romfs-index.bin` in place of `romfs-patches` (`runtime/romfs.c`, `overlay_init`): the same 60 fps patches, with the swap made from the game's own files by `tools/make_romfs_patches.py --xbox-prompts`, which the setup runs next to the plain one: + +- the action prompt (`DoCommand_00`) draws the letter images `Nt_KeyTexA/B/X/Y_00`, whose image data is swapped, in a diamond that names the other face buttons. The diamond's letters are text the game takes from `LayoutMsg/DoCommand_00.msbt` of each language's `Mals` archive (and the layout's own text panes behind it); both are swapped. The diamond's shape shows the button's position and stays. +- every other prompt is text in the button font (`scft/External.bfotf` in each `Font/Font*.Nin_NX_NVN.bfarc.zs`, glyphs U+E040 to U+E043); the A and B, X and Y glyphs are swapped. + +Without the folder (an install made by an older setup, or one where the setup could not prepare it and showed a warning) the menu says the files are missing, the log has `RomFS overlay: pad_layout = xbox_prompts, but ...`, and the Switch prompts stay; running the setup again on the install folder writes it. In a source build, `.venv-py314\Scripts\python tools\make_romfs_patches.py --version --xbox-prompts` writes `build\\romfs-patches-xbox`. + ## Window keys | Key | Gamepad | Action | |---|---|---| | F1 or Esc | Minus and Plus together | open or close the settings menu; the game gets no input while it is open | -| F10 | | performance overlay: off, frame rate, full panel | +| F10 | L3 + R3 (both stick clicks) | performance overlay: off, frame rate, full panel | | F11 | | borderless fullscreen on or off, saved to `totk.ini` | The Minus+Plus combo never reaches the game. In the menu the arrow keys, the D-pad or the left stick move, Enter or A selects, Q and E or L and R switch tabs, Esc or B goes back (Switch button names, as in the table above). @@ -46,7 +60,7 @@ The *Controls* tab of the settings menu lists, under the button layout, every Sw The tab writes `totk.ini` lines, one per Switch button or stick direction: `bind_