From a4bb2d535577896d20ceb73afec96084cde7c402 Mon Sep 17 00:00:00 2001 From: Valiantsin Dzerakh Date: Sat, 24 Oct 2026 19:35:41 +0300 Subject: [PATCH] Document the UTF-8 manifests of hactool and totk.exe and the damaged file message Co-Authored-By: Claude Opus 5.5 --- THIRD_PARTY_NOTICES.md | 2 +- docs/installer.md | 7 +++++++ docs/toolchain-msys2-python.md | 9 +++++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index bec4fc7d..b5fc4fcf 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -50,7 +50,7 @@ These tools run on the developer's or player's machine during the build. They ar | Vulkan-Headers and Vulkan-Loader 1.4.363.0 (MSYS2 packages; headers and `libvulkan-1.dll.a` only, no loader DLL) | https://github.com/KhronosGroup | Apache-2.0 OR MIT | `runtime/deps/` | | Vulkan Memory Allocator 3.4.0 (MSYS2 package), header | https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator | MIT | `runtime/deps/` | | Ninja 1.13.2 (`ninja-win.zip`) | https://github.com/ninja-build/ninja/releases/tag/v1.13.2 | Apache-2.0 | `bin/ninja.exe` | -| hactool (SciresM @ `1d64a83450e025622f3468c28fc4164dad2c5ef6` plus MacNativeGames' `portable-patches.diff`: `MNG_TITLE_KEYS` and a relaxed BKTR size check), built with MSYS2 UCRT64 clang; imports only KERNEL32 and the UCRT | https://github.com/SciresM/hactool | ISC | `bin/hactool.exe` | +| hactool (SciresM @ `1d64a83450e025622f3468c28fc4164dad2c5ef6` plus MacNativeGames' `portable-patches.diff`: `MNG_TITLE_KEYS` and a relaxed BKTR size check; our `unicode-patches.diff`: an embedded manifest that makes UTF-8 the process code page), built with MSYS2 UCRT64 gcc; imports only KERNEL32 and the UCRT | https://github.com/SciresM/hactool | ISC | `bin/hactool.exe` | | `totk_xlate.exe` (Ryujinx shader translator, NativeAOT build of `tools/xlate/`, see the Ryujinx row above) | built by `tools/build_xlate.py` | MIT | `bin/totk_xlate.exe` | | MacNativeGames shader variant list | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/assets/shader_list.tsv`), Copyright (c) 2026 MacNativeGames contributors | MIT | `data/shader_list.tsv` | diff --git a/docs/installer.md b/docs/installer.md index c0738dea..8f240a76 100644 --- a/docs/installer.md +++ b/docs/installer.md @@ -104,6 +104,13 @@ totk.ini.example, VERSION the default ini; the release string (git describe of - The keys file path is passed to the tools; key values never appear in the UI, logs or files. - The window is a front end over the CLI (`python -m installer --base ... --update ... --keys ... --lang ... --dest ...`), which prints one JSON progress event per line. Tests and CI use the CLI. +- Paths may hold any Unicode characters (a Cyrillic user name puts the temp and install folders under it). Every exe + the setup runs or builds reads its arguments and paths as UTF-8 through an `activeCodePage` manifest: hactool + (`toolchain/hactool/unicode-patches.diff`), totk.exe (`runtime/totk.rc`), ninja (upstream). The Python steps write + their files and logs as UTF-8. +- A game file whose data fails its hash checks or ends early stops the setup with `check.input_damaged`, which names + the file and asks to copy it again (`extract.DamagedFile`). A failed merge checks the base Program NCA alone to tell + which file is damaged. - Unsigned exe: SmartScreen warns on first start. ## Later diff --git a/docs/toolchain-msys2-python.md b/docs/toolchain-msys2-python.md index fd9b5cc7..3b0b0a82 100644 --- a/docs/toolchain-msys2-python.md +++ b/docs/toolchain-msys2-python.md @@ -95,6 +95,15 @@ Applied the portable part of MacNativeGames `build_app.py:build_hactool()` (diff 2. BKTR: allow 0x200-aligned padding after the subsection table (`nca.c`). 3. BKTR: accept the subsection total-size layout of newer cumulative updates (`nca.c`). +Ours (`toolchain\hactool\unicode-patches.diff`): `hactool.manifest` with `UTF-8`, +embedded through `hactool.rc` (Makefile rule `hactool.res.o`, `windres`). hactool converts argv, `HOME` and +`USERPROFILE` from UTF-8 to UTF-16 (`filepath.c`); without the manifest they arrive in the ANSI code page, and a +non-ASCII user profile or path stops it with `Failed to convert ... to UTF-16!`. Needs Windows 10 1903 or later. + +Rebuild: in an MSYS2 UCRT64 shell (`MSYSTEM=UCRT64 C:\msys64\usr\bin\bash.exe -lc ...`) in `toolchain\src\hactool`, +`make clean && make -j8`, then copy `hactool.exe` to `toolchain\hactool\`. gcc writes temporary files to `TMP`; where +it cannot (sandboxed shells), point `TMP` and `TEMP` at a writable folder. + Not applied: the `MNG_BASE_RANGE_OFFSET/SIZE` base-file hook. It uses `funopen` (BSD/macOS only); on Windows the base NCA must be extracted to a file first.