From f7d563db05eefcfc12afeb12d7490ac60dd89051 Mon Sep 17 00:00:00 2001 From: Deon Fouche Date: Sat, 31 Oct 2026 18:55:39 +1000 Subject: [PATCH 1/6] Draw Xbox button icons into the prompts overlay with make_romfs_patches.py --prompts tools/button_icons.py, adapted from MacNativeGames' make_button_icons.py, draws LB, RB, LT, RT, Menu, View and a d-pad cross into the button font of every font archive and into the button guide's L and ZL images, with the letters cut out in the game's own text fonts, and raises the resource size table for the archives that grow. --prompts xbox swaps the face buttons as --xbox-prompts did (now its older name); --prompts xbox-label keeps their letters for the xbox and auto pad layouts. The plain overlay is unchanged. tests/romfs_prompts_test.py runs both on a RomFS it builds itself. Co-Authored-By: Claude Opus 5.5 --- THIRD_PARTY_NOTICES.md | 3 +- tests/romfs_prompts_test.py | 537 +++++++++++++++++++++++++++++++ tools/button_icons.py | 319 ++++++++++++++++++ tools/make_romfs_patches.py | 233 +++++--------- tools/requirements-installer.txt | 2 + tools/requirements-py314.txt | 1 + 6 files changed, 944 insertions(+), 151 deletions(-) create mode 100644 tests/romfs_prompts_test.py create mode 100644 tools/button_icons.py diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 2d52b952..0a074358 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -11,6 +11,7 @@ Code copied or adapted from another project is listed here with its source, the | MacNativeGames TotK frame-rate patches: 60 fps present-interval controller, real-time frame step and its present sampling, hair and cloth clock, trampoline slow-motion check (instruction words and 1.2.1 sites), player position chain; quick-menu capture background RomFS overlay | https://github.com/ly05010419/MacNativeGames @ `ebe621de99bfc48f3c42f2d2aa73e04fbcc6eae2` (`games/totk/runtime/game_patches.c`, `game_version.h`, `world_survey.c`, `games/totk/scripts/make_romfs_patches.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `runtime/frame_rate.c`, `recomp/patches-v121.txt`, `tools/make_romfs_patches.py` | | MacNativeGames TotK 1.4.3 fade fix: the four distance / LOD and pre-actor fade sites, their instruction words and the floor of the fade step at 1 | https://github.com/ly05010419/MacNativeGames @ `3e9e7f67fdfdd3404134dcfa0f24969a9425a33f` (`games/totk/runtime/game_patches_v143.c` patch_fade_increment), Copyright (c) 2026 MacNativeGames contributors | MIT | `recomp/patches-v143.txt` (set fade; the floor without flags, clz >> 5, as in pull request #78) | | MacNativeGames TotK aspect ratio: the camera and viewport sites that hand the perspective projection its aspect ratio, and the UI layout edits for a canvas wider than 16:9 | https://github.com/ly05010419/MacNativeGames @ `933b6a0d1438de160208a0a13eb5e8d7a2cb8094` (`games/totk/runtime/aspect_patch.h`, `aspect_math.h`, `game_patches_v143.c`, `games/totk/scripts/ultrawide_layout.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `recomp/patches-v143.txt` (set aspect), `tools/ultrawide_layout.py`, `tools/make_romfs_patches.py` (--aspect) | +| MacNativeGames TotK button icons: the Xbox controller's shoulder buttons, triggers, Menu, View and d-pad drawn with letters from the game's own text fonts, the .bfotf key, the button guide's images and their block-linear layout, the resource size table raised for the archives that grew | https://github.com/ly05010419/MacNativeGames @ `933b6a0d1438de160208a0a13eb5e8d7a2cb8094` (`games/totk/scripts/make_button_icons.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `tools/button_icons.py`, `tools/make_romfs_patches.py` (--prompts) | | TOTK-Ultrawide, the community mod for emulators: the list of UI panes the ultrawide layout edits and the distances they move (data only, no code copied; the edit code is our own, and MacNativeGames' `ultrawide_layout.py` in the row above already carries the same values). The mod's repository URL and licence version are not recorded in this repository | TOTK-Ultrawide (community mod; URL not recorded here) | GPL (version not recorded here) | `tools/ultrawide_layout.py` (`KEEP_ROOT`, `edits`, `anim_edits`) | | MacNativeGames TotK virtual-core preemption: asynchronous preemption of an owner in top-level game code, priority inheritance through guest mutexes, the SDK's preemption-disable protocol (TLR+0x100/+0x102, SVC 0x36), the core-wait diagnostics | https://github.com/ly05010419/MacNativeGames @ `33e8fa83052ea1ae09a3583137008e9797e12950` (`games/totk/runtime/threads.c`, commits 8bf0893 and 19730cd), Copyright (c) 2026 MacNativeGames contributors | MIT | `runtime/kernel.c` (SuspendThread and a register-saving stub replace SIGUSR2), `runtime/hostfn.c`, `runtime/svc.c` (host depth) | | MacNativeGames RomFS index builder (`TOTKRFS1` layout) | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/scripts/prepare_romfs_backend.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `tools/romfs_index.py` | @@ -65,7 +66,7 @@ These tools run on the developer's or player's machine during the build. They ar | Component | Source | License | Where in the release | |---|---|---|---| | CPython 3.14.9 embeddable package, with `tkinter` and Tcl/Tk 9.0.4 taken from the python.org 3.14.9 installer's component MSI (unpacked, never run); includes `vcruntime140*.dll` (Microsoft, redistributable with Python), OpenSSL, libffi, SQLite, zlib, libtommath | https://www.python.org/downloads/release/python-3149/ | PSF-2.0 (Tcl/Tk: Tcl/Tk licence, BSD-style) | `python/` | -| numpy 2.5.4 (with its bundled OpenBLAS), lz4 4.4.5, cryptography 50.0.2 (with its bundled OpenSSL), cffi 2.1.1, pycparser 3.11 | PyPI binary wheels, pins in `tools/requirements-installer.txt` | BSD-3-Clause; lz4 BSD-3-Clause; cryptography Apache-2.0 OR BSD-3-Clause; cffi MIT; pycparser BSD-3-Clause | `python/Lib/site-packages/` | +| numpy 2.5.4 (with its bundled OpenBLAS), lz4 4.4.5, cryptography 50.0.2 (with its bundled OpenSSL), fontTools 4.68.0, cffi 2.1.1, pycparser 3.11 | PyPI binary wheels, pins in `tools/requirements-installer.txt` | BSD-3-Clause; lz4 BSD-3-Clause; cryptography Apache-2.0 OR BSD-3-Clause; fontTools MIT; cffi MIT; pycparser BSD-3-Clause | `python/Lib/site-packages/` | | llvm-mingw 20261020 (LLVM/Clang/LLD 23, mingw-w64, libc++, libunwind, compiler-rt), x86_64 subset | https://github.com/mstorsjo/llvm-mingw/releases/tag/20261020 | Apache-2.0 with LLVM exception; mingw-w64 ZPL-2.1/BSD/public domain; libc++ and libunwind Apache-2.0 with LLVM exception and MIT/UIUC; winpthreads MIT | `llvm/` | | SDL 3.4.18 upstream Windows release (`SDL3.dll`) and mingw development package (headers, `libSDL3.dll.a`) | https://github.com/libsdl-org/SDL/releases/tag/release-3.4.18 | zlib | `bin/SDL3.dll`, `runtime/deps/` | | libopus 1.6.1 (MSYS2 `mingw-w64-ucrt-x86_64-opus` 1.6.1-2; `libopus-0.dll` imports only KERNEL32 and the UCRT api-ms-win-crt DLLs), headers and import library | https://repo.msys2.org/mingw/ucrt64/ | BSD-3-Clause | `bin/libopus-0.dll`, `runtime/deps/` | diff --git a/tests/romfs_prompts_test.py b/tests/romfs_prompts_test.py new file mode 100644 index 00000000..e68b89bc --- /dev/null +++ b/tests/romfs_prompts_test.py @@ -0,0 +1,537 @@ +"""tools/make_romfs_patches.py --prompts and tools/button_icons.py on a RomFS made here, with no game files: each prompt +names the label of the Xbox button that the pad layout presses, the shoulder buttons, triggers, Plus, Minus and the +d-pad are drawn as on an Xbox controller, the resource size table grows with the fonts, every other file and member +stays byte for byte, the same input gives the same output, and a RomFS the tool does not recognise fails before it +writes anything. Both game versions: 1.2.1 (Common.Product.110, 72x72 button guide images) and 1.4.3 +(Common.Product.140, 108x108). + +Run: .venv-py314\\Scripts\\python tests\\romfs_prompts_test.py +""" +import compression.zstd as zstd +import hashlib +import importlib.util +import io +import json +import random +import struct +import subprocess +import sys +import tempfile +import unittest +import zlib +from pathlib import Path + +from fontTools.fontBuilder import FontBuilder +from fontTools.pens.recordingPen import RecordingPen +from fontTools.pens.t2CharStringPen import T2CharStringPen +from fontTools.ttLib import TTFont + +ROOT = Path(__file__).resolve().parent.parent +TOOL = ROOT / 'tools' / 'make_romfs_patches.py' +spec = importlib.util.spec_from_file_location('button_icons', ROOT / 'tools' / 'button_icons.py') +icons = importlib.util.module_from_spec(spec) +spec.loader.exec_module(icons) + +# The face button each Switch button is in a pad layout (docs/controls.md, runtime/input_map.c), the label an Xbox pad +# prints on it, and the layout each overlay is made for (runtime/input_map.c input_prompts_folder). +POSITION = {'nintendo': {'A': 'east', 'B': 'south', 'X': 'north', 'Y': 'west'}, + 'xbox': {'A': 'south', 'B': 'east', 'X': 'west', 'Y': 'north'}} +XBOX_LABEL = {'south': 'A', 'east': 'B', 'west': 'X', 'north': 'Y'} +STYLE_LAYOUT = {'xbox': 'nintendo', 'xbox-label': 'xbox'} +FACE = {'A': 0xE040, 'B': 0xE041, 'X': 0xE042, 'Y': 0xE043} +# The glyphs drawn as Xbox icons, with their advance: LB RB LT RT, Menu and View, the d-pad. +DRAWN = {0xE044: 820, 0xE045: 820, 0xE046: 760, 0xE047: 760, 0xE04B: 640, 0xE04C: 640, 0xE049: 640, + **{cp: 640 for cp in range(0xE060, 0xE066)}} + +VERSIONS = {'v121': dict(common='110', size=72, newest=121), 'v143': dict(common='140', size=108, newest=143)} +DEFAULT_FONT, REGIONAL_FONT = 'Font/Font.Nin_NX_NVN.bfarc.zs', 'Font/Font_KRko.Nin_NX_NVN.bfarc.zs' +PANES = {'T_A_00': 'A', 'T_B_00': 'B', 'T_X_00': 'X', 'T_Y_00': 'Y', 'T_Name_00': 'A'} +MESSAGES = {**PANES, 'Greeting': 'Hello'} +R8_UNORM, BC1 = 0x201, 0x1a01 +IMAGE = 16384 # an R8 square up to 128 rows, two GOBs wide, block height 16: the size of the game's key images + + +# ---- the formats, written here from their layout so the tools' readers are checked against an independent writer +def sarc(members, align=0x80): + names, sfnt, name_at = list(members), b'', {} + for name in names: + name_at[name] = len(sfnt) // 4 + sfnt += name.encode() + b'\0' + sfnt += bytes(-len(sfnt) % 4) + head = 0x14 + 12 + 16 * len(names) + 8 + len(sfnt) + start = head + -head % align + data, nodes = b'', b'' + for name in names: + data += bytes(-len(data) % align) + nodes += struct.pack('<4I', zlib.crc32(name.encode()), 0x01000000 | name_at[name], len(data), len(data) + len(members[name])) + data += members[name] + out = (b'SARC' + struct.pack(' [(frame, value)] of a FLAN made by bflan().""" + pos, out = 0x14, {} + count, table = struct.unpack_from(' (format, size, image).""" + out = bytearray(0x30) + out[0:4], out[0x20:0x24] = b'BNTX', b'NX ' + table = len(out) + out += bytes(8 * len(textures)) + struct.pack_into(' (format, size, image) of a BNTX made by bntx().""" + count, table = struct.unpack_from(' text) after a pan1 section.""" + sections = b'pan1' + struct.pack(' text).""" + labels = b''.join(bytes([len(name)]) + name.encode() + struct.pack(' size).""" + crcs = sorted((zlib.crc32(path.removesuffix('.zs').encode()), size) for path, size in entries.items()) + return b'RESTBL' + struct.pack(' (advance, draw(pen)).""" + names = {cp: f'uni{cp:04X}' for cp in glyphs} + fb = FontBuilder(1000, isTTF=False) + fb.setupGlyphOrder(['.notdef'] + list(names.values())) + fb.setupCharacterMap(names) + charstrings, metrics = {}, {} + for gname, (advance, draw) in [('.notdef', (500, lambda pen: rect(pen, 50, 0, 450, 700)))] + \ + [(names[cp], glyphs[cp]) for cp in glyphs]: + pen = T2CharStringPen(advance, None) + draw(pen) + charstrings[gname], metrics[gname] = pen.getCharString(), (advance, 0) + fb.setupCFF(name, {'FullName': name}, charstrings, {}) + fb.setupHorizontalMetrics(metrics) + fb.setupHorizontalHeader(ascent=800, descent=-200) + fb.setupNameTable({'familyName': name, 'styleName': 'Regular'}) + fb.setupOS2(sTypoAscender=800, usWinAscent=800, usWinDescent=200) + fb.setupPost() + out = io.BytesIO() + fb.save(out) + return out.getvalue() + + +# A distinct rectangle and advance for every button glyph, so a glyph that moved is found by its outline. +BUTTONS = {cp: (600 + 4 * (cp - 0xE040), lambda pen, i=cp - 0xE040: rect(pen, 20 + 5 * i, 20, 300 + 5 * i, 400 + 3 * i)) + for cp in range(0xE040, 0xE069)} +LETTERS_1 = {ord('L'): (560, polygon((60, 0), (480, 0), (480, 100), (160, 100), (160, 700), (60, 700))), + ord('B'): (560, lambda pen: [rect(pen, 60, 0, 460, 700), rect(pen, 160, 100, 360, 300, True), + rect(pen, 160, 400, 360, 600, True)]), + ord('R'): (560, lambda pen: [rect(pen, 60, 0, 460, 700), rect(pen, 160, 400, 360, 600, True)])} +LETTERS_2 = {ord('T'): (560, polygon((230, 0), (330, 0), (330, 600), (520, 600), (520, 700), (40, 700), (40, 600), (230, 600)))} + + +class Romfs: + """A RomFS (romfs.bin, index.json) holding the files make_romfs_patches.py reads, the originals kept to compare.""" + + def __init__(self, folder, version, text_fonts=True, restbl_skips=(), l_format=R8_UNORM): + v, rnd = VERSIONS[version], random.Random(version) + words = [b'Layout', b'Pane', b'Font', b'Msg', b'Texture', b'Anim', b'Group', b'Text', b'Base', b'Key'] + samples = [b' '.join(rnd.choice(words) + str(rnd.randrange(999)).encode() for _ in range(rnd.randrange(20, 80))) + for _ in range(400)] + self.dictionary = zstd.train_dict(samples, 4096) + self.common = f'UI/LayoutArchive/Common.Product.{v["common"]}.Nin_NX_NVN.blarc.zs' + self.mals = [f'Mals/EUen.Product.{v["newest"]}.sarc.zs', f'Mals/USen.Product.{v["newest"]}.sarc.zs'] + self.old_mals = 'Mals/USen.Product.110.sarc.zs' + self.restbl = f'System/Resource/ResourceSizeTable.Product.{v["newest"]}.Nin_NX_NVN.rsizetable.zs' + self.old_restbl = 'System/Resource/ResourceSizeTable.Product.100.Nin_NX_NVN.rsizetable.zs' + size = v['size'] + images = {f'Nt_KeyTex{b}_00': (R8_UNORM, size, rnd.randbytes(IMAGE)) for b in 'ABXY'} + images.update({'Nt_KeyTexL_00^z': (l_format, size, rnd.randbytes(IMAGE)), + 'Nt_KeyTexZL_00': (R8_UNORM, size, rnd.randbytes(IMAGE)), + 'Nt_KeyTexLSt_00': (R8_UNORM, size, rnd.randbytes(IMAGE))}) + self.text_fonts = [bfotf(cff_font('Text1', LETTERS_1), 0x01020304), bfotf(cff_font('Text2', LETTERS_2), 0x0a0b0c0d)] + default = {'scft/External.bfotf': bfotf(cff_font('External', BUTTONS), 0x5a17c3e9), 'fnt/rest.bin': rnd.randbytes(700)} + if text_fonts: + default.update(zip(icons.TEXT_FONTS, self.text_fonts)) + mals = sarc({'LayoutMsg/DoCommand_00.msbt': msbt(MESSAGES), 'LayoutMsg/Other_00.msbt': msbt({'T_A_00': 'A'})}) + self.contents = { # RomFS path -> contents before compression + self.common: sarc({'anim/ShortCutBG_00_In.bflan': bflan(('C_CaptureBG_00', 'C_CaptureClearBG_00', + 'P_CaptureClearBG_00', 'N_Other_00')), + 'anim/AutoBuilder_00_In.bflan': bflan(('C_CaptureBG_00',)), + 'blyt/DoCommand_00.bflyt': bflyt(PANES), + 'blyt/Title_00.bflyt': bflyt({'N_Title_00': 'Z'}), + 'timg/__Combined.bntx': bntx(images)}), + DEFAULT_FONT: sarc(default), + # The same font under another key: drawn once, stored under each archive's own key. + REGIONAL_FONT: sarc({'scft/External.bfotf': bfotf(cff_font('External', BUTTONS), 0x77777777), + 'fnt/rest.bin': rnd.randbytes(500)}), + **{path: mals for path in self.mals + [self.old_mals]}, + } + sizes = {path: len(data) + 256 for path, data in self.contents.items() if path not in restbl_skips} + self.contents[self.restbl] = restbl({**sizes, **{f'Other/{i}.bin': 1000 + i for i in range(20)}}) + self.contents[self.old_restbl] = restbl({DEFAULT_FONT: 1}) + plain = (REGIONAL_FONT, self.restbl, self.old_restbl) # stored without a dictionary + self.files = {'Pack/ZsDic.pack.zs': zstd.compress(sarc({'zs.zsdic': self.dictionary.dict_content}))} + self.files.update({path: zstd.compress(data, zstd_dict=None if path in plain else self.dictionary) + for path, data in self.contents.items()}) + self.files['Other/file.bin'] = rnd.randbytes(100) + image, entries = bytearray(), [] + for path, data in self.files.items(): + image += bytes(-len(image) % 16) + entries.append(dict(path=path, offset=len(image), size=len(data))) + image += data + folder.mkdir(parents=True) + (folder / 'romfs.bin').write_bytes(image) + (folder / 'index.json').write_text(json.dumps(dict(status='pass', files=entries))) + self.folder, self.version = folder, version + + def run(self, out, *flags): + return subprocess.run([sys.executable, '-I', str(TOOL), '--version', self.version, '--romfs', str(self.folder), + '--out', str(out), *flags], capture_output=True, text=True, encoding='utf-8') + + def read(self, out, path): + """The overlay's file, decompressed.""" + raw = (out / path).read_bytes() + return zstd.decompress(raw, zstd_dict=self.dictionary if zstd.get_frame_info(raw).dictionary_id else None) + + +def font(blob): + return TTFont(io.BytesIO(unbfotf(blob))) + + +def glyph(f, cp): + """(outline, advance) of the glyph of code point cp.""" + name = f.getBestCmap()[cp] + pen = RecordingPen() + f.getGlyphSet()[name].draw(pen) + return pen.value, f['hmtx'][name][0] + + +def manifest(out): + return {line.split('\t')[0]: line.split('\t')[1:] for line in (out / 'manifest.tsv').read_text().splitlines() + if not line.startswith('#')} + + +class PromptsTest(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.tmp = tempfile.TemporaryDirectory(prefix='totk-prompts-') + cls.base = Path(cls.tmp.name) + cls.romfs = {v: Romfs(cls.base / v / 'romfs', v) for v in VERSIONS} + cls.outs = {} + for v, romfs in cls.romfs.items(): + for style in ('plain', 'xbox', 'xbox-label'): + out = cls.base / v / style + proc = romfs.run(out, *(() if style == 'plain' else ('--prompts', style))) + assert proc.returncode == 0, proc.stderr + cls.outs[v, style] = (out, json.loads(proc.stdout.strip().splitlines()[-1])) + + @classmethod + def tearDownClass(cls): + cls.tmp.cleanup() + + def each(self, styles=('xbox', 'xbox-label')): + """(RomFS, style, overlay folder, summary) of every run of these styles.""" + return [(self.romfs[v], style, out, summary) for (v, style), (out, summary) in self.outs.items() + if style in styles] + + def test_plain_overlay_has_only_the_frame_rate_patch(self): + for romfs, _, out, summary in self.each(('plain',)): + with self.subTest(version=romfs.version, style="plain"): + self.assertEqual(set(manifest(out)), {romfs.common}) + self.assertIsNone(summary['prompts']) + keys = visibility(sarc_files(romfs.read(out, romfs.common))['anim/ShortCutBG_00_In.bflan']) + self.assertEqual(keys['C_CaptureBG_00'], [(0.0, 0), (0.0, 1)]) + self.assertEqual(keys['N_Other_00'], [(0.0, 0), (1.0, 1)]) + + def test_face_buttons_name_the_xbox_label_of_the_button_pressed(self): + for romfs, style, out, _ in self.each(): + with self.subTest(version=romfs.version, style=style): + layout = STYLE_LAYOUT[style] + label = {b: XBOX_LABEL[POSITION[layout][b]] for b in FACE} # Switch button -> the label of its button + before, after = sarc_files(romfs.contents[romfs.common]), sarc_files(romfs.read(out, romfs.common)) + old, new = bntx_images(before['timg/__Combined.bntx']), bntx_images(after['timg/__Combined.bntx']) + panes, messages = pane_texts(after['blyt/DoCommand_00.bflyt']), {} + for path in romfs.mals if layout == 'nintendo' else []: + messages[path] = message_texts(sarc_files(romfs.read(out, path))['LayoutMsg/DoCommand_00.msbt']) + original = font(sarc_files(romfs.contents[DEFAULT_FONT])['scft/External.bfotf']) + for path in (DEFAULT_FONT, REGIONAL_FONT): + drawn = font(sarc_files(romfs.read(out, path))['scft/External.bfotf']) + for b, cp in FACE.items(): + self.assertEqual(glyph(drawn, cp), glyph(original, FACE[label[b]]), f'{path}: glyph of {b}') + for b in FACE: + self.assertEqual(new[f'Nt_KeyTex{b}_00'], old[f'Nt_KeyTex{label[b]}_00'], f'action prompt image of {b}') + self.assertEqual(panes[f'T_{b}_00'], label[b], f'diamond letter of {b}') + for path, texts in messages.items(): + self.assertEqual(texts[f'T_{b}_00'], label[b], f'{path}: diamond message of {b}') + # Text that only looks like a letter pane, and every other message, stays. + self.assertEqual((panes['T_Name_00'], pane_texts(after['blyt/Title_00.bflyt'])), ('A', {'N_Title_00': 'Z'})) + for path, texts in messages.items(): + self.assertEqual((texts['T_Name_00'], texts['Greeting']), ('A', 'Hello')) + self.assertEqual(sarc_files(romfs.read(out, path))['LayoutMsg/Other_00.msbt'], msbt({'T_A_00': 'A'})) + + def test_other_buttons_are_drawn_as_xbox_icons(self): + for romfs, style, out, _ in self.each(): + with self.subTest(version=romfs.version, style=style): + original = font(sarc_files(romfs.contents[DEFAULT_FONT])['scft/External.bfotf']) + default = sarc_files(romfs.read(out, DEFAULT_FONT)) + drawn = font(default['scft/External.bfotf']) + for cp in range(0xE040, 0xE069): + outline, advance = glyph(drawn, cp) + if cp in DRAWN: + self.assertNotEqual(outline, glyph(original, cp)[0], f'U+{cp:04X} drawn') + self.assertEqual(advance, DRAWN[cp], f'U+{cp:04X} advance') + elif cp not in FACE.values(): + self.assertEqual((outline, advance), glyph(original, cp), f'U+{cp:04X} kept') + # The letters are holes in the key: LB is the key, L, and B with its two counters. + moves = sum(op == 'moveTo' for op, _ in glyph(drawn, 0xE044)[0]) + self.assertEqual(moves, 5) + self.assertEqual(sum(op == 'moveTo' for op, _ in glyph(drawn, 0xE047)[0]), 1 + 1 + 1 + 1) # RT: key, R + counter, T + self.assertEqual(sum(op == 'moveTo' for op, _ in glyph(drawn, 0xE060)[0]), 2) # d-pad up: cross, arrow + # The text fonts and the archives' other members stay byte for byte; both archives hold the same font. + before = sarc_files(romfs.contents[DEFAULT_FONT]) + self.assertEqual({k: v for k, v in default.items() if k != 'scft/External.bfotf'}, + {k: v for k, v in before.items() if k != 'scft/External.bfotf'}) + regional = sarc_files(romfs.read(out, REGIONAL_FONT)) + self.assertEqual(regional['fnt/rest.bin'], sarc_files(romfs.contents[REGIONAL_FONT])['fnt/rest.bin']) + self.assertEqual(unbfotf(regional['scft/External.bfotf']), unbfotf(default['scft/External.bfotf'])) + self.assertEqual(struct.unpack_from(' 0 and grown[REGIONAL_FONT] > 0, grown) + self.assertEqual(summary['prompts']['grown'], {p: g for p, g in grown.items() if g > 0}) + for crc, entry in original.items(): + paths = [p for p in grown if zlib.crc32(p.removesuffix('.zs').encode()) == crc and grown[p] > 0] + self.assertEqual(table[crc], entry + (grown[paths[0]] if paths else 0)) + self.assertEqual(set(table), set(original)) + + def test_manifest_lists_what_changed_with_both_hashes(self): + for romfs, style, out, _ in self.each(('plain', 'xbox', 'xbox-label')): + with self.subTest(version=romfs.version, style=style): + expect = {romfs.common} + if style != 'plain': + expect |= {DEFAULT_FONT, REGIONAL_FONT, romfs.restbl} | (set(romfs.mals) if style == 'xbox' else set()) + lines = manifest(out) + self.assertEqual(set(lines), expect) + for path, (size, sha, original) in lines.items(): + data = (out / path).read_bytes() + self.assertEqual((int(size), sha), (len(data), hashlib.sha256(data).hexdigest()), path) + self.assertEqual(original, hashlib.sha256(romfs.files[path]).hexdigest(), path) + + def test_same_input_same_output_and_the_older_flag(self): + romfs = self.romfs['v121'] + for flags in (('--prompts', 'xbox'), ('--xbox-prompts',)): + out = self.base / 'again' / '-'.join(flags) + proc = romfs.run(out, *flags) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual((out / 'manifest.tsv').read_bytes(), (self.outs['v121', 'xbox'][0] / 'manifest.tsv').read_bytes()) + + def test_unrecognised_romfs_fails_before_writing(self): + cases = {'no text fonts': (dict(text_fonts=False), icons.TEXT_FONTS[0]), + 'no size entry': (dict(restbl_skips=(REGIONAL_FONT,)), 'has no entry in the resource size table'), + 'compressed L image': (dict(l_format=BC1), 'Nt_KeyTexL_00: format')} + for i, (case, (options, message)) in enumerate(cases.items()): + with self.subTest(case): + romfs = Romfs(self.base / 'bad' / str(i) / 'romfs', 'v121', **options) + out = self.base / 'bad' / str(i) / 'out' + proc = romfs.run(out, '--prompts', 'xbox-label') + self.assertNotEqual(proc.returncode, 0) + self.assertIn(message, proc.stderr) + self.assertFalse(out.exists(), 'a failed run writes nothing') + + +if __name__ == '__main__': + unittest.main() diff --git a/tools/button_icons.py b/tools/button_icons.py new file mode 100644 index 00000000..c062080e --- /dev/null +++ b/tools/button_icons.py @@ -0,0 +1,319 @@ +# Adapted from MacNativeGames games/totk/scripts/make_button_icons.py @ 933b6a0 (MIT, Copyright (c) 2026 +# MacNativeGames contributors). +"""Xbox button icons for the game's prompts, drawn on the player's machine for tools/make_romfs_patches.py --prompts. +The outlines are our own shapes with letters cut out of them in the game's own text fonts: nothing of the game is in +the repository. + +Every prompt except the button guide in play is text in the button font, scft/External.bfotf of each +Font/Font*.Nin_NX_NVN.bfarc.zs: private-use code points from U+E040 (A, B, X, Y, then L, R, ZL, ZR, the d-pad, Plus and +Minus, the sticks). A .bfotf is an OpenType (CFF) font after an 8-byte header; from offset 4 on, every byte is XORed +with a 4-byte key, which the size in the header's second word gives away (bfotf_decode). redraw_font draws the shoulder +buttons, the triggers, Plus, Minus and the d-pad as an Xbox controller names them (LB RB, LT RT, Menu and View, a +cross), can swap the glyphs of the face buttons (A with B, X with Y), and has fontTools write the font again. The +archive then grows by a kilobyte or two, more than the game's resource size table allows it: grow_restbl raises those +entries (without it the game stops at start, MacNativeGames found). + +The button guide in play (bottom right) draws L and ZL from images of the common layout archive instead +(timg/__Combined.bntx, Nt_KeyTexL_00 and Nt_KeyTexZL_00): bntx_redraw paints the same icons into them, the size +unchanged. The sticks keep the Switch icons, and so do prompts drawn as fixed images (some tutorial animations). + +Needs fontTools (tools/requirements-installer.txt, tools/requirements-py314.txt). +""" +import io +import struct +import zlib + +from fontTools.pens.basePen import BasePen +from fontTools.pens.boundsPen import BoundsPen +from fontTools.pens.recordingPen import RecordingPen +from fontTools.pens.reverseContourPen import ReverseContourPen +from fontTools.pens.t2CharStringPen import T2CharStringPen +from fontTools.pens.transformPen import TransformPen +from fontTools.ttLib import TTFont + +FONT = 'scft/External.bfotf' +# The letters on the keys come from the text fonts of the default archive; the regional archives lack them. +TEXT_ARCHIVE = 'Font/Font.Nin_NX_NVN.bfarc.zs' +TEXT_FONTS = ('scft/ninP_RodinNTLG-B_DH_003_subset.bfotf', 'scft/nintendo_NTLG-DB_DH_002.bfotf') +# The button guide's images and the glyph each one shows. +HUD_TEXTURES = {'Nt_KeyTexL_00': 0xE044, 'Nt_KeyTexZL_00': 0xE046} + + +def xor(data, key): + """data with byte i XORed with key[i % 4].""" + pad = (key * (len(data) // 4 + 1))[:len(data)] + return (int.from_bytes(data, 'little') ^ int.from_bytes(pad, 'little')).to_bytes(len(data), 'little') + + +def bfotf_decode(blob): + """(OpenType font, key) of a .bfotf (the module docstring). The font fills the rest of the file.""" + key = xor(blob[4:8], struct.pack('>I', len(blob) - 8)) + otf = xor(blob[8:], key) + assert otf[:4] == b'OTTO', 'not a CFF OpenType font' + return otf, key + + +def bfotf_encode(otf, key, head): + """The .bfotf of otf: head (the original's first word) and the size, the rest XORed with key as bfotf_decode reads + it. The size field holds the whole font, so its length has to be a multiple of 4 (fontTools pads its tables).""" + assert len(otf) % 4 == 0, len(otf) + return head[:4] + xor(struct.pack('>I', len(otf)) + otf, key) + + +# ---- drawing, in units of the font's 1000-unit em; the round A/B/X/Y icons fill y 20..585 +Y0, Y1 = 20, 585 +CY, R = (Y0 + Y1) / 2, (Y1 - Y0) / 2 +K = 0.5523 # control point distance of a quarter circle drawn with one cubic curve, per unit of radius + + +class Glyphs: + """Outlines of letters from the game's own text fonts, drawn as holes in a shape.""" + def __init__(self, fonts): + self.fonts = fonts + + def outline(self, ch): + for font in self.fonts: + cmap = font.getBestCmap() + if ord(ch) in cmap: + glyphs = font.getGlyphSet(); g = glyphs[cmap[ord(ch)]] + rec = RecordingPen(); g.draw(rec); box = BoundsPen(glyphs); g.draw(box) + return rec, box.bounds, font['head'].unitsPerEm, g.width + raise KeyError(f'no text font has {ch!r}') + + def hole(self, pen, text, cx, cy, cap, max_width): + """text centred on (cx, cy), at most cap high and max_width wide, its contours reversed so they cut holes.""" + parts, x, ink = [], 0.0, [1e9, 1e9, -1e9, -1e9] + for ch in text: + rec, b, upm, advance = self.outline(ch); k = 1000 / upm + parts.append((rec, k, x)) + ink = [min(ink[0], x + b[0] * k), min(ink[1], b[1] * k), max(ink[2], x + b[2] * k), max(ink[3], b[3] * k)] + x += advance * k * 0.96 + width, height = ink[2] - ink[0], ink[3] - ink[1] + scale = min(cap / height, max_width / width) + ox, oy = cx - (ink[0] + width / 2) * scale, cy - (ink[1] + height / 2) * scale + reverse = ReverseContourPen(pen) + for rec, k, x in parts: + rec.replay(TransformPen(reverse, (scale * k, 0, 0, scale * k, ox + x * scale, oy))) + + +def text_glyphs(blobs): + """Glyphs over the text fonts in blobs (.bfotf files), the first that has a letter giving it.""" + return Glyphs([TTFont(io.BytesIO(bfotf_decode(blob)[0])) for blob in blobs]) + + +def circle(pen, cx, cy, r): + k = K * r + pen.moveTo((cx + r, cy)) + pen.curveTo((cx + r, cy + k), (cx + k, cy + r), (cx, cy + r)) + pen.curveTo((cx - k, cy + r), (cx - r, cy + k), (cx - r, cy)) + pen.curveTo((cx - r, cy - k), (cx - k, cy - r), (cx, cy - r)) + pen.curveTo((cx + k, cy - r), (cx + r, cy - k), (cx + r, cy)) + pen.closePath() + + +def rounded(pen, x0, y0, x1, y1, bottom, top=None): + """Counter-clockwise rounded rectangle (bottom and top corner radii).""" + top = bottom if top is None else top + kb, kt = K * bottom, K * top + pen.moveTo((x0 + bottom, y0)); pen.lineTo((x1 - bottom, y0)) + pen.curveTo((x1 - bottom + kb, y0), (x1, y0 + bottom - kb), (x1, y0 + bottom)); pen.lineTo((x1, y1 - top)) + pen.curveTo((x1, y1 - top + kt), (x1 - top + kt, y1), (x1 - top, y1)); pen.lineTo((x0 + top, y1)) + pen.curveTo((x0 + top - kt, y1), (x0, y1 - top + kt), (x0, y1 - top)); pen.lineTo((x0, y0 + bottom)) + pen.curveTo((x0, y0 + bottom - kb), (x0 + bottom - kb, y0), (x0 + bottom, y0)); pen.closePath() + + +def polygon(pen, points): + pen.moveTo(points[0]) + for p in points[1:]: + pen.lineTo(p) + pen.closePath() + + +def xbox_icons(g): + """Code point -> (advance, draw(pen)) for the buttons an Xbox controller names differently from a Switch: L R = LB + RB, ZL ZR = LT RT, Plus and Minus = Menu and View, the d-pad as a cross. g: Glyphs for the letters. The face + buttons keep the game's own glyphs (redraw_font swaps them for the Nintendo layout).""" + def bumper(label, advance=820): + def draw(pen): + rounded(pen, 40, Y0 + 70, advance - 40, Y1 - 70, 70, 150); g.hole(pen, label, advance / 2, CY, 230, advance - 200) + return advance, draw + + def trigger(label, advance=760): + def draw(pen): + rounded(pen, 60, Y0, advance - 60, Y1, 60, 230); g.hole(pen, label, advance / 2, CY - 20, 230, advance - 220) + return advance, draw + + def menu(advance=640): # three bars, as on the Menu button + def draw(pen): + circle(pen, advance / 2, CY, R); reverse = ReverseContourPen(pen) + for dy in (-115, 0, 115): + rounded(reverse, advance / 2 - 150, CY + dy - 38, advance / 2 + 150, CY + dy + 38, 38) + return advance, draw + + def view(advance=640): # two overlapping windows, as on the View button + def draw(pen): + circle(pen, advance / 2, CY, R); reverse = ReverseContourPen(pen); s = 150 + rounded(reverse, advance / 2 - s - 15, CY + 15, advance / 2 - 15, CY + s + 15, 16) + rounded(reverse, advance / 2 + 15, CY - s - 15, advance / 2 + s + 15, CY - 15, 16) + return advance, draw + + def dpad(*lit, advance=640): # a cross with an arrow cut into each direction the prompt means + def draw(pen): + cx, length, arm = advance / 2, R, R * 0.40 + polygon(pen, [(cx + length, CY - arm), (cx + length, CY + arm), (cx + arm, CY + arm), (cx + arm, CY + length), + (cx - arm, CY + length), (cx - arm, CY + arm), (cx - length, CY + arm), (cx - length, CY - arm), + (cx - arm, CY - arm), (cx - arm, CY - length), (cx + arm, CY - length), (cx + arm, CY - arm)]) + m, t, h = (length + arm) / 2, arm * 0.72, arm * 0.80 # arm centre, half base and height of the arrow + heads = {'up': [(cx - t, CY + m - h / 2), (cx, CY + m + h / 2), (cx + t, CY + m - h / 2)], + 'down': [(cx + t, CY - m + h / 2), (cx, CY - m - h / 2), (cx - t, CY - m + h / 2)], + 'left': [(cx - m + h / 2, CY - t), (cx - m - h / 2, CY), (cx - m + h / 2, CY + t)], + 'right': [(cx + m - h / 2, CY + t), (cx + m + h / 2, CY), (cx + m - h / 2, CY - t)]} + for d in lit: + polygon(pen, heads[d]) # already clockwise: a hole + return advance, draw + + return { + 0xE044: bumper('LB'), 0xE045: bumper('RB'), 0xE046: trigger('LT'), 0xE047: trigger('RT'), + 0xE04B: menu(), 0xE04C: view(), + 0xE049: dpad('up', 'down', 'left', 'right'), 0xE060: dpad('up'), 0xE061: dpad('down'), + 0xE062: dpad('left'), 0xE063: dpad('right'), 0xE064: dpad('up', 'down'), 0xE065: dpad('left', 'right'), + } + + +def redraw_font(otf, icons, swaps=()): + """The OpenType font otf with the glyphs of each (code point, code point) pair of swaps trading places (outlines and + horizontal metrics), then every code point of icons (code point -> (advance, draw)) drawn anew, written again by + fontTools. The timestamps stay, so the same input gives the same bytes.""" + font = TTFont(io.BytesIO(otf), recalcTimestamp=False) + assert font['head'].unitsPerEm == 1000, f'em of {font["head"].unitsPerEm} units, the drawings assume 1000' + cff = font['CFF '].cff + top = cff.topDictIndex[0] + assert not hasattr(top, 'ROS'), 'CID-keyed CFF' # its glyphs can use different private dictionaries + charstrings, private, hmtx, cmap = top.CharStrings, top.Private, font['hmtx'], font.getBestCmap() + nominal = getattr(private, 'nominalWidthX', 0) + for a, b in swaps: + ga, gb = cmap[a], cmap[b] + outline_a, outline_b = charstrings[ga], charstrings[gb] + charstrings[ga], charstrings[gb] = outline_b, outline_a + hmtx[ga], hmtx[gb] = hmtx[gb], hmtx[ga] + for cp, (advance, draw) in icons.items(): + pen = T2CharStringPen(advance - nominal, None); draw(pen) # the width operand is relative to nominalWidthX + charstrings[cmap[cp]] = pen.getCharString(private=private, globalSubrs=cff.GlobalSubrs) + glyphs = font.getGlyphSet() + for cp, (advance, _) in icons.items(): + box = BoundsPen(glyphs); glyphs[cmap[cp]].draw(box) + hmtx[cmap[cp]] = (advance, int(round(box.bounds[0]))) + out = io.BytesIO(); font.save(out) + return out.getvalue() + + +# ---- the button guide's images: square R8_UNORM masks (white key, black letter), one mip, block-linear, 72x72 up to +# 1.2.1 and 108x108 in 1.4.3. Each is drawn from the icon of the same Switch button, fitted into the square. +R8_UNORM, HUD_MARGIN, SUPERSAMPLE = 0x201, 4, 4 + + +def gob_offset(x, y, width_gobs, block_height): + """Byte offset of (x, y) in a Tegra block-linear surface of 1 byte per pixel (GOBs of 64 bytes x 8 rows).""" + return ((y // (8 * block_height)) * 512 * block_height * width_gobs + (x // 64) * 512 * block_height + + ((y % (8 * block_height)) // 8) * 512 + ((x % 64) // 32) * 256 + ((y % 8) // 2) * 64 + + ((x % 32) // 16) * 32 + (y % 2) * 16 + (x % 16)) + + +class PolygonPen(BasePen): + """Outlines flattened to closed polygons.""" + def __init__(self): + super().__init__(None); self.polygons = [] + + def _moveTo(self, pt): + self.polygons.append([pt]) + + def _lineTo(self, pt): + self.polygons[-1].append(pt) + + def _curveToOne(self, pt1, pt2, pt3): + p0 = self.polygons[-1][-1] + for i in range(1, 17): + t = i / 16; u = 1 - t + self.polygons[-1].append(tuple(u ** 3 * p0[k] + 3 * u * u * t * pt1[k] + 3 * u * t * t * pt2[k] + t ** 3 * pt3[k] + for k in (0, 1))) + + def _qCurveToOne(self, pt1, pt2): + p0 = self.polygons[-1][-1] + for i in range(1, 17): + t = i / 16; u = 1 - t + self.polygons[-1].append(tuple(u * u * p0[k] + 2 * u * t * pt1[k] + t * t * pt2[k] for k in (0, 1))) + + +def rasterize(draw, size): + """A size x size coverage mask (0..255, rows top down) of draw's outline, fitted into the square, nonzero winding.""" + rec = RecordingPen(); draw(rec) + box = BoundsPen(None); rec.replay(box); x0, y0, x1, y1 = box.bounds + inner = size - 2 * HUD_MARGIN; scale = inner / max(x1 - x0, y1 - y0) + ox = HUD_MARGIN + (inner - (x1 - x0) * scale) / 2 - x0 * scale + oy = HUD_MARGIN + (inner - (y1 - y0) * scale) / 2 - y0 * scale + flat = PolygonPen(); rec.replay(TransformPen(flat, (scale, 0, 0, -scale, ox, size - oy))) # font y up -> rows down + edges = [(a, b) for poly in flat.polygons for a, b in zip(poly, poly[1:] + poly[:1]) if a[1] != b[1]] + n = size * SUPERSAMPLE; cover = [0] * (size * size) + for row in range(n): + y = (row + 0.5) / SUPERSAMPLE + crossings = sorted((a[0] + (y - a[1]) * (b[0] - a[0]) / (b[1] - a[1]), 1 if b[1] > a[1] else -1) + for a, b in edges if min(a[1], b[1]) <= y < max(a[1], b[1])) + winding, start = 0, None + for x, d in crossings: + if winding == 0: + start = x + winding += d + if winding == 0: + for col in range(max(0, round(start * SUPERSAMPLE)), min(n, round(x * SUPERSAMPLE))): + cover[(row // SUPERSAMPLE) * size + col // SUPERSAMPLE] += 1 + return bytes(min(255, c * 255 // (SUPERSAMPLE * SUPERSAMPLE)) for c in cover) + + +def bntx_redraw(data, base, textures, icons): + """Paint the icon of each texture of textures (name without the ^ suffix -> code point; icons: code point -> + (advance, draw)) over its image in the BNTX at data[base:] (a bytearray), in place. Returns the names painted.""" + assert data[base:base + 4] == b'BNTX' and data[base + 0x20:base + 0x24] == b'NX ' + count, table = struct.unpack_from('.Nin_NX_NVN.rsizetable.zs, a RESTBL whose +# entries are keyed by the CRC32 of the RomFS path without .zs. The game sizes the buffer it loads a file into from its +# entry, so an archive that grew needs its entry raised by as much. +def restbl_offsets(table): + """CRC32 of a path -> offset of its size in the RESTBL table.""" + assert table[:6] == b'RESTBL', 'not a RESTBL' + count = struct.unpack_from(' bytes it grew) raised by that much.""" + at, out = restbl_offsets(table), bytearray(table) + for path, extra in growth.items(): + h = zlib.crc32(path.removesuffix('.zs').encode()) + assert h in at, f'{path} has no entry in the resource size table' + struct.pack_into('x when the game resolution is that size (runtime/romfs.c overlay_init); the camera's side of it is the patch set aspect (recomp/patches-v143.txt). ---xbox-prompts: the same overlay with the A and B, and the X and Y button prompts swapped, for pad_layout = xbox_prompts -(runtime/romfs.c overlay_init loads romfs-patches-xbox in place of romfs-patches). That layout plays an Xbox pad by -position, as nintendo does: the button that does Switch A is labelled B, and the prompts then name the label. Built from -the game's own images and glyphs, nothing added: -- the action prompt (blyt/DoCommand_00.bflyt) draws the letter images Nt_KeyTexA/B/X/Y_00 of timg/__Combined.bntx in - the Common archive: their image data is swapped (same format, size and tiling). Around it, a diamond names the - other face buttons with one-letter text panes (T_B_00, T_X_00, ...). The game fills them from the messages of the - same name in LayoutMsg/DoCommand_00.msbt of each Mals/ archive, over the layout's own text: both have - their letters swapped the same way. The diamond's shape (Nt_KeyBtnSubBase*) shows the button's position, which - stays right. -- every other prompt is text: the button glyphs U+E040 A, U+E041 B, U+E042 X, U+E043 Y of scft/External.bfotf, the - same font in each Font/Font*.Nin_NX_NVN.bfarc.zs. Their CFF outlines and horizontal metrics are swapped. +--prompts xbox | xbox-label: the same overlay with Xbox button icons in the game's prompts, for prompts = xbox +(runtime/romfs.c overlay_init loads romfs-patches-xbox or romfs-patches-xbox-label in place of romfs-patches). Drawn +from the game's own files on the player's machine, nothing added (tools/button_icons.py, which needs fontTools): +- both: L, R, ZL, ZR, Plus, Minus and the d-pad become LB, RB, LT, RT, Menu, View and a cross in the button font + (scft/External.bfotf, the same font in each Font/Font*.Nin_NX_NVN.bfarc.zs), and L and ZL in the button guide's images + (Nt_KeyTexL_00 and Nt_KeyTexZL_00 of timg/__Combined.bntx in the Common archive). The fonts grow, so the overlay also + carries the resource size table (System/Resource/ResourceSizeTable.Product....rsizetable.zs) with their entries + raised by as much. +- xbox, for pad_layout = nintendo. That layout plays an Xbox pad by position: the button that does Switch A is + labelled B, so the face buttons' prompts are swapped to name the labels, A with B and X with Y: + - the action prompt (blyt/DoCommand_00.bflyt) draws the letter images Nt_KeyTexA/B/X/Y_00 of timg/__Combined.bntx in + the Common archive: their image data is swapped (same format, size and tiling). Around it, a diamond names the + other face buttons with one-letter text panes (T_B_00, T_X_00, ...). The game fills them from the messages of the + same name in LayoutMsg/DoCommand_00.msbt of each Mals/ archive, over the layout's own text: both have + their letters swapped the same way. The diamond's shape (Nt_KeyBtnSubBase*) shows the button's position, which + stays right. + - every other prompt: the glyphs U+E040 A, U+E041 B, U+E042 X, U+E043 Y of the button font trade places. +- xbox-label, for pad_layout = xbox and auto. Those layouts put Switch A on the button labelled A (auto picks xbox for + every pad not labelled the Nintendo way), so the face buttons keep their letters; the action prompt's diamond keeps + the Switch positions (A on the right). +--xbox-prompts is the older name of --prompts xbox. Input: build/extracted//romfs (romfs.bin and index.json, verified by tools/extract.py). Output: -build//romfs-patches/ by default (romfs-patches-xbox with --xbox-prompts), and /manifest.tsv (path, size, SHA-256 of the overlay, SHA-256 of the original). Run with .venv-py314. +build//romfs-patches/ by default (romfs-patches-