Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
42 KiB
MacNativeGames / d2r-mobile / nxnative: research notes
Scope: what the three reference ports do, what transfers to an x86-64 Windows port of TotK, what is Apple-only. Checkouts (read-only, HEAD at time of study, 2026-10-20):
| Repo | Path | HEAD |
|---|---|---|
| MacNativeGames | refs/MacNativeGames |
b75be8c |
| d2r-mobile | refs/d2r-mobile |
53e7c5b (MacNativeGames copied its generic parts from 9f0c257) |
| nxnative | refs/nxnative |
a35e964 |
All paths below are relative to refs/MacNativeGames unless prefixed.
0. Architecture in one paragraph
Not an emulator and not a recompiler. The game's three ARM64 NSOs (main, sdk, subsdk0; rtld is dropped and the
host links) are rewritten offline so they can run natively on Apple ARM64 (games/totk/third_party/d2r/patch_nso.py).
A C host maps them at original relative layout, resolves imports itself, and replaces Horizon at two levels:
(a) kernel SVCs via a "gate" trampoline, (b) whole nn::* SDK functions overridden by mangled name at relocation time.
NVN is replaced entirely: nvnBootstrapLoader is overridden, all 523 NVN entry points route to a host implementation
that records commands into game memory and replays them on Metal 4. Shaders: Maxwell code -> Ryujinx translator ->
SPIR-V -> SPIRV-Cross -> MSL -> Metal, cached on disk, partially pre-built at app build time.
Consequence for us: the CPU side of their work (rewriter, gate) is ARM64-host-only and useless on x86-64 as code, but it
is an exact, verified census of every place game code touches the OS. Everything above the CPU boundary (SVC
semantics, nn::* overrides, NVN command model, shader pipeline, TotK patches, data formats) is reusable knowledge, and
much of it is plain C.
1. Build pipeline: XCI/NSP + prod.keys -> app
Entry: build_app.py (1028 lines). Steps documented at build_app.py:1-30 and docs/how-it-works.md:73-125.
1.1 Inputs
- Base game 1.0.0 only (
0100F2C0115B6000), XCI or NSP. NSZ/XCZ base unsupported. Personalized tickets unsupported. - Optional cumulative update NSP/NSZ via
--update: 1.1.2 (v262144), 1.2.1 (v393216), 1.4.3 (v720896). No intermediate update needed. DLC and combined packages unsupported (README.md:66-67, 204). prod.keys. NSP needs--firmware <dir of .nca>or--shared-fonts <dir>(NSP has no system fonts; XCI reads its update partition) (README.md:67, 100).- Python deps (
games/totk/requirements.txt):cryptography==50.0.2,lz4==4.4.5,capstone==5.0.9,nsz==5.0.0. Python 3.14 is required because the build uses stdlibcompression.zstd(requirements.txt:1,scripts/locate_shaders.py:24).
1.2 Code (step 1, build_app.py:384-426)
- Container detection by header: XCI = HFS0 secure partition, NSP = PFS0 (
docs/how-it-works.md:75). games/totk/scripts/extract_exefs.py: own pure-Python reader, no hactool. NCA3, key-area or common-ticket AES-CTR, hierarchical SHA-256 PFS0. Title key from ticket decrypted with titlekek fromprod.keys, in memory only (extract_exefs.py:1-5, 93-146). Ticket signature type0x10004is little-endian04 00 01 00(docs/how-it-works.md:132).- ExeFS hashes compared to
expected-<ver>.json. A differentmain.npdmis accepted if its ACID signature verifies against Nintendo's public ACID modulus (build_app.py:57-70, 146-171). scripts/prepare_modules.py-> ELF inventory;third_party/d2r/patch_nso.py <nso> <out> <name>for sdk, subsdk0, main withDM_RULESET=native DM_PRIVATE_GATES=0; output hash must equalrewritten_sha256(build_app.py:410-418).scripts/prepare_runtime.pybinds the C loader to the verified images: emitsbuild/runtime_tables.h(symbols, relocations incl. RELR, gates) (scripts/prepare_runtime.py:1-25).
1.3 Data (step 2, build_app.py:429-470, 573-597)
- Base RomFS:
scripts/extract_romfs.py, decrypts the 17 GB RomFS into one flatromfs.bin+index.json, verified through all six IVFC levels, then whole-image SHA-256 compared (build_app.py:433-441). - Update merge (BKTR): hactool, pinned
https://github.com/SciresM/hactool.git@1d64a83450e025622f3468c28fc4164dad2c5ef6(build_app.py:44).build_hactool()patches the fetched source beforemake CC=clang(build_app.py:473-570):MNG_TITLE_KEYSenv: private title-key file for both tickets, mode 0600, deleted after merge.MNG_BASE_RANGE_OFFSET/SIZE: funopen-backed bounded view of the base NCA inside the original XCI/NSP, avoiding a second 17 GB copy (Darwinfunopen; Windows needs another mechanism or a temp copy).- Two BKTR layout checks relaxed: newer updates leave sector-aligned padding after the subsection table
(
build_app.py:542-565). Output via--sectionN(not--romfs, which does not export the BKTR section) (docs/how-it-works.md:119). Driver:scripts/extract_update_romfs.py.
- Update version is chosen by exact ExeFS hashes, not file name; BKTR section marker identifies an update NCA
(
docs/how-it-works.md:117). - Shared fonts:
scripts/extract_shared_fonts.py. Six BFTTF from system data titles starting0100000000000811; BFTTF = big-endian words XORed withkey = word0 ^ 0x7f9a0218, word1 = size (extract_shared_fonts.py:19-28, 122-128). Other firmware fonts accepted if structurally valid. - RomFS overlays from the player's files (manifest: path, size, sha256 overlay, sha256 original):
scripts/make_romfs_patches.py: quick-menu capture timing (UI/LayoutArchive/Common.Product.100.Nin_NX_NVN.blarc.zsanim keys moved to frame 0) and 1.0.0 game-over retry (Event/EventFlow/DmF_SY_GameOver.bfevfl.zs) (header lines 1-25).scripts/make_output_res_overlay.py: HeapDef for 2560x1440 (see section 6.3).
1.4 Shaders and packaging (steps 3-4)
- SPIRV-Cross @
aa217aeb6c9f0ace7a0ab233b28807edf45eb165, compiled from 11 sources with plain clang, no CMake (build_app.py:49-54, 600-612). - Prebuilt
games/totk/prebuilt/macos-arm64/totk_xlatechecked againsttotk_xlate.sha256(build_app.py:615-630). scripts/prepare_shader_seed.pyextracts every shader from the verified RomFS + main module, translates default-bind variants plus everyVrow fromassets/shader_list.tsv; refuses to ship fewer than 8,000 (docs/first-launch-shaders.md). 1.4.3: 8,290 unique shaders, 24,871 runtime files, ~334 MiB seed.- Optional:
metalcompiles MSL ->.metallibper source hash (build_app.py:674);tools/warm_metal_cache.m. - Runtime binary built per version (
games/totk/Makefile:71-72, 341, 351, 361:-DTOTK_GAME_V112/V121/V143), bundled, codesigned,--checkrun.
1.5 Verification hashes (games/totk/assets/expected-v*.json)
| 1.0.0 | 1.1.2 | 1.2.1 | 1.4.3 | |
|---|---|---|---|---|
main sha256 |
faf81f8a…a3eaab |
f04fbebd…08d1ac |
e5aa8d48…efcb6 |
e84a90d5…2dc9e |
sdk |
91573d44…4eea |
same | same | 1d187f29…e4ad |
subsdk0 |
9d04c989…3f99 |
same | same | 767c35bc…79b1 |
rtld |
28317605…a44d |
same | same | 49308064…2817 |
| RomFS size | 17,069,139,640 | 17,051,988,992 | 17,052,047,360 | 17,052,625,232 |
| RomFS sha256 | 4751542f…dafcf |
d98fde08…c43 |
c2beb1c2…7c11 |
797e66f2…19e9b |
Full hashes in the JSON files. Fonts identical across versions. Base card image 17,536,620,544 bytes, program NCA
9d827368992a2faafa9062aa5458b7ec.nca (expected-v1.0.0.json). 1.0.0-1.2.1 share sdk/subsdk0/rtld; 1.4.3 replaced
all three (new SDK, RELR relocs, GNU-hash-only SDK modules) (docs/how-it-works.md:125). nxnative records 1.4.3 build IDs:
main=277178b7dba1b6d4949a84778d4abc58 subsdk0=f39f4c431234c72eb520cfdd1001bbcf sdk=574284b40c06cf1821a44857939083be,
SDK 20.5.22 (refs/nxnative/dm/games/0100F2C0115B6000/game.tsv).
These hashes are directly reusable for our own input validation.
2. NSO rewriter: every place game code touches the OS
File: games/totk/third_party/d2r/patch_nso.py (1036 lines, Python + capstone), plus call_window.py, relr.py.
Verification: structck.py (structural, independent llvm-objdump recount), equiv.py (Unicorn per-site differential
with mutation negative controls).
2.1 Why
Apple's kernel clobbers x18 (platform register) and owns tpidr_el0; Horizon's TLR (tpidrro_el0), SVCs and the
system counter have no Darwin counterpart (patch_nso.py:12-22). iOS forbids writable code, so the rewrite is static and
never patched at runtime; thread-slot indices come from a read-only "key page" (patch_nso.py:21, 245-248).
2.2 What is rewritten (linear scan of .text, patch_nso.py:609-619)
| Instruction | Encoding mask | Replacement |
|---|---|---|
svc #n |
w & 0xFFE0001F == 0xD4000001 |
host gate, arg = imm16 |
mrs xN, tpidr_el0 |
0xD53BD040 |
load thread slot KTLS |
msr tpidr_el0, xN |
0xD51BD040 |
store KTLS |
mrs xN, tpidrro_el0 |
0xD53BD060 |
load KTLR (Horizon ThreadLocalRegion ptr) |
mrs xN, cntpct_el0 |
0xD53BE020 |
host gate (timer) |
mrs xN, cntvct_el0 |
0xD53BE040 |
host gate (timer) |
any x18/w18 operand |
capstone regex | load/modify/store slot K18 via scratch regs |
x18 handling variants: plain (site_plain, 331-341), conditional branch on x18 (site_cond, 343-358), adr/adrp x18
(site_pcrel, 360-369), br x18 jump tables with landing pads (site_br, 371-391; 1.4.3 adds signed 32-bit offset
tables at five sites, SIGNED_TABLES 631-637 keyed by build ID 277178b7…), blr x18 call windows (call_window.py,
1.0.0 main+0x9c9858), LDXR/STXR exclusive windows containing x18 copied whole into the patch area so no stores land
inside the reservation (find_windows/site_window, 457-582). Function bounds from .eh_frame_hdr (584-605).
Local TotK fixes vs upstream: jump-table look-back 8 -> 32 instructions for main+0x590574 (1.0.0)
(third_party/d2r/README.md, patch_nso.py:649); exclusive window fallthrough exit fix for main+0x72afdc,
main+0x7aa630 (patch_nso.py:560-562).
Not rewritten in code but handled at runtime: mrs CTR_EL0 and dc civac trap on Darwin and are emulated in the
SIGILL handler, SDK only (games/totk/runtime/loader.c:370-374).
The CUTS, OBS, PARAMS tables in the copied rewriter are D2R-only (keyed by D2R build ID db0507f3…,
patch_nso.py:54-167, 916-928); TotK uses none of them. TotK game patches live in the runtime (section 6).
2.3 Counts (OS touch-points)
- 1.2.1 main: 31,475 rewrite sites; sdk: 100 SVC/timer gates (
docs/v1.2.1-port.md). sdk has 33cntpctsites, 14 with destination x19..x23 (runtime/gate.Scomment at the x0..x29 restore). - 1.1.2: 55,018 site checks across modules (
docs/how-it-works.md:130). 1.4.3: 68,968 semantic sites, 554,281 relocations applied at load (docs/v1.4.3-port.md:20-21). - Game
mainitself makes almost no SVCs; the SVCs live insdk.maintouches the OS mostly throughx18(compiler uses it as a general register), TLS (tpidr_el0) and importednn::*/nvn*functions.
2.4 Implication for an x86-64 recompiler
x18is an ordinary GPR in Nintendo's ABI: a recompiler must model it as a normal register; no special handling.tpidr_el0(user TLS, written by the game) andtpidrro_el0(TLR, read-only) need per-guest-thread storage; TLR layout: host allocates 0x200 bytes TLR, 8 KiB TLS, and stores a dummy pointer at TLR+0x1F8 (loader.c:277-290). IPC message buffer is in the TLR (first 0x100 bytes,runtime/ipc.c:16).cntpct_el0/cntvct_el0must return 19.2 MHz ticks: host computesns * 12 / 625(loader.c:299-300).- Exclusive load/store pairs (LDXR/STXR, LDAXR/STLXR) appear in both game and SDK; a recompiler must lower them to host CAS or equivalent with matching semantics.
- Jump tables via
br x18with byte/halfword/signed-32 offsets exist; static recompilation must recover them (the rewriter's recovery logic and the 1.4.3 table list are a ready reference). - The SVC list in section 3.2 is the complete kernel surface actually used.
3. Horizon host
Files: games/totk/runtime/ (9,856 lines total). Generic parts adapted from d2r-mobile dm/host.
3.1 Entry and gate
runtime/gate.S(124 lines, ARM64 asm): switches to the per-thread host stack, saves x0-x30, q0-q31, NZCV, FPSR, FPCR into an 800-byte frame, callsruntime_dispatch(return_pc, regs), restores (x18 skipped).runtime_dispatch(loader.c:296-304): finds the gate by return address in<module>.gates.tsv; timer gates write 19.2 MHz ticks to the target register; SVC gates callruntime_svc(services.c:304).- Host functions handed to guest code run on the thread's host stack (
runtime/hle_stack.c,hle_stack.S, 4096 thunks).
3.2 SVCs implemented (complete list)
runtime/services.c:304-382, runtime/threads.c:1-20, 354-560, runtime/ipc.c:169-210. Unknown SVC = fatal.
| SVC | Name | Where | Notes |
|---|---|---|---|
| 0x01 | SetHeapSize | services.c:369 | 2 MiB granularity, cap 4 GiB; 1.4.3 sets ~2.2 GiB in nninitStartup |
| 0x02 | SetMemoryPermission | services.c:315 -> loader.c:66 | RELRO on module data, 4 KiB logical on 16 KiB host pages |
| 0x03 | SetMemoryAttribute | loader.c:91 | only the RELRO lock (mask=attr=16) |
| 0x04/0x05 | MapMemory/UnmapMemory | services.c:131 | stack moves; copy + protect source |
| 0x06 | QueryMemory | services.c:343 | synthesizes MemoryInfo for heap, alias, stack, modules |
| 0x08-0x0A | Create/Start/ExitThread | threads.c:354-378 | |
| 0x0B | SleepThread | threads.c:379 | 0/-1/-2 = yields |
| 0x0C/0x0D | Get/SetThreadPriority | threads.c:409-410 | prio 28..59 |
| 0x0E/0x0F | Get/SetThreadCoreMask | threads.c:411-418 | bookkeeping |
| 0x10 | GetCurrentProcessorNumber | threads.c:419 | virtual core |
| 0x11/0x12/0x17 | SignalEvent/ClearEvent/ResetSignal | threads.c:506 | |
| 0x15 | CreateTransferMemory | ipc.c:202 | |
| 0x16 | CloseHandle | threads.c:549 | |
| 0x18 | WaitSynchronization | threads.c:522 | |
| 0x19 | CancelSynchronization | threads.c:542 | |
| 0x1A/0x1B | ArbitrateLock/Unlock | threads.c:422-430 | tag = owner handle, bit 0x40000000 = waiters |
| 0x1C/0x1D | WaitProcessWideKeyAtomic/SignalProcessWideKey | threads.c:431-469 | condvars |
| 0x1F | ConnectToNamedPort | ipc.c:188 | only sm: |
| 0x21/0x22 | SendSyncRequest(+WithUserBuffer) | ipc.c:193 | in-process CMIF/HIPC/TIPC server |
| 0x25 | GetThreadId | threads.c:420 | |
| 0x26 | Break | services.c:377 | |
| 0x27 | OutputDebugString | services.c:378 | |
| 0x29 | GetInfo | services.c:317-338 | ids 0-8,10-18,20-23,28; 18 = program id 0x0100f2c0115b6000 |
| 0x2C/0x2D | MapPhysicalMemory/Unmap | services.c:339, 159-191 | alias region, 4 KiB page bitmap; LimitReached 0x10801 on pool exhaustion |
| 0x34/0x35 | WaitForAddress/SignalToAddress | threads.c:470-500 | |
| 0x45 | CreateEvent | threads.c:501 |
Semantics follow Atmosphère mesosphère (KConditionVariable, KAddressArbiter, KSynchronization) (threads.c:12).
3.3 Services
Only one real IPC service is served in-process: audren:u (runtime/audio_renderer.c:343), plus sm: itself.
Everything else is replaced at the nn::* function level by mangled-name override during relocation
(loader.c:209-228: runtime_observer for JUMP_SLOT, then runtime_override, wrapped to run on the host stack).
~163 distinct symbols across:
| Area | File | Count | Examples |
|---|---|---|---|
| oe/aoc/pctl/err/diag/fs init | services.c:289-302 |
24 | GetDesiredLanguage, GetOperationMode, GetPerformanceMode, aoc list = empty |
| fs | fs_host.c, romfs.c |
31+2 | MountRom, MountSaveData, MountCacheStorage, file/dir ops; saves = host dir |
| hid | input.c, hid_feedback.c |
16+13 | GetNpadStates (FullKey/Handheld/JoyDual/...), six-axis, vibration |
| applet/oe/nfp/err | applet_host.c |
31 | notification messages, launch params, nfp (amiibo) stubs |
| time | time_host.c |
13 | clocks, ToCalendarTime, ClockSnapshot (0xD0 layout) |
| codec | opus_codec.c |
7 | HardwareOpusDecoder -> libopus |
| movie | movie_host.c |
vtable | movie::Decoder (subsdk0 = NintendoSDK_movie) -> libvpx VP9 |
| pl | shared_font.c |
3 | shared font address/size/state |
| swkbd | swkbd_host.c |
7 | ShowKeyboard -> host text entry |
| account | account.c |
6 | preselected user, nickname |
| vi | frontend.m |
5 | OpenDefaultDisplay, CreateLayer, GetNativeWindow, vsync event |
| os | threads.c, services.c |
few | CreateSystemEvent, AllocateAddressRegion (observed) |
| nvn | hle_nvn.c |
523 | via nvnBootstrapLoader |
Startup: host calls the game's own nn::init::Start(main_thread_handle, 0, rtld_cb, module_inits, module_finis) in sdk
(loader.c:470-477); DT_INIT for each module is run from the callback (loader.c:458-462). nnMain return PC is a
version-table checkpoint.
TotK-specific vs generic: thread/sync/IPC/gate/stack code is generic (adapted from D2R). Memory layout constants,
nn::* override list (driven by what TotK main imports), audio REV11/REV15 renderer, movie decoder, fonts, swkbd hook
by PLT/GOT address are TotK-shaped. Roughly half of runtime/ is game-agnostic Horizon emulation.
3.4 Threading model
- One guest thread = one host pthread running on the guest stack the SDK allocated (
threads.c:4-14). - All waits under one global mutex
G, one condvar per thread; correctness first. - Virtual cores: TotK keys per-core allocators/job queues by
GetCurrentProcessorNumberwithout locks, so guest threads must hold one of 3 virtual application cores while running guest code; core released only where Horizon would switch (waits, sleep, yield, exit, blocking host calls).TOTK_FREE_CORES=1disables (threads.c:120-150). Critical correctness constraint for any port. - Guest threads run at
QOS_CLASS_USER_INTERACTIVE(threads.c:235).
3.5 Memory layout
runtime/services.c:18-123, docs/how-it-works.md:27-32.
- All guest memory must be below 2^39: the SDK encodes IPC buffer addresses in 39 bits (HIPC descriptor bits 36..38).
- macOS 26: alias region fixed at
0x70_0000_0000, size 64 GiB (real Horizon 39-bit size). Stack region 256 MiB, heap region below it; 6 GiB low reservation searched from0x3_0000_0000. - Fallback (macOS 27/iOS): one hole for stack+heap+alias, alias shrunk in 4 GiB steps down to 8 GiB; game uses < 3 GiB.
- Application pool reported:
0xCD500000(~3.2 GiB, retail) + extra for 2K heaps (services.c:60-63). - System resource 32 MiB; NPDM declares 24 MB system resource, so the game uses the alias region via MapPhysicalMemory
(
refs/nxnative/dm/games/0100F2C0115B6000/game.tsv). - Module images: page-aligned, main gets one extra writable page after its image for patch data (
loader.c:182-185). - Host page size 16 KiB asserted (
loader.c:489); Windows has 4 KiB pages / 64 KiB allocation granularity, which makes guest-exact 4 KiB permission tracking easier than on macOS.
3.6 TLS
Per thread: 8 KiB TLS block (tpidr_el0 target), 0x200-byte TLR (tpidrro_el0 target), dummy pointer at TLR+0x1F8,
256 KiB sigaltstack (loader.c:269-290). Slot storage = Darwin pthread keys read through tpidrro_el0 + key*8.
4. NVN layer (NVN on Metal)
Location: games/totk/third_party/d2r_gfx/gfx/nvn/ and gfx/tex/. Mapping reference:
refs/d2r-mobile/dm/gfx/nvn_metal_mapping.md (78 lines, good one-page spec).
4.1 Files and size
| File | Lines | Role | Backend |
|---|---|---|---|
hle_nvn.c |
2,727 | API front: 258 NVN(...) implementations, object semantics, command recording |
agnostic C (uses mach for VM, pthread) |
totk_nvn_ext.c |
275 | NVN 55 extras (debug labels, deferred binds, timestamps) | agnostic |
nvn_obj.h |
203 | in-place object layouts + command record format | agnostic |
include/nvn.h |
1,756 | NVN types/structs | agnostic |
nvn_protos.h, names.txt |
577, 523 | prototypes, 523 entry names | agnostic |
nvn_thunks.S, nvn_entry.S |
6,813, 79 | generated ARM64 counting thunks per entry | ARM64 asm, regenerate |
nvn_tex.c/.h |
164 | texture layout sizes (deko3d-derived, zlib) | agnostic |
nvn_dirty.c |
253 | write-protect dirty-page tracking of pool memory | OS VM API (mprotect/mach) |
totk_vpool.c |
121 | virtual memory pools via mach_vm_remap |
OS VM API |
nvn_hmap.h, nvn_pool_index.h, totk_serial_set.h, nvn_cube_group.h, nvn_pso_skey.h ... |
~1,000 | containers, pool index, PSO state keys | agnostic |
nvn_mtl.m |
13,696 | replay of command streams onto Metal 4, PSO cache, render passes, present | Metal / ObjC |
totk_mfx.m, totk_mfx_up.m |
621 | MetalFX upscale + frame interpolation | Apple-only |
totk_shader_rt.m |
279 | runtime shader translation client, cache, variants | Foundation + spawn |
nvn_refl.m |
108 | reflection JSON -> bindings | Foundation |
fps_overlay.c, totk_sysload.c |
428 | HUD via CoreText, IOKit load | Apple-only |
gfx/tex/nvn_deswizzle.c, nvn_bcdec.c, nvn_bc1astc.c |
~270 | block-linear deswizzle, BC decode, BC1->ASTC | agnostic |
gfx/tex/nvn_deswizzle_gpu.m, nvn_bctc.metal, nvn_fmt_mtl.c |
~980 | GPU deswizzle, BC transcode shaders, NVN->MTLPixelFormat | Metal |
Totals: gfx/nvn 29,295 lines (incl. 6,813 generated asm); Metal-specific ~15,000; backend-agnostic C ~6,000.
4.2 Model
hle_nvn.c:1-20, nvn_metal_mapping.md:
mainimports onlynvnBootstrapLoader; host overrides it; the NVN driver in sdk never runs. All 523 names resolve to counting thunks; unimplemented = fatal (hle_nvn.c:54-66). TotK requires NVN API 55, minor >= 15 (hle_nvn.c:153, check atmain+0xe9d614). Device integer queries return Tegra X1 values (hle_nvn.c:150-165).- GPU address = CPU address. Memory pools are game memory wrapped as zero-copy MTLBuffers (16 KiB-rounded, overlapping allowed). Buffer address -> (MTLBuffer, offset) via interval table.
- Commands recorded into the game-provided command memory in
nvn_obj.hformat; replayed atQueueSubmitCommands. CPU-visible side effects (event writes, counters, sync) happen after GPU completion. - Sync object = submit fence value;
SyncWaitreturns after the GPU reaches it (MTLSharedEvent). - Textures: block-linear, deswizzled on upload; CPU writes detected by write-protect dirty tracking and re-uploaded.
TotK streams textures by CPU write +
nvnMemoryPoolFlushMappedRange(nxnative game.tsv: tex_flush_upload). - Window: game-owned swapchain textures presented by a present thread with
WindowSetCrop. - Shader bindings: driver constant buffers c0 (SSBO table) and c2 (texture/sampler handles) are filled by the replay
layer. TotK offsets: combined texture handles c2+0x20 (32 slots x 8 B), separate textures c2+0x168, samplers
c2+0x568, images c2+0x120; c0 SSBO tables vs=0x110 ps=0x510, compute c0 0x200 (group counts) / 0x210 (UBO table),
vertex point-size limit c0+0x610 (
tools/totk_xlate/Service.cs:26-37); nxnative lists cs SSBO base 0x310 (game.tsv). - Conditional rendering in GPU order (
nvn_mtl.m:1679), occlusion queries via visibility buffer (nvn_mtl.m:724), layered passes (nvn_mtl.m:7252), store-action lookahead (nvn_mtl.m:3786-3830), PSO prediction on submit thread (docs/pipeline-prediction-stalls.md). nvn_mtl.mstill contains D2R-only paths (shop drag image8662, 3D brightening9948, world freeze10380, power-save10426): dead weight for TotK.
4.3 Retarget to Vulkan or D3D12
Reusable as-is or with small edits: hle_nvn.c, nvn_obj.h, nvn.h, nvn_protos.h, names.txt, nvn_tex.c,
gfx/tex/*.c, PSO state key logic, pool index, serial set, the c0/c2 slot conventions, reflection format.
Replace: nvn_mtl.m (13.7k lines of ObjC/Metal 4), MetalFX, CoreText overlay, nvn_dirty.c/totk_vpool.c VM calls
(VirtualProtect + vectored exception handler, MapViewOfFile3/placeholders for remap), the ARM64 thunk asm
(x86-64 or plain C function table).
Vulkan is the natural target: SPIR-V from the Ryujinx translator feeds it directly (drop SPIRV-Cross and MSL rewriting),
Ryujinx's own Vulkan backend is a working reference for Maxwell-on-Vulkan state mapping, and features Metal lacked
(quads via index emulation, depth clamp, geometry stages, subgroup ops in vertex stage, border colors) are mostly native.
Zero-copy "GPU address = CPU address" maps to VK_EXT_external_memory_host or to explicit buffer mirroring with dirty
tracking; on discrete GPUs (no unified memory) the pool-as-buffer model needs staging, which is the biggest design change.
D3D12 would need SPIR-V -> DXIL (spirv-cross HLSL or dxil-spirv); more work, no benefit.
Estimate: a Vulkan replay layer of similar scope to nvn_mtl.m (10-15k lines), reusing the front end unchanged.
5. Shader path
5.1 Translator
vendor/ryujinx: Ryujinx 1.3.2Ryujinx.Graphics.Shader,Ryujinx.Common,Spv.Generator, unmodified in-tree (vendor/ryujinx/README.md).games/totk/tools/totk_xlate/:Program.cs(215, IGpuAccessor answering texture/attr/output types from a job file, c1 constant segment needed for BRX jump tables),Service.cs(468,--serviceline protocol: JSON request -> SPIR-V -> spirv-cross MSL + reflection JSON),VertexCounterLowering.cs(61).publish_native.sh: copies vendor/ryujinx, applies patches,dotnet publish -r osx-arm64 -p:PublishAot=true. For Windows:-r win-x64. .NET SDK 10.0.401 used (prebuilt/macos-arm64/README.md).- Patches (
tools/totk_xlate/patches/), all to Ryujinx shader optimizer, all real correctness bugs, all relevant to any SPIR-V consumer:0001-keep-call-arguments-in-variables: don't propagate expressions into Call args (SPIR-V passes args by pointer).0002-keep-live-call-sources-when-removing-dead-node: DCE must not strip sources of side-effecting ops (TotK sun-visibility kernelc7830faclost its atomicMin).0003-keep-side-effect-assignments-in-the-ast-optimizer: unused-result atomics (Maxwell RED, ATOM to RZ) kept.0004-no-else-if-flattening: else-if flattening gave shared merge blocks; SPIRV-Cross then dropped the trailing else (fragment shader1870878alost loops with atomic max + discard; 12 variants lost atomics, 83 changed discards). Note: 0004 is a SPIRV-Cross interaction; check whether the Vulkan path needs it (probably harmless to keep).
- Exact-hash lowerings: visibility-counter vertex shader uses subgroup ballot + atomic; lowered to one atomic add per
passing invocation, restricted to code SHA-256
EABE9FDD…24AD(1.1.2/1.2.1) andF5F4D231…1FF7B(1.4.3) (VertexCounterLowering.cs:1-20). Needed on Metal only because vertex-stage subgroup builtins were unavailable through SPIRV-Cross; Vulkan supports subgroup ops in vertex shaders, so likely not needed. - MSL post-processing in Service.cs: depth textures sampled as plain textures rewritten to
depth2detc. (Service.cs:39-46, 118+), FP64 detection (Service.cs:47). Metal-only. - Window origin upper-left, depth 0..1 (game sets via
nvnDeviceSetWindowOriginMode/SetDepthMode); some shaders useNEAR_IS_MINUS_W(totk_shader_rt.m:29).
5.2 Shader identity and extraction
- Key = SHA-256(first 0x880 bytes of NVN control section + code) (
totk_shader_rt.m:1,nvn_mtl.m:4962-4963). Control sections are 0x880 or 0x88c bytes; only 0x880 is hashed. Graphics code has a 0x30-byte NVN header (+ 0x50-byte SPH inside code per Program.cs) (scripts/locate_shaders.py). - Variant = shader + translator "answers" (texture slot types, image formats, attr/output component types); variants reused only if answers match current bind state.
- Sources (
scripts/prepare_shader_seed.py:75-210,docs/first-launch-shaders.mdstep 1):- BNSH files (
BNSHmagic,grscat +96) insideShader/*.bfsha.zs(BFSHA;material.bfshaholds most),Effect/*.esetb, UI layouts. - agl
.sharcb(BAHSmagic) inLib/agl,Shader/ApplicationPackage,Lib/Terrain,Lib/gsys: records<u64 code offset, u32 control size, i32 control offset>; controls may be shared. Missing these = magenta title. - BNSH embedded in
main. - RomFS
.zs= zstd with dictionaries fromPack/ZsDic.pack.zs(SARC of dicts); SARC nesting up to 4 deep.
- BNSH files (
- Counts: 1.0.0 9,360; 1.1.2 8,286; 1.4.3 8,290 unique shaders, 0 offline translation failures.
5.3 Shared shader list and pipeline cache
games/totk/assets/shader_list.tsv: 41,414V(shader key + deps JSON), 1,557S(libraries to warm), 63C(compute pipelines), 4,737R(render pipelines, hex-encoded key),Llocator rows (docs/shader-list.md). Identifiers only, no game data.tflag = title screen set (1,445 rows across versions),m= background warm. TheVrows (shader hash + bind state) are backend-agnostic and directly reusable to pre-translate SPIR-V for Vulkan.Rrows encode Metal-pipeline-specific state keys and are not portable.- Runtime cache:
~/Library/Application Support/MacNativeGames/TOTK/shader-cache; missing variants appended toshader-list-contribution.txt. - First-compile stalls (100-900 ms) remain the main perf issue; long sessions showed footprint growth 4.5 -> 6.25 GiB
(
docs/v1.4.3-port.md:55-72).
6. TotK-specific knowledge (CPU-approach independent)
6.1 Version address table (games/totk/runtime/game_version.h)
Keys are 1.0.0 main offsets; values per version. Generated by scripts/map_version_sites.py (instruction-window search
ignoring PC-relative fields; data via ADRP+LDR users; functions via callers) (docs/v1.2.1-port.md, "补丁地址").
| Role | 1.0.0 | 1.1.2 | 1.2.1 | 1.4.3 |
|---|---|---|---|---|
| present-interval controller clamp (fps60) | 0x8f6c60.. | 0x8f66e4.. | 0x8f273c.. | 0x2db204.. (game_patches_v143.c:16-22) |
| frame-step producer (realtime) | 0x7a83f4.. | 0x7f7a38.. | 0x7ede7c.. | 0x2daa54, 0x3e5658 (game_patches_v143.c:74-75) |
| global ratio | 0x8f68b8.. | 0x8f633c.. | 0x8f2394.. | in 0x2daa54 block |
| scene render size store | 0xc630bc | 0xcdd3e8 | 0xcc3f68 | 0x585e48 |
| output size getter call | 0xc630a4 | 0xcdd3d0 | 0xcc3f50 | 0x3dcb28 (display cfg +0x10/+0x508) |
| swapchain size const | 0xe996ac | 0xea7efc | 0xe7aa3c | 0xbdb578 |
| nnMain return checkpoint | 0xe995c4 | 0xea7e14 | 0xe7a954 | 0xbdb48c |
| forceSetMatrix | 0x69a18c | 0x70376c | 0x6ba86c | 0x45ce70 |
| scene slot (data) | 0x4663f08 | 0x4725058 | 0x4728538 | 0x3add238 |
| stale-ref crash site | 0x15ecb0c | 0x1627f10 | 0x1622bf0 | 0x1bd01c4 |
| ShowKeyboard PLT / GOT | 0x2aa4fa0 / 0x453f648 | 0x2b16100 / 0x46129e0 | 0x2b18000 / 0x4615d78 | 0x2bb79f0 / 0x398b348 |
| frame controller return | 0x8f6cbc | 0x8f6740 | 0x8f2798 | 0x2db2ec |
| actor name/pos/rot offsets | 0x210/0x2ac/0x2b8 | 0x218/0x2b4/0x2c0 | same | same |
| ultrawide camera/viewport | - | - | 0xcb4b10, 0x12521cc, 0x19b36f0, 0x1ecdf68 | 0x628c74, 0xb43460, 0x1ff04f4, 0x253fbb0 |
Exact original/replacement words per site: game_patches.c:27-35, 121-126, 169-173, 298-304;
game_patches_v143.c:16-22, 74-75, 193-194, 260-263. Every patch verifies original words before writing.
6.2 Frame timing (the 30 -> 60 fps unlock)
- Adaptive present-interval controller
main+0x8f6bec(1.0.0), called once per frame frommain+0x8f68b4, converts measured frame work time (system object +0x118) into 60 Hz ticks clamped to [min, max]; result goes tonvnWindowSetPresentIntervaland to the variable frame-rate stepmain+0x7a83e4(ratio = interval*0.5, frame time = interval/60). Shipping min = 2 (30 fps). Patch lowers min to 1 (7 instructions) (game_patches.c:13-35). - Real-time step: replace the producer with two floats read from a data page (ratio = dt*30, dt seconds), updated each
present from a 5-sample moving average of present intervals, clamped to [1/120, 1/15] s (
game_patches.c:42-154). - 1.4.3: FrameRateMgr has two call sites fed from
0x3e5658with a 0.5/1.0 coefficient selected bysys+0x1dabit 0; replace the nominal 60/FPS value before the multiply, not the callee entry (docs/v1.4.3-port.md:48). - nxnative alternative for 1.4.3 (
refs/nxnative/dm/tools/patch_nso.py:1037-1168): interval pickermain+0x2db174, int fps at timing +0x50 read atmain+0x2daa54andmain+0x3e562c, float fps at +0x24; host chooses whole-vsync steps; FPSLocker-derived fixes: D-pad menu fademain+0x1fbc80, swapchain argumentmain+0xbe14a8= 1, movie start/end 30 fps cap atmain+0x2b36100/main+0x2b33918, view render scale atmain+0xd5c24. - Side effects of high fps: game-over "Retry" hung on 1.0.0 because
DmF_SY_GameOverwaits one frame after pausing; fixed by RomFS overlay. 1.1.2+ useDmF_SY_GameOver.110(EventWaitUpdate), unaffected (scripts/make_romfs_patches.pyheader). Quick-menu capture background black at 60 fps; fixed by moving bflan visibility keys to frame 0. TOTK_PCHTXTloader accepts community emulator pchtxt patches (by@nsobid) (game_patches.c:387-423).
6.3 Resolution
- Scene render size from parameter tree (
main+0xc6325c: handheld 1280x720, docked 1600x900), stored in renderer +0xcc4/+0xcc8 (game_patches.c:156-168). 1.4.3 actual scene RT is 1600x896. - Output/swapchain 1920x1080 constant before "NvInitializeGraphics" (
main+0xe996ac); canvas from graphics params (+0x38 output, +0x30 scene) (game_patches.c:207-235). 1.4.3: display config object +0x10 canvas, +0x508 crop, +0x528 scale, pending bit 21 of +8 (game_patches_v143.c:144-148). - Runtime mode switch path: post
nn::oemessage 30 (operation mode changed) + set renderer +0xca4 bit 0x200 ->main+0x1185310handler relayouts (game_patches.c:222-230). - 2560x1440 needs bigger heaps:
System/HeapSize/Product.Nin_NX_NVN.HeapDef.byml.zs:DynamicTextureAllocatorMem113,664 KiB (parentGraphics201,728 KiB); scale by pixel ratio, grow Graphics and the app pool by the same amount (+89,088 KiB at 1440p) (scripts/make_output_res_overlay.py,game_patches.c:271-284). - UI canvas width cap, 19:9 aspect constants (D2R; not TotK). TotK ultrawide: patch every producer of the projection
aspect ratio (sead camera +0xb0), DefaultAspect 1.777778 in rodata only copied at camera creation
(
runtime/aspect_patch.h:1-13). - Final upscale shader hashes: EASU
a3a570eb…(old),ac58fc18…(1.4.3, multiplies by 1/3.5 then 3.5, R11G11B10F) (docs/v1.4.3-port.md:39).
6.4 Game data and engine facts
- Engine: sead / agl / gsys, NintendoWare G3d2 / Ui2d / Vfx2, Havok;
mainimportsnn::ro(NRO loading) (refs/nxnative/dm/games/0100F2C0115B6000/game.tsv). subsdk0 = NintendoSDK_movie (Android MediaCodec over NvMM). - Heaps:
sead::ExpHeapuse list at +0xc8, heap self ptr +0x28, size +0x30 (loader.c:394-419). - Movies: 28 WebM in
Event/Movie, VP9 profile 0, 1920x1080, 30 fps, video only; NVDEC path aborts without a host decoder;movie::Decoder17 vtable methods, output format 4 = NV12; pacing by PTS required at >30 fps (runtime/movie_host.c:1-24). Identical across 1.0.0-1.2.1. - Audio:
audren:uREV11 (Q14 biquads) and REV15 (1.4.3: 0x188-byte voice input, float biquads, 0xa8 splitter destination) (docs/v1.4.3-port.md:26); effects (delay, reverb, compressor) ported from Ryujinx.Audio. Voice streams are Opus viann::codec::HardwareOpusDecoderwith 8-byte big-endian header {payload size, final range} (runtime/opus_codec.c:1-5). 1.4.3 English voice: 280,723 packets. - Streamed audio bug:
content://Sound/Resource/Stream/Dm_OP_0001.bwavhas a double separator after the mount name; path parser must skip repeated/(docs/audio-troubleshooting.md,runtime/romfs_path.h). - Save format (1.1.2/1.2.1): 2,307,656 bytes, header
0x47E0F4; 54 files in a profile (docs/v1.2.1-port.md). - Language codes the game accepts via
GetDesiredLanguage: ja, de, fr, es, it, nl, ru, ko, zh-Hans, pt (pt-BR on 1.4.3), en-US (services.c:246-272). - RomFS file counts: 1.1.2 304,471; 1.2.1 304,473; 1.4.3 304,243.
6.5 Known pitfalls (from docs and loader)
- Virtual cores: per-core lock-free data in the game; must emulate 3 application cores (section 3.4).
- 39-bit guest address limit for IPC (section 3.5).
- MapPhysicalMemory must map only unmapped pages and leave mapped ones intact; TotK heaps grow by remapping overlapping
ranges. Pool exhaustion must return LimitReached (132 ->
0x10801), not0xd401(abort) (services.c:164-174). - Texture auto block height: NVN truncates
height_in_blocks*1.5/8; deko3d rounds up; wrong at boundaries (shrine warp pad 160x160 BC3-sRGB shows black) (docs/how-it-works.md:47-55). - D16 load-clear vs fragment depth rounding differ (0.5 -> 32767 vs 32768); Ultrahand/Recall classification grid
(
docs/how-it-works.md:57-63). Check on Vulkan too. - Aliased memory (virtual pools remapping physical pools, two pools on same CPU memory) needs alias-aware barriers
(
docs/how-it-works.md:67-71). - Crash workarounds in game code: stale optional target reference at
GAME_ADDR(0x15ecb0c)(object+0x6b0) after long play (#57); tagged invalid reference0xffff0000ffffffffduring save load (loader.c:313-369). A recompiler must reproduce or patch these. - TotK reserves only 16 bytes for
NVNtextureViewon its stack (main+0x823398), host objects must fit NVN's reserved sizes (nvn_obj.hFITS asserts). - Shader translator optimizer bugs (section 5.1).
- Long-session metadata growth (dead-serial hash sets ~1 GiB after 3 h) (
docs/serial-history.md). - Mods: RomFS-only overlay via manifest; ExeFS/IPS not supported (
README.md:200).
7. Apple-only vs portable
| Component | Files | Status |
|---|---|---|
| NSO rewriter, gate, thread_enter, hle_stack asm, NVN thunks | third_party/d2r/*.py, runtime/*.S, nvn_thunks.S |
ARM64-host-only. Useless for x86-64 execution; valuable as OS-touch census and verification method |
| Loader | runtime/loader.c |
logic portable (reloc kinds 1027/257/1025/1026, RELR, symbol override); mmap/mach/CommonCrypto/signals Apple/POSIX |
| Kernel emulation | runtime/threads.c, ipc.c, services.c |
portable C over pthreads + mach_vm; Windows needs VirtualAlloc2 placeholders, SRWLock/CondVar or winpthreads |
nn::* overrides |
fs_host.c, romfs.c, time_host.c, account.c, applet_host.c, opus_codec.c, movie_host.c, shared_font.c, swkbd_host.c, hid_feedback.c, input.c, audio_renderer.c, audio_effects.c, audio_reverb.c |
portable C (libopus, libvpx); CommonCrypto SHA-256 to replace |
| Audio output | audio_mixer.c |
AudioToolbox; replace with WASAPI/XAudio2/SDL |
| Input, haptics, motion | gamepad.m, gamepad_feedback.m |
GameController/CoreHaptics/IOKit; replace with SDL3/XInput |
| Window, front end | frontend.m, app/launcher.m, frontend_ios.m |
AppKit/UIKit |
| Graphics replay | nvn_mtl.m, nvn_refl.m, totk_shader_rt.m, gfx/tex/*.m, .metal |
Metal 4 |
| Upscale / interpolation | totk_mfx*.m |
MetalFX; replace with FSR/DLSS/XeSS or drop |
| HUD | fps_overlay.c, totk_sysload.c |
CoreText/IOKit |
| Shader translator | vendor/ryujinx, tools/totk_xlate |
portable C#; NativeAOT for win-x64 works |
| Build | build_app.py, scripts/*.py |
mostly portable Python; Apple bits: xcrun, codesign, lsregister, APFS clone, funopen hactool hook |
8. Toolchain
- macOS 26+ on Apple silicon (Metal 4), Xcode command-line tools (
xcrun clang, C11,-fobjc-arc). - Homebrew:
opus,libvpx(linked statically from/opt/homebrew/opt/*/lib/*.a),python@3.14. - Python 3.14 (stdlib
compression.zstd), venv withcryptography,lz4,capstone,nsz; dev:unicorn==2.1.4,Pillow,brotli(requirements-dev.txt). - .NET 10 SDK (10.0.401) only to rebuild
totk_xlate(NativeAOT). - Fetched at build: hactool @
1d64a83, SPIRV-Cross @aa217ae. - Optional: Xcode Metal toolchain for offline
.metallib; iOS: Xcode 26, xcodegen, cmake, ninja. - Single-instance "game slot" lock for all GPU tests (
runtime/game_slot.c).
9. Licenses
From THIRD_PARTY_NOTICES.md and LICENSE files:
| Code | License |
|---|---|
| MacNativeGames (repo) | MIT (LICENSE) |
third_party/d2r (rewriter, verification), third_party/d2r_gfx (NVN on Metal), tools/totk_xlate |
MIT (d2r-mobile) |
nvn_tex.c texture layout sizes |
zlib (deko3d) |
| bcdec 0.985 | MIT or public domain |
vendor/ryujinx (Shader, Common, Spv.Generator 1.3.2) |
MIT |
| audio effects/mixer ported from Ryujinx.Audio | MIT (third_party/licenses/Ryujinx-Audio.txt) |
| SPIRV-Cross | Apache-2.0 |
| hactool (+ bundled mbedTLS) | ISC (+ Apache-2.0) |
| libopus, libvpx | BSD-3-Clause |
| d2r-mobile | MIT |
| nxnative | MIT (refs/nxnative/LICENSE); bundles libvpx, stb_vorbis, libdeflate, Vulkan headers, NVIDIA FLIP with their own licenses |
The TotK-Ultrawide community mod is GPL; MacNativeGames states none of its code is used (runtime/aspect_patch.h:6).
Everything we would reuse (host C, NVN front end, translator patches, addresses, hashes) is MIT/zlib/Apache: compatible
with any license we choose.
10. nxnative and d2r-mobile notes
- d2r-mobile
dm/hostis the generic origin (loader,hle_svc.c,hle_thread.c,hle_table.cfor libc passthrough,HLE/HLE_FORCEregistration with mandatory reasons,UNIMPLnamed stubs). Offline shader route: everything pre-translated to metallib, no runtime translation.dm/gfx/nvn_metal_mapping.mdis the best concise NVN spec. - nxnative generalizes to 14 games with per-game profiles (
dm/games/<title id>/game.tsv), including TotK 1.4.3 and BotW 1.9.0 (shares sdk/subsdk0 build with TotK per its comment), Vulkan-on-Metal path for Hades (dm/gfx/vk/hle_vk.m), LibHac-baseddm/tools/nx_extract(.NET) for NSP + update layering. Its TotK profile has memory numbers (mem_alias 0x98000000, heap0x86000000, app pool0xCD500000) and NVN constant-buffer bases. One-script packaging:scripts/build_ipa_from_nsp.py(1,229 lines).
11. Takeaways for the Windows x86-64 port
- The CPU strategy must change (no native ARM64), but the rewriter's site census (
sites.tsv,gates.tsvper module) is a precise list of every SVC, timer read, TLS access and x18 use; reusepatch_nso.py's scanner as an analysis tool for the recompiler front end. - The host boundary is mostly at
nn::*function level, not IPC: a recompiler can link recompiledmainagainst host implementations of ~163nn::*symbols plus 523 NVN entries, and either recompilesdk(for os/thread/heap code that uses the 38 SVCs in section 3.2) or HLE more of it. - Port as C:
threads.c,ipc.c,services.cmemory logic,fs_host.c,romfs.c,time_host.c, audio renderer and effects, opus, movie, fonts, swkbd,hle_nvn.c+nvn_obj.h+ texture code. - Write a Vulkan replay to replace
nvn_mtl.m; keep Ryujinx translator + the four patches, emit SPIR-V directly. - Reuse version hashes, address table, fps/realtime/resolution patch semantics, RomFS overlays, HeapDef math, shader
extraction formats and the shared shader list
Vrows.