After the design in #845 by itaskaev2. New patch lines carry an exlaunch plugin whose init runs at every
start: `plugin` lifts the module from its image after the init (the hook stubs and trampolines the file
has as zeros) and names the hash of that .text, `expect` lists the words the init writes into the game,
`stub` lifts an inline-hook stub's branch to its trampoline as a return, `seed` adds the callbacks, impls
and targets, and `orig` lists an unpatched copy of a hooked main function (fo_<address>) at the replace
hook's trampoline address in the plugin's function table, so the plugin's call of the original does not
reach the hook again and rt_dispatch stays as it is.
A replacement word @call:<module>+<offset> calls the stub without writing x30, with every register in
the Cpu before it and read back after it; the site's own instruction runs after it in place. patches.h
gets RT_PLUGINS, RT_PLUGIN_WRITES and RT_PLUGIN_ROOTS for the runtime. Without plugin lines the output is
byte-identical to dev (generated C, patches.h and layout of 1.2.1 and 1.4.3).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
recomp/layout.py appends a mod's code modules (exlaunch plugins such as exefs/subsdk9) after sdk as plugin0..N in
mod load order, so the game modules keep their bases; layout.json names each plugin's NSO and its SHA-256, and without
plugins layout.json and layout.h stay byte-identical. gen.py --plugins lifts them with DT_INIT and the init_array
entries as extra seeds, and Module.eh_frame_functions accepts a MOD0 with an empty .eh_frame_hdr range. build_game.py
takes the plugin rows from layout.json into modules.c instead of asking the ExeFS hash table for them.
No-mod generated C is byte-identical to dev; with MSFix in the layout main, subsdk0 and sdk are too.
After the design in #845 by itaskaev2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A patch set marked `ordered` (recomp/patches.py) belongs to something the player orders; tools/exefs_patches.py marks
the mods' ExeFS patch sets so. A site two or more ordered sets patch is contested: its ordered alternatives test one
byte, rt_patch_pick[<site>], instead of each set's switch, and patches.h lists the contested sites with their sets for
the runtime. Other alternatives keep their place and switch, so a cheat before the mods' sets still wins while it is
on. Sites with at most one ordered set, and tables without contested sites, are emitted as before, patches.h
included. The pick goes into the lift cache key.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FN_ENTER (runtime/cpu.h) opened all 282,702 lifted functions of 1.4.3 with a load of g_fnprof and a branch, for one
reader: runtime/resolution.c res_fn_hit, which waits for main+0x585bd0 (SET_RENDER_SIZE). A `hook` line of the patch
table (recomp/patches.py) now names the functions that keep it; patches-v143.txt hooks main+0x585bd0, and gen.py stops
on a hook that is no function start. Tables without hook lines (1.2.1, 1.1.0) keep FN_ENTER in every function, and
gen.py --fn-enter-all puts it everywhere for a whole-game function-entry profile. The hooked addresses of a chunk are
in its lift cache key (tests/lift_cache_test.py test_hooks_are_in_the_key).
On 1.4.3 the generated C is otherwise byte-identical to dev's (282,701 FN_ENTER lines fewer, nothing else differs).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The conflict in recomp/gen.py was the Usage line and the options next to --v-locals and --v-trust; both are kept, and
v_locals and v_trust are in the Emitter options the cache key holds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
K = 8 cut guest time 3.9-6.2% in the town on a 7900X against K = 0. func.py tests the same default; --v-locals 0
gives the previous output.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
recomp/vlocals.py rewrites each emitted function: c->v forms become the RT_VLOCALS accessors of runtime/simd.h, the
K registers the function names most get bits in VMASK, they are loaded at entry, stored where the dirty mask may hold
them before calls and exits, and reloaded after calls; --v-trust skips the reload of V8-V15. A function with a form the
rewrite does not know keeps every register in the Cpu. K = 0, the default, emits the same C as before. The manifest
counts the rewritten functions and syncs. tests/diff/func.py takes --v-locals, --v-trust and --work (one work
directory per concurrent run).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gen.py lifted every chunk on each run and only skipped rewriting the ones whose text came out the same, so a rebuild
after one changed code cheat still spent the whole lift (102 s on 12 cores, 178 s with 4 workers) to recompile two
chunks. A chunk's text depends only on the generator's sources and options, the module and the function starts
discovered from it, the chunk's number and functions, and the patch sites and spin waits inside its address range, so
the SHA-256 of these is now its key. <mod>/lift_cache.json records each chunk's key with the SHA-256, size and mtime of
its file and the statistics and folded ranges the manifest and folded.txt need, under a digest of the entry. A chunk
whose entry is intact, whose key matches and whose file is unchanged is reused without a worker; no pool starts when
nothing is left to lift. The cache is written after the manifest, and a failed write only warns. --no-cache, or
TOTK_LIFT_CACHE=0 for the installer's lift, lifts everything.
On 1.4.3 with the installed mods' cheats, a run with unchanged inputs takes 4 s and one with a changed cheat word lifts
one chunk in 7 s. Its output equals a --no-cache lift into an empty folder and the previous gen.py's output byte for
byte. tests/lift_cache_test.py checks reuse and invalidation on a module it makes itself.
An IPS, IPS32 or pchtxt file for main that writes main's .rodata, .data or .bss was refused as a whole. Such writes
need no lifting: tools/exefs_patches.py now puts them in build-cache/exefs.data, and runtime/mods.c stores them into
the loaded main before any game code runs, as emulators and Atmosphere apply a patch file to the module before its
dynamic linker runs. A DT_RELR word gets main's base added; bytes over words an explicit-addend relocation fills change
nothing and are dropped; a write into the dynamic linker's tables, a lazily bound function pointer or outside main's
image keeps the file unsupported. A mod whose data lines do not all fit the build keeps its code patches off too.
The mods are applied from the bottom one up (mods_exefs_apply), so where two mods write the same bytes the higher one
wins, as it does for files. --probe-mods checks it with two mods writing one byte.
The lifted code reads some of main's bytes at lift time (the jump tables). recomp/gen.py lists those ranges in
<gen>/main/folded.txt, and the setup checks the data writes against them after the lift: a file that writes them
becomes unsupported, since the change would not reach the generated code. Generated C without mods is unchanged.
This is the data part of #790 (ericdallo's ExeFS work), split out on its own; the plugin part follows separately.
Co-Authored-By: itaskaev2 <165683571+itaskaev2@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gen.py started one emit worker per CPU, about 1.3 GiB each on 1.4.3, so many-core machines with modest RAM died with a MemoryError during "Translating code to C" (issues 17, 28, probably 66). The default is now min(CPU count, 80% of the free physical memory / 1.5 GiB), at least 1, capped by the memory limit of the enclosing job object if there is one. --jobs still overrides it, and the chosen count and its reason are logged. The memory helper is tools/hostmem.py, shipped with the release tools.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Players bring their own cheats\<build id>.txt next to totk.exe; none ship with the port. The setup turns cheats that
write main's code into patch sets (tools/cheats.py), cheats that write data run in a cheat VM after dmnt's every
presented frame (runtime/cheats.c), and the menu's Cheats tab switches each one, saved under [cheats] in totk.ini.
The patch table now takes several sets per site, first set in file order winning, and replacements that end in a
branch or ret, so cheats that patch the same instruction or end in a jump can be built.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The generator emits a guest poll such as `ldr w9, [x19]; cmp w9, #1; b.eq` as a plain C load, and clang hoists it out
of the loop: the lock wait in f_7001fa0e2c compiled to `cmp; jne; jmp .`, so a thread that found the lock held spun
forever after its holder released it. A fight run hung this way in 3 of 13 launches on an RX 9070, with the UBO
snapshot on and off; gdb showed the thread on that `jmp` in every sample. The gate is an opaque call on every
back-edge of the 868 loops tools/spin_waits.py lists, so each pass reloads memory: the exe goes from 182 self-jumps to
148, and 144 of those are the game's own `b .` instructions. 8 of 8 fight runs then passed; tests/diff/func.py with
--patches passed 5470 cases on 878 functions, and the four probes passed on 1.4.3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every lifted instruction read and wrote c->x[n] and c->sp, so each register passed from one guest instruction to the
next went through a store and a load, and clang could not keep it in a host register across a guest store.
recomp/regs.py now copies the six most used X registers and SP of a function into locals at its entry. A forward pass
over the function's flow tracks which locals changed since the last sync, and only those are stored back before a
call, an SVC, a trap, a return or a tail exit. After an SVC every local reloads; after a call only X0-X18 and X30,
since AAPCS64 has the callee preserve X19-X29 and SP. TotK does not break that: main and subsdk0 import neither
longjmp nor the unwinder, sdk's have no direct callers, and the runtime writes no register above X3 except X30.
gen.py --no-callee-saved-trust reloads them too, --reg-locals sets the count. Functions with an unresolved computed br
keep their registers in the Cpu, as they keep NZCV.
All registers in locals was slower than none: clang spills them to the stack and the syncs stay. Over 300 hot
functions, 4-6 locals gave the fewest instructions and memory operands.
1.4.3 .text 299 -> 266 MiB. Fight bench (bench-fight2-2026-10-27b, frames 5400-7500, uncapped): 11.30 and 11.92 ms vs
13.5-13.9 on main; frames 2400-5300 11.24 and 11.36 vs 13.25. Without the callee-saved trust 12.58 ms.
tests/diff/func.py: 0 failing on 1.4.3 (889 functions, 583 with calls) and 1.2.1 (810); a plan with no spills fails
at once. Probes kernel, vkexec, nvn-tex and shader pass in both versions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every flag-setting instruction packed N, Z, C and V into c->nzcv and every reader unpacked them. recomp/flags.py
now runs a reaching-definitions pass per function: a compare stores its operands in a local Flags F, and a reader
whose definitions are all of one kind computes its condition from them, so cmp and b.cond become one host compare.
CCMP, FCMP, MSR NZCV and ADCS write the packed form, which readers reached by several kinds also use. Calls and
returns publish nothing (AAPCS64 neither passes nor preserves NZCV); tail branches, falls into the next function and
br through rt_dispatch publish c->nzcv, since discovery can start a function inside another one. Functions with an
unresolved computed br keep flags in c->nzcv.
1.4.3 main: 492k direct and 89k packed flag reads, .text 342 -> 299 MiB. Fight bench (bench-fight2-2026-10-27b,
frames 5400-7500, uncapped) 13.15 and 13.63 ms vs 13.9 before. tests/flags_check.py checks the new helpers against
the packed ones; tests/diff/func.py no longer compares NZCV, which generated code leaves untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>