Commit Graph
12 Commits
Author SHA1 Message Date
Valiantsin DzerakhandClaude Opus 5.5 44bc5887f0 gen.py keeps 8 V registers per function in locals by default
K = 8 cut guest time 3.9-6.2% in the town on a 7900X against K = 0. func.py tests the same default; --v-locals 0
gives the previous output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 20:11:39 +03:00
Valiantsin DzerakhandClaude Opus 5.5 eb8c7ea888 gen.py --v-locals K keeps the K most used V registers of each function in C locals
recomp/vlocals.py rewrites each emitted function: c->v forms become the RT_VLOCALS accessors of runtime/simd.h, the
K registers the function names most get bits in VMASK, they are loaded at entry, stored where the dirty mask may hold
them before calls and exits, and reloaded after calls; --v-trust skips the reload of V8-V15. A function with a form the
rewrite does not know keeps every register in the Cpu. K = 0, the default, emits the same C as before. The manifest
counts the rewritten functions and syncs. tests/diff/func.py takes --v-locals, --v-trust and --work (one work
directory per concurrent run).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 19:40:05 +03:00
fa94e36549 Store the data writes of mods' ExeFS patch files into main before the game runs
An IPS, IPS32 or pchtxt file for main that writes main's .rodata, .data or .bss was refused as a whole. Such writes
need no lifting: tools/exefs_patches.py now puts them in build-cache/exefs.data, and runtime/mods.c stores them into
the loaded main before any game code runs, as emulators and Atmosphere apply a patch file to the module before its
dynamic linker runs. A DT_RELR word gets main's base added; bytes over words an explicit-addend relocation fills change
nothing and are dropped; a write into the dynamic linker's tables, a lazily bound function pointer or outside main's
image keeps the file unsupported. A mod whose data lines do not all fit the build keeps its code patches off too.

The mods are applied from the bottom one up (mods_exefs_apply), so where two mods write the same bytes the higher one
wins, as it does for files. --probe-mods checks it with two mods writing one byte.

The lifted code reads some of main's bytes at lift time (the jump tables). recomp/gen.py lists those ranges in
<gen>/main/folded.txt, and the setup checks the data writes against them after the lift: a file that writes them
becomes unsupported, since the change would not reach the generated code. Generated C without mods is unchanged.

This is the data part of #790 (ericdallo's ExeFS work), split out on its own; the plugin part follows separately.

Co-Authored-By: itaskaev2 <165683571+itaskaev2@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 15:51:42 +01:00
Valiantsin DzerakhandClaude Sonnet 5.5 526e9f32bd Retry the lift's emit pool with half the workers after a MemoryError, and read the worker count from TOTK_LIFT_JOBS
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 23:27:46 +03:00
conongnang 44105a95ff Merge branch 'valentderah:main' into cheats 2026-10-31 13:41:59 +07:00
Valiantsin DzerakhandClaude Opus 5.5 a5711539e1 Size the code generator's worker pool by free memory so it stops running out of RAM
gen.py started one emit worker per CPU, about 1.3 GiB each on 1.4.3, so many-core machines with modest RAM died with a MemoryError during "Translating code to C" (issues 17, 28, probably 66). The default is now min(CPU count, 80% of the free physical memory / 1.5 GiB), at least 1, capped by the memory limit of the enclosing job object if there is one. --jobs still overrides it, and the chosen count and its reason are logged. The memory helper is tools/hostmem.py, shipped with the release tools.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 12:43:50 +03:00
conongnangandClaude Opus 5.5 3df82b8842 Load Atmosphere cheat files and switch their cheats from a Cheats tab in the menu
Players bring their own cheats\<build id>.txt next to totk.exe; none ship with the port. The setup turns cheats that
write main's code into patch sets (tools/cheats.py), cheats that write data run in a cheat VM after dmnt's every
presented frame (runtime/cheats.c), and the menu's Cheats tab switches each one, saved under [cheats] in totk.ini.

The patch table now takes several sets per site, first set in file order winning, and replacements that end in a
branch or ret, so cheats that patch the same instruction or end in a jump can be built.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 10:28:37 +07:00
Valiantsin DzerakhandClaude Opus 5.5 c2943970cc Merge the spin wait gates: they also end the fight hangs
The generator emits a guest poll such as `ldr w9, [x19]; cmp w9, #1; b.eq` as a plain C load, and clang hoists it out
of the loop: the lock wait in f_7001fa0e2c compiled to `cmp; jne; jmp .`, so a thread that found the lock held spun
forever after its holder released it. A fight run hung this way in 3 of 13 launches on an RX 9070, with the UBO
snapshot on and off; gdb showed the thread on that `jmp` in every sample. The gate is an opaque call on every
back-edge of the 868 loops tools/spin_waits.py lists, so each pass reloads memory: the exe goes from 182 self-jumps to
148, and 144 of those are the game's own `b .` instructions. 8 of 8 fight runs then passed; tests/diff/func.py with
--patches passed 5470 cases on 878 functions, and the four probes passed on 1.4.3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:05:44 +03:00
Valiantsin DzerakhandClaude Opus 5.5 0ca11dfba2 Keep the six most used general registers of each generated function in C locals
Every lifted instruction read and wrote c->x[n] and c->sp, so each register passed from one guest instruction to the
next went through a store and a load, and clang could not keep it in a host register across a guest store.
recomp/regs.py now copies the six most used X registers and SP of a function into locals at its entry. A forward pass
over the function's flow tracks which locals changed since the last sync, and only those are stored back before a
call, an SVC, a trap, a return or a tail exit. After an SVC every local reloads; after a call only X0-X18 and X30,
since AAPCS64 has the callee preserve X19-X29 and SP. TotK does not break that: main and subsdk0 import neither
longjmp nor the unwinder, sdk's have no direct callers, and the runtime writes no register above X3 except X30.
gen.py --no-callee-saved-trust reloads them too, --reg-locals sets the count. Functions with an unresolved computed br
keep their registers in the Cpu, as they keep NZCV.

All registers in locals was slower than none: clang spills them to the stack and the syncs stay. Over 300 hot
functions, 4-6 locals gave the fewest instructions and memory operands.

1.4.3 .text 299 -> 266 MiB. Fight bench (bench-fight2-2026-10-27b, frames 5400-7500, uncapped): 11.30 and 11.92 ms vs
13.5-13.9 on main; frames 2400-5300 11.24 and 11.36 vs 13.25. Without the callee-saved trust 12.58 ms.
tests/diff/func.py: 0 failing on 1.4.3 (889 functions, 583 with calls) and 1.2.1 (810); a plan with no spills fails
at once. Probes kernel, vkexec, nvn-tex and shader pass in both versions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 22:21:18 +03:00
Valiantsin DzerakhandClaude Opus 5.5 27056b212a Keep NZCV in a local of each generated function instead of c->nzcv
Every flag-setting instruction packed N, Z, C and V into c->nzcv and every reader unpacked them. recomp/flags.py
now runs a reaching-definitions pass per function: a compare stores its operands in a local Flags F, and a reader
whose definitions are all of one kind computes its condition from them, so cmp and b.cond become one host compare.
CCMP, FCMP, MSR NZCV and ADCS write the packed form, which readers reached by several kinds also use. Calls and
returns publish nothing (AAPCS64 neither passes nor preserves NZCV); tail branches, falls into the next function and
br through rt_dispatch publish c->nzcv, since discovery can start a function inside another one. Functions with an
unresolved computed br keep flags in c->nzcv.

1.4.3 main: 492k direct and 89k packed flag reads, .text 342 -> 299 MiB. Fight bench (bench-fight2-2026-10-27b,
frames 5400-7500, uncapped) 13.15 and 13.63 ms vs 13.9 before. tests/flags_check.py checks the new helpers against
the packed ones; tests/diff/func.py no longer compares NZCV, which generated code leaves untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 18:07:58 +03:00
Valiantsin DzerakhandClaude Opus 5.5 3984344635 Spin wait gates: tools/spin_waits.py (the nxnative search) lists the loops that only another thread can end, the patch tables carry them as spin lines (857 in 1.4.3, 864 in 1.2.1), the generator sends their back-edges through rt_spin_gate, and the gate pauses the core once per pass and hands the virtual core to a waiter after 50 us of polling (TOTK_SPIN, TOTK_SPIN_HANDOFF_US)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 15:51:47 +03:00
Valiantsin DzerakhandClaude Opus 5.5 19c9b648d1 Add the recompiler, Windows runtime, tools and docs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 22:27:54 +03:00