Build the game code with an execution profile when the release has one

build_game.py --pgo gen builds an instrumented totk.exe (llvm-mingw: MSYS2's clang has no profile runtime) that adds
its counts to <out>/pgo/*.profraw every minute and at exit; plat_exit ends the process with ExitProcess, which skips
compiler-rt's atexit write. tools/pgo.py merges them into build/pgo/pgo-<ver>.profdata with a record of the recomp/
tree it fits. build_game.py --pgo FILE compiles the generated code at -O2 and the runtime with that profile.

make_release.py ships build/pgo's profiles in data/ and builds the runtime libraries with them; the installer passes
the update's profile to build_game.py, and builds at -O1 without one. Profiles name lifted functions by address and
hold branch counts and control-flow hashes of the generated C, no bytes of the game.

1.4.3 on machine A, mean frame by world clock: the fight2 bench 11.49 -> 10.06 ms (-12.5%), a fight outside the
training 13.00 -> 12.24 ms (-5.8%); every window CPU-bound. The profile was trained on scripted runs and 24 minutes
of play.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/building.md

@@ -150,6 +150,33 @@ your machines without MSYS2, Python or .NET. `romfs.bin` stays out unless you pa
 `romfs\` next to the exe or point `TOTK_ROMFS` at its folder. Run `totk.exe --prepare-shaders` once in the copied
 folder. The folder holds code and data derived from your game: for your own machines only.
 
+### Profile-guided build
+
+`build_game.py --pgo <file>.profdata` compiles the generated code at -O2 with an execution profile, and the runtime
+with it too: hot paths laid out straight, hot calls inlined, code that never ran compiled for size. On 1.4.3 the fight
+bench ran 12-15% faster per frame and a fight outside the training about 6%. The installer builds with
+`data/pgo-<ver>.profdata` of the release when the update has one, else at -O1.
+
+A profile only fits the generated C it was trained on: functions are matched by name and control-flow hash, so a
+function whose C changes loses its counts without a warning. Retrain after a lifter or generator change, from a clean
+`recomp/`. The instrumented build needs llvm-mingw, as MSYS2's clang has no profile runtime:
+
+```powershell
+$llvm = "toolchain\llvm-mingw-<date>\llvm-mingw-<date>-ucrt-x86_64"
+.venv\Scripts\python tools\build_game.py --version v143 --out build\v143-pgogen --toolchain $llvm --pgo gen
+```
+
+Play `build\v143-pgogen\totk.exe` through varied scenes (fights, towns, riding, building, the Depths, menus) for 20-30
+minutes; it is slower than a normal build. It writes `build\v143-pgogen\pgo\*.profraw` every minute and at exit.
+Scripted runs from saves (`tools/run_game.py`) add to the same file. Then:
+
+```powershell
+.venv\Scripts\python tools\pgo.py merge build\v143-pgogen --version v143
+```
+
+writes `build\pgo\pgo-v143.profdata` and its record (`.json`: the `recomp/` tree it fits). `make_release.py` ships every
+profile of `build\pgo` in `data\` and warns when a record names another `recomp/` tree than the one released.
+
 ## Linux (in progress)
 
 The runtime builds and runs on x86-64 Linux with a window, input and menu on SDL3 (`*_sdl.c`) in place of the Win32

docs/installer.md

@@ -147,15 +147,17 @@ llvm/                 llvm-mingw, x86_64-only subset (Apache-2.0 with LLVM excep
 runtime/              libtotkrt.a, libtotkrt-v2.a and the headers the generated code and modules.c/romfs.c/patches.c
                       include
 bin/                  totk_xlate.exe, hactool.exe, SDL3.dll, libopus-0.dll, ninja.exe
-data/                 shader_list.tsv (MacNativeGames, MIT)
+data/                 shader_list.tsv (MacNativeGames, MIT); pgo-<ver>.profdata + .json, the execution profile an
+                      update's code and the runtime libraries compile with (building.md, "Profile-guided build")
 THIRD_PARTY_NOTICES.md
 totk.ini.example, VERSION  the default ini; the release string (the repository's VERSION plus the commit)
 ```
 
 - SDL3 comes from the upstream release, not MSYS2: MSYS2's DLL imports the LGPL libiconv.
 - Every bundled third-party piece gets a row in `THIRD_PARTY_NOTICES.md`.
-- The release contains nothing game-derived; `make_release.py` runs `tools/check_content.py` over the folder before it
-  finishes.
+- The release contains no code, data, text or assets of the game; `make_release.py` runs `tools/check_content.py` over
+  the folder before it finishes. The execution profiles name lifted functions by address (`f_<address>`) and hold
+  branch counts and control-flow hashes of the generated C.
 
 ## Switch system fonts
 

installer/layout.py

@@ -12,6 +12,8 @@ Release (docs/installer.md "Release layout"), root = the unpacked folder:
                             include/ + lib/ of Vulkan, SDL3, Opus, libvpx and VMA (build_game.py --deps)
     bin/                    totk_xlate.exe (+ totk_xlate.id), hactool.exe, ninja.exe, SDL3.dll, libopus-0.dll
     data/shader_list.tsv    the MacNativeGames shader list
+    data/pgo-<v>.profdata   the execution profile the update's code is compiled with (tools/pgo.py; Windows only);
+                            without one the game is built at -O1
     totk.ini.example
     VERSION                 the release version, one line
     temp/                   the default temp folder (created)
@@ -24,7 +26,7 @@ Dev mode, root = a repository checkout (no python/python.exe): build/py314-embed
 Python the release ships, with the same packages), else this Python; toolchain/llvm-mingw-*/ and toolchain/deps-mingw;
 runtime/ as the sources with build/install-rtlib/libtotkrt.a and libtotkrt-v2.a (tools/build_game.py --make-runtime-lib
 --out build/install-rtlib, once per --cpu); MSYS2 UCRT64 ninja.exe; toolchain/hactool; build/xlate/totk_xlate.exe; the shader list from
-refs/MacNativeGames; build/install-temp. The version is VERSION with -dev and the commit: 0.1.0-dev+973c626. On Linux,
+refs/MacNativeGames; build/pgo/pgo-<v>.profdata when there is one; build/install-temp. The version is VERSION with -dev and the commit: 0.1.0-dev+973c626. On Linux,
 dev mode takes clang, lld and ninja from PATH and the system's libraries (no deps folder, no SDL3 copy).
 """
 import os
@@ -66,6 +68,7 @@ class Layout:
     temp: Path | None = None
     sysroot: Path | None = None       # Linux release: what the game links against, instead of the host's
     runtime_lib_v2: Path | None = None   # libtotkrt-v2.a: the runtime for CPUs without AVX2 (build_game.py --cpu v2)
+    pgo_dir: Path | None = None       # pgo-<v>.profdata (tools/pgo.py); None: no profiles (Linux)
 
     @property
     def clang(self):
@@ -79,6 +82,12 @@ class Layout:
             return lib
         return None if self.dev else lib
 
+    def pgo(self, version):
+        """The execution profile build_game.py --pgo compiles the update's code with, None without one (the game is
+        then built at -O1)."""
+        p = self.pgo_dir / f'pgo-{version_tag(version)}.profdata' if self.pgo_dir else None
+        return p if p and p.is_file() else None
+
     def patches(self, version):
         """The code patch table for recomp/gen.py."""
         return self.recomp / f'patches-{version_tag(version)}.txt'
@@ -129,6 +138,7 @@ def release_layout(root):
         xlate=root / 'bin' / ('totk_xlate' + EXE),
         dlls=(root / 'bin' / 'SDL3.dll', root / 'bin' / 'libopus-0.dll') if WIN else (root / 'bin' / 'libSDL3.so.0',),
         shader_list=root / 'data' / 'shader_list.tsv', ini_example=root / 'totk.ini.example',
+        pgo_dir=root / 'data' if WIN else None,
         # Linux: None, the install folder's .setup-temp (installer/cli.py): the AppImage is mounted read-only, and on
         # a Steam Deck /home is often too full for the generated C while the install folder is on the SD card
         temp=root / 'temp' if WIN else None,
@@ -154,7 +164,8 @@ def dev_layout(root):
         ninja=UCRT / ('ninja' + EXE), hactool=root / 'toolchain' / 'hactool' / ('hactool' + EXE),
         xlate=root / 'build' / 'xlate' / ('totk_xlate' + EXE),
         dlls=tuple(sorted((deps / 'bin').glob('*.dll'))),
-        shader_list=None, ini_example=root / 'totk.ini.example', temp=root / 'build' / 'install-temp')
+        shader_list=None, ini_example=root / 'totk.ini.example', temp=root / 'build' / 'install-temp',
+        pgo_dir=root / 'build' / 'pgo')
 
 
 def dev_commit(root):

installer/steps.py

@@ -944,7 +944,8 @@ class Compile(Step):
     id = 'compile'
 
     def key(self, ctx):
-        return fingerprint('compile-3', ctx.keys['lift'], runtime_key(ctx))
+        return fingerprint('compile-3', ctx.keys['lift'], runtime_key(ctx),
+                           ctx.layout.pgo(game_version(ctx)) or 'no profile')
 
     def current(self, ctx, record):
         exe = ctx.dest / ('totk' + EXE)
@@ -970,10 +971,12 @@ class Compile(Step):
         rep.progress('compile', 'compile', force=True)
         cpu = getattr(ctx, 'cpu', 'v3')
         lib = layout.runtime_lib_for(cpu)
+        pgo = layout.pgo(game_version(ctx))
         code, log = run_tool(ctx, 'compile', layout.tools / 'build_game.py', [
             '--gen', ctx.temp / 'gen', '--out', build, '--toolchain', layout.llvm, '--cpu', cpu,
             *(['--deps', layout.deps] if layout.deps else []), *(['--sysroot', layout.sysroot] if layout.sysroot else []),
-            '--runtime', layout.runtime, *(['--runtime-lib', lib] if lib else ()), '--ninja', layout.ninja,
+            '--runtime', layout.runtime, *(['--runtime-lib', lib] if lib else ()), *(['--pgo', pgo] if pgo else ()),
+            '--ninja', layout.ninja,
             '--version', version_tag(game_version(ctx)), '--expected', layout.expected(game_version(ctx)),
             '--exefs', dest / 'exefs', '--romfs', dest / 'romfs',
             '--profile', dest / 'profile'], parse)

runtime/platform/linux.c

@@ -1275,7 +1275,16 @@ int plat_stderr_path(char *out, size_t cap) {
 
 // ---- process
 
+#ifdef TOTK_PGO_GEN
+// compiler-rt's profile runtime (an instrumented build, tools/build_game.py --pgo gen). It writes its counters from an
+// atexit handler, which _exit skips; __llvm_profile_dump writes them once and makes later writes no-ops.
+int __llvm_profile_dump(void);
+#endif
+
 void plat_exit(int code) {
+#ifdef TOTK_PGO_GEN
+    __llvm_profile_dump();
+#endif
     fflush(stdout);
     fflush(stderr);
     _exit(code);

runtime/platform/win32.c

@@ -791,7 +791,33 @@ void plat_aligned_free(void *p) { _aligned_free(p); }
 
 // ---- process
 
+#ifdef TOTK_PGO_GEN
+// compiler-rt's profile runtime (an instrumented build, tools/build_game.py --pgo gen). It writes its counters from an
+// atexit handler, which ExitProcess skips; __llvm_profile_dump writes them once and makes later writes no-ops.
+int __llvm_profile_dump(void);
+int __llvm_profile_write_file(void);
+void __llvm_profile_reset_counters(void);
+
+// A long played session must not lose its counts to a crash or a reboot: every minute the counters are added to the
+// .profraw (the file merges, -fprofile-generate's %m) and zeroed. Counts taken between the write and the reset are lost.
+static DWORD WINAPI pgo_flush_thread(void *arg) {
+    (void)arg;
+    for (;;) {
+        Sleep(60 * 1000);
+        if (__llvm_profile_write_file() == 0) __llvm_profile_reset_counters();
+    }
+}
+
+__attribute__((constructor)) static void pgo_flush_start(void) {
+    HANDLE h = CreateThread(NULL, 0, pgo_flush_thread, NULL, 0, NULL);
+    if (h) CloseHandle(h);
+}
+#endif
+
 void plat_exit(int code) {
+#ifdef TOTK_PGO_GEN
+    __llvm_profile_dump();
+#endif
     fflush(stdout);
     fflush(stderr);
     ExitProcess((UINT)code);

tools/build_game.py

@@ -109,6 +109,36 @@ def set_cpu(cpu):
     GEN_FLAGS[GEN_FLAGS.index('-march=x86-64-v3')] = march
     RT_FLAGS[RT_FLAGS.index('-march=x86-64-v3')] = march
     RT_FLAGS.append(f'-DRT_CPU_LEVEL={3 if cpu == "v3" else 2}')
+
+
+# Profile-guided optimisation (--pgo). Both modes compile the generated code at -O2, the level the profile's inlining and
+# hot/cold layout work at. A static function's profile name carries its source path by default; the base name alone
+# keeps the names equal between the training build and a build from another tree (a worktree, an install).
+PGO_NAMES = ['-mllvm', '-static-func-full-module-prefix=false']
+PGO_INPUT = None   # the .profdata of --pgo FILE: every compile depends on it, so a new profile rebuilds them
+
+
+def set_pgo(pgo, out):
+    """--pgo gen: instrument the generated code and the runtime; totk.exe adds its counts to
+    out/pgo/default_<id>.profraw every minute and at exit (runtime/platform win32.c), across runs. --pgo FILE: compile
+    with that merged profile (tools/pgo.py merge). Functions the profile never saw run are optimised for size."""
+    global PGO_INPUT
+    GEN_FLAGS[GEN_FLAGS.index('-O1')] = '-O2'
+    if pgo == 'gen':
+        # absolute: the instrumented exe resolves the directory against its own working directory
+        flags = [f'-fprofile-generate={cmd_arg((out / "pgo").resolve().as_posix())}']
+        RT_FLAGS.append('-DTOTK_PGO_GEN')
+        LINK_FLAGS.append('-fprofile-generate')   # links clang_rt.profile
+    else:
+        PGO_INPUT = Path(pgo).resolve()
+        if not PGO_INPUT.is_file():
+            sys.exit(f'--pgo {pgo}: no such profile')
+        # unprofiled: chunks the training never entered; out-of-date and backend-plugin: functions whose code changed
+        # since the training, which then compile without a profile
+        flags = [f'-fprofile-use={cmd_path(PGO_INPUT)}', '-Wno-profile-instr-unprofiled',
+                 '-Wno-profile-instr-out-of-date', '-Wno-backend-plugin']
+    GEN_FLAGS.extend(flags + PGO_NAMES)
+    RT_FLAGS.extend(flags + PGO_NAMES)
 if WIN:
     # _WIN32_WINNT: Windows 10, the port's minimum. MSYS2's headers default to it, llvm-mingw's (the release toolchain)
     # to Windows 7, which hides the per-monitor DPI functions gfx_window_win.c and dpi_probe_win.c call.
@@ -384,12 +414,13 @@ def ninja_file(gen, out, mods, pool_depth, skip=(), prebuilt=False, runtime_lib=
         '',
     ]
     objs = {}
+    pgo_dep = f' | {ninja_path(PGO_INPUT)}' if PGO_INPUT else ''
     for mod, files in mods.items():
         objs[mod] = []
         for f in files:
             o = out / 'obj' / mod / (f.stem + '.o')
             if not prebuilt:
-                lines.append(f'build {ninja_path(o)}: cc_gen {ninja_path(f)}')
+                lines.append(f'build {ninja_path(o)}: cc_gen {ninja_path(f)}{pgo_dep}')
                 lines.append('  pool = gen')
             objs[mod].append(o)
     rt = []
@@ -406,7 +437,7 @@ def ninja_file(gen, out, mods, pool_depth, skip=(), prebuilt=False, runtime_lib=
             lines.append(f'build {ninja_path(o)}: rc {ninja_path(src)} | {ninja_path(RUNTIME / "totk.manifest")} '
                          f'{ninja_path(RUNTIME / "totk.ico")}')
         else:
-            lines.append(f'build {ninja_path(o)}: cc_rt {ninja_path(src)}')
+            lines.append(f'build {ninja_path(o)}: cc_rt {ninja_path(src)}{pgo_dep}')
         rt.append(o)
     objs['runtime'] = rt
     every = [o for v in objs.values() for o in v]
@@ -769,6 +800,10 @@ def main():
                     help=f'build only <out>/{LIB_NAME} from runtime/ (no generated C, no totk.exe)')
     ap.add_argument('--runtime-lib', type=Path, help=f'link against this {LIB_NAME} instead of compiling runtime/ '
                                                     f'({", ".join(sorted(INSTALL_SIDE))} and modules.c still compile)')
+    ap.add_argument('--pgo', metavar='gen|FILE',
+                    help='profile-guided optimisation, generated code at -O2: gen builds an instrumented totk.exe '
+                         '(needs --toolchain on Windows: MSYS2 clang ships no profile runtime); FILE builds with that '
+                         '.profdata (tools/pgo.py merge)')
     ap.add_argument('--expected', type=Path,
                     help='expected hashes (default tools/data/expected-<display version>.json)')
     args = ap.parse_args()
@@ -776,6 +811,11 @@ def main():
 
     gen = (args.gen or ROOT / 'generated' / args.version).resolve()
     out = (args.out or ROOT / 'build' / args.version).resolve()
+    if args.pgo:
+        if args.pgo == 'gen' and WIN and not args.toolchain:
+            sys.exit('--pgo gen: MSYS2 UCRT64 clang ships no profile runtime (clang_rt.profile); pass --toolchain with '
+                     'an llvm-mingw directory')
+        set_pgo(args.pgo, out)
     extracted = None if args.exefs and args.romfs else work_dir() / 'extracted' / args.version
     paths = {'exefs': (args.exefs or extracted / 'exefs').resolve(),
              'romfs': (args.romfs or extracted / 'romfs').resolve(),

tools/make_release.py

@@ -15,6 +15,8 @@ cached in toolchain/release-dl/ and checked against the sha256 in PINS; a mismat
                libraries of SDL3, Opus, libvpx, Vulkan and VMA)
     bin/       totk_xlate.exe (+ .id), hactool.exe, SDL3.dll (upstream), libopus-0.dll, ninja.exe
     recomp/, tools/, data/, installer/, licenses/, THIRD_PARTY_NOTICES.md, totk.ini.example
+    data/pgo-<v>.profdata  build/pgo's execution profile of an update (tools/pgo.py) and its .json record: the
+               install compiles that update's code and the runtime libraries with it
     VERSION    the repository's VERSION plus the commit, 0.1.0+973c626; the installer's release string
     TotK-Setup.exe  installer/launcher/launcher.c built with llvm/ (-mwindows): starts python/pythonw.exe -m installer.gui
 
@@ -27,9 +29,11 @@ runtime library reads only what both versions' layout.h share (MOD_COUNT, MOD_MA
 
 A release also needs build/xlate built from the current sources (tools/build_xlate.py); the script stops first if not.
 
-The folder contains nothing derived from the game: check_content runs over it before the script finishes. Then the
-folder's own python runs a check-only installer pass (no game files: it must get through the release, toolchain and host
-checks and stop at the missing game files) and the window's --selftest.
+The folder contains no code, data, text or assets of the game: check_content runs over it before the script finishes.
+The execution profiles name the lifted functions by their address (f_<address>) and hold branch counts and
+control-flow hashes of the generated C, nothing of the game's own bytes. Then the folder's own python runs a check-only
+installer pass (no game files: it must get through the release, toolchain and host checks and stop at the missing game
+files) and the window's --selftest.
 """
 import argparse
 import hashlib
@@ -55,6 +59,7 @@ ROOT = Path(__file__).resolve().parent.parent
 MAIN_ROOT = (ROOT / 'build').resolve().parent   # build/, generated/, toolchain/ and refs/ are junctions in a worktree
 DL = MAIN_ROOT / 'toolchain' / 'release-dl'
 WORK = MAIN_ROOT / 'toolchain' / 'release-work'   # unpacked downloads
+PGO_DIR = MAIN_ROOT / 'build' / 'pgo'   # pgo-<version>.profdata + .json (tools/pgo.py merge)
 
 MSYS2 = 'https://repo.msys2.org/mingw/ucrt64/'
 SDL_VERSION = '3.4.18'
@@ -378,17 +383,39 @@ def lifted_version():
              '(tools/extract.py, recomp/gen.py; docs/building.md)')
 
 
-def runtime_lib(out, llvm):
+def profiles():
+    """{version tag: .profdata} of build/pgo (tools/pgo.py merge). A profile trained on another recomp/ than the one
+    shipped still works but misses the functions whose generated C changed since: warn, so the next release retrains."""
+    tree = subprocess.run(['git', '-C', str(ROOT), 'rev-parse', 'HEAD:recomp'], capture_output=True, text=True).stdout.strip()
+    dirty = subprocess.run(['git', '-C', str(ROOT), 'status', '--porcelain', '--', 'recomp'], capture_output=True,
+                           text=True).stdout.strip()
+    found = {}
+    for p in sorted(PGO_DIR.glob('pgo-*.profdata')):
+        rec = p.with_suffix('.json')
+        record = json.loads(rec.read_text(encoding='utf-8')) if rec.is_file() else {}
+        if record.get('recomp_tree') != tree or dirty:
+            print(f'make_release: warning: {p.name} was trained on recomp tree {record.get("recomp_tree", "unknown")[:12]}, '
+                  f'this is {tree[:12]}{" with local changes" if dirty else ""}: functions whose generated C changed '
+                  'build without a profile; retrain (docs/building.md, "Profile-guided build")', file=sys.stderr)
+        found[p.stem.removeprefix('pgo-')] = p
+    if not found:
+        print(f'make_release: no execution profile in {PGO_DIR}: installs build the game at -O1', file=sys.stderr)
+    return found
+
+
+def runtime_lib(out, llvm, pgo):
     """Build libtotkrt.a (x86-64-v3) and libtotkrt-v2.a (x86-64-v2) with the folder's llvm/, and copy them, the headers
-    and the install-side sources into runtime/."""
+    and the install-side sources into runtime/. pgo: profiles() of the release."""
     version = lifted_version()
     gen = MAIN_ROOT / 'generated' / version
     rt = out / 'runtime'
+    # the runtime is the same for every update: any update's profile covers it (the newest update's, as players run it)
+    newest = sorted(pgo.items())[-1:]
     for cpu, name in (('v3', 'libtotkrt.a'), ('v2', 'libtotkrt-v2.a')):
         build = MAIN_ROOT / 'build' / 'release-check' / f'rtlib-{cpu}'
         run([sys.executable, ROOT / 'tools' / 'build_game.py', '--make-runtime-lib', '--toolchain', llvm, '--cpu', cpu,
              '--version', version, '--deps', out / 'runtime' / 'deps', '--ninja', out / 'bin' / 'ninja.exe',
-             '--gen', gen, '--out', build],
+             '--gen', gen, '--out', build, *(['--pgo', newest[0][1]] if newest else [])],
             env=clean_env(llvm / 'bin', out / 'bin'))
         shutil.copy2(build / 'libtotkrt.a', rt / name)
     for p in sorted((ROOT / 'runtime').rglob('*')):
@@ -399,7 +426,7 @@ def runtime_lib(out, llvm):
         shutil.copy2(ROOT / 'runtime' / name, rt / name)
 
 
-def build_sources(out):
+def build_sources(out, pgo):
     copy_tree(ROOT / 'recomp', out / 'recomp', lambda r: not r.endswith(('.py', '.txt')) or '__pycache__' in r)
     for f in TOOLS_FILES:
         (out / 'tools').mkdir(exist_ok=True)
@@ -409,6 +436,10 @@ def build_sources(out):
     (out / 'data').mkdir(exist_ok=True)
     text = shader_list.history(MAIN_ROOT / 'refs' / 'MacNativeGames', MNG_COMMIT)
     (out / 'data' / 'shader_list.tsv').write_text(text, encoding='utf-8', newline='\n')
+    for p in pgo.values():
+        for f in (p, p.with_suffix('.json')):
+            if f.is_file():
+                shutil.copy2(f, out / 'data' / f.name)
     copy_tree(ROOT / 'installer', out / 'installer', lambda r: '__pycache__' in r or r.startswith('launcher/'))
     shutil.copy2(ROOT / 'totk.ini.example', out / 'totk.ini.example')
     shutil.copy2(ROOT / 'THIRD_PARTY_NOTICES.md', out / 'THIRD_PARTY_NOTICES.md')
@@ -527,12 +558,14 @@ def prove(out, cpu='v3'):
     ext = MAIN_ROOT / 'build' / 'extracted' / version
     chk = MAIN_ROOT / 'build' / 'release-check'
     env = clean_env(out / 'python', out / 'llvm' / 'bin', out / 'bin')
+    pgo = out / 'data' / f'pgo-{version}.profdata'
     t0 = time.time()
     run([out / 'python' / 'python.exe', out / 'tools' / 'build_game.py', '--toolchain', out / 'llvm', '--cpu', cpu,
          '--deps', out / 'runtime' / 'deps', '--runtime', out / 'runtime',
          '--runtime-lib', out / 'runtime' / ('libtotkrt-v2.a' if cpu == 'v2' else 'libtotkrt.a'),
          '--ninja', out / 'bin' / 'ninja.exe', '--version', version, '--gen', gen, '--out', chk, '--exefs', ext / 'exefs',
-         '--romfs', ext / 'romfs', '--profile', chk / 'profile'], env=env)
+         '--romfs', ext / 'romfs', '--profile', chk / 'profile', *(['--pgo', pgo] if pgo.is_file() else [])],
+        env=env)
     print(f'prove: build {time.time() - t0:.0f} s')
     for dll in ('SDL3.dll', 'libopus-0.dll'):
         shutil.copy2(out / 'bin' / dll, chk / dll)
@@ -568,8 +601,9 @@ def main():
     llvm = build_llvm(out)
     build_bin(out)
     build_deps(out)
-    runtime_lib(out, llvm)
-    build_sources(out)
+    pgo = profiles()
+    runtime_lib(out, llvm, pgo)
+    build_sources(out, pgo)
     build_launcher(out, llvm)
     collect_licenses(out)
     check_folder(out)

tools/pgo.py

@@ -0,0 +1,84 @@
+"""Merge the raw profiles of an instrumented build into the .profdata that build_game.py --pgo FILE reads.
+
+    python tools/pgo.py merge build/v143-pgogen --version v143 [--out build/pgo/pgo-v143.profdata]
+
+An instrumented build (build_game.py --pgo gen) writes <build>/pgo/default_<id>.profraw while totk.exe runs (every
+minute and at exit); successive runs of the same exe add to that file. llvm-profdata comes from the llvm-mingw toolchain
+the instrumented build used: its raw format is tied to the compiler version.
+
+Beside the profile goes <out>.json, the training record: the game version and the git tree of recomp/ at the merge.
+The profile matches the generated C of that generator; tools/make_release.py ships it only while recomp/ is unchanged,
+since llvm-profdata matches functions by name and control-flow hash and a regenerated function silently loses its
+counts. Merge right after the training runs, from the commit they were built at.
+"""
+import argparse
+import datetime
+import json
+import subprocess
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent.parent
+EXE = '.exe' if sys.platform == 'win32' else ''
+DEFAULT_DIR = ROOT / 'build' / 'pgo'   # tools/make_release.py takes pgo-<version>.profdata from here
+
+
+def llvm_profdata(toolchain):
+    if toolchain:
+        tool = toolchain / 'bin' / f'llvm-profdata{EXE}'
+    else:
+        found = sorted((ROOT / 'toolchain').glob(f'llvm-mingw-*/llvm-mingw-*/bin/llvm-profdata{EXE}'))
+        tool = found[-1] if found else Path(f'llvm-profdata{EXE}')
+    if not tool.is_file():
+        sys.exit(f'{tool} not found: pass --toolchain')
+    return tool
+
+
+def recomp_tree():
+    """The git tree id of recomp/ at HEAD; None when recomp/ has uncommitted changes."""
+    dirty = subprocess.run(['git', '-C', str(ROOT), 'status', '--porcelain', '--', 'recomp'], capture_output=True,
+                           text=True).stdout.strip()
+    if dirty:
+        return None
+    return subprocess.run(['git', '-C', str(ROOT), 'rev-parse', 'HEAD:recomp'], capture_output=True,
+                          text=True).stdout.strip() or None
+
+
+def merge(build, out, version, toolchain):
+    raw = sorted((build / 'pgo').glob('*.profraw'))
+    if not raw:
+        sys.exit(f'no .profraw in {build / "pgo"}: run the instrumented totk.exe first')
+    tree = recomp_tree()
+    if not tree:
+        sys.exit('recomp/ has uncommitted changes: the record could not name the generator the profile matches')
+    out.parent.mkdir(parents=True, exist_ok=True)
+    tmp = out.with_suffix('.profdata.tmp')
+    cmd = [str(llvm_profdata(toolchain)), 'merge', '--failure-mode=all', '-o', str(tmp), *map(str, raw)]
+    p = subprocess.run(cmd, capture_output=True, text=True)
+    if p.returncode:
+        tmp.unlink(missing_ok=True)
+        sys.exit(f'llvm-profdata merge failed: {(p.stdout + p.stderr).strip()[-2000:]}')
+    tmp.replace(out)
+    record = {'version': version, 'recomp_tree': tree, 'merged': datetime.date.today().isoformat(),
+              'raw_files': len(raw)}
+    out.with_suffix('.json').write_text(json.dumps(record, indent=1) + '\n', encoding='utf-8')
+    size = sum(f.stat().st_size for f in raw)
+    print(f'{out}: {len(raw)} raw files ({size / 2**20:.1f} MiB) -> {out.stat().st_size / 2**20:.1f} MiB; '
+          f'recomp tree {tree[:12]}')
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    sub = ap.add_subparsers(dest='cmd', required=True)
+    m = sub.add_parser('merge', help='merge <build>/pgo/*.profraw into one .profdata and its training record')
+    m.add_argument('build', type=Path, help='the instrumented build directory')
+    m.add_argument('--version', required=True, help='game version the build was made for (v121, v143)')
+    m.add_argument('--out', type=Path, help=f'default {DEFAULT_DIR.relative_to(ROOT).as_posix()}/pgo-<version>.profdata')
+    m.add_argument('--toolchain', type=Path, help='llvm-mingw directory (default: the newest under toolchain/)')
+    a = ap.parse_args()
+    out = (a.out or DEFAULT_DIR / f'pgo-{a.version}.profdata').resolve()
+    merge(a.build.resolve(), out, a.version, a.toolchain)
+
+
+if __name__ == '__main__':
+    main()