Crash logs ended in fatal errors when the system's commit ran out for a moment: a staging buffer (-2), a guest commit
with the system at 24442 of 24447 MB, image creation (-2). Windows grows a system-managed page file and some drivers
charge device memory to the process commit, so the same request often succeeds a moment later.
oom.c holds the wait nvn_api_cmd.c realloc_wait already used: it requests a release, retries every 10 ms for up to
30 s, logs the first 8 with the commit numbers. rt_oom_request sets a flag that vk_texture_new_submit takes (one atomic
load per submit): it drops up to 256 released textures of the two kinds sweep_pressure drops (overwritten, clean
mirrored), never GPU-written content, and frees the headroom blocks. vk_oom_release does at once what needs no submit:
headroom blocks, finished upload batches and deferred objects.
- memory.c commit: waits and retries without frz_lock between tries; every caller waits, boot ones too.
- vk_staging and the draw-data chunks (vk_exec_draw.c scratch): release, then wait; the open upload batch is not
submitted, its caller still records into it.
- create_tex: release and one retry; no wait under the texture lock, the texture is asked for again at its next use.
- the texture write-back buffer: release, then wait.
- the headroom thread keeps no spare block for 30 s after a failure.
TOTK_TEST_OOM=<site>:<n> (commit, staging, image, chunk) fails the first n attempts; probe-kernel forces 3 refused
commits, probe-vkexec 2 staging buffers, 1 image and 1 chunk, and both pass through them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
79c6951 left the closing brace of a debug block in the per-heap loop, so the function ended early and the build failed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
c838d96 counted system memory for every discrete GPU but NVIDIA's, an assumption. An RTX 5080 log (#398) showed 10.4 GB
of process commit at frame 1200, about what the RX 9070 holds with its VRAM counted, and the RTX 3060 Ti in #670 ran
out of system commit (17.8 GB) at 4K with its VRAM full. vk_mem_host_probe now allocates 128 MiB of device-local memory
once after device creation and compares the process commit before and after; half the size or more turns the system
memory cap on. The log line carries the measured amount, so user logs show what each driver does.
RX 9070: "a 128 MiB device-local allocation took 128 MiB of process commit".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The size rule for TOTK_ASTC_BC looked at the largest device-local heap only. On an integrated GPU that heap is system
RAM reported as VRAM (10.6 GiB on the Ally X, 14.9 GiB on the 880M), so RGBA8 stayed on exactly where memory is
shortest. vk_mem_uma() now turns BC3 on there as well.
7900X iGPU, field save, same exe, frame 1800: RGBA8 commit 7047 MB, GPU 49-53 ms, 16.8-18.4 fps; BC3 commit 6155 MB,
GPU 42.6 ms, 21.6-21.8 fps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Headroom woke for every memory type that got new device memory, the staging pool's own 64 MiB blocks included, and
then kept a 256 MiB spare block in that type's default pool. On the RX 9070 the default pools of host memory types 1
and 3 held 4 MiB, beside 512 MiB of spare blocks. Textures and mirrors, whose block allocations headroom moves off the
render thread, live in device-local types.
RX 9070, field save, 60 s: heap 0 VMA blocks 192 MiB instead of 704, driver usage 1384 MiB instead of 1896; 60 fps,
1% low 51.9.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The paths that give memory back under pressure (headroom's spare block, released textures, pool mirrors) watched the
Vulkan heap budget only. On a UMA GPU that budget is gigabytes of the same RAM the game runs in, and on Windows the
AMD driver charges device-local allocations to the process commit one to one (RX 9070: VRAM use and private bytes
grew by the same 1164, 256 and 256 MiB). Crash logs show the result: the process at 4.4-10.5 GB of commit, 16 GB
machines at 80-94% RAM, std::bad_alloc inside amdvlk64.dll (#285), allocation failures on Intel UHD.
vk_mem_host_room (vk_memory.c) is what the system can still give beyond a reserve (TOTK_MEM_RESERVE_MIB, 1536):
commit, and on a UMA GPU physical memory too; Linux overcommits, so there only a UMA GPU's available memory counts.
NVIDIA is left out, not measured. vk_mem_vram caps the budget at use plus that room, and headroom keeps its spare
block only while four blocks fit in it.
RX 9070, field save, 60 s with the room near zero from the start (TOTK_MEM_RESERVE_MIB=21000): commit 7124 MB instead
of 8965, VRAM 2631 MiB instead of 3781, released textures 153 instead of 1866; mirrors refused for 423 MiB of pools,
GPU 4.45 ms a frame instead of 3.34 ms, 60 fps either way. Without pressure nothing changes. The census logs the
room, the device class line the mode.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A driver that runs out of memory faults or throws std::bad_alloc instead of returning an error: #285's log shows
bad_alloc inside amdvlk64.dll under gpl_link, and nothing in the report told that apart from a driver bug. The crash
report now prints the process and system commit, as rt_fatal does for our own failed allocations; --probe-kernel
checks the line.
The census at frames 600 and 1800 logs the commit as a baseline and, per heap, VMA's blocks and allocations beside
the driver's usage and budget, so a report shows whose memory grew.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The crash report named the lifted function and printed the guest registers, but the lifted code keeps guest
registers in host locals and writes them back only at calls, so the dump showed stale values and the faulting
instruction stayed unknown. Logs from #1005, #967 and #998 could not be traced further than the function.
The report now adds the x86 registers of the faulting thread (Windows and Linux), 64 bytes of machine code around
rip with the offset into the lifted function, so the instruction can be found in a local build of the same version,
64 bytes of memory around every register value that points at readable memory, and the first 128 bytes of the
guest stack. --probe-kernel checks the new lines with a child that faults with known values in rbx and x19.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ported onto vk_dlaa_plan (#794): the scene end in the inventory also records its submission and op (end_fence,
end_pos) so chunked replays see it at the same point. DLSS-only: everything sits behind dl_active().
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The arrow alone read as a direction (review of #585). The key keeps the width of the game's own glyph, so the name goes
under a solid arrow rather than on a wider key.
replay_workers stays 0. On the 7900X 4 threads cut the replay by a quarter (town 6.7 -> 5.0 ms, fight2 5.3 -> 3.9 ms)
with no fps change: the game's own threads bound the frame there. Weak CPUs may gain.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
entry_of decides at record time whether a new texture takes the content of a live GPU-written texture at its storage
(alias_copy) or the storage bytes. In a chunked replay the source's GPU ownership is set when a chunk records its
render, so the choice followed the order the threads got to the chunks: a later chunk could take the old bytes before
an earlier chunk recorded the render, and a mirrored texture keeps them. Now a chunk that makes an entry while a live
colour texture of its size sits at the storage waits until the chunks before it are recorded (vk_exec_chunk_settle,
the texture lock released for the wait), skips sources the scan revives at a later op, and ignores GPU ownership a
later chunk set in this submission. slot_resolve and vk_texture_by_id_locked no longer read tslots through a pointer
taken before the wait.
TOTK_REPLAY_SKEW_US (debug) makes the even chunks wait before they record, so the chunk after each one records
first in every submission. With it at 3000 us and 4 threads the field save runs clean (grass drawn, 0 validation errors); the alias path
did not fire there (0 copied, 0 filled from storage), so this does not reproduce the missing grass reported on
RADV, and the fix there stays unconfirmed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DLSS kept the replay on one thread: its decisions follow the passes in replay order, its jitter and camera are noted
draw by draw, and its evaluation reads the per-object motion list the scene's draws build. Now the scan takes DLSS's
decisions on its own state (vk_dlaa_plan, split out of vk_dlaa_pass) and notes the scene camera in replay order
(vk_dlaa_scan_draw), and the copy of the lit scene and the evaluation are records of the chunk that holds the op
(vk_dlaa_exec). A chunk's draws are jittered while they come before the op the scan ended the scene at. The
evaluation waits until the chunks before it are recorded, and the chunks after it wait for it (a hold), since they
sample its output. Chunks note their draws for the per-object motion one at a time (mo_lock in vk_exec_draw.c); the
uniform overrides of a motion draw and shader_for's last-frame variant flag are per thread, as the other chunks' draws
go on meanwhile; the FXAA shader table takes concurrent lookups. fxaa = 0 runs in chunks too.
With dlaa = 1 at 4K (DLSS Quality) on 1.2.1 and an RTX 5070, replay_workers = 4 takes the frame rate from 70 to 78
fps where the CPU is the limit, with no one-frame glitch in 241 consecutive frames.
(cherry picked from commit 04f8bb1b57)
replay_workers = N (0, the default, keeps the sequential replay) records a submission of enough draws in chunks that start between passes, on N threads and in twice as many chunks, and submits them in order. In the town at 4K on an RTX 5070, 4 threads take the replay from 8.2 to 4.5 ms a frame and the frame rate from 71 to 79.
A sequential scan on the submitting thread applies only what is queue state (fixed-function and bind ops, render targets, render enable and conditions, samples-passed counting) and keeps a copy of it where each chunk starts, with how far its lists got: inline uniform updates, RENDER_COND results, contact shadow marches and the ops that drop uniform copies. A chunk takes those from the lists instead of from the chunks before it, starts with every image in GENERAL and ends with a full barrier, so its recording depends only on the copy and the lists.
Each thread records with a context of its own: the replay's memos, descriptor state, occlusion query segments and per-command-buffer state moved from file statics into VkExec and VkExecDraw, and the tables the contexts share (stand-ins, depth twins, views, mirrors, timestamps, completion callbacks, crumbs) take locks or atomics. The scan makes render targets live in replay order and notes the op (vk_texture_revive), since a chunk can sample a storage before the chunk that renders it got there or after the scan went past a later target there: a chunk's texture slots resolve as of its own op, and the chunks after a texture with no entry yet wait for the chunk that makes it. Uniform copies are dropped at the op that writes them (vk_usnap_drop_at).
Contact shadows plan their march in replay order (vk_contact_plan) and the chunk that holds its op records it; the HDR source keeps the copy that comes last in replay order. DLSS and fxaa = 0 act on the frame draw by draw and keep the replay sequential. TOTK_REPLAY_STATE_CHECK=1 compares the state after the last chunk with the scan's.
A draw or dispatch whose shaders write storage buffers drops the uniform copies they cover from its own op on, as the other GPU writes do (scan_writes). The scan still translates no shader: it reads the shaders the recording contexts translated from a table shared with them (vk_exec_draw_writes), and an op whose shader no context translated yet holds the chunks after it until it is recorded. Before, the chunk that recorded such a write dropped the copies for every position at that moment, so a later chunk recorded first bound the copy taken at the submit call: on an RX 7800 XT on RADV the grass near the camera came out dark or unshaded in some runs with 4 threads.
TOTK_TRACE=perf counts the chunks held per submission by reason and logs when each chunk of the last submission began. The settings menu has the thread count under Advanced, Scheduling, with the count this run uses under it.
The recorder thread begins, ends and submits the submission's open part (vk_rec.c rec_end_cb), so a chunk's mirror prologue stays off it: vk_sync_part waits until the recorder is idle, as the part pool allows one thread at a time, then records, ends and submits the prologue itself. Through the recorder the prologue could reach the queue before it was ended, and the GPU lost the device at the title screen.
(cherry picked from commit b94e2e9642)
FCMP and the vector compares no longer flush denormal operands under FPCR.FZ: rt_fpcr_sync keeps MXCSR.DAZ equal to
FPCR.FZ, and x86 compares under DAZ read a denormal as a zero of its sign, as ARM does under FZ. The directed RMode
check of FN(op) and FN(vop) moved into the result check's branch, and min/max skip it, since they do not round.
Hot lifted functions shrink by about 10% of their x86 instructions; the town camera-pan bench (1080p, two runs each)
goes from 10.25 to 9.78 ms per frame. simd.py (60000 encodings), lift.py and func.py report 0 failures.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With vb_used the streams a draw does not read got the zero buffer, so draws in a row that read different streams
bound all 17 slots again almost every draw. On an RTX 5060 Laptop that cost 16 % of the frame rate (issue #974). Now
only the read streams are compared, one bind covers the changed ones, and the others keep the buffer bound earlier
in the command buffer, which the deferred frees keep alive until the submission is done.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The setup wrote romfs-patches-pc for 16:9 only, so prompts = pc on a 21:9, 32:9, 16:10 or 3:2 screen kept the game
resolution at 1920x1080 (cfg.c) or, with output_res set to the screen's shape, fell back to the Switch prompts.
- make_romfs_patches.py --reuse <folder>: a file of an earlier overlay whose contents match is hard-linked (copied
where links fail) instead of compressed again; the output is byte for byte the same.
- The setup builds romfs-patches-pc-<w>x<h> for each of ASPECT_SIZES from romfs-patches-pc with --reuse: only the
Common archive is compressed, so the fifteen folders take about two minutes and 6 MiB each instead of 22 minutes
and 73 MiB each.
- cfg.c: the display-shaped output_res default applies with prompts = pc when romfs-patches-pc-<w>x<h> is there
(romfs_shaped_pc_ready); Xbox prompts keep 1920x1080 as before.
An i5-1235U or Core 3 100U has 2 P-cores. Binding the five frame-critical threads there, with a spinning recorder
among them, crowds two cores and their hyperthreads while the E-cores stay idle. plat_perf_core_setup now counts the
allowed physical P-cores and selects nothing below PC_MIN_CORES (4), so the scheduler places those threads as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rec_submit queued each part's submission and then drained the ring until the recorder had submitted it: 0.29 ms a
frame in the town on a Ryzen 7900X, 0.34 ms on 4 cores. Now it publishes the submission and returns
(TOTK_REC_ASYNC=1, default). The queue order the present, uploads, readbacks and device waits rely on is kept by
vk_queue_lock: a thread's first lock waits until the recorder submitted what was queued before (rec_queue_wait).
The recorder itself and a thread that already holds the lock skip the wait. rec_submit reports the first failed
submission the recorder made. Town, in-run A/B: replay -4.1% on all cores, -2.9% on 4 cores / 8 threads; queue
users waited about 0.02 ms a frame. fight2 with TOTK_REC_CHECK and the town under sync validation are clean.
Where the recorder does not poll (fewer than 6 usable logical processors) the producer now publishes the ring head
every 8192 words instead of 64, as Savage Wind Recomp does (runtime/nvn/vulkan/record.c rec_init): each publish
there costs an event set and a wake-up. Town on 4 logical processors: frame interval -1.3%. TOTK_REC_PUB=ab
compares both; the perf log counts the recorder's wake-ups and the queue waits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On an Intel CPU with P-cores and E-cores a thread on an E-core runs at about 60% of a P-core's speed, and the frame
waits for its slowest critical thread. The game's main thread, the thread that presents, and the vk replay, recorder
and present threads now select the CPU sets of the P-core logical processors the process affinity allows. Nothing is
selected on a CPU with one efficiency class or when the affinity leaves only one kind of core. TOTK_PCORE=0 turns it
off; "power:" log lines say what was selected and which threads took it. LAS Recomp does the same for its game,
render and vsync threads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The executable exports NvOptimusEnablement and AmdPowerXpressRequestHighPerformance, which NVIDIA Optimus and AMD
switchable graphics read at start to run it on the dedicated GPU; Optimus can otherwise hide its GPU from a program
it put on the integrated one. At start the process opts out of EcoQoS (E-cores and low clocks on hybrid Intel CPUs
when Windows deems it in the background) and of Windows 11 ignoring its timer resolution while the window is hidden.
One "power:" log line says whether Windows accepted it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Guest waits and waits for a virtual core usually end within microseconds. A thread now polls for 20 us with the
global lock released before it sleeps on its condition variable, so a quick wake costs no OS wake-up
(TOTK_KSPIN_US, 0 turns it off). The recorder polls by the clock for 60 us instead of 4096 pauses, which last a few
us on Zen 4 and about 140 us on Coffee Lake (TOTK_REC_SPIN_US). Both stay off with fewer than 6 usable logical
processors. Savage Wind Recomp polls the same way before it parks.
Machine A, town save, in-run A/B: frame interval -1.8% on average for the kernel poll (5 of 7 block pairs), -0.7%
with the process limited to 4 cores / 8 threads; the recorder poll is neutral there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Shadow storage is on by default under Wine as well, and the region snapshots were only measured on a native Linux
host. TOTK_SHADOW_SNAP=1 still turns them on under Wine.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each 2 MiB region is now queried at most once per guest submission, shared with the texture checks, instead of once
per pool at every part of a submission. On a Steam Deck at Lookout Landing the replay thread's write tracking fell
from about 4.4 to 1.3 ms a frame and the frame rate rose about 7.5%. TOTK_SHADOW_SNAP=0 restores the old queries.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vk_memory.c: dev now keeps every finalized pool until retire_value (at
least the submission being recorded), which also covers a shadow with a
deferred copy, so shadow_had_deferred goes. The deferred shadow prologue
and dev's upload scratch pool both stay.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cfg.c: the key list takes dev's DESWIZZLE and keeps this branch's
DYNAMIC_RES and FIXED_RES_PATCH.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vk_pool_fini freed import, physical and virtual pools right after vkQueueWaitIdle, which covers submitted work only;
with the synchronous replay another game thread may be recording a submission that names the pool. Every pool kind
now retires through the completion queue at the later of the recording's value and the next one.
destroy_tex freed the texture record while other threads still held &e->pub (render targets for a pass, storage
bindings writing wr_fence through it): they read a stale VkImage and cached views by that handle. The record is now
freed with its image's stamp (vk_defer_free), and views of a dropped entry are deferred at once instead of cached.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the synchronous replay (the default since 7be68ec) a game thread finalizing a texture destroys released texture
cache entries while another game thread records a submission that slot_resolve may already have handed one of them.
defer() saw the recording's value only on the recording thread and stamped the others with the last submitted value,
so the completion thread freed the image before the submission naming it ran: "write invalid" device faults on NVIDIA
(#788, #802). vk_sync_submit now notes the value at the start of the recording, and every deferral made from then on
waits for it. A fight run counted 10 such deferrals during the world load.
Device faults now log the deferred-free state, and with VK_EXT_device_address_binding_report the object that held the
faulting address and whether it was unbound before the fault (TOTK_ADDR_REPORT=0 turns the map off). The census line
counts deferrals made off the recording thread during a recording.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
