With sync_ahead an invalidate of pool storage waits for the GPU write of a value the game was told is done when the range holds no older result, so the pixels TotK reads back once through a buffer per request are no longer zeros or an answer from the ring's last round, which in the Depths put Link in gloom on clean ground; buffers the game reads every frame still get the result of the frame before without a wait

This commit is contained in:
ipavlin98 committed 2026-11-18 00:21:46 +03:00
1 parent 6b0974f08d
commit 68078948f5
8 files changed
+58 -6

No files matched your search

+1
View File
@@ -39,6 +39,7 @@ GPU semantics behind the graphics rows: [maxwell.md](../maxwell.md). The method
| frame counter wanders 50-60 | two 60 Hz clocks (vsync event and present acquire) drifting apart | whether both follow one display tick |
| a 30 Hz thread runs at 27 Hz, slow cutscenes | host sleep overshoot: Windows waitable timers wake 0.3-0.5 ms late | `TOTK_TRACE=sched` sleep asked versus took |
| 20 fps after a scheduler change | a yield back-off that starts too early | yield handling against the console's |
| in the Depths Link takes gloom damage and wears the gloom haze while standing on clean ground, some runs only; only with `sync_ahead = 1` | the game read a GPU result of the frame it was told is done before the GPU wrote it: its one-pixel readbacks go through a buffer per request, which then holds zeros or an answer from the ring's last round (fixed: `vk_memory.c` `vk_mem_shadow_read` waits at the invalidate) | `sync_ahead = 0`; the shadow log line's "the game waited for a GPU write it reads" count |
| climbing or movement wrong only above some fps | game code that assumes a fixed step count per frame | compare at 30, 60 and the failing rate |
## Threads and crashes
+1 -1
View File
@@ -90,7 +90,7 @@ Every guest thread is a host thread. TotK assumes Horizon's scheduler: three app
- Memory pools. Apple's unified memory let MacNativeGames use the game's pools zero-copy. The back end imports the pools as host memory (`VK_EXT_external_memory_host`). On a discrete GPU, pools bound as buffers are mirrored in device-local memory and the pages the CPU wrote (write watch, above) are uploaded before a replay; idle pool regions are made read-only so the write-watch queries skip them (`vk_memory.c`). GPU-rendered textures are written back when the CPU reads them: on a pool invalidate, on a CPU copy, and for small targets once the CPU waits for a fence (`vk_texture.c`).
- Replay thread. `TOTK_REPLAY_ASYNC=1` moves the replay off the submitting thread onto a queue at most one frame behind the game; reads of GPU results wait for the newest writer instead of draining the queue (`vk_backend.c`). It is off by default. Where the game waits for the GPU it overlaps the replay with the game threads (a GTX 1650 in the Depths, issue #88), a gain measured only there; while it was the default, a GTX 1660 Ti at the 60 fps cap with the GPU at 26 % reported uneven motion (#659). It adds up to a frame of latency; `replay_async = 1` turns it on.
- Chunked replay. A large submission is recorded in chunks by several threads (`TOTK_REPLAY_WORKERS`, 0 by default: the sequential replay; 4 is a good value from 8 logical processors) and the chunks go to the queue in order (`vk_exec.c`). A sequential scan keeps a copy of the queue state where each chunk starts and lists what a chunk would otherwise learn from the chunks before it: inline uniform updates, conditions, contact shadow marches, uniform drops. It also makes render targets live in replay order and notes the op, since a chunk can sample a storage before the chunk that renders it got there or after the scan went past a later target there: a chunk's texture slots resolve as of its own op, and the chunks after a texture with no entry yet wait for the chunk that makes it. DLSS's decisions (the G-buffer, the light target, the end of the scene, the scaling pass) are the scan's too, and its copy of the lit scene and its evaluation are records of the chunk that holds the op; a chunk's draws are jittered while they come before the scan's end of the scene. The evaluation waits until the chunks before it are recorded, as it reads what their draws noted (the camera block, the jitter, the per-object motion list, which chunks add to under one lock), and the chunks after it wait for it, as they sample its output. `TOTK_REPLAY_STATE_CHECK=1` compares the state after the last chunk with the scan's.
- Frame ahead. `TOTK_SYNC_AHEAD=1` (`sync_ahead`, off by default) makes the one `nvnSyncWait` per frame that waits for the frame presented last wait for the frame before it, so the game builds the next frame while the GPU still runs the previous one (`nvn_api_core.c`, `nvnSyncWait`). The game then writes memory that in-flight work still reads. This is safe only with shadow storage (`vk_memory.c`, the CPU-visible pools are host-visible copies of guest memory, on by default on Linux and Wine, `shadow = 1` (`TOTK_SHADOW=1`) on native Windows; without it the switch is ignored): a CPU write reaches the GPU only through the shadow copy made before each submit (`vk_mem_shadow_sync`), and a copy that would overwrite bytes an unfinished submission reads is queued as a GPU copy in the prologue command buffer of the part submitted next (`vk_mem_shadow_prologue`, `TOTK_SHADOW_DEFER=1`, the default with the feature) or, with `=0`, waited for on the replay thread (`shadow_reader_wait`). Only that one wait lags (identified by its call site, known for 1.1.0, 1.2.1 and 1.4.3); results the game reads back from the GPU after it can be a frame old. The fence write-back of small render targets (`vk_texture.c`, `write_back_n`) does not wait for the GPU with `TOTK_WB_EAGER=1` (`wb_eager`, the default, but only while frame-ahead is in effect: with `sync_ahead` off or refused the setting does nothing and the write-back reads the image when it runs): `vk_sync_submit` has `vk_texture_eager_record` copy the targets that value rendered into staging buffers inside the same command buffer, so once the value has passed the write-back only copies from staging into the target's storage. Only a target that was written back within the last 64 timeline values is copied (the first write-back of a target takes the blocking path), and the staging of a target past that window, no longer wanted or released goes through the deferred free (`vk_defer_buffer_until`). A snapshot is used only when its value is the one that last rendered the target, nothing recorded a write to the image since (`wr_fence`) and the GPU has finished it; otherwise the write-back reads the image as before, and the exit line `vk: wb_eager:` counts the reasons. With frame-ahead in effect the exit line `vk: fence write-backs waited for the GPU` counts the write-backs that waited for a copy and the time they waited, for every value of the setting. `check` also reads the old way and compares.
- Frame ahead. `TOTK_SYNC_AHEAD=1` (`sync_ahead`, off by default) makes the one `nvnSyncWait` per frame that waits for the frame presented last wait for the frame before it, so the game builds the next frame while the GPU still runs the previous one (`nvn_api_core.c`, `nvnSyncWait`). The game then writes memory that in-flight work still reads. This is safe only with shadow storage (`vk_memory.c`, the CPU-visible pools are host-visible copies of guest memory, on by default on Linux and Wine, `shadow = 1` (`TOTK_SHADOW=1`) on native Windows; without it the switch is ignored): a CPU write reaches the GPU only through the shadow copy made before each submit (`vk_mem_shadow_sync`), and a copy that would overwrite bytes an unfinished submission reads is queued as a GPU copy in the prologue command buffer of the part submitted next (`vk_mem_shadow_prologue`, `TOTK_SHADOW_DEFER=1`, the default with the feature) or, with `=0`, waited for on the replay thread (`shadow_reader_wait`). Only that one wait lags (identified by its call site, known for 1.1.0, 1.2.1 and 1.4.3). GPU writes into pool storage come back with their fence, so a buffer the game reads back every frame after that wait holds the result of the frame before; a buffer it has not read lately holds none, and its `nvnMemoryPoolInvalidateMappedRange` waits for the write (`vk_mem_shadow_read`): TotK reads single pixels through a buffer per request, and in the Depths zeros in one of them put Link in gloom. The fence write-back of small render targets (`vk_texture.c`, `write_back_n`) does not wait for the GPU with `TOTK_WB_EAGER=1` (`wb_eager`, the default, but only while frame-ahead is in effect: with `sync_ahead` off or refused the setting does nothing and the write-back reads the image when it runs): `vk_sync_submit` has `vk_texture_eager_record` copy the targets that value rendered into staging buffers inside the same command buffer, so once the value has passed the write-back only copies from staging into the target's storage. Only a target that was written back within the last 64 timeline values is copied (the first write-back of a target takes the blocking path), and the staging of a target past that window, no longer wanted or released goes through the deferred free (`vk_defer_buffer_until`). A snapshot is used only when its value is the one that last rendered the target, nothing recorded a write to the image since (`wr_fence`) and the GPU has finished it; otherwise the write-back reads the image as before, and the exit line `vk: wb_eager:` counts the reasons. With frame-ahead in effect the exit line `vk: fence write-backs waited for the GPU` counts the write-backs that waited for a copy and the time they waited, for every value of the setting. `check` also reads the old way and compares.
- Recorder thread. The Vulkan calls the replay makes into its command buffer are made by a thread of their own (`vk_rec.c`, after Savage Wind Recomp and LAS Recomp): each `vkCmd*` copies its arguments into a packet of a ring and the recorder makes the calls in order; calls into other command buffers are made at once; the recorder also ends the command buffer, and the submit waits until it has. In the town on an RX 9070 the replay takes 17% less time on the submitting thread. It is off by default on AMD GPUs, where the driver loses the device or hangs the GPU at startup with it (`TOTK_VK_RECORDER=1` forces it on, `=0` makes every call on the submitting thread).
- Uniform snapshot. The game rewrites per-object uniform blocks on other threads soon after it submits the frame
that reads them, while the replay still records it. At the start of a submission's replay every uniform range it
+1
View File
@@ -30,6 +30,7 @@ If a PR touches this code, confirm the fix still holds.
| lime silhouettes in grass; rectangles in the sky | a target outside the output map must keep its texel; zero-filling it wiped the half-res depth | `vk_pipeline.c` `target_wmask` |
| stale grass maps in the far terrain for a second after a hit | write-back of a texture the CPU had rewritten | `vk_texture.c` `write_back` |
| cut grass stays standing; fire does not spread (#132) | CPU reads of GPU-rendered textures without an invalidate | `vk_texture.c` `vk_texture_write_back_small`, `nvn_ext.c` `nvn_pool_invalidate` |
| gloom damage on clean ground in the Depths with `sync_ahead` | the game read a one-request pixel buffer before the GPU of the frame in flight wrote it | `vk_memory.c` `vk_mem_shadow_read`, from `nvn_ext.c` `nvn_pool_invalidate` |
| cloud and fog volumes empty for the session | a one-shot draw skipped instead of waiting for its pipeline | `vk_exec_draw.c` `prepare_draw` |
| tile rectangles | storage buffers bound by Ryujinx first-use order, not the c0 slot | `vk_exec_draw.c` `ssbo_slot` |
| black dialog background and loading map | 3D targets rejected, the grading LUT never written | `vk_exec.c` `resolve_target` |
+3
View File
@@ -52,6 +52,9 @@ uint64_t nvn_sync_ahead_guard(void);
int nvn_sync_ahead_on(void); // 1 when the frame-ahead wait is in effect: TOTK_SYNC_AHEAD on and not refused (no shadow storage, no known wait site)
int vk_mem_shadow_on(void); // vk_memory.c: 1 with shadow storage on (valid once the Vulkan back end is up)
void vk_mem_shadow_fence(void); // vk_memory.c: returns once every shadow sync that read the old guard is done
// vk_memory.c: the game is about to read [addr, addr+size) of pool storage; waits for a GPU write into it that the game
// was told is done, unless the range still holds the result of the write before
void vk_mem_shadow_read(uint64_t addr, uint64_t size);
void gfx_replay_wait_value(uint64_t v); // vk_backend.c: the replay thread submitted value v
uint64_t nvn_fence_last(void); // fence value of the newest submit (0 before the first)
int nvn_depth_minus_w(void); // 1 after DeviceSetDepthMode(NEAR_IS_MINUS_W), else 0
+2 -1
View File
@@ -475,7 +475,8 @@ NVN_API(nvnQueueFenceSync) {
// the frame it presented last; on the Switch that work is long done by then, here it is still in the replay and on the
// GPU (about 11 ms from the submit), and the wait sets the frame time. With the switch on, that wait, and only that
// one, waits for the frame before it instead. What the game reads back of GPU results after the wait can then be a
// frame old. The game's CPU writes to memory the GPU still reads reach the GPU only through shadow copies, which are
// frame old; a buffer that holds no older result is waited for when the game invalidates it (vk_memory.c
// vk_mem_shadow_read). The game's CPU writes to memory the GPU still reads reach the GPU only through shadow copies, which are
// queued for the GPU or wait (nvn_sync_ahead_guard). The wait still lasts until the replay thread submitted the asked
// value, so no copy of that value itself picks up those writes. pa_fence: the newest fence at the latest present and
// the one before, present thread only for writes. pa_guard: the newest value the game was told is done early.
+1
View File
@@ -17,6 +17,7 @@
void nvn_pool_invalidate(const pool_t *p, uint64_t off, uint64_t size) {
if (!p->mem || nvn_gpu_only((uintptr_t)p->mem) || off >= p->size || !size) return;
if (size > p->size - off) size = p->size - off;
vk_mem_shadow_read((uintptr_t)p->mem + off, size);
gfx_backend()->guest_read((uintptr_t)p->mem + off, size);
}
+48 -3
View File
@@ -1303,6 +1303,50 @@ void vk_mem_shadow_fence(void) {
plat_mutex_unlock(&shadow_lock);
}
// TOTK_SYNC_AHEAD, nvnMemoryPoolInvalidateMappedRange: the game is about to read [addr, addr+size) of what the GPU
// writes into pool storage. While the guard is not done, a write into the range that is still pending can belong to a
// value the game was told is done, and its bytes come back only with that value's fence (vk_gpu_written). A range the
// game reads again and again (its depth queries and visibility maps: one buffer written every frame or every few)
// then still holds the result of the write before, and the game gets that one, a sample late: what the switch trades
// for the frame in flight. A range it has not read lately holds no such result. TotK reads single pixels back
// (main+0x407ed8 on 1.4.3) through a 64-byte buffer per request, taken in turn from rings of 64 to 256 and read once,
// right after the frame wait: read early, the pixel is 0 0 0 0 in a buffer never used, and the answer to another
// request, about 27 s old, once the ring came round. In the Depths the zeros put Link in gloom where there is none. So
// a range with a pending write that is not known waits for the guard here. Known: the game invalidated this very
// range within the last SREAD_AGE fence values (about 30 frames), when it held a result or was waited for.
#define SREAD_N 64
#define SREAD_AGE 64
typedef struct { uint64_t addr, size, at; } SRead;
static SRead sread[SREAD_N]; // shadow_lock: the ranges the game invalidated last, with the value issued then
static uint64_t s_read_waits, s_read_ticks; // shadow_lock: since the last log
void vk_mem_shadow_read(uint64_t addr, uint64_t size) {
if (!shadow_on || !size || !nvn_sync_ahead_guard()) return;
plat_mutex_lock(&shadow_lock);
uint64_t now = vk_sync_issued();
SRead *e = &sread[0]; // the range's entry, else the oldest
for (int i = 0; i < SREAD_N; i++) {
if (sread[i].addr == addr && sread[i].size == size) { e = &sread[i]; break; }
if (sread[i].at < e->at) e = &sread[i];
}
int known = e->addr == addr && e->size == size && e->at + SREAD_AGE >= now, pending = 0;
for (int i = 0; i < nspend && !pending; i++) pending = spend[i].lo < addr + size && spend[i].hi > addr;
uint64_t g = nvn_sync_ahead_guard(), wait = pending && !known && g > vk_sync_fence_done() ? g : 0;
// A pending write of a value the game was not told is done, in a range not known: the game reads ahead of its own
// wait and the range stays unknown.
if (known || !pending || wait) *e = (SRead){addr, size, now};
plat_mutex_unlock(&shadow_lock);
if (!wait) return;
uint64_t t0 = qpc();
{
NVN_BLOCKING();
vk_sync_fence_wait(wait, UINT64_MAX);
}
plat_mutex_lock(&shadow_lock);
s_read_waits++, s_read_ticks += qpc() - t0;
plat_mutex_unlock(&shadow_lock);
}
int vk_mem_shadow_on(void) { return shadow_on; }
void vk_mem_shadow_sync(uint64_t fence) {
@@ -1408,10 +1452,11 @@ static void shadow_log(uint64_t frame) {
qpc_ms(s_ticks), nspend);
if (defer_on || s_waits) // only with TOTK_SYNC_AHEAD: the shadow log of everyone else stays as it was
rt_log("mem: shadow deferred (%s): %llu pools, %.1f MiB in %llu copies; replay waited for a reader %llu times, "
"%.2f ms", defer_on ? "on" : "off", (unsigned long long)s_defer_pools, s_defer_bytes / 1048576.0,
(unsigned long long)s_defer_copies, (unsigned long long)s_waits, qpc_ms(s_wait_ticks));
"%.2f ms; the game waited for a GPU write it reads %llu times, %.2f ms", defer_on ? "on" : "off",
(unsigned long long)s_defer_pools, s_defer_bytes / 1048576.0, (unsigned long long)s_defer_copies,
(unsigned long long)s_waits, qpc_ms(s_wait_ticks), (unsigned long long)s_read_waits, qpc_ms(s_read_ticks));
s_syncs = s_queries = s_copied = s_untracked = s_wb_bytes = s_ticks = 0;
s_defer_pools = s_defer_bytes = s_defer_copies = s_wait_ticks = s_waits = 0;
s_defer_pools = s_defer_bytes = s_defer_copies = s_wait_ticks = s_waits = s_read_waits = s_read_ticks = 0;
plat_mutex_unlock(&shadow_lock);
}
+1 -1
View File
@@ -59,7 +59,7 @@
# Let the game build the next frame while the GPU still draws the previous one, 0 or 1. Default: off. Each frame the
# game waits once for the GPU to finish the frame it presented last; with 1 that wait covers the frame before it
# instead. On Great Sky Island (Linux, RX 9070 XT) with shadow_defer = 1 this took about 73 to 100 fps. Costs one more
# frame of input delay, and frame times vary more from frame to frame than with 0. GPU results the game reads back after that wait, for example occlusion queries, can be a frame old.
# frame of input delay, and frame times vary more from frame to frame than with 0. GPU results the game reads back every frame after that wait, for example its depth queries, can be a frame old; a result it reads only once is waited for.
# Works on 1.1.0, 1.2.1 and 1.4.3. Needs shadow storage (shadow below), which is on by default on Linux and
# Wine and needs shadow = 1 on native Windows; without it the setting is ignored and the log says so.
# sync_ahead = 0