mirror of
https://github.com/valentderah/totk-pc-recomp.git
synced 2026-11-21 12:55:57 +00:00
The in-place frame ahead merges every mirror write-back by words, is refused with import memory, and the probe checks its pending-write list
Without shadow storage a plain GPU copy back into a mirrored range lands after the game, told the value is done a frame early, may have written other words of that range for the next frame; with TOTK_SYNC_AHEAD_WBMERGE (1 by default) the copies' and updates' write-backs go through the word merge as a shader's do. With memory = import the GPU reads vertex, index and storage data in place, and a check run on the islands saw the game rewrite about 10 KiB of them a frame after the wait, so the switch is refused there unless shadow storage is on. --probe-vkmem checks the pending-write list (newest writer below a bound, pruning, joining when full). The one-off readback rule copied from Savage Wind gets its THIRD_PARTY_NOTICES.md row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b558317a7a
commit
54b4e092cd
10 files changed
+79
-16
No files matched your search
@@ -33,6 +33,7 @@ Code copied or adapted from another project is listed here with its source, the
|
||||
| AMD FidelityFX SDK 1.1.4 subset (public host and shader headers, frame interpolation, optical flow and FSR 3 upscaler components, the Vulkan backend `ffx_vk.cpp` and its GLSL shaders, shared helpers), with four marked Savage Wind patches (64-byte aligned backend scratch arrays, doubled context size where `wchar_t` is 4 bytes, all six descriptor pool sizes, the upscaler's Catmull-Rom history reprojection) and one of ours (the upscaler's luma history declared `rgba16f`) | https://github.com/GPUOpen-LibrariesAndSDKs/FidelityFX-SDK/tree/v1.1.4, Copyright (C) 2024 Advanced Micro Devices, Inc., via https://github.com/Revivedx/SavageWindRecomp @ `bf6146c14c37fee8781a601e5a544b93dff6d75e` (`third_party/FidelityFX/`) | MIT | `runtime/third_party/FidelityFX/` (compiled into `totk.exe`; licence text in `runtime/third_party/FidelityFX/LICENSE.txt`, shipped as `licenses/FidelityFX-SDK-LICENSE.txt`) |
|
||||
| SavageWindRecomp FidelityFX build support, frame generation and FSR 3 upscaler set-up: the SDK-to-clang compat header, the glslang SPIR-V shader generator with its resource reflection and its FSR 3 upscaler permutations, the blob lookup and the backend and context creation of `runtime/nvn/vulkan/framegen.cpp`, the upscaler context with its shared images and the dispatch parameters of `runtime/nvn/vulkan/fsr3.cpp`, the pair's present spacing of `runtime/nvn/vulkan/device.c` `present_wait_turn` | https://github.com/Revivedx/SavageWindRecomp @ `bf6146c14c37fee8781a601e5a544b93dff6d75e` (`third_party/FidelityFX/ffx_compat.h`, `tools/ffx_shaders.py`, `runtime/nvn/vulkan/framegen.cpp`, `runtime/nvn/vulkan/fsr3.cpp`, `runtime/nvn/vulkan/device.c`) | MIT | `runtime/third_party/FidelityFX/ffx_compat.h`, `runtime/third_party/FidelityFX/spirv/ffx_shaders_vk.h` and `ffx_fsr3upscaler_vk.h` (output of `tools/ffx_shaders.py`), `tools/ffx_shaders.py`, `runtime/vk_fg.cpp`, `runtime/vk_fsr3.cpp`, `runtime/vk_backend.c` (`present_pair`) |
|
||||
| SavageWindRecomp settings menu: two-pane option rows (row look, left / right stepping, hold speed-up, description pane), the menu style and tab bar | https://github.com/Revivedx/SavageWindRecomp @ `575ad88e02f9a89a9a885d90039d48af33c982f4` (`runtime/ui/option_rows.mm`, `runtime/ui/menu.mm`), Copyright (c) 2026 Revivedx and the Savage Wind Recomp contributors, Copyright (c) 2026 Peiikai | MIT | `runtime/ui/option_rows.cpp`, `runtime/ui/option_rows.h`, `runtime/ui/menu.cpp` |
|
||||
| SavageWindRecomp GPU overlap one-off readback rule: a target not read back within the last 16 frames (64-entry table) makes the frame wait wait up to the value that rendered it | https://github.com/Revivedx/SavageWindRecomp @ `bf6146c14c37fee8781a601e5a544b93dff6d75e` (`runtime/nvn/vulkan/exec.c` readback_schedule, renderer_wait_seq), Copyright (c) 2026 Revivedx and the Savage Wind Recomp contributors, Copyright (c) 2026 Peiikai | MIT | `runtime/vk_texture.c` (oneoff_streaming, vk_texture_oneshot) |
|
||||
| SavageWindRecomp descriptor sets allocated in growing batches per layout | https://github.com/Revivedx/SavageWindRecomp @ `740b18025fe11ba03c117dbb56668db304b5043e` (`runtime/nvn/vulkan/exec.c` vk_alloc_set), Copyright (c) 2026 Revivedx and the Savage Wind Recomp contributors, Copyright (c) 2026 Peiikai | MIT | `runtime/vk_exec_draw.c` (alloc_set_batched) |
|
||||
| SavageWindRecomp performance overlay: the frame ring and its one-second summary, the NVML calls, the sampling loop, the full panel's layout | https://github.com/Revivedx/SavageWindRecomp @ `575ad88e02f9a89a9a885d90039d48af33c982f4` (`runtime/perfmon.c`, `runtime/perfmon.h`, `perf_overlay_draw` in `runtime/ui/menu.mm`), Copyright (c) 2026 Revivedx and the Savage Wind Recomp contributors, Copyright (c) 2026 Peiikai | MIT | `runtime/perfmon.c`, `runtime/perfmon.h`, `runtime/ui/menu.cpp` |
|
||||
| Roboto Medium font (Dear ImGui's `misc/fonts/Roboto-Medium.ttf`, unmodified) | https://fonts.google.com/specimen/Roboto, via https://github.com/peiikai/LASRecomp @ `da35b83a9611d046a9a681aae668d7085954d37f` (`third_party/imgui/misc/fonts/`), Copyright 2011 Google Inc. | Apache-2.0 | `runtime/ui/fonts/Roboto-Medium.ttf` (licence text in `runtime/ui/fonts/LICENSE-Roboto.txt`), embedded by `runtime/ui/menu.cpp` |
|
||||
|
||||
@@ -90,7 +90,8 @@ Every guest thread is a host thread. TotK assumes Horizon's scheduler: three app
|
||||
- Memory pools. Apple's unified memory let MacNativeGames use the game's pools zero-copy. The back end imports the pools as host memory (`VK_EXT_external_memory_host`). On a discrete GPU, pools bound as buffers are mirrored in device-local memory and the pages the CPU wrote (write watch, above) are uploaded before a replay; idle pool regions are made read-only so the write-watch queries skip them (`vk_memory.c`). GPU-rendered textures are written back when the CPU reads them: on a pool invalidate, on a CPU copy, and for small targets once the CPU waits for a fence (`vk_texture.c`).
|
||||
- Replay thread. `TOTK_REPLAY_ASYNC=1` moves the replay off the submitting thread onto a queue at most one frame behind the game; reads of GPU results wait for the newest writer instead of draining the queue (`vk_backend.c`). It is off by default. Where the game waits for the GPU it overlaps the replay with the game threads (a GTX 1650 in the Depths, issue #88), a gain measured only there; while it was the default, a GTX 1660 Ti at the 60 fps cap with the GPU at 26 % reported uneven motion (#659). It adds up to a frame of latency; `replay_async = 1` turns it on.
|
||||
- Chunked replay. A large submission is recorded in chunks by several threads (`TOTK_REPLAY_WORKERS`, 0 by default: the sequential replay; 4 is a good value from 8 logical processors) and the chunks go to the queue in order (`vk_exec.c`). A sequential scan keeps a copy of the queue state where each chunk starts and lists what a chunk would otherwise learn from the chunks before it: inline uniform updates, conditions, contact shadow marches, uniform drops. It also makes render targets live in replay order and notes the op, since a chunk can sample a storage before the chunk that renders it got there or after the scan went past a later target there: a chunk's texture slots resolve as of its own op, and the chunks after a texture with no entry yet wait for the chunk that makes it. DLSS's decisions (the G-buffer, the light target, the end of the scene, the scaling pass) are the scan's too, and its copy of the lit scene and its evaluation are records of the chunk that holds the op; a chunk's draws are jittered while they come before the scan's end of the scene. The evaluation waits until the chunks before it are recorded, as it reads what their draws noted (the camera block, the jitter, the per-object motion list, which chunks add to under one lock), and the chunks after it wait for it, as they sample its output. `TOTK_REPLAY_STATE_CHECK=1` compares the state after the last chunk with the scan's.
|
||||
- Frame ahead. `TOTK_SYNC_AHEAD=1` (`sync_ahead`, off by default) makes the one `nvnSyncWait` per frame that waits for the frame presented last wait for the frame before it, so the game builds the next frame while the GPU still runs the previous one (`nvn_api_core.c`, `nvnSyncWait`). The game then writes memory that in-flight work still reads. This is safe only with shadow storage (`vk_memory.c`, the CPU-visible pools are host-visible copies of guest memory, on by default on Linux and Wine, `shadow = 1` (`TOTK_SHADOW=1`) on native Windows; without it the switch is ignored): a CPU write reaches the GPU only through the shadow copy made before each submit (`vk_mem_shadow_sync`), and a copy that would overwrite bytes an unfinished submission reads is queued as a GPU copy in the prologue command buffer of the part submitted next (`vk_mem_shadow_prologue`, `TOTK_SHADOW_DEFER=1`, the default with the feature) or, with `=0`, waited for on the replay thread (`shadow_reader_wait`). Only that one wait lags (identified by its call site, known for 1.1.0, 1.2.1 and 1.4.3). GPU writes into pool storage come back with their fence, so a buffer the game reads back every frame after that wait holds the result of the frame before; a buffer it has not read lately holds none, and its `nvnMemoryPoolInvalidateMappedRange` waits for the write (`vk_mem_shadow_read`): TotK reads single pixels through a buffer per request, and in the Depths zeros in one of them put Link in gloom. The exit line `mem: the game waited for a GPU write it reads` counts these waits and their time. On 1.4.3 the patch set `qstamp` (`recomp/patches-v143.txt`) lets the game take its depth query results one frame older; it follows the switch and is off when the switch is refused. The fence write-back of small render targets (`vk_texture.c`, `write_back_n`) does not wait for the GPU with `TOTK_WB_EAGER=1` (`wb_eager`, the default, but only while frame-ahead is in effect: with `sync_ahead` off or refused the setting does nothing and the write-back reads the image when it runs): `vk_sync_submit` has `vk_texture_eager_record` copy the targets that value rendered into staging buffers inside the same command buffer, so once the value has passed the write-back only copies from staging into the target's storage. Only a target that was written back within the last 64 timeline values is copied (the first write-back of a target takes the blocking path), and the staging of a target past that window, no longer wanted or released goes through the deferred free (`vk_defer_buffer_until`). A snapshot is used only when its value is the one that last rendered the target, nothing recorded a write to the image since (`wr_fence`) and the GPU has finished it; otherwise the write-back reads the image as before, and the exit line `vk: wb_eager:` counts the reasons. With frame-ahead in effect the exit line `vk: fence write-backs waited for the GPU` counts the write-backs that waited for a copy and the time they waited, for every value of the setting. `check` also reads the old way and compares.
|
||||
- Frame ahead. `TOTK_SYNC_AHEAD=1` (`sync_ahead`, off by default) makes the one `nvnSyncWait` per frame that waits for the frame presented last wait for the frame before it, so the game builds the next frame while the GPU still runs the previous one (`nvn_api_core.c`, `nvnSyncWait`). The game then writes memory that in-flight work still reads. Two forms guard that. With shadow storage (`vk_memory.c`, the CPU-visible pools are host-visible copies of guest memory, on by default on Linux and Wine, `shadow = 1` (`TOTK_SHADOW=1`) on native Windows) a CPU write reaches the GPU only through the shadow copy made before each submit (`vk_mem_shadow_sync`), and a copy that would overwrite bytes an unfinished submission reads is queued as a GPU copy in the prologue command buffer of the part submitted next (`vk_mem_shadow_prologue`, `TOTK_SHADOW_DEFER=1`, the default with the feature) or, with `=0`, waited for on the replay thread (`shadow_reader_wait`). Only that one wait lags (identified by its call site, known for 1.1.0, 1.2.1 and 1.4.3). GPU writes into pool storage come back with their fence, so a buffer the game reads back every frame after that wait holds the result of the frame before; a buffer it has not read lately holds none, and its `nvnMemoryPoolInvalidateMappedRange` waits for the write (`vk_mem_shadow_read`): TotK reads single pixels through a buffer per request, and in the Depths zeros in one of them put Link in gloom. The exit line `mem: the game waited for a GPU write it reads` counts these waits and their time. On 1.4.3 the patch set `qstamp` (`recomp/patches-v143.txt`) lets the game take its depth query results one frame older; it follows the switch and is off when the switch is refused. The fence write-back of small render targets (`vk_texture.c`, `write_back_n`) does not wait for the GPU with `TOTK_WB_EAGER=1` (`wb_eager`, the default, but only while frame-ahead is in effect: with `sync_ahead` off or refused the setting does nothing and the write-back reads the image when it runs): `vk_sync_submit` has `vk_texture_eager_record` copy the targets that value rendered into staging buffers inside the same command buffer, so once the value has passed the write-back only copies from staging into the target's storage. Only a target that was written back within the last 64 timeline values is copied (the first write-back of a target takes the blocking path), and the staging of a target past that window, no longer wanted or released goes through the deferred free (`vk_defer_buffer_until`). A snapshot is used only when its value is the one that last rendered the target, nothing recorded a write to the image since (`wr_fence`) and the GPU has finished it; otherwise the write-back reads the image as before, and the exit line `vk: wb_eager:` counts the reasons. With frame-ahead in effect the exit line `vk: fence write-backs waited for the GPU` counts the write-backs that waited for a copy and the time they waited, for every value of the setting. `check` also reads the old way and compares.
|
||||
- Frame ahead in place. Without shadow storage the GPU reads the game's own memory, as in Savage Wind's overlap, and `vk_ahead_init` (`runtime/vk_ahead.c`) turns on guards for what the GPU reads or writes there after the recording, each with a switch `TOTK_SYNC_AHEAD_<NAME>` (1 by default unless noted) for A/B runs. `MSYNC`: the VRAM mirrors' sync copies take the synced pages from a staging copy the replay thread makes at submit (`vk_memory.c` `stage_ranges`), not from guest memory when the GPU runs them; the frame wait returns only after the replay submitted the asked value, so those are that frame's bytes. A synced range that an earlier value still writes back into waits for that value first, or the copy would drop its GPU words. `WBMERGE`: every GPU write into a mirror goes back through the word merge (`wb_merge_record`), so a write-back that lands after the game wrote the range for the next frame puts back only the words the GPU changed. `DESWZ`: a texture the CPU rewrote after its last upload is deswizzled on the CPU (`upload_gpu`, reason `frame ahead`). `READS`: GPU writes into CPU-visible memory are listed with their value until it completes (bounded, `aw_add`), and an invalidate of a range not read lately waits for the writer's own value (`ahead_read`). `ONESHOT`: the frame wait waits up to the value that rendered a dirty small target not written back within the last 16 frames (Savage Wind's one-off rule, `vk_texture_oneshot`), so a photo or a pick reads its own frame, while targets read every frame or in turns get the frame before. `QUERIES` (0 by default): the same for samples-passed reports; TotK reads those two updates after it asks (1.2.1 `f_70010c3298`). With import memory (`TOTK_MEMORY=import`, the integrated-GPU mode) the GPU reads vertex, index and storage data in place, which no guard covers (a check run saw the game rewrite about 10 KiB of them a frame after the wait, particles mostly), so the switch is refused there without shadow storage. The exit lines `mem: sync-ahead in place` and `sync-ahead: frame waits made exact` give the guards' waits and copies. `TOTK_SYNC_AHEAD_CHECK=1` runs the detector below with the late wait itself.
|
||||
- Frame-ahead check. `TOTK_SYNC_AHEAD=check` keeps the frame wait exact and measures what a one-frame-late wait without shadow storage would race (`runtime/vk_ahead.c`). The replay notes, with each submission's value, every access of guest memory the GPU makes after the recording: in-place bindings of imported pools without a VRAM mirror (`VK_AHEAD_READ`), the mirror's sync copies (`vk_mem_mirror_prologue`), GPU deswizzle sources (`vk_texture.c` `upload_gpu`) and the mirror write-backs (`vk_mem_mirror_end`). At the frame wait for value A, whose late form would have waited for L, the notes of values in (L, A] form the frame's read set: merged per class, gap-joined into at most 512 write-watch queries that stamp the writes made so far into the shared page generations (`vk_mem_ahead_observe`). At the next frame-wait entry the queries run again, and every page of the set stamped since is a CPU write the late wait would have raced; memory without a write watch is compared by page hashes. GPU writes into CPU-visible memory and reports are listed with their value: an invalidate of a range written by a value in (L, A] of the last frame wait, and the fence write-back of a small target rendered by one, count as stale readbacks. One line `sync-ahead check:` per second of host time and a run summary with the largest offenders by pool at exit. `--probe-vkmem` checks the detector on a watched pool (`vk_ahead_selftest`); `tests/sync_ahead_check.py` checks that the wait stays exact.
|
||||
- Recorder thread. The Vulkan calls the replay makes into its command buffer are made by a thread of their own (`vk_rec.c`, after Savage Wind Recomp and LAS Recomp): each `vkCmd*` copies its arguments into a packet of a ring and the recorder makes the calls in order; calls into other command buffers are made at once; the recorder also ends the command buffer, and the submit waits until it has. In the town on an RX 9070 the replay takes 17% less time on the submitting thread. It is off by default on AMD GPUs, where the driver loses the device or hangs the GPU at startup with it (`TOTK_VK_RECORDER=1` forces it on, `=0` makes every call on the submitting thread).
|
||||
- Uniform snapshot. The game rewrites per-object uniform blocks on other threads soon after it submits the frame
|
||||
|
||||
@@ -51,6 +51,7 @@ void nvn_sync_ahead_present(void);
|
||||
uint64_t nvn_sync_ahead_guard(void);
|
||||
int nvn_sync_ahead_on(void); // 1 when the frame-ahead wait is in effect: TOTK_SYNC_AHEAD=1 and a wait site known for this version
|
||||
int vk_mem_shadow_on(void); // vk_memory.c: 1 with shadow storage on (valid once the Vulkan back end is up)
|
||||
int vk_mem_mirror_on(void); // vk_memory.c: 1 when pools used as buffers get VRAM mirrors (memory = mirror or auto on a discrete GPU)
|
||||
void vk_mem_shadow_fence(void); // vk_memory.c: returns once every shadow sync that read the old guard is done
|
||||
// vk_memory.c: the game is about to read [addr, addr+size) of pool storage; waits for a GPU write into it that the game
|
||||
// was told is done, unless the range still holds the result of the write before
|
||||
|
||||
@@ -509,6 +509,14 @@ static int sync_mode(void) {
|
||||
if (mode < 0) {
|
||||
const char *v = cfg_value("TOTK_SYNC_AHEAD");
|
||||
int on = strcmp(v, "0") != 0 && strcmp(v, "check") != 0, check = !strcmp(v, "check");
|
||||
// In place (no shadow storage) the guards cover the VRAM mirrors' copies; with memory = import the GPU reads
|
||||
// vertex, index and storage data straight from the game's memory, and a check run (TOTK_MEMORY=import,
|
||||
// islands) saw the game rewrite ~10 KiB of them a frame after the wait, particles mostly.
|
||||
if (on && !vk_mem_shadow_on() && !vk_mem_mirror_on()) {
|
||||
rt_log("nvn: TOTK_SYNC_AHEAD needs shadow storage or the VRAM mirrors (memory = import reads the game's "
|
||||
"vertex and storage data in place); the game waits for the GPU as before");
|
||||
on = 0;
|
||||
}
|
||||
if ((on || check) && !frame_wait_lr()) {
|
||||
rt_log("nvn: TOTK_SYNC_AHEAD: no frame wait site is known for this game version; the game waits for the "
|
||||
"GPU as before");
|
||||
|
||||
@@ -376,6 +376,11 @@ static int test_import(pool_t *pool) {
|
||||
GCHECK(!ah, "sync-ahead check self test failed at check %d", ah);
|
||||
printf("probe-vkmem: sync-ahead check: a write after the frame wait into the read set is flagged, one before it is not\n");
|
||||
g_gpu_checks++;
|
||||
int aw = vk_mem_ahead_selftest();
|
||||
GCHECK(!aw, "sync-ahead pending-write list self test failed at check %d", aw);
|
||||
printf("probe-vkmem: sync-ahead in place: the pending-write list names the newest writer below a bound, prunes at "
|
||||
"completion and joins when full\n");
|
||||
g_gpu_checks++;
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
@@ -283,6 +283,7 @@ uint64_t vk_mem_ahead_written(uint64_t lo, uint64_t hi, uint32_t since, void (*f
|
||||
void *arg);
|
||||
int vk_mem_pool_info(uint64_t addr, uint64_t *lo, uint64_t *size, uint32_t *flags, int *mirrored); // 0: no pool
|
||||
void vk_mem_ahead_log(void); // TOTK_SYNC_AHEAD=1 in place: the guards' waits and copies over the run, at exit
|
||||
int vk_mem_ahead_selftest(void); // --probe-vkmem: the in-place form's pending-write list; 0 passed
|
||||
void vk_mem_gpu_wrote(const VkMemSpan *sp, uint64_t size);
|
||||
// A shader's storage-buffer write: goes back through the write-back merge (vk_memory.c), which keeps the words the GPU
|
||||
// did not change.
|
||||
|
||||
+5
-7
@@ -107,6 +107,7 @@ void vk_ahead_init(void) {
|
||||
VkAheadGuards *g = &vk_ahead_g;
|
||||
g->lag = 1;
|
||||
g->msync = rt_knob("SYNC_AHEAD_MSYNC", "1");
|
||||
g->wbmerge = rt_knob("SYNC_AHEAD_WBMERGE", "1");
|
||||
g->deswz = rt_knob("SYNC_AHEAD_DESWZ", "1");
|
||||
g->reads = rt_knob("SYNC_AHEAD_READS", "1");
|
||||
g->oneshot = rt_knob("SYNC_AHEAD_ONESHOT", "1");
|
||||
@@ -114,13 +115,10 @@ void vk_ahead_init(void) {
|
||||
// goes 1 -> 2 or 3 at one update, 2 -> 3 at the next, and the report is read in state 3), by then the frame
|
||||
// wait one frame late has covered that frame. Savage Wind (BOTW reads them right after the wait) has it on Mali.
|
||||
g->queries = rt_knob("SYNC_AHEAD_QUERIES", "0");
|
||||
rt_log("sync-ahead: in place (no shadow storage), memory %s; guards: mirror syncs staged %d, deswizzle of "
|
||||
"rewritten textures on the CPU %d, invalidates wait for the writer %d, one-off small targets %d, "
|
||||
"samples-passed reports %d", vk_mirror_on ? "mirror" : "import", g->msync, g->deswz, g->reads, g->oneshot,
|
||||
g->queries);
|
||||
if (!vk_mirror_on)
|
||||
rt_log("sync-ahead: warning: import memory: vertex, index and storage data are read in place on the GPU, "
|
||||
"no guard covers them");
|
||||
rt_log("sync-ahead: in place (no shadow storage), memory %s; guards: mirror syncs staged %d, write-backs "
|
||||
"merged %d, deswizzle of rewritten textures on the CPU %d, invalidates wait for the writer %d, one-off "
|
||||
"small targets %d, samples-passed reports %d", vk_mirror_on ? "mirror" : "import", g->msync, g->wbmerge,
|
||||
g->deswz, g->reads, g->oneshot, g->queries);
|
||||
}
|
||||
vk_ahead_on = check || (vk_ahead_g.lag && rt_knob("SYNC_AHEAD_CHECK", "0"));
|
||||
if (!vk_ahead_on) return;
|
||||
|
||||
+2
-1
@@ -16,11 +16,12 @@ enum { AHW_WRITE, AHW_COUNTER, AHW_SAMPLES, AHW_EVENT, AHW_N };
|
||||
extern int vk_ahead_on;
|
||||
// The in-place form's switches, read once by vk_ahead_init; every guard is 0 without lag. A/B: TOTK_SYNC_AHEAD_<NAME>.
|
||||
// msync: the mirror syncs copy the synced pages into staging on the CPU at submit (vk_memory.c vk_mem_mirror_prologue);
|
||||
// wbmerge: every GPU write into a mirror goes back through the word merge, not only a shader's (vk_memory.c gpu_wrote);
|
||||
// deswz: a texture re-uploaded after the CPU rewrote it takes the CPU deswizzle (vk_texture.c upload_gpu); reads: an
|
||||
// invalidate of a range not read lately waits for the pending GPU write into it (vk_memory.c vk_mem_shadow_read);
|
||||
// oneshot: a one-off small target is waited for, up to its value (vk_ahead_oneshot); queries (default 0): so is a
|
||||
// samples-passed report.
|
||||
typedef struct { int lag, msync, deswz, reads, oneshot, queries; } VkAheadGuards;
|
||||
typedef struct { int lag, msync, wbmerge, deswz, reads, oneshot, queries; } VkAheadGuards;
|
||||
extern VkAheadGuards vk_ahead_g;
|
||||
void vk_ahead_init(void); // vk_memory.c mirror_mode, once at device start, shadow mode known: reads TOTK_SYNC_AHEAD
|
||||
// The newest value in (lo, hi] whose GPU result the game reads right after the frame wait (0: none): the in-place
|
||||
|
||||
+43
-2
@@ -1504,6 +1504,43 @@ static void ahead_read(uint64_t addr, uint64_t size) {
|
||||
ah_time(&ah_rd_waits, &ah_rd_ns, &ah_rd_max, qpc() - t0);
|
||||
}
|
||||
|
||||
// --probe-vkmem: the pending-write list with values no submission has: the newest overlapping writer below a bound,
|
||||
// pruning at completion, and a full list joining a write into its nearest entry, which still covers it. The list is
|
||||
// restored after. 0: passed, else the failed check.
|
||||
int vk_mem_ahead_selftest(void) {
|
||||
static AWrite saved[AW_N];
|
||||
plat_mutex_lock(&aw_lock);
|
||||
int n0 = naw, peak0 = aw_peak, bad = 0;
|
||||
uint64_t joined0 = aw_joined, f = vk_sync_issued() + 1000000, a = 0x10000000;
|
||||
memcpy(saved, aw, sizeof aw);
|
||||
naw = 0;
|
||||
plat_mutex_unlock(&aw_lock);
|
||||
aw_add(a, a + 0x100, f);
|
||||
aw_add(a + 0x80, a + 0x200, f + 1);
|
||||
aw_add(a, a + 0x100, f); // the same range again: no new entry
|
||||
plat_mutex_lock(&aw_lock);
|
||||
if (naw != 2) bad = 1;
|
||||
else if (aw_writer(a, a + 0x40, 0, UINT64_MAX) != f) bad = 2;
|
||||
else if (aw_writer(a + 0xc0, a + 0xd0, 0, UINT64_MAX) != f + 1) bad = 3;
|
||||
else if (aw_writer(a + 0xc0, a + 0xd0, 0, f + 1) != f) bad = 4; // below the value being recorded
|
||||
else if (aw_writer(a + 0x200, a + 0x300, 0, UINT64_MAX)) bad = 5; // touching, not overlapping
|
||||
if (!bad) {
|
||||
aw_prune(f);
|
||||
if (naw != 1 || aw_writer(a, a + 0x40, f, UINT64_MAX)) bad = 6;
|
||||
}
|
||||
int fill = AW_N - naw; // the entry check 6 left stays: fill up to exactly AW_N, so only the next add joins
|
||||
plat_mutex_unlock(&aw_lock);
|
||||
for (int i = 0; !bad && i < fill; i++) aw_add(a + 0x1000 * (uint64_t)(i + 1), a + 0x1000 * (uint64_t)(i + 1) + 0x10, f + 2);
|
||||
uint64_t lo = a + 0x1000 * (AW_N + 8);
|
||||
if (!bad) aw_add(lo, lo + 0x10, f + 3); // full: joined with the nearest
|
||||
plat_mutex_lock(&aw_lock);
|
||||
if (!bad && (naw != AW_N || aw_joined != joined0 + 1 || aw_writer(lo, lo + 0x10, 0, UINT64_MAX) != f + 3)) bad = 7;
|
||||
memcpy(aw, saved, sizeof aw);
|
||||
naw = n0, aw_peak = peak0, aw_joined = joined0;
|
||||
plat_mutex_unlock(&aw_lock);
|
||||
return bad;
|
||||
}
|
||||
|
||||
void vk_mem_ahead_log(void) {
|
||||
plat_mutex_lock(&aw_lock);
|
||||
rt_log("mem: sync-ahead in place: the game waited for a GPU write it reads %llu times, %.1f ms in all, longest "
|
||||
@@ -1527,6 +1564,7 @@ void vk_mem_shadow_read_log(void) {
|
||||
}
|
||||
|
||||
int vk_mem_shadow_on(void) { return shadow_on; }
|
||||
int vk_mem_mirror_on(void) { return vk_mirror_on; }
|
||||
|
||||
void vk_mem_shadow_sync(uint64_t fence) {
|
||||
if (!shadow_on) return;
|
||||
@@ -2382,8 +2420,11 @@ static void gpu_wrote(const VkMemSpan *sp, uint64_t size, int shader) {
|
||||
if (r && size) gpu_writes(r, sp->offset, size);
|
||||
if (!r || r->mstate != MS_ON || sp->buffer != r->mbuf || !size) return;
|
||||
plat_mutex_lock(&mirror_lock);
|
||||
// A binding to the end of its pool (no size) can span hundreds of MiB: those stay copies.
|
||||
if (shader && !(sp->offset & 3) && !(size & 3) && size <= MERGE_MAX && merging() && snap_add(r, sp->offset, size)) {
|
||||
// A binding to the end of its pool (no size) can span hundreds of MiB: those stay copies. TOTK_SYNC_AHEAD=1 in place
|
||||
// (vk_ahead_g.wbmerge): the copies' and updates' writes merge too. A plain copy back lands when the GPU runs it, after
|
||||
// the game, told the value is done a frame early, may have written other words of the range for the next frame,
|
||||
// and would put the older words back; the merge writes only the words this value changed.
|
||||
if ((shader || vk_ahead_g.wbmerge) && !(sp->offset & 3) && !(size & 3) && size <= MERGE_MAX && merging() && snap_add(r, sp->offset, size)) {
|
||||
plat_mutex_unlock(&mirror_lock);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -56,7 +56,9 @@ static void vk_ahead_frame_lag(uint64_t asked, uint64_t lagged, int behind) {
|
||||
lag_calls++, lag_asked = asked, lag_value = lagged, lag_behind = behind;
|
||||
}
|
||||
static void vk_ahead_frame_done(int ok) { assert(lag_calls == 1); done_calls++, done_ok = ok; }
|
||||
static int mirror;
|
||||
static int vk_mem_shadow_on(void) { return shadow; }
|
||||
static int vk_mem_mirror_on(void) { return mirror; }
|
||||
static uint64_t nvn_fence_last(void) { return last; }
|
||||
static void rt_log(const char *format, ...) { (void)format; }
|
||||
// The patch table: set qstamp exists on 1.4.3 only and starts as TOTK_SYNC_AHEAD says (runtime/patches.c).
|
||||
@@ -77,9 +79,10 @@ PRODUCTION
|
||||
static uint32_t seed = 0x19ed634b;
|
||||
static uint32_t random_word(void) { seed ^= seed << 13; seed ^= seed >> 17; return seed ^= seed << 5; }
|
||||
int main(int argc, char **argv) {
|
||||
assert(argc == 5);
|
||||
assert(argc == 6);
|
||||
rt_game_version = atoi(argv[1]); enabled = atoi(argv[2]); shadow = atoi(argv[3]);
|
||||
int det = atoi(argv[4]);
|
||||
mirror = atoi(argv[5]);
|
||||
int supported = rt_game_version == 110 || rt_game_version == 121 || rt_game_version == 143;
|
||||
uint64_t site = rt_game_version == 110 ? 0x7000b39aa4ull : rt_game_version == 121 ? 0x7000b23dfcull :
|
||||
rt_game_version == 143 ? 0x70002db99cull : 0;
|
||||
@@ -87,7 +90,8 @@ int main(int argc, char **argv) {
|
||||
qstamp_on = rt_game_version == 143 && enabled;
|
||||
last = 20; nvn_sync_ahead_present();
|
||||
last = 40; nvn_sync_ahead_present();
|
||||
int on = enabled == 1 && supported, check = enabled == 2 && supported; // shadow off: the in-place form
|
||||
// shadow off: the in-place form, which needs the VRAM mirrors
|
||||
int on = enabled == 1 && supported && (shadow || mirror), check = enabled == 2 && supported;
|
||||
vk_ahead_on = check || (on && det);
|
||||
assert(pa_fence[0] == (on || check ? 40 : 0));
|
||||
assert(pa_fence[1] == (on || check ? 20 : 0));
|
||||
@@ -155,8 +159,8 @@ int main(int argc, char **argv) {
|
||||
nvnSyncWait(&cpu);
|
||||
assert(pa_guard == 39);
|
||||
}
|
||||
printf("sync-ahead version %d enabled %d shadow %d detector %d: 100000 wait selections passed (%d relaxed, %d up "
|
||||
"to a one-off value)\n", rt_game_version, enabled, shadow, det, relaxed_cases, oneshot_cases);
|
||||
printf("sync-ahead version %d enabled %d shadow %d detector %d mirror %d: 100000 wait selections passed (%d relaxed, %d up "
|
||||
"to a one-off value)\n", rt_game_version, enabled, shadow, det, mirror, relaxed_cases, oneshot_cases);
|
||||
}
|
||||
'''
|
||||
|
||||
@@ -178,7 +182,9 @@ def main():
|
||||
for enabled in (0, 1, 2):
|
||||
for shadow in (0, 1):
|
||||
for det in (0, 1):
|
||||
subprocess.run([str(exe), str(version), str(enabled), str(shadow), str(det)], check=True)
|
||||
for mirror in (0, 1):
|
||||
subprocess.run([str(exe), str(version), str(enabled), str(shadow), str(det), str(mirror)],
|
||||
check=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
Reference in new issue
Block a user