mirror of
https://github.com/valentderah/totk-pc-recomp.git
synced 2026-11-19 04:15:57 +00:00
Merge pull request #1253: sync_ahead works on game 1.1.0, and game-time scripts find the player there (Marjaperse)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
commit
f53b86c61d
6 files changed
+231
-9
No files matched your search
@@ -90,7 +90,7 @@ Every guest thread is a host thread. TotK assumes Horizon's scheduler: three app
|
||||
- Memory pools. Apple's unified memory let MacNativeGames use the game's pools zero-copy. The back end imports the pools as host memory (`VK_EXT_external_memory_host`). On a discrete GPU, pools bound as buffers are mirrored in device-local memory and the pages the CPU wrote (write watch, above) are uploaded before a replay; idle pool regions are made read-only so the write-watch queries skip them (`vk_memory.c`). GPU-rendered textures are written back when the CPU reads them: on a pool invalidate, on a CPU copy, and for small targets once the CPU waits for a fence (`vk_texture.c`).
|
||||
- Replay thread. `TOTK_REPLAY_ASYNC=1` moves the replay off the submitting thread onto a queue at most one frame behind the game; reads of GPU results wait for the newest writer instead of draining the queue (`vk_backend.c`). It is off by default. Where the game waits for the GPU it overlaps the replay with the game threads (a GTX 1650 in the Depths, issue #88), a gain measured only there; while it was the default, a GTX 1660 Ti at the 60 fps cap with the GPU at 26 % reported uneven motion (#659). It adds up to a frame of latency; `replay_async = 1` turns it on.
|
||||
- Chunked replay. A large submission is recorded in chunks by several threads (`TOTK_REPLAY_WORKERS`, 0 by default: the sequential replay; 4 is a good value from 8 logical processors) and the chunks go to the queue in order (`vk_exec.c`). A sequential scan keeps a copy of the queue state where each chunk starts and lists what a chunk would otherwise learn from the chunks before it: inline uniform updates, conditions, contact shadow marches, uniform drops. It also makes render targets live in replay order and notes the op, since a chunk can sample a storage before the chunk that renders it got there or after the scan went past a later target there: a chunk's texture slots resolve as of its own op, and the chunks after a texture with no entry yet wait for the chunk that makes it. DLSS's decisions (the G-buffer, the light target, the end of the scene, the scaling pass) are the scan's too, and its copy of the lit scene and its evaluation are records of the chunk that holds the op; a chunk's draws are jittered while they come before the scan's end of the scene. The evaluation waits until the chunks before it are recorded, as it reads what their draws noted (the camera block, the jitter, the per-object motion list, which chunks add to under one lock), and the chunks after it wait for it, as they sample its output. `TOTK_REPLAY_STATE_CHECK=1` compares the state after the last chunk with the scan's.
|
||||
- Frame ahead. `TOTK_SYNC_AHEAD=1` (`sync_ahead`, off by default) makes the one `nvnSyncWait` per frame that waits for the frame presented last wait for the frame before it, so the game builds the next frame while the GPU still runs the previous one (`nvn_api_core.c`, `nvnSyncWait`). The game then writes memory that in-flight work still reads. This is safe only with shadow storage (`vk_memory.c`, the CPU-visible pools are host-visible copies of guest memory, on by default on Linux and Wine, `shadow = 1` (`TOTK_SHADOW=1`) on native Windows; without it the switch is ignored): a CPU write reaches the GPU only through the shadow copy made before each submit (`vk_mem_shadow_sync`), and a copy that would overwrite bytes an unfinished submission reads is queued as a GPU copy in the prologue command buffer of the part submitted next (`vk_mem_shadow_prologue`, `TOTK_SHADOW_DEFER=1`, the default with the feature) or, with `=0`, waited for on the replay thread (`shadow_reader_wait`). Only that one wait lags (identified by its call site, known for 1.2.1 and 1.4.3); results the game reads back from the GPU after it can be a frame old.
|
||||
- Frame ahead. `TOTK_SYNC_AHEAD=1` (`sync_ahead`, off by default) makes the one `nvnSyncWait` per frame that waits for the frame presented last wait for the frame before it, so the game builds the next frame while the GPU still runs the previous one (`nvn_api_core.c`, `nvnSyncWait`). The game then writes memory that in-flight work still reads. This is safe only with shadow storage (`vk_memory.c`, the CPU-visible pools are host-visible copies of guest memory, on by default on Linux and Wine, `shadow = 1` (`TOTK_SHADOW=1`) on native Windows; without it the switch is ignored): a CPU write reaches the GPU only through the shadow copy made before each submit (`vk_mem_shadow_sync`), and a copy that would overwrite bytes an unfinished submission reads is queued as a GPU copy in the prologue command buffer of the part submitted next (`vk_mem_shadow_prologue`, `TOTK_SHADOW_DEFER=1`, the default with the feature) or, with `=0`, waited for on the replay thread (`shadow_reader_wait`). Only that one wait lags (identified by its call site, known for 1.1.0, 1.2.1 and 1.4.3); results the game reads back from the GPU after it can be a frame old.
|
||||
- Recorder thread. The Vulkan calls the replay makes into its command buffer are made by a thread of their own (`vk_rec.c`, after Savage Wind Recomp and LAS Recomp): each `vkCmd*` copies its arguments into a packet of a ring and the recorder makes the calls in order; calls into other command buffers are made at once; the recorder also ends the command buffer, and the submit waits until it has. In the town on an RX 9070 the replay takes 17% less time on the submitting thread. It is off by default on AMD GPUs, where the driver loses the device or hangs the GPU at startup with it (`TOTK_VK_RECORDER=1` forces it on, `=0` makes every call on the submitting thread).
|
||||
- Uniform snapshot. The game rewrites per-object uniform blocks on other threads soon after it submits the frame
|
||||
that reads them, while the replay still records it. At the start of a submission's replay every uniform range it
|
||||
|
||||
@@ -185,14 +185,16 @@ static uint64_t read_ptr(uint64_t a) {
|
||||
uint64_t p = 0;
|
||||
return read_guest(a, &p, 8) ? p : 0;
|
||||
}
|
||||
// 1.2.1: [main+0x4728538] (1.1.0: [main+0x4630a68], 1.4.3: [main+0x3add238]) +0x1e8 scene, its component 13 lists the
|
||||
// 1.2.1: [main+0x4728538] (1.1.0: [[main+0x4630a68]], 1.4.3: [main+0x3add238]) +0x1e8 scene, its component 13 lists the
|
||||
// resident actors; the one named "Player" has its position at +0x2b4 (MacNativeGames world_survey.c, game_version.h
|
||||
// TOTK_GAME_V121/V143). The 1.1.0 slot was measured from its own dereference chain: 181 scene accesses in that build
|
||||
// read it, and one of them reproduces this walk (the 14-component check, component 13, the 0x70 stride and the name
|
||||
// compare) instruction for instruction.
|
||||
// compare) instruction for instruction. Its slot is a GOT entry pointing to the singleton's storage, so it needs
|
||||
// one more dereference than the direct singleton slots of 1.2.1 and 1.4.3.
|
||||
static int player_pos(float xyz[3]) {
|
||||
uint64_t module = read_ptr(MOD_MAIN_BASE + (RT_V143 ? 0x3add238 : RT_V110 ? 0x4630a68 : 0x4728538)),
|
||||
scene = module ? read_ptr(module + 0x1e8) : 0;
|
||||
uint64_t module = read_ptr(MOD_MAIN_BASE + (RT_V143 ? 0x3add238 : RT_V110 ? 0x4630a68 : 0x4728538));
|
||||
if (RT_V110) module = read_ptr(module);
|
||||
uint64_t scene = module ? read_ptr(module + 0x1e8) : 0;
|
||||
int32_t count = 0;
|
||||
if (!scene || !read_guest(scene + 0x50, &count, 4) || count < 14 || count > 256) return 0;
|
||||
uint64_t comps = read_ptr(scene + 0x58), resident = comps ? read_ptr(comps + 13 * 8) : 0;
|
||||
|
||||
@@ -480,10 +480,11 @@ NVN_API(nvnQueueFenceSync) {
|
||||
// value, so no copy of that value itself picks up those writes. pa_fence: the newest fence at the latest present and
|
||||
// the one before, present thread only for writes. pa_guard: the newest value the game was told is done early.
|
||||
static uint64_t pa_fence[2], pa_guard;
|
||||
// The return address of that wait. 1.2.1: 0x7000b23dfc. 1.4.3: 0x70002db99c, the second of the two nvnSyncWait calls
|
||||
// in f_70002db8ac; like on 1.2.1 its sync comes from a two-entry ring indexed per frame, and the first call waits on
|
||||
// one object's sync behind a mode check. 0: no site known, the switch stays off.
|
||||
// The return address of that wait. 1.1.0: 0x7000b39aa4, 1.2.1: 0x7000b23dfc, 1.4.3: 0x70002db99c. The second
|
||||
// nvnSyncWait in 1.1.0 f_7000b39a00 and 1.4.3 f_70002db8ac uses the per-frame sync, like on 1.2.1; the first
|
||||
// call waits on one object's sync behind a mode check. 0: no site known, the switch stays off.
|
||||
static uint64_t frame_wait_lr(void) {
|
||||
if (rt_game_version == 110) return 0x7000b39aa4ull;
|
||||
return rt_game_version == 121 ? 0x7000b23dfcull : rt_game_version == 143 ? 0x70002db99cull : 0;
|
||||
}
|
||||
static int sync_ahead(void) {
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
"""Check the production player-position walk used to start the game-time clock.
|
||||
|
||||
python tests/frame_world_check.py --cc path/to/clang
|
||||
"""
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
HARNESS = r'''
|
||||
#include <assert.h>
|
||||
#include <math.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
#define MOD_MAIN_BASE 0x7000000000ull
|
||||
#define RT_V110 (version == 110)
|
||||
#define RT_V143 (version >= 143)
|
||||
static int version;
|
||||
static uint8_t memory[0x8000];
|
||||
static uint64_t singleton;
|
||||
static int plat_mem_read(uint64_t address, void *out, size_t size) {
|
||||
uint64_t slot = MOD_MAIN_BASE + (RT_V143 ? 0x3add238 : RT_V110 ? 0x4630a68 : 0x4728538);
|
||||
if (address == slot && size == 8) { memcpy(out, &singleton, 8); return 1; }
|
||||
if (address > sizeof memory || size > sizeof memory - address) return 0;
|
||||
memcpy(out, memory + address, size);
|
||||
return 1;
|
||||
}
|
||||
static void put64(uint64_t address, uint64_t value) { memcpy(memory + address, &value, 8); }
|
||||
static void put32(uint64_t address, int32_t value) { memcpy(memory + address, &value, 4); }
|
||||
PRODUCTION
|
||||
static void fixture(void) {
|
||||
memset(memory, 0, sizeof memory);
|
||||
singleton = RT_V110 ? 0x800 : 0x1000;
|
||||
put64(0x800, 0x1000); // 1.1.0's GOT entry points to the singleton's storage
|
||||
put64(0x1000 + 0x1e8, 0x2000);
|
||||
put32(0x2000 + 0x50, 14); put64(0x2000 + 0x58, 0x3000);
|
||||
put64(0x3000 + 13 * 8, 0x4000);
|
||||
put32(0x4000 + 0x20, 2); put64(0x4000 + 0x28, 0x5000);
|
||||
// A missing first actor must not hide the valid player in the second entry.
|
||||
put64(0x5000 + 0x70 + 8, 0x5800); put64(0x5800 + 0x40, 0x6000);
|
||||
put64(0x6000 + 0x218, 0x7000); memcpy(memory + 0x7000, "Player", 7);
|
||||
float position[3] = {-700.0f, 140.0f, 870.0f};
|
||||
memcpy(memory + 0x6000 + 0x2b4, position, sizeof position);
|
||||
}
|
||||
int main(void) {
|
||||
const int versions[] = {110, 121, 143};
|
||||
for (unsigned i = 0; i < sizeof versions / sizeof *versions; i++) {
|
||||
version = versions[i]; fixture();
|
||||
float position[3] = {0};
|
||||
assert(player_pos(position));
|
||||
assert(position[0] == -700.0f && position[1] == 140.0f && position[2] == 870.0f);
|
||||
singleton = 0; assert(!player_pos(position));
|
||||
fixture(); put64(0x800, 0); if (RT_V110) assert(!player_pos(position));
|
||||
fixture(); put32(0x2000 + 0x50, 13); assert(!player_pos(position));
|
||||
fixture(); put32(0x4000 + 0x20, 0); assert(!player_pos(position));
|
||||
fixture(); put64(0x4000 + 0x28, 0); assert(!player_pos(position));
|
||||
fixture(); memory[0x7000] = 'X'; assert(!player_pos(position));
|
||||
fixture(); float bad = NAN; memcpy(memory + 0x6000 + 0x2b4, &bad, 4);
|
||||
assert(!player_pos(position));
|
||||
printf("world-clock player walk: version %d passed\n", version);
|
||||
}
|
||||
}
|
||||
'''
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
parser.add_argument("--cc", default=shutil.which("clang") or r"C:\msys64\ucrt64\bin\clang.exe")
|
||||
parser.add_argument("--source", type=Path, default=ROOT / "runtime/frame_rate.c")
|
||||
args = parser.parse_args()
|
||||
source = args.source.read_text()
|
||||
start = source.index("static int read_guest(")
|
||||
end = source.index("static void speed_report(", start)
|
||||
with tempfile.TemporaryDirectory(prefix="world-clock-", dir=ROOT / "build") as directory:
|
||||
directory = Path(directory)
|
||||
path, exe = directory / "check.c", directory / "check.exe"
|
||||
path.write_text(HARNESS.replace("PRODUCTION", source[start:end]))
|
||||
subprocess.run([args.cc, "-std=c11", "-O2", "-Wall", "-Werror", str(path), "-o", str(exe)], check=True)
|
||||
subprocess.run([str(exe)], check=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,133 @@
|
||||
"""Exercise the production sync-ahead wait selection with a deterministic back end.
|
||||
|
||||
python tests/sync_ahead_check.py [--cc path/to/clang]
|
||||
"""
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
HARNESS = r'''
|
||||
#include <assert.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
typedef struct { uint64_t x[31]; } Cpu;
|
||||
typedef struct { uint32_t magic, fenced; uint64_t fence; } sync_t;
|
||||
enum { NVN_SYNC_WAIT_RESULT_ALREADY_SIGNALED, NVN_SYNC_WAIT_RESULT_CONDITION_SATISFIED,
|
||||
NVN_SYNC_WAIT_RESULT_TIMEOUT_EXPIRED };
|
||||
static int rt_game_version, enabled, shadow, succeed, relaxed;
|
||||
static uint64_t completed, last, replay_value, synced, waited, waited_timeout;
|
||||
static int replay_calls, shadow_calls, sync_calls, wait_calls;
|
||||
uint64_t nvn_sync_ahead_guard(void);
|
||||
static uint64_t done(void) { return completed; }
|
||||
static void cpu_synced(uint64_t value) { synced = value; sync_calls++; }
|
||||
static int wait_for(uint64_t value, uint64_t timeout) {
|
||||
assert(shadow_calls == relaxed);
|
||||
wait_calls++; waited = value; waited_timeout = timeout;
|
||||
if (succeed) completed = value;
|
||||
return succeed;
|
||||
}
|
||||
typedef struct { uint64_t (*fence_done)(void); void (*cpu_synced)(uint64_t); int (*fence_wait)(uint64_t, uint64_t); } GfxBackend;
|
||||
static const GfxBackend backend = {done, cpu_synced, wait_for};
|
||||
static const GfxBackend *gfx_backend(void) { return &backend; }
|
||||
static int cfg_bool(const char *name) { assert(!strcmp(name, "TOTK_SYNC_AHEAD")); return enabled; }
|
||||
static int vk_mem_shadow_on(void) { return shadow; }
|
||||
static uint64_t nvn_fence_last(void) { return last; }
|
||||
static void rt_log(const char *format, ...) { (void)format; }
|
||||
static void gfx_replay_wait_value(uint64_t value) { replay_value = value; replay_calls++; }
|
||||
static void vk_mem_shadow_fence(void) {
|
||||
assert(nvn_sync_ahead_guard() >= replay_value);
|
||||
shadow_calls++;
|
||||
}
|
||||
#define NVN_BLOCKING() ((void)0)
|
||||
#define NVN_API(name) static void name(Cpu *c)
|
||||
#define arg_p(c, i) ((void *)(uintptr_t)(c)->x[i])
|
||||
#define arg_x(c, i) ((c)->x[i])
|
||||
#define ret_x(c, value) ((c)->x[0] = (value))
|
||||
PRODUCTION
|
||||
static uint32_t seed = 0x19ed634b;
|
||||
static uint32_t random_word(void) { seed ^= seed << 13; seed ^= seed >> 17; return seed ^= seed << 5; }
|
||||
int main(int argc, char **argv) {
|
||||
assert(argc == 4);
|
||||
rt_game_version = atoi(argv[1]); enabled = atoi(argv[2]); shadow = atoi(argv[3]);
|
||||
int supported = rt_game_version == 110 || rt_game_version == 121 || rt_game_version == 143;
|
||||
uint64_t site = rt_game_version == 110 ? 0x7000b39aa4ull : rt_game_version == 121 ? 0x7000b23dfcull :
|
||||
rt_game_version == 143 ? 0x70002db99cull : 0;
|
||||
assert(frame_wait_lr() == site);
|
||||
last = 20; nvn_sync_ahead_present();
|
||||
last = 40; nvn_sync_ahead_present();
|
||||
assert(pa_fence[0] == (enabled && shadow && supported ? 40 : 0));
|
||||
assert(pa_fence[1] == (enabled && shadow && supported ? 20 : 0));
|
||||
int relaxed_cases = 0, polls = 0, other_waits = 0;
|
||||
for (int i = 0; i < 100000; i++) {
|
||||
sync_t sync = {.fenced = random_word() % 2, .fence = random_word() % 65};
|
||||
uint64_t timeout = i % 3 == 0 ? 0 : i % 3 == 1 ? 1 : UINT64_MAX;
|
||||
uint64_t lr = i / 3 % 3 == 0 ? site : i / 3 % 3 == 1 ? site + 4 : site - 0x18;
|
||||
completed = random_word() % 65;
|
||||
uint64_t initial_done = completed, target = sync.fence;
|
||||
succeed = random_word() % 2;
|
||||
relaxed = sync.fenced && enabled && shadow && supported && timeout && lr == site &&
|
||||
completed < target && target > 20 && target <= 40;
|
||||
relaxed_cases += relaxed; polls += timeout == 0; other_waits += lr != site;
|
||||
if (relaxed) target = 20;
|
||||
replay_calls = shadow_calls = sync_calls = wait_calls = 0;
|
||||
synced = waited = waited_timeout = replay_value = UINT64_MAX;
|
||||
pa_guard = 0;
|
||||
Cpu cpu = {0}; cpu.x[0] = (uintptr_t)&sync; cpu.x[1] = timeout; cpu.x[30] = lr;
|
||||
nvnSyncWait(&cpu);
|
||||
assert(replay_calls == relaxed && shadow_calls == relaxed);
|
||||
assert(pa_guard == (relaxed ? sync.fence : 0));
|
||||
if (relaxed) assert(replay_value == sync.fence);
|
||||
if (!sync.fenced) {
|
||||
assert(cpu.x[0] == NVN_SYNC_WAIT_RESULT_TIMEOUT_EXPIRED && !sync_calls && !wait_calls);
|
||||
} else if (initial_done >= target) {
|
||||
assert(cpu.x[0] == NVN_SYNC_WAIT_RESULT_ALREADY_SIGNALED && sync_calls == 1 && !wait_calls);
|
||||
assert(synced == (relaxed ? initial_done : sync.fence));
|
||||
} else {
|
||||
assert(wait_calls == 1 && waited == target && waited_timeout == timeout);
|
||||
assert(cpu.x[0] == (succeed ? NVN_SYNC_WAIT_RESULT_CONDITION_SATISFIED : NVN_SYNC_WAIT_RESULT_TIMEOUT_EXPIRED));
|
||||
assert(sync_calls == succeed);
|
||||
if (succeed) assert(synced == target);
|
||||
}
|
||||
}
|
||||
assert(polls > 0 && other_waits > 0);
|
||||
assert((relaxed_cases > 0) == (enabled && shadow && supported));
|
||||
// Raising the guard for an older request must not lower a guard already published by another waiter.
|
||||
if (enabled && shadow && supported) {
|
||||
sync_t sync = {.fenced = 1, .fence = 35};
|
||||
completed = 25; pa_guard = 39; relaxed = 1;
|
||||
Cpu cpu = {0}; cpu.x[0] = (uintptr_t)&sync; cpu.x[1] = UINT64_MAX; cpu.x[30] = site;
|
||||
nvnSyncWait(&cpu);
|
||||
assert(pa_guard == 39);
|
||||
}
|
||||
printf("sync-ahead version %d enabled %d shadow %d: 100000 wait selections passed (%d relaxed)\n", rt_game_version, enabled, shadow, relaxed_cases);
|
||||
}
|
||||
'''
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
parser.add_argument("--cc", default=shutil.which("clang") or r"C:\msys64\ucrt64\bin\clang.exe")
|
||||
args = parser.parse_args()
|
||||
source = (ROOT / "runtime/nvn_api_core.c").read_text()
|
||||
start = source.index("static uint64_t pa_fence[2], pa_guard;")
|
||||
end = source.index("// void nvnEventBuilderSetDefaults", start)
|
||||
(ROOT / "build").mkdir(exist_ok=True)
|
||||
with tempfile.TemporaryDirectory(prefix="sync-ahead-", dir=ROOT / "build") as directory:
|
||||
directory = Path(directory)
|
||||
path, exe = directory / "check.c", directory / "check.exe"
|
||||
path.write_text(HARNESS.replace("PRODUCTION", source[start:end]))
|
||||
subprocess.run([args.cc, "-std=c11", "-O2", "-Wall", "-Werror", str(path), "-o", str(exe)], check=True)
|
||||
for version in (110, 121, 143, 999):
|
||||
for enabled in (0, 1):
|
||||
for shadow in (0, 1):
|
||||
subprocess.run([str(exe), str(version), str(enabled), str(shadow)], check=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+1
-1
@@ -60,7 +60,7 @@
|
||||
# game waits once for the GPU to finish the frame it presented last; with 1 that wait covers the frame before it
|
||||
# instead. On Great Sky Island (Linux, RX 9070 XT) with shadow_defer = 1 this took about 73 to 100 fps. Costs one more
|
||||
# frame of input delay, and frame times vary more from frame to frame than with 0. GPU results the game reads back after that wait, for example occlusion queries, can be a frame old.
|
||||
# Works on 1.2.1 and 1.4.3; 1.1.0 ignores it. Needs shadow storage (shadow below), which is on by default on Linux and
|
||||
# Works on 1.1.0, 1.2.1 and 1.4.3. Needs shadow storage (shadow below), which is on by default on Linux and
|
||||
# Wine and needs shadow = 1 on native Windows; without it the setting is ignored and the log says so.
|
||||
# sync_ahead = 0
|
||||
# Shadow storage, 0 or 1: each CPU-visible memory pool of the game gets a host-visible copy that the GPU reads, in
|
||||
|
||||
Reference in new issue
Block a user