diff --git a/docs/architecture.md b/docs/architecture.md index 3849e025..949aff73 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -84,7 +84,7 @@ Every guest thread is a host thread. TotK assumes Horizon's scheduler: three app - NVN front end from MacNativeGames (`nvn_api_*.c`, `nvn_obj.h`, texture layout and deswizzle). The game records command buffers through it; nothing reaches the GPU until the game submits. - Vulkan 1.3 back end in place of MacNativeGames' Metal one. At `QueueSubmitCommands` the recorded NVN commands are replayed into Vulkan command buffers (`vk_exec*.c`). - Memory pools. Apple's unified memory let MacNativeGames use the game's pools zero-copy. The back end imports the pools as host memory (`VK_EXT_external_memory_host`). On a discrete GPU, pools bound as buffers are mirrored in device-local memory and the pages the CPU wrote (write watch, above) are uploaded before a replay; idle pool regions are made read-only so the write-watch queries skip them (`vk_memory.c`). GPU-rendered textures are written back when the CPU reads them: on a pool invalidate, on a CPU copy, and for small targets once the CPU waits for a fence (`vk_texture.c`). -- Replay thread. `TOTK_REPLAY_ASYNC=1` moves the replay off the submitting thread onto a queue at most one frame behind the game; reads of GPU results wait for the newest writer instead of draining the queue (`vk_backend.c`). It is off by default because no timing has shown a gain yet. +- Replay thread. `TOTK_REPLAY_ASYNC=1` moves the replay off the submitting thread onto a queue at most one frame behind the game; reads of GPU results wait for the newest writer instead of draining the queue (`vk_backend.c`). It is on by default: where the game waits for the GPU it overlaps the replay with the game threads (a GTX 1650 in the Depths, issue #88), elsewhere it is neutral; it adds up to a frame of latency, and `replay_async = 0` turns it off. - Recorder thread. The Vulkan calls the replay makes into its command buffer are made by a thread of their own (`vk_rec.c`, after Savage Wind Recomp and LAS Recomp): each `vkCmd*` copies its arguments into a packet of a ring and the recorder makes the calls in order; calls into other command buffers are made at once; the recorder also ends the command buffer, and the submit waits until it has. In the town on an RX 9070 the replay takes 17% less time on the submitting thread. It is off by default on AMD GPUs, where the driver loses the device or hangs the GPU at startup with it (`TOTK_VK_RECORDER=1` forces it on, `=0` makes every call on the submitting thread). - Uniform snapshot. The game rewrites per-object uniform blocks on other threads soon after it submits the frame that reads them, while the replay still records it. At the start of a submission's replay every uniform range it diff --git a/runtime/cfg.c b/runtime/cfg.c index c53dfa49..8df21755 100644 --- a/runtime/cfg.c +++ b/runtime/cfg.c @@ -146,7 +146,7 @@ static const CfgMeta metas[] = { {"MENU_LANG", "", "en|ru|pt-BR|zh-Hans", LIVE}, {"AMIIBO_DIR", "", NULL, LIVE}, // Advanced: scheduling - {"ASYNC_PREEMPT", "1", "0|1|2", RESTART}, {"REPLAY_ASYNC", "0", "0|1", RESTART}, + {"ASYNC_PREEMPT", "1", "0|1|2", RESTART}, {"REPLAY_ASYNC", "1", "0|1", RESTART}, {"FREE_CORES", "0", "0|1", RESTART}, {"TIMER_1MS", "1", "0|1", RESTART}, // shaders and pipelines {"PIPE_WAIT", "default", "default|draw", RESTART}, {"PIPE_SKIP", "fullscreen", "strict|any|fullscreen", RESTART}, diff --git a/runtime/vk_backend.c b/runtime/vk_backend.c index 62eeaef2..798d8331 100644 --- a/runtime/vk_backend.c +++ b/runtime/vk_backend.c @@ -166,7 +166,10 @@ static uint64_t ck_reads, ck_diff, ck_diff_unexplained, ck_diff_idle; int vk_replay_async(void) { static int on = -1; if (on < 0) { - on = cfg_bool("TOTK_REPLAY_ASYNC"); // off by default: no measured gain yet, one more frame of latency + // On by default: where the game waits for the GPU it overlaps the replay with the game threads (a GTX 1650 in + // the Depths went from 450-686 frames over 50 ms a minute to 10-13, issue #88); where the replay or the game + // threads bound the frame it neither gains nor loses. It adds up to a frame of latency. + on = cfg_bool("TOTK_REPLAY_ASYNC"); rt_log("vk: replay %s (TOTK_REPLAY_ASYNC=%d)", on ? "on its own thread, at most one frame behind the game" : "on the submitting thread", on); } diff --git a/totk.ini.example b/totk.ini.example index e2eb6146..d7e1ee91 100644 --- a/totk.ini.example +++ b/totk.ini.example @@ -32,8 +32,9 @@ # A thread waiting for a virtual core interrupts the owner when it runs game code: 1 = on, 2 = priority inheritance # without the interrupt, 0 = off (the old scheduling). # async_preempt = 1 -# Replay NVN submissions on their own thread, at most one frame behind the game (1 = on). Adds up to a frame of latency. -# replay_async = 0 +# Replay NVN submissions on their own thread, at most one frame behind the game, 0 or 1. Default: on. Adds up to a +# frame of latency; 0 replays them on the game thread. +# replay_async = 1 # EXPERIMENTAL, THE GAME CRASHES WITH 1. 1 lifts the virtual core gate: guest threads run without the console's limit # of one thread per core, and the game keeps per-core data without locks. With 1, 1.2.1 and 1.4.3 crash or hang, # usually at start (#212). Keep 0 to play.