mirror of
https://github.com/valentderah/totk-pc-recomp.git
synced 2026-11-20 12:45:57 +00:00
Record large submissions in chunks on several threads
replay_workers = N (0, the default, keeps the sequential replay) records a submission of enough draws in chunks that start between passes, on N threads and in twice as many chunks, and submits them in order. In the town at 4K on an RTX 5070, 4 threads take the replay from 8.2 to 4.5 ms a frame and the frame rate from 71 to 79.
A sequential scan on the submitting thread applies only what is queue state (fixed-function and bind ops, render targets, render enable and conditions, samples-passed counting) and keeps a copy of it where each chunk starts, with how far its lists got: inline uniform updates, RENDER_COND results, contact shadow marches and the ops that drop uniform copies. A chunk takes those from the lists instead of from the chunks before it, starts with every image in GENERAL and ends with a full barrier, so its recording depends only on the copy and the lists.
Each thread records with a context of its own: the replay's memos, descriptor state, occlusion query segments and per-command-buffer state moved from file statics into VkExec and VkExecDraw, and the tables the contexts share (stand-ins, depth twins, views, mirrors, timestamps, completion callbacks, crumbs) take locks or atomics. The scan makes render targets live in replay order and notes the op (vk_texture_revive), since a chunk can sample a storage before the chunk that renders it got there or after the scan went past a later target there: a chunk's texture slots resolve as of its own op, and the chunks after a texture with no entry yet wait for the chunk that makes it. Uniform copies are dropped at the op that writes them (vk_usnap_drop_at).
Contact shadows plan their march in replay order (vk_contact_plan) and the chunk that holds its op records it; the HDR source keeps the copy that comes last in replay order. DLSS and fxaa = 0 act on the frame draw by draw and keep the replay sequential. TOTK_REPLAY_STATE_CHECK=1 compares the state after the last chunk with the scan's.
A draw or dispatch whose shaders write storage buffers drops the uniform copies they cover from its own op on, as the other GPU writes do (scan_writes). The scan still translates no shader: it reads the shaders the recording contexts translated from a table shared with them (vk_exec_draw_writes), and an op whose shader no context translated yet holds the chunks after it until it is recorded. Before, the chunk that recorded such a write dropped the copies for every position at that moment, so a later chunk recorded first bound the copy taken at the submit call: on an RX 7800 XT on RADV the grass near the camera came out dark or unshaded in some runs with 4 threads.
TOTK_TRACE=perf counts the chunks held per submission by reason and logs when each chunk of the last submission began. The settings menu has the thread count under Advanced, Scheduling, with the count this run uses under it.
The recorder thread begins, ends and submits the submission's open part (vk_rec.c rec_end_cb), so a chunk's mirror prologue stays off it: vk_sync_part waits until the recorder is idle, as the part pool allows one thread at a time, then records, ends and submits the prologue itself. Through the recorder the prologue could reach the queue before it was ended, and the GPU lost the device at the title screen.
(cherry picked from commit b94e2e9642)
This commit is contained in:
1 parent
4fcedda5e6
commit
6b4d84ec50
24 files changed
+2013
-627
No files matched your search
@@ -85,6 +85,7 @@ Every guest thread is a host thread. TotK assumes Horizon's scheduler: three app
|
||||
- Vulkan 1.3 back end in place of MacNativeGames' Metal one. At `QueueSubmitCommands` the recorded NVN commands are replayed into Vulkan command buffers (`vk_exec*.c`).
|
||||
- Memory pools. Apple's unified memory let MacNativeGames use the game's pools zero-copy. The back end imports the pools as host memory (`VK_EXT_external_memory_host`). On a discrete GPU, pools bound as buffers are mirrored in device-local memory and the pages the CPU wrote (write watch, above) are uploaded before a replay; idle pool regions are made read-only so the write-watch queries skip them (`vk_memory.c`). GPU-rendered textures are written back when the CPU reads them: on a pool invalidate, on a CPU copy, and for small targets once the CPU waits for a fence (`vk_texture.c`).
|
||||
- Replay thread. `TOTK_REPLAY_ASYNC=1` moves the replay off the submitting thread onto a queue at most one frame behind the game; reads of GPU results wait for the newest writer instead of draining the queue (`vk_backend.c`). It is off by default. Where the game waits for the GPU it overlaps the replay with the game threads (a GTX 1650 in the Depths, issue #88), a gain measured only there; while it was the default, a GTX 1660 Ti at the 60 fps cap with the GPU at 26 % reported uneven motion (#659). It adds up to a frame of latency; `replay_async = 1` turns it on.
|
||||
- Chunked replay. A large submission is recorded in chunks by several threads (`TOTK_REPLAY_WORKERS`, 0 by default: the sequential replay; 4 is a good value from 8 logical processors) and the chunks go to the queue in order (`vk_exec.c`). A sequential scan keeps a copy of the queue state where each chunk starts and lists what a chunk would otherwise learn from the chunks before it: inline uniform updates, conditions, contact shadow marches, uniform drops. It also makes render targets live in replay order and notes the op, since a chunk can sample a storage before the chunk that renders it got there or after the scan went past a later target there: a chunk's texture slots resolve as of its own op, and the chunks after a texture with no entry yet wait for the chunk that makes it. DLSS and `fxaa = 0` act on the frame draw by draw and keep the replay sequential. `TOTK_REPLAY_STATE_CHECK=1` compares the state after the last chunk with the scan's.
|
||||
- Recorder thread. The Vulkan calls the replay makes into its command buffer are made by a thread of their own (`vk_rec.c`, after Savage Wind Recomp and LAS Recomp): each `vkCmd*` copies its arguments into a packet of a ring and the recorder makes the calls in order; calls into other command buffers are made at once; the recorder also ends the command buffer, and the submit waits until it has. In the town on an RX 9070 the replay takes 17% less time on the submitting thread. It is off by default on AMD GPUs, where the driver loses the device or hangs the GPU at startup with it (`TOTK_VK_RECORDER=1` forces it on, `=0` makes every call on the submitting thread).
|
||||
- Uniform snapshot. The game rewrites per-object uniform blocks on other threads soon after it submits the frame
|
||||
that reads them, while the replay still records it. At the start of a submission's replay every uniform range it
|
||||
|
||||
@@ -182,6 +182,9 @@ Many rows below are developer tools that only act together with another setting
|
||||
| `audio_off` | Any value mixes audio without opening an output device. | The game runs in silence and reports two channels; same as `audio = none`. | Caution |
|
||||
| `one_per_core` (default `0`) | `1` keeps the process on the first logical processor of each physical core; `ab` switches it with the `ab` blocks. | Busy threads stop sharing a core with a sibling thread, which can help or hurt on the CPU; it only applies when the CPU has hyperthreads. | Caution |
|
||||
| `reexec` | Counter of address-space retries that the runtime sets itself when it starts the process again. | The counter is cleared when the window is recreated; a value of 20 or more stops the start with a fatal error. | Risky |
|
||||
| `replay_chunks` (default `0`) | Most chunks a submission is cut into when `replay_workers` is set; `0` uses twice the number of recording threads. | Fewer chunks leave threads idle at the end of a submission, more add scan and submit work per chunk; no effect while `replay_workers` is `0`. | Caution |
|
||||
| `replay_chunk_draws` (default `128`) | Least draws a chunk of the chunked replay holds before it may end at a pass. | Small values make many short chunks, large ones leave big submissions on few threads; no effect while `replay_workers` is `0`. | Caution |
|
||||
| `replay_state_check` (default `0`) | `1` compares the replay state after the last chunk of a chunked submission with the state of the scan and logs differences. | Adds a comparison per submission and log lines on a mismatch; no effect while `replay_workers` is `0`. | Caution |
|
||||
| `restart_wait` | Set by the runtime when it restarts itself; names the old process by id and creation time so the new process waits for it to exit. | Meant for the runtime's own use; the code does not describe a manual value. | Unknown |
|
||||
| `spin` (default `1`) | `0` makes a guest thread that polls memory keep its virtual core. | Polling threads can starve the thread whose item they wait for, so frames can stall. | Caution |
|
||||
| `spin_handoff_us` (default `50`) | Microseconds a polling wait lasts before it gives its virtual core to a waiting thread. | A shorter value hands cores over sooner; a longer one keeps them longer. | Caution |
|
||||
|
||||
+4
-2
@@ -53,7 +53,7 @@ static const char *const known[] = {
|
||||
"ONE_PER_CORE", "OPERATION_MODE", "OUTPUT_RES", "PAD_LAYOUT", "PASS_CRUMBS", "PATCH_OUTPUT", "PATCH_SCENE", "PCORE", "PCSS", "PCSS_K", "PERF_OVERLAY", "PERF_SLOW_MS", "PIPELINE_CACHE", "PIPE_MEMO", "PIPE_PREBUILD", "PIPE_SKIP", "PIPE_SYNC", "PIPE_WAIT",
|
||||
"PREWARM_XLATE", "PROBE_MOVIE", "PROBE_MOVIE_BENCH", "PROBE_MOVIE_SHOT", "PROFILE", "PROFILE_CHAINS", "PROFILE_DIR", "PROFILE_FRAMES",
|
||||
"PROFILE_MAIN", "PROFILE_RENDER", "PROMPTS", "PUSH_SKIP", "PUSH_STAGE", "RDOC", "RDOC_DIR", "RDOC_DLL", "REALTIME", "REC_ASYNC", "REC_END", "REC_SPIN_US", "REEXEC",
|
||||
"RENDER_SCALE", "REPLAY_ASYNC",
|
||||
"RENDER_SCALE", "REPLAY_ASYNC", "REPLAY_CHUNKS", "REPLAY_CHUNK_DRAWS", "REPLAY_STATE_CHECK", "REPLAY_WORKERS",
|
||||
"REPLAY_CHECK", "RESOLVE_CHECK", "RESTART_WAIT", "ROMFS", "ROMFS_INDEX", "ROMFS_MMAP", "ROMFS_MODS", "ROMFS_PATCHES",
|
||||
"ROMFS_PROOF_DIR", "ROMFS_VERIFY", "RSTICK_DEADZONE", "RUMBLE", "RUMBLE_CURVE", "RUMBLE_STRENGTH", "RUMBLE_TRIGGERS", "SHADER_CACHE",
|
||||
"SHADER_MEMO", "SHADOW_RANGE", "SHADOW_RES", "SHADOW_SCALE", "SHARED_FONTS", "SHARPNESS", "SHOTS", "SHOTS_DIR",
|
||||
@@ -95,7 +95,7 @@ static const struct { const char *key, *section; } sections[] = {
|
||||
{"BIND_RIGHT", "input"}, {"BIND_DOWN", "input"}, {"BIND_LSTICK_LEFT", "input"}, {"BIND_LSTICK_UP", "input"},
|
||||
{"BIND_LSTICK_RIGHT", "input"}, {"BIND_LSTICK_DOWN", "input"}, {"BIND_RSTICK_LEFT", "input"},
|
||||
{"BIND_RSTICK_UP", "input"}, {"BIND_RSTICK_RIGHT", "input"}, {"BIND_RSTICK_DOWN", "input"}, {"BIND_SHAKE", "input"},
|
||||
{"ASYNC_PREEMPT", "performance"}, {"REPLAY_ASYNC", "performance"}, {"FREE_CORES", "performance"},
|
||||
{"ASYNC_PREEMPT", "performance"}, {"REPLAY_ASYNC", "performance"}, {"REPLAY_WORKERS", "performance"}, {"FREE_CORES", "performance"},
|
||||
{"TIMER_1MS", "performance"}, {"PIPE_WAIT", "performance"}, {"PIPE_SKIP", "performance"},
|
||||
{"LIB_PREBUILD", "performance"}, {"LIB_PREBUILD_THREADS", "performance"}, {"PIPE_PREBUILD", "performance"},
|
||||
{"PREWARM_XLATE", "performance"}, {"GPL", "performance"}, {"ASTC", "performance"}, {"MEMORY", "performance"},
|
||||
@@ -163,6 +163,8 @@ static const CfgMeta metas[] = {
|
||||
{"DISCORD_IMAGE", "totk_512x512", NULL, LIVE},
|
||||
// Advanced: scheduling
|
||||
{"ASYNC_PREEMPT", "1", "0|1|2", RESTART}, {"REPLAY_ASYNC", "0", "0|1", RESTART},
|
||||
{"REPLAY_WORKERS", "0", "0..16", RESTART}, {"REPLAY_CHUNKS", "0", "0..64", RESTART},
|
||||
{"REPLAY_CHUNK_DRAWS", "128", "1..100000", RESTART}, {"REPLAY_STATE_CHECK", "0", "0|1", RESTART},
|
||||
{"FREE_CORES", "0", "0|1", RESTART}, {"TIMER_1MS", "1", "0|1", RESTART},
|
||||
// shaders and pipelines
|
||||
{"PIPE_WAIT", "default", "default|draw", RESTART}, {"PIPE_SKIP", "fullscreen", "strict|any|fullscreen", RESTART},
|
||||
|
||||
@@ -51,6 +51,7 @@ extern "C" {
|
||||
void rt_log(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
|
||||
void vk_queue_lock(void);
|
||||
void vk_queue_unlock(void);
|
||||
int vk_exec_replay_workers(void);
|
||||
void res_sizes(unsigned out[2], unsigned scene[2]);
|
||||
double frame_step_limit_hz(void);
|
||||
extern const int rt_game_version;
|
||||
@@ -905,6 +906,23 @@ static void row_threads(const char *key, UiTextId label, UiTextId desc) {
|
||||
opt_choice_list(key, label, vals, names, N, desc);
|
||||
}
|
||||
|
||||
// The chunked replay's recording threads: Off, then the counts; the note under the value is what this run uses, which
|
||||
// a change replaces only after a restart.
|
||||
static void row_replay_workers(void) {
|
||||
static const char *const vals[] = {"0", "2", "3", "4", "6", "8", "12", "16"};
|
||||
enum { N = sizeof vals / sizeof *vals };
|
||||
const char *names[N];
|
||||
names[0] = menu_text(UI_MENU_OFF);
|
||||
for (int i = 1; i < N; i++) names[i] = vals[i];
|
||||
static char now[64];
|
||||
char n[16];
|
||||
int w = vk_exec_replay_workers();
|
||||
snprintf(n, sizeof n, "%d", w);
|
||||
if (w >= 0) snprintf(now, sizeof now, menu_text(UI_MENU_REPLAY_WORKERS_NOW), w ? n : menu_text(UI_MENU_OFF));
|
||||
opt_choice_list("TOTK_REPLAY_WORKERS", UI_MENU_REPLAY_WORKERS, vals, names, N, UI_MENU_REPLAY_WORKERS_DESC, -1,
|
||||
w >= 0 ? now : nullptr);
|
||||
}
|
||||
|
||||
// A free-text key shown as Off (unset) / On (value set): On writes on_value, any set value reads as On.
|
||||
static void row_set_or_not(const char *key, UiTextId label, UiTextId desc, const char *on_value) {
|
||||
const char *vals[] = {"", on_value}, *names[] = {menu_text(UI_MENU_OFF), menu_text(UI_MENU_ON)};
|
||||
@@ -954,6 +972,7 @@ static void tab_advanced(float scale) {
|
||||
static const UiTextId preempt[] = {UI_MENU_PREEMPT_OFF, UI_MENU_PREEMPT_ON, UI_MENU_PREEMPT_INHERIT}; // 0|1|2
|
||||
opt_choice("TOTK_ASYNC_PREEMPT", UI_MENU_ASYNC_PREEMPT, preempt, 3, UI_MENU_ASYNC_PREEMPT_DESC);
|
||||
opt_toggle("TOTK_REPLAY_ASYNC", UI_MENU_REPLAY_ASYNC, UI_MENU_REPLAY_ASYNC_DESC);
|
||||
row_replay_workers();
|
||||
opt_toggle("TOTK_TIMER_1MS", UI_MENU_TIMER_1MS, UI_MENU_TIMER_1MS_DESC);
|
||||
|
||||
opt_group(UI_MENU_GROUP_SHADERS);
|
||||
|
||||
@@ -386,6 +386,9 @@ static const char *const text_en[UI_TEXT_COUNT] = {
|
||||
[UI_MENU_PREEMPT_OFF] = "Off", [UI_MENU_PREEMPT_ON] = "On", [UI_MENU_PREEMPT_INHERIT] = "Inheritance only",
|
||||
[UI_MENU_REPLAY_ASYNC] = "GPU frame prep on its own thread",
|
||||
[UI_MENU_REPLAY_ASYNC_DESC] = "Prepares frames for the GPU on a separate thread instead of the game thread. Adds up to one frame of latency.",
|
||||
[UI_MENU_REPLAY_WORKERS] = "GPU frame prep threads",
|
||||
[UI_MENU_REPLAY_WORKERS_DESC] = "Records large frames for the GPU on several threads at once. Helps when the CPU limits the frame rate; needs 8 or more logical processors. Off records on one thread.",
|
||||
[UI_MENU_REPLAY_WORKERS_NOW] = "now: %s",
|
||||
[UI_MENU_TIMER_1MS] = "1 ms Windows timer",
|
||||
[UI_MENU_TIMER_1MS_DESC] = "Raises the system timer resolution to 1 ms for steadier frame pacing.",
|
||||
[UI_MENU_PIPE_WAIT] = "Waiting draw handling",
|
||||
@@ -862,6 +865,9 @@ static const char *const text_ru[UI_TEXT_COUNT] = {
|
||||
[UI_MENU_PREEMPT_OFF] = "Выкл.", [UI_MENU_PREEMPT_ON] = "Вкл.", [UI_MENU_PREEMPT_INHERIT] = "Только наследование",
|
||||
[UI_MENU_REPLAY_ASYNC] = "Подготовка кадра для GPU в отдельном потоке",
|
||||
[UI_MENU_REPLAY_ASYNC_DESC] = "Кадр для GPU готовится в отдельном потоке, а не в потоке игры. Добавляет до одного кадра задержки.",
|
||||
[UI_MENU_REPLAY_WORKERS] = "Потоки подготовки кадра для GPU",
|
||||
[UI_MENU_REPLAY_WORKERS_DESC] = "Большие кадры для GPU записываются сразу в нескольких потоках. Помогает, когда частоту кадров ограничивает процессор; нужно 8 логических процессоров или больше. «Выкл.» — запись в одном потоке.",
|
||||
[UI_MENU_REPLAY_WORKERS_NOW] = "сейчас: %s",
|
||||
[UI_MENU_TIMER_1MS] = "Таймер Windows 1 мс",
|
||||
[UI_MENU_TIMER_1MS_DESC] = "Повышает точность системного таймера до 1 мс для ровного темпа кадров.",
|
||||
[UI_MENU_PIPE_WAIT] = "Обработка ожидающих отрисовок",
|
||||
@@ -1235,6 +1241,9 @@ static const char *const text_pt_br[UI_TEXT_COUNT] = {
|
||||
[UI_MENU_PREEMPT_OFF] = "Desligado", [UI_MENU_PREEMPT_ON] = "Ligado", [UI_MENU_PREEMPT_INHERIT] = "Só herança",
|
||||
[UI_MENU_REPLAY_ASYNC] = "Preparo de quadro da GPU em thread própria",
|
||||
[UI_MENU_REPLAY_ASYNC_DESC] = "Prepara os quadros para a GPU numa thread separada em vez da thread do jogo. Adiciona até um quadro de latência.",
|
||||
[UI_MENU_REPLAY_WORKERS] = "Threads de preparo de quadro da GPU",
|
||||
[UI_MENU_REPLAY_WORKERS_DESC] = "Grava quadros grandes para a GPU em várias threads ao mesmo tempo. Ajuda quando a CPU limita a taxa de quadros; precisa de 8 ou mais processadores lógicos. Desligado grava numa thread só.",
|
||||
[UI_MENU_REPLAY_WORKERS_NOW] = "agora: %s",
|
||||
[UI_MENU_TIMER_1MS] = "Timer do Windows de 1 ms",
|
||||
[UI_MENU_TIMER_1MS_DESC] = "Aumenta a resolução do timer do sistema para 1 ms, para um ritmo de quadros mais estável.",
|
||||
[UI_MENU_PIPE_WAIT] = "Tratamento de desenhos em espera",
|
||||
@@ -1842,6 +1851,10 @@ static const char *const text_zh_hans[UI_TEXT_COUNT] = {
|
||||
[UI_MENU_REPLAY_ASYNC] = "GPU 帧准备独立线程",
|
||||
[UI_MENU_REPLAY_ASYNC_DESC] = "在单独的线程而不是游戏线程上为 GPU 准备帧。"
|
||||
"最多增加一帧延迟。",
|
||||
[UI_MENU_REPLAY_WORKERS] = "GPU 帧准备线程数",
|
||||
[UI_MENU_REPLAY_WORKERS_DESC] = "同时在多个线程上为 GPU 录制大帧。CPU 限制帧率时有帮助;需要 8 个或更多逻辑处理器。"
|
||||
"关:在单个线程上录制。",
|
||||
[UI_MENU_REPLAY_WORKERS_NOW] = "当前:%s",
|
||||
[UI_MENU_TIMER_1MS] = "1 ms Windows 计时器",
|
||||
[UI_MENU_TIMER_1MS_DESC] = "将系统计时器精度提高到 1 ms,使帧间隔更平稳。",
|
||||
[UI_MENU_PIPE_WAIT] = "等待中绘制的处理",
|
||||
|
||||
@@ -175,6 +175,7 @@ typedef enum {
|
||||
UI_MENU_ASYNC_PREEMPT, UI_MENU_ASYNC_PREEMPT_DESC, UI_MENU_PREEMPT_OFF, UI_MENU_PREEMPT_ON,
|
||||
UI_MENU_PREEMPT_INHERIT,
|
||||
UI_MENU_REPLAY_ASYNC, UI_MENU_REPLAY_ASYNC_DESC,
|
||||
UI_MENU_REPLAY_WORKERS, UI_MENU_REPLAY_WORKERS_DESC, UI_MENU_REPLAY_WORKERS_NOW,
|
||||
UI_MENU_TIMER_1MS, UI_MENU_TIMER_1MS_DESC,
|
||||
UI_MENU_PIPE_WAIT, UI_MENU_PIPE_WAIT_DESC, UI_MENU_PIPE_WAIT_DEFAULT, UI_MENU_PIPE_WAIT_DRAW,
|
||||
UI_MENU_PIPE_SKIP, UI_MENU_PIPE_SKIP_DESC, UI_MENU_PIPE_SKIP_STRICT, UI_MENU_PIPE_SKIP_ANY,
|
||||
|
||||
+29
-1
@@ -226,6 +226,10 @@ enum { VK_MCLS_NONE, VK_MCLS_HOST_NOACC, VK_MCLS_HOST_UNC, VK_MCLS_HOST_CACHED,
|
||||
enum { VK_MUSE_VB, VK_MUSE_IB, VK_MUSE_UBO, VK_MUSE_SSBO, VK_MUSE_IND, VK_MUSE_COPY_SRC, VK_MUSE_GPU_WRITE,
|
||||
VK_MUSE_CPU_READ, VK_MUSE_N };
|
||||
void vk_mem_count(int cls, int use, uint64_t bytes);
|
||||
// A thread recording a chunk of a submission (vk_exec.c) counts into its own acc from vk_mem_count_local(acc) on, until
|
||||
// vk_mem_count_local(NULL); vk_mem_count_flush adds acc into the census and clears it.
|
||||
void vk_mem_count_local(int64_t (*acc)[VK_MUSE_N]);
|
||||
void vk_mem_count_flush(int64_t (*acc)[VK_MUSE_N]);
|
||||
// TOTK_TRACE=perf dirty census (vk_memory.c): the replay notes each imported pool it binds as a vertex, index, uniform,
|
||||
// storage or indirect buffer; at the start of every submit the pages the CPU wrote in such pools since the previous
|
||||
// submit are counted from the shared page generations and logged as `mem: dirty` every 60 frames. Nothing when tracing
|
||||
@@ -258,10 +262,14 @@ void vk_mem_gpu_wrote_shader(const VkMemSpan *sp, uint64_t size);
|
||||
// it records, ubo bindings take the copy (0: not copied, bind memory), GPU writes of the replay drop the copies they touch.
|
||||
void vk_usnap_begin(const uint64_t *handles, int n, uint64_t fence);
|
||||
void vk_usnap_end(uint64_t fence);
|
||||
int vk_usnap_find(uint64_t addr, uint64_t n, uint64_t align, VkDescriptorBufferInfo *out);
|
||||
// at: the copies dropped at or before it are not found (vk_usnap_drop_at); UINT32_MAX outside a chunked replay.
|
||||
int vk_usnap_find(uint64_t addr, uint64_t n, uint64_t at, uint64_t align, VkDescriptorBufferInfo *out);
|
||||
const uint8_t *vk_usnap_map(void); // a host copy of the current arena, at the offsets vk_usnap_find gives
|
||||
const uint8_t *vk_usnap_bytes(uint64_t addr, uint64_t n); // the copy of [addr, addr+n) as of the submit, NULL: none
|
||||
void vk_usnap_drop(uint64_t addr, uint64_t n); // the replay makes the GPU write [addr, addr+n) of guest memory
|
||||
// The state scan of a chunked replay (vk_exec.c): the op that writes [addr, addr+n) is the at-th that may; the chunks
|
||||
// pass at with the ops they record (VkExec.drop_n) and find the copies only before it.
|
||||
void vk_usnap_drop_at(uint64_t addr, uint64_t n, uint32_t at);
|
||||
uint64_t vk_usnap_gen(void); // changes whenever a binding may resolve differently
|
||||
void vk_usnap_shutdown(void); // vk_exec_draw_shutdown: frees the arenas
|
||||
// vk_mvprev.c: last frame's copy of the scene's object (c7) and bone (c5) blocks, for per-object motion vectors.
|
||||
@@ -426,6 +434,7 @@ typedef struct {
|
||||
void vk_hdr_present_source(const VkTexImage *win, VkImage *image, VkFormat *format);
|
||||
void vk_hdr_image_gone(VkImage image);
|
||||
const VkTexImage *vk_texture_get(const tex_t *t); // created and made current; NULL for BUFFER or unmapped formats
|
||||
const VkTexImage *vk_texture_revive(const tex_t *t, uint32_t pos); // the entry vk_texture_get would return if it exists, made live at op pos
|
||||
// A texture slot; a slot whose texture was finalized follows the live texture at the same storage (hle_nvn.c).
|
||||
const VkTexImage *vk_texture_by_id(int id, view_t *view, int *has_view);
|
||||
// The replay looks up many slots per draw: it takes the texture lock once around them (vk_texture_lock) and uses the
|
||||
@@ -433,6 +442,12 @@ const VkTexImage *vk_texture_by_id(int id, view_t *view, int *has_view);
|
||||
void vk_texture_lock(void);
|
||||
void vk_texture_unlock(void);
|
||||
const VkTexImage *vk_texture_by_id_locked(int id, view_t *view, int *has_view);
|
||||
// 1 when slot id's entry had its freshness check in this submission, so vk_texture_by_id_locked would only look it up.
|
||||
// No lock: a context of a chunked replay asks before taking the texture lock.
|
||||
int vk_texture_slot_checked(int id);
|
||||
// 1 when the state scan revived a texture at the slot's storage in this submission: the slot's answer depends on the op
|
||||
// a chunk records, so no memo holds it (vk_exec_draw.c); a chunk sees every revival up to its last op.
|
||||
int vk_texture_slot_moves(int id);
|
||||
VkSampler vk_sampler_by_id_locked(int id);
|
||||
// The slot's sampler with nearest filtering and no anisotropy: a depth texture read without a compare lands on one texel
|
||||
// on the Switch although the game binds a linear sampler, and MacNativeGames samples it the same way. VK_NULL_HANDLE
|
||||
@@ -495,9 +510,22 @@ int vk_exec_record(VkCommandBuffer cb, const uint64_t *handles, int n, uint64_t
|
||||
// batch) without a signal, and returns a new begun one for the rest of the submission. The submission's timeline value
|
||||
// is signalled by its last part, whose signal covers every earlier part in submission order.
|
||||
VkCommandBuffer vk_sync_split(void);
|
||||
// Inside vk_exec_record: submits cb, a part recorded and ended on another thread, after the upload batch and the
|
||||
// mirror copies queued so far, without a signal (the submitting thread, in the order of the submission).
|
||||
void vk_sync_part(VkCommandBuffer cb);
|
||||
// GPU time of a part recorded on another thread (vk_sync.c "GPU time"): the start timestamp after vkBeginCommandBuffer,
|
||||
// the pair or -1; the end before vkEndCommandBuffer.
|
||||
int vk_sync_ts_open(VkCommandBuffer cb);
|
||||
void vk_sync_ts_close(VkCommandBuffer cb, int pair, uint64_t fence);
|
||||
// Runs fn(arg) on the completion thread once fence is done on the GPU (event writes, counter reports, sync state).
|
||||
// Callbacks run in the order they were added, after those of earlier values; a value already done runs at once there.
|
||||
void vk_completion_add(uint64_t fence, void (*fn)(void *), void *arg);
|
||||
// Callbacks of a part recorded on another thread: between vk_completion_defer(d) and vk_completion_defer(NULL) the
|
||||
// calling thread's vk_completion_add calls collect in d, and vk_completion_flush adds them, in their order, when the
|
||||
// part takes its place in the submission (vk_exec.c record_chunks).
|
||||
typedef struct { void *items; int n, cap; } VkDeferred;
|
||||
void vk_completion_defer(VkDeferred *d);
|
||||
void vk_completion_flush(VkDeferred *d);
|
||||
// gfx_vulkan_backend (vk_backend.c): presents go to a present thread and run after the submissions made before them.
|
||||
void rdoc_init(void); // runtime/rdoc.c: TOTK_RDOC debug capture; before vkCreateInstance
|
||||
int rdoc_active(void);
|
||||
|
||||
+71
-48
@@ -7,9 +7,13 @@
|
||||
// passes behind nearby geometry: the small shadows at feet, under stones and in grass that the shadow maps are too
|
||||
// coarse for. Transparent surfaces, the sky and post-processing come after, untouched.
|
||||
//
|
||||
// Inputs from the game, read from the fragment stage's uniform block c4 bound in the composite pass (guest memory,
|
||||
// so it may be one frame newer than the replay): the projection scales c4[7].x and c4[8].y, the far plane c4[14].y,
|
||||
// and the first sun cascade's depth row c4[44].xyz, whose negated direction points from a view-space position toward
|
||||
// vk_contact_plan decides that in replay order and vk_contact_exec records it: a chunked replay's scan
|
||||
// plans the march (vk_exec.c scan_contact) and the chunk that holds its op records it.
|
||||
//
|
||||
// Inputs from the game, read where the march is planned from the fragment stage's uniform block c4 bound in the
|
||||
// composite pass (guest memory, so it may be one frame newer than the replay): the projection scales c4[7].x and
|
||||
// c4[8].y, the far plane c4[14].y, and the first sun cascade's depth row c4[44].xyz, whose negated direction points from
|
||||
// a view-space position toward
|
||||
// the sun (the G-buffer shaders compare dot(c4[44].xyz, pos) + c4[44].w with the cascade map). The scene colour images
|
||||
// get STORAGE usage when the setting is on (vk_texture.c create_tex).
|
||||
#include "vk.h"
|
||||
@@ -23,22 +27,22 @@
|
||||
#include "vk_exec.h"
|
||||
|
||||
static struct {
|
||||
int tried, ok, strength, debug, logged;
|
||||
int read, tried, ok, strength, debug, logged;
|
||||
PFN_vkCmdPushDescriptorSetKHR push;
|
||||
VkDescriptorSetLayout dsl;
|
||||
VkPipelineLayout layout;
|
||||
VkPipeline pipe;
|
||||
VkSampler sampler;
|
||||
const VkTexImage *scene, *depth; // the last G-buffer's scene colour and linear depth
|
||||
uint32_t w, h;
|
||||
VkExecTarget scene, depth; // the last G-buffer's scene colour and linear depth
|
||||
int pending; // the composite into scene ran; the march waits for the next pass
|
||||
uint64_t c4;
|
||||
} C;
|
||||
|
||||
int vk_contact_strength(void) {
|
||||
if (!C.tried) {
|
||||
if (!C.read) { // once: the scan asks at every draw
|
||||
int s = cfg_int("TOTK_CONTACT_SHADOWS");
|
||||
C.strength = s < 0 ? 0 : s > 100 ? 100 : s;
|
||||
C.read = 1;
|
||||
}
|
||||
return C.strength;
|
||||
}
|
||||
@@ -90,16 +94,16 @@ static int init(VkCtx *c) {
|
||||
}
|
||||
|
||||
// A G-buffer pass: a B10G11R11 scene colour, an R32F linear depth and a D32F depth, all one size, at least 640 wide.
|
||||
static int is_gbuffer(const VkExec *ex, const VkTexImage **scene, const VkTexImage **depth) {
|
||||
static int is_gbuffer(const VkExec *ex, const VkExecTarget **scene, const VkExecTarget **depth) {
|
||||
*scene = *depth = NULL;
|
||||
if (ex->pass_noatt || ex->pass_w < 640 || ex->depth.format != VK_FORMAT_D32_SFLOAT || ex->depth.level) return 0;
|
||||
for (uint32_t i = 0; i < ex->pass_ncolor && i < 8; i++) {
|
||||
const VkExecTarget *t = &ex->color[i];
|
||||
if (!t->image || t->level || t->width != ex->pass_w || t->height != ex->pass_h) continue;
|
||||
if (t->format == VK_FORMAT_B10G11R11_UFLOAT_PACK32 && !*scene) *scene = t->img;
|
||||
if (t->format == VK_FORMAT_R32_SFLOAT && !*depth) *depth = t->img;
|
||||
if (t->format == VK_FORMAT_B10G11R11_UFLOAT_PACK32 && !*scene) *scene = t;
|
||||
if (t->format == VK_FORMAT_R32_SFLOAT && !*depth) *depth = t;
|
||||
}
|
||||
return *scene && *depth && ((*scene)->usage & VK_IMAGE_USAGE_STORAGE_BIT);
|
||||
return *scene && *depth && (*scene)->img && ((*scene)->img->usage & VK_IMAGE_USAGE_STORAGE_BIT);
|
||||
}
|
||||
|
||||
static void mem_barrier(VkCommandBuffer cb) {
|
||||
@@ -112,28 +116,62 @@ static void mem_barrier(VkCommandBuffer cb) {
|
||||
vkCmdPipelineBarrier2(cb, &d);
|
||||
}
|
||||
|
||||
void vk_contact_flush(VkExec *ex) {
|
||||
if (!C.pending) return;
|
||||
// The lighting composite: the remembered scene colour as the only target, with a D16 depth of its size.
|
||||
static int is_composite(const VkExec *ex) {
|
||||
return C.scene.image && ex->pass_ncolor >= 1 && ex->color[0].image == C.scene.image && !ex->color[0].level &&
|
||||
ex->depth.format == VK_FORMAT_D16_UNORM && ex->pass_w == C.scene.width && ex->pass_h == C.scene.height;
|
||||
}
|
||||
|
||||
int vk_contact_plan(const VkExec *ex, int pass, VkExecJob *j) {
|
||||
if (!vk_contact_strength()) return 0;
|
||||
if (pass) {
|
||||
const VkExecTarget *scene, *depth;
|
||||
// The camera block: c4 of the fragment stage, 0x940 bytes. The composite binds it only after its pass begins, so
|
||||
// the newest one seen at any pass start serves.
|
||||
const p_bind_buf *b = &ex->ubo[1][1];
|
||||
if (b->addr && b->size == 0x940) C.c4 = b->addr;
|
||||
if (is_gbuffer(ex, &scene, &depth)) {
|
||||
C.pending = 0;
|
||||
C.scene = *scene, C.depth = *depth;
|
||||
return 0;
|
||||
}
|
||||
if (is_composite(ex)) {
|
||||
C.pending = C.c4 != 0;
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
if (!C.pending) return 0;
|
||||
C.pending = 0;
|
||||
if (ex->in_pass) return;
|
||||
const float *u = (const float *)(uintptr_t)C.c4;
|
||||
float lx = -u[44 * 4 + 0], ly = -u[44 * 4 + 1], lz = -u[44 * 4 + 2], ll = sqrtf(lx * lx + ly * ly + lz * lz);
|
||||
float px = u[7 * 4 + 0], py = u[8 * 4 + 1], zfar = u[14 * 4 + 1];
|
||||
if (!(ll > 1e-6f) || !(px > 0) || !(py > 0) || !(zfar > 1)) return;
|
||||
VkImageView in = vk_exec_target_view(C.depth, VK_FORMAT_R32_SFLOAT, VK_IMAGE_VIEW_TYPE_2D, VK_IMAGE_ASPECT_COLOR_BIT,
|
||||
if (!(ll > 1e-6f) || !(px > 0) || !(py > 0) || !(zfar > 1)) return 0;
|
||||
float w = (float)C.scene.width, h = (float)C.scene.height;
|
||||
*j = (VkExecJob){.scene = C.scene, .depth = C.depth,
|
||||
.pc = {px, py, zfar, (float)C.strength / 100.0f,
|
||||
lx / ll, ly / ll, lz / ll, 0.6f,
|
||||
w, h, 1.0f / w, 1.0f / h,
|
||||
0, 0, 0, 0}};
|
||||
return 1;
|
||||
}
|
||||
|
||||
void vk_contact_exec(VkExec *ex, const VkExecJob *j) {
|
||||
if (!C.tried) init(vk_ctx());
|
||||
if (!C.ok) return;
|
||||
VkImageView in = vk_exec_target_view(j->depth.img, VK_FORMAT_R32_SFLOAT, VK_IMAGE_VIEW_TYPE_2D, VK_IMAGE_ASPECT_COLOR_BIT,
|
||||
0, 0, 1, 0);
|
||||
VkImageView out = vk_exec_target_view(C.scene, VK_FORMAT_B10G11R11_UFLOAT_PACK32, VK_IMAGE_VIEW_TYPE_2D,
|
||||
VkImageView out = vk_exec_target_view(j->scene.img, VK_FORMAT_B10G11R11_UFLOAT_PACK32, VK_IMAGE_VIEW_TYPE_2D,
|
||||
VK_IMAGE_ASPECT_COLOR_BIT, 0, 0, 1, 0);
|
||||
if (!in || !out) return;
|
||||
vk_exec_pass_end(ex);
|
||||
VkCommandBuffer cb = ex->cb;
|
||||
float pcv[16] = {px, py, zfar, (float)C.strength / 100.0f,
|
||||
lx / ll, ly / ll, lz / ll, 0.6f,
|
||||
(float)C.w, (float)C.h, 1.0f / (float)C.w, 1.0f / (float)C.h,
|
||||
(float)C.debug, 0, 0, 0};
|
||||
float pcv[16];
|
||||
memcpy(pcv, j->pc, sizeof pcv);
|
||||
pcv[12] = (float)C.debug;
|
||||
if (!C.logged) {
|
||||
C.logged = 1;
|
||||
rt_log("vk: contact shadows on a %ux%u scene, sun toward %.3f %.3f %.3f (view space), far %g", C.w, C.h,
|
||||
pcv[4], pcv[5], pcv[6], zfar);
|
||||
rt_log("vk: contact shadows on a %ux%u scene, sun toward %.3f %.3f %.3f (view space), far %g", j->scene.width, j->scene.height,
|
||||
pcv[4], pcv[5], pcv[6], pcv[2]);
|
||||
}
|
||||
mem_barrier(cb); // the compose passes' writes, before the march reads and writes them
|
||||
VkDescriptorImageInfo ii = {C.sampler, in, VK_IMAGE_LAYOUT_GENERAL}, oi = {VK_NULL_HANDLE, out, VK_IMAGE_LAYOUT_GENERAL};
|
||||
@@ -145,36 +183,21 @@ void vk_contact_flush(VkExec *ex) {
|
||||
vkCmdBindPipeline(cb, VK_PIPELINE_BIND_POINT_COMPUTE, C.pipe);
|
||||
rec_push(C.push, cb, VK_PIPELINE_BIND_POINT_COMPUTE, C.layout, 0, 2, w);
|
||||
vkCmdPushConstants(cb, C.layout, VK_SHADER_STAGE_COMPUTE_BIT, 0, sizeof pcv, pcv);
|
||||
vkCmdDispatch(cb, (C.w + 7) / 8, (C.h + 7) / 8, 1);
|
||||
vkCmdDispatch(cb, (j->scene.width + 7) / 8, (j->scene.height + 7) / 8, 1);
|
||||
mem_barrier(cb); // and what follows sees the darkened colour
|
||||
ex->recorded += 6;
|
||||
vk_exec_draw_new_cb(); // the game's bound compute pipeline and descriptors were replaced: bind them again
|
||||
}
|
||||
|
||||
// The lighting composite: the remembered scene colour as the only target, with a D16 depth of its size.
|
||||
static int is_composite(const VkExec *ex) {
|
||||
return C.scene && ex->pass_ncolor >= 1 && ex->color[0].img == C.scene && !ex->color[0].level &&
|
||||
ex->depth.format == VK_FORMAT_D16_UNORM && ex->pass_w == C.w && ex->pass_h == C.h;
|
||||
vk_exec_draw_new_cb(ex); // the game's bound compute pipeline and descriptors were replaced: bind them again
|
||||
}
|
||||
|
||||
// The sequential replay's hooks; a chunk leaves the marches to the scan.
|
||||
void vk_contact_pass_begin(VkExec *ex) {
|
||||
if (!C.tried && !init(vk_ctx())) return;
|
||||
if (!C.ok) return;
|
||||
const VkTexImage *scene, *depth;
|
||||
// The camera block: c4 of the fragment stage, 0x940 bytes. The composite binds it only after its pass begins, so
|
||||
// the newest one seen at any pass start serves.
|
||||
const p_bind_buf *b = &ex->ubo[1][1];
|
||||
if (b->addr && b->size == 0x940) C.c4 = b->addr;
|
||||
if (is_gbuffer(ex, &scene, &depth)) {
|
||||
C.pending = 0;
|
||||
C.scene = scene, C.depth = depth, C.w = ex->pass_w, C.h = ex->pass_h;
|
||||
return;
|
||||
}
|
||||
if (is_composite(ex)) {
|
||||
C.pending = C.c4 != 0;
|
||||
return;
|
||||
}
|
||||
vk_contact_flush(ex);
|
||||
VkExecJob j;
|
||||
if (!ex->chunked && vk_contact_plan(ex, 1, &j)) vk_contact_exec(ex, &j);
|
||||
}
|
||||
|
||||
void vk_contact_flush(VkExec *ex) {
|
||||
VkExecJob j;
|
||||
if (!ex->chunked && C.pending && vk_contact_plan(ex, 0, &j)) vk_contact_exec(ex, &j);
|
||||
}
|
||||
|
||||
void vk_contact_release(void) {
|
||||
|
||||
+3
-3
@@ -91,7 +91,7 @@ void vk_crumbs_report(void) {
|
||||
// records changed pipeline state also notes the shaders of a pass's first draw. A full barrier already precedes every
|
||||
// pass and dispatch (vk_exec.c vk_exec_pass_begin, vk_exec_draw.c op_dispatch), so the end-of-pipe write adds no wait.
|
||||
static VkPassMark ring[VK_PASS_MARKS];
|
||||
static uint32_t seq; // marks made; only the replay thread writes it
|
||||
static uint32_t seq; // marks made (atomic: the recording threads of a chunked replay)
|
||||
|
||||
// TOTK_PASS_CRUMBS: unset, buffer marks where VK_AMD_buffer_marker exists; 1, also NVIDIA checkpoints; 0, neither.
|
||||
int vk_pass_marks_want_nv(void) {
|
||||
@@ -124,8 +124,8 @@ void vk_pass_mark_cmd(VkCommandBuffer cb, uint32_t id, int end) {
|
||||
|
||||
VkPassMark *vk_pass_mark_begin(VkCommandBuffer cb, int kind, uint64_t fence) {
|
||||
if (!vk_pass_marks_on || !cb) return NULL;
|
||||
uint32_t id = ++seq;
|
||||
if (!id) id = ++seq; // 0 means no mark
|
||||
uint32_t id = __atomic_add_fetch(&seq, 1, __ATOMIC_RELAXED); // the recording threads of a chunked replay
|
||||
if (!id) id = __atomic_add_fetch(&seq, 1, __ATOMIC_RELAXED); // 0 means no mark
|
||||
VkPassMark *m = &ring[id & (VK_PASS_MARKS - 1)];
|
||||
m->id = id, m->kind = (uint8_t)kind, m->fence = fence, m->draws = UINT32_MAX;
|
||||
rec_pass_mark(cb, id, 0);
|
||||
|
||||
+3
-2
@@ -557,7 +557,7 @@ static int mv_pass(VkExec *ex, struct Size *e, VkImageView zv, float jx, float j
|
||||
VkCommandBuffer cb = ex->cb;
|
||||
uint32_t w = e->w, h = e->h;
|
||||
VkDescriptorBufferInfo cam;
|
||||
if (!mv_init() || !D.cam_addr || !vk_usnap_find(D.cam_addr, CAM_BLOCK, 4, &cam)) {
|
||||
if (!mv_init() || !D.cam_addr || !vk_usnap_find(D.cam_addr, CAM_BLOCK, UINT32_MAX, 4, &cam)) {
|
||||
D.mv_none++;
|
||||
D.cam_prev_ok = 0;
|
||||
e->chars_ok = 0;
|
||||
@@ -1045,7 +1045,7 @@ static void evaluate(VkExec *ex, const VkExecTarget *dst) {
|
||||
full_barrier(cb);
|
||||
if (dst != t) vk_dlaa_swap_img = t->img, vk_dlaa_swap_view = e->out_view; // the scaling draw reads DLSS's picture
|
||||
ex->recorded += 4;
|
||||
vk_exec_draw_new_cb(); // NGX bound pipelines and descriptor sets of its own: the next draw binds everything again
|
||||
vk_exec_draw_new_cb(ex); // NGX bound pipelines and descriptor sets of its own: the next draw binds everything again
|
||||
D.jdepth = z->image;
|
||||
next_jitter(w, ow);
|
||||
if (++D.evals == 1 || D.evals == 300 || D.evals % 3600 == 0) // 300: one early line with the frames counted
|
||||
@@ -1195,6 +1195,7 @@ static uint64_t spirv_hash(const GfxShader *s) {
|
||||
}
|
||||
|
||||
int vk_dlaa_skip_draw(VkExec *ex, const GfxShader *fs) {
|
||||
if (ex->chunked) return 0; // a chunked replay runs only with fxaa = 1 and without DLSS (vk_exec.c chunked_on)
|
||||
static uint64_t frame_read = ~0ull;
|
||||
static int fxaa = 1;
|
||||
// fxaa = 0 reports once whether it found the game's FXAA draw: the first skip, or a minute of frames (3600) with
|
||||
|
||||
+788
-108
File diff suppressed because it is too large.
Load diff
+84
-13
@@ -2,10 +2,12 @@
|
||||
// handles of a submission, keeps render targets, passes, clears and fixed-function state; draws, binds and dispatches
|
||||
// (vk_exec_draw.c) and copies, counters, events and conditional rendering (vk_exec_xfer.c) plug in through the two
|
||||
// hooks at the bottom.
|
||||
// Runs on the submitting thread under vk_sync_submit's lock: no locking inside.
|
||||
// Runs on the submitting thread under vk_sync_submit's lock; a chunked replay (TOTK_REPLAY_WORKERS, vk_exec.c) records
|
||||
// on several threads, each with a VkExec of its own, and shares only the locked tables named where they are declared.
|
||||
// The op loop follows MacNativeGames nvn_mtl.m (MIT, Copyright (c) 2026 jhaizhou) and the pass handling LAS Recomp
|
||||
// (MIT, Copyright (c) 2026 Peiikai); vk_exec.c names the sources.
|
||||
#pragma once
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#include "nvn_api.h"
|
||||
@@ -38,8 +40,31 @@ typedef struct {
|
||||
uint8_t scale; // VkTexImage.scale
|
||||
} VkExecTarget;
|
||||
|
||||
// The replay state. One instance lives for the run: NVN state is queue state and carries over from one submission to
|
||||
// the next, as in MacNativeGames' and LAS Recomp's replays. The "per command buffer" fields are reset by vk_exec_record.
|
||||
// TOTK_TRACE=gfx: replay counters of the current frame, logged and reset at each present (vk_exec_frame_log).
|
||||
typedef struct {
|
||||
uint64_t passes, draws_seen, draws, clears, copies, dispatches, submits;
|
||||
uint64_t fb_draws, fb_barriers, fb_restarts, fb_layouts; // feedback loops: draws sampling a written target of
|
||||
// their pass, in-pass barriers, pass restarts, layout
|
||||
// transitions
|
||||
} VkExecFrame;
|
||||
|
||||
// A contact shadow march (vk_contact.c) the scan of a chunked replay planned before an op: vk_contact_plan decides it in
|
||||
// replay order, vk_contact_exec records it on the chunk that holds the op.
|
||||
typedef struct VkExecJob {
|
||||
const void *op; // vk_exec.c: the op it precedes in a scanned submission
|
||||
VkExecTarget scene, depth; // the scene colour and the linear depth
|
||||
float pc[16]; // the march's push constants
|
||||
} VkExecJob;
|
||||
|
||||
// An OP_UPDATE_UB of the submission: [lo, hi) of guest memory takes data (vk_exec_draw.c ubo_ptr reads them).
|
||||
typedef struct { uint64_t lo, hi; const uint8_t *data; } VkExecUpd;
|
||||
|
||||
struct VkExecDraw; // vk_exec_draw.c: the memos and command buffer state of one recording context
|
||||
|
||||
// The replay state and one recording context. NVN state is queue state and carries over from one submission to the
|
||||
// next, as in MacNativeGames' and LAS Recomp's replays. The "per command buffer" fields are reset by vk_exec_record.
|
||||
// The fields from rt up to views are that queue state (VK_EXEC_STATE_BEGIN/END); the rest belongs to the context that
|
||||
// records with it.
|
||||
typedef struct VkExec {
|
||||
// ---- per command buffer
|
||||
VkCommandBuffer cb; // being recorded
|
||||
@@ -94,8 +119,14 @@ typedef struct VkExec {
|
||||
int render_disabled; // OP_RENDER_ENABLE false or a failed OP_RENDER_COND: clears and draws skip
|
||||
p_render_cond render_cond; // last OP_RENDER_COND
|
||||
int has_render_cond;
|
||||
// samples-passed counting (vk_exec_xfer.c): a reset since the last report, a report since the last reset, the game
|
||||
// reported twice without a reset (every draw counts from then on)
|
||||
int oq_counting, oq_reported, oq_keep;
|
||||
|
||||
// ---- internal to vk_exec.c
|
||||
VkExecFrame fr; // counters of this context, added to vk_exec_frame at the end of a submission
|
||||
int split_draws, part_start; // draws and dispatches since the last part; recorded at its start
|
||||
uint64_t part_t0; // replay time at its start
|
||||
void *views; // image views created for this command buffer (freed at its fence)
|
||||
int nviews, cap_views;
|
||||
int barrier_at; // recorded right after the last full barrier (-1: none yet)
|
||||
@@ -112,8 +143,30 @@ typedef struct VkExec {
|
||||
const void *fb_fs; // feedback_scan's memo: fragment shader, texture bind serial, pass serial
|
||||
uint32_t fb_serial, fb_pass, fb_sampled;
|
||||
uint32_t tex_serial, pass_serial; // bumped by every texture or image bind and pass begin
|
||||
|
||||
// ---- vk_exec_xfer.c: the occlusion query of the open pass and the queries closed since the last report or reset
|
||||
int oq_open; // slot of the active query, -1 none
|
||||
uint32_t *oq_seg, oq_nseg, oq_capseg;
|
||||
int oq_lost; // the segment missed a draw (ring full)
|
||||
|
||||
struct VkExecDraw *draw; // vk_exec_draw.c, made on the context's first draw op
|
||||
|
||||
// ---- a chunk of a scanned submission (vk_exec.c): the scan's answers, taken in op order
|
||||
int chunked; // records a chunk: what depends on earlier chunks comes from the scan
|
||||
const VkExecUpd *upd; // the submission's inline uniform updates
|
||||
const uint8_t *cond; // its RENDER_COND results (1 renders)
|
||||
const VkExecJob *jobs; // its contact shadow marches
|
||||
uint32_t njobs;
|
||||
uint32_t upd_n, cond_i, job_i; // updates, results and records before the op being recorded
|
||||
uint32_t drop_n; // ops before it that may write uniform ranges (vk_usnap_drop_at)
|
||||
const uint32_t *wpos; // the positions of the draws and dispatches among those ops (vk_exec_draw_writes)
|
||||
uint32_t wpos_i, wpos_end; // the next of them, and the end of the chunk's
|
||||
uint32_t pos; // ops of the submission up to and with it (the scan's too)
|
||||
} VkExec;
|
||||
|
||||
#define VK_EXEC_STATE_BEGIN offsetof(VkExec, rt)
|
||||
#define VK_EXEC_STATE_END offsetof(VkExec, fr)
|
||||
|
||||
// Begins dynamic rendering with the current targets unless a pass is open (a full memory barrier precedes it). Call
|
||||
// before a draw or an attachment clear.
|
||||
void vk_exec_pass_begin(VkExec *ex);
|
||||
@@ -136,9 +189,16 @@ VkCommandBuffer vk_exec_inline_begin(int *reopen);
|
||||
void vk_exec_inline_end(int reopen);
|
||||
// vk_exec_draw.c: forget what the command buffer being recorded has bound (pipeline, sets, dynamic state, vertex
|
||||
// buffers): the replay continues in a new one (vk_sync_split).
|
||||
void vk_exec_draw_new_cb(void);
|
||||
void vk_exec_draw_new_cb(VkExec *ex);
|
||||
// The recording threads of the chunked replay as this run read TOTK_REPLAY_WORKERS (0 off), -1 before the first
|
||||
// submission read it (the settings menu shows it).
|
||||
int vk_exec_replay_workers(void);
|
||||
// vk_exec_draw.c, for the state scan of a chunked replay: the storage ranges the shaders of a draw (dispatch 0) or a
|
||||
// dispatch at state s write, as {address, size} into out; their count, or -1 when there are more than max or a stage's
|
||||
// shader is not known yet (no context has translated it). Reads uniform blocks through s->upd like a chunk.
|
||||
int vk_exec_draw_writes(const VkExec *s, int dispatch, uint64_t (*out)[2], int max);
|
||||
// vk_exec_draw.c: the vertex and fragment shader of the last draw recorded into this command buffer (NULL: none yet).
|
||||
void vk_exec_draw_shaders(const GfxShader **vs, const GfxShader **fs);
|
||||
void vk_exec_draw_shaders(const VkExec *ex, const GfxShader **vs, const GfxShader **fs);
|
||||
VkBuffer vk_exec_inline_buffer(VkDeviceSize size);
|
||||
// vk_exec.c: the stand-in a 3D BC render target is drawn through goes with its image (vk_exec_image_gone).
|
||||
void vk_exec_bc_gone(VkImage image);
|
||||
@@ -155,16 +215,10 @@ int vk_exec_targets_recent(VkExec *ex, int any);
|
||||
// The current pass's targets with their formats, extents and the frames they were last rendered in (diagnostics).
|
||||
void vk_exec_targets_describe(VkExec *ex, char *buf, size_t cap);
|
||||
|
||||
// TOTK_TRACE=gfx: replay counters of the current frame, logged and reset at each present (vk_exec_frame_log).
|
||||
typedef struct {
|
||||
uint64_t passes, draws_seen, draws, clears, copies, dispatches, submits;
|
||||
uint64_t fb_draws, fb_barriers, fb_restarts, fb_layouts; // feedback loops: draws sampling a written target of
|
||||
// their pass, in-pass barriers, pass restarts, layout
|
||||
// transitions
|
||||
} VkExecFrame;
|
||||
extern VkExecFrame vk_exec_frame;
|
||||
void vk_exec_frame_written(const tex_t *t); // vk_texture_gpu_written: t rendered or copied into
|
||||
uint64_t vk_exec_recording_fence(void); // fence of the submission this thread records, else 0
|
||||
uint32_t vk_exec_replay_pos(void); // the op this thread records in a chunk, else UINT32_MAX
|
||||
void vk_exec_frame_log(const tex_t *presented, uint64_t frame);
|
||||
|
||||
// TOTK_DUMP=<frame> (vk_dump.c): between presents frame-1 and frame, every pass's targets and every dispatch's storage
|
||||
@@ -188,6 +242,7 @@ void vk_c0_fill(uint8_t bank[VK_C0_BYTES], int stage, const p_bind_buf ssbo[16],
|
||||
// vk_exec_draw.c: OP_DRAW, OP_DRAW_IDX, OP_DISPATCH, OP_BIND_* (program, state, vattr, vstream, vbuf, ubo, ssbo, tex,
|
||||
// samp, img), OP_UPDATE_UB, OP_DRAW_IDX_IND, OP_DISPATCH_IND. payload: nvn_op_payload of the record.
|
||||
void vk_exec_op_draw(VkExec *ex, uint32_t op, const void *payload);
|
||||
int vk_exec_bind(VkExec *ex, uint32_t op, const void *payload); // a bind op's writes into ex; 1: VkExecDraw.bind_gen
|
||||
// The aspect a sampled view of img reads (vk_exec_draw.c get_view): stencil for a view in STENCIL depth-stencil mode.
|
||||
VkImageAspectFlags vk_exec_view_aspect(const VkTexImage *img, const view_t *v, int hv);
|
||||
VkImageView vk_exec_target_view(const VkTexImage *img, VkFormat format, VkImageViewType type, uint32_t aspect,
|
||||
@@ -199,10 +254,26 @@ void vk_exec_op_xfer(VkExec *ex, uint32_t op, const void *payload);
|
||||
// sample DLSS's output vk_dlaa_swap_view in its place, so the game's scaling draw scales DLSS's picture.
|
||||
extern const VkTexImage *vk_dlaa_swap_img;
|
||||
extern VkImageView vk_dlaa_swap_view;
|
||||
// vk_contact.c: a pass about to begin (its targets resolved, no pass open), and a dispatch or copy about to be recorded
|
||||
// The state scan's part of the transfer ops: render enable, conditional rendering (returns the condition: 1 renders),
|
||||
// samples-passed counting state and the pending reports conditions read. Returns 1 for every other op.
|
||||
// The state scan of a chunked replay (vk_exec.c): the textures op uses go live in replay order (vk_texture_revive); pos:
|
||||
// the op's place in the submission. 1 when one of them has no entry yet.
|
||||
int vk_exec_xfer_revive(uint32_t op, const void *payload, uint32_t pos);
|
||||
int vk_exec_xfer_scan(VkExec *ex, uint32_t op, const void *payload);
|
||||
// The guest memory a COPY_B2B, COPY_T2B or CLEAR_BUF op makes the GPU write, as far as the scan can tell without the
|
||||
// texture's image: [*addr, *addr + return value).
|
||||
uint64_t vk_exec_xfer_written(const VkExec *ex, uint32_t op, const void *payload, uint64_t *addr);
|
||||
// vk_contact.c: a pass about to begin (its targets resolved, no pass open), and a dispatch or copy about to be recorded.
|
||||
// vk_contact_plan is what they do in replay order (pass: the first; else the second), for the scan; 1: a march is due (j).
|
||||
void vk_contact_pass_begin(VkExec *ex);
|
||||
void vk_contact_flush(VkExec *ex);
|
||||
int vk_contact_plan(const VkExec *ex, int pass, VkExecJob *j);
|
||||
void vk_contact_exec(VkExec *ex, const VkExecJob *j);
|
||||
// vk_exec_xfer.c, samples-passed counting: a draw about to be recorded in the open pass (begins an occlusion query
|
||||
// when counting and none is active), and the open pass about to end (ends the active query).
|
||||
void vk_exec_oq_draw(VkExec *ex);
|
||||
void vk_exec_oq_pass_end(VkExec *ex);
|
||||
void vk_exec_oq_forget(VkExec *ex);
|
||||
void vk_exec_oq_flush(VkExec *ex); // the end of a chunk: its closed queries counted as a segment without a report // vk_exec_xfer_shutdown: the context's queries belonged to the old pool
|
||||
// vk_exec.c: fn on every recording context (teardown, the device idle).
|
||||
void vk_exec_contexts(void (*fn)(VkExec *ex));
|
||||
+600
-346
File diff suppressed because it is too large.
Load diff
+114
-38
@@ -72,7 +72,7 @@ static void guest_written_cb(void *arg) {
|
||||
|
||||
// The GPU wrote [addr, addr+size) of CPU-visible memory: tell the texture cache once the fence is reached.
|
||||
static void note_gpu_write(VkExec *ex, uint64_t addr, uint64_t size) {
|
||||
vk_usnap_drop(addr, size);
|
||||
if (!ex->chunked) vk_usnap_drop(addr, size); // a chunk: the scan dropped it in its place
|
||||
vk_exec_ub_foreign(ex, addr, size);
|
||||
if (nvn_gpu_only(addr)) return;
|
||||
Range *r = malloc(sizeof *r);
|
||||
@@ -129,13 +129,6 @@ static int oq_state; // 0 not set up, 1 counting available,
|
||||
static VkQueryControlFlags oq_flags;
|
||||
static volatile uint8_t oq_busy[OQ_N]; // from begin until the completion thread read and reset the slot
|
||||
static uint32_t oq_next; // replay: next ring slot
|
||||
static int oq_counting; // replay: draws are counted (a reset since the last report, or oq_keep)
|
||||
static int oq_reported; // replay: a samples-passed report since the last reset
|
||||
static int oq_keep; // replay: the game reported twice without a reset: count every draw
|
||||
static int oq_open = -1; // replay: the slot whose query is active in the open pass
|
||||
static uint32_t *oq_seg; // replay: slots closed since the last report or reset
|
||||
static uint32_t oq_nseg, oq_capseg;
|
||||
static int oq_lost; // replay: the segment missed a draw (ring full)
|
||||
static uint64_t oq_count; // completion thread: samples since the last reset
|
||||
|
||||
typedef struct { uint64_t addr; int reset, lost; uint32_t n; uint32_t q[]; } OqSegment;
|
||||
@@ -166,39 +159,49 @@ static int oq_ready(void) {
|
||||
}
|
||||
|
||||
static void oq_close(VkExec *ex) {
|
||||
if (oq_open < 0) return;
|
||||
vkCmdEndQuery(ex->cb, oq_pool, (uint32_t)oq_open);
|
||||
if (ex->oq_open < 0) return;
|
||||
vkCmdEndQuery(ex->cb, oq_pool, (uint32_t)ex->oq_open);
|
||||
ex->recorded++;
|
||||
if (oq_nseg == oq_capseg) {
|
||||
oq_capseg = oq_capseg ? oq_capseg * 2 : 64;
|
||||
oq_seg = realloc(oq_seg, sizeof *oq_seg * oq_capseg);
|
||||
if (!oq_seg) rt_fatal("vk: out of memory");
|
||||
if (ex->oq_nseg == ex->oq_capseg) {
|
||||
ex->oq_capseg = ex->oq_capseg ? ex->oq_capseg * 2 : 64;
|
||||
ex->oq_seg = realloc(ex->oq_seg, sizeof *ex->oq_seg * ex->oq_capseg);
|
||||
if (!ex->oq_seg) rt_fatal("vk: out of memory");
|
||||
}
|
||||
oq_seg[oq_nseg++] = (uint32_t)oq_open;
|
||||
oq_open = -1;
|
||||
ex->oq_seg[ex->oq_nseg++] = (uint32_t)ex->oq_open;
|
||||
ex->oq_open = -1;
|
||||
}
|
||||
|
||||
void vk_exec_oq_draw(VkExec *ex) {
|
||||
if (!oq_counting || oq_open >= 0) return;
|
||||
uint32_t q = oq_next % OQ_N;
|
||||
if (__atomic_load_n(&oq_busy[q], __ATOMIC_ACQUIRE)) {
|
||||
if (!ex->oq_counting || ex->oq_open >= 0) return;
|
||||
uint32_t q = __atomic_fetch_add(&oq_next, 1, __ATOMIC_RELAXED) % OQ_N; // threads recording chunks take slots too
|
||||
if (__atomic_exchange_n(&oq_busy[q], 1, __ATOMIC_ACQUIRE)) {
|
||||
warn_once(W_QUERY, "occlusion query ring full: draws left uncounted");
|
||||
oq_lost = 1;
|
||||
ex->oq_lost = 1;
|
||||
return;
|
||||
}
|
||||
oq_busy[q] = 1;
|
||||
oq_next++;
|
||||
vkCmdBeginQuery(ex->cb, oq_pool, q, oq_flags);
|
||||
ex->recorded++;
|
||||
oq_open = (int)q;
|
||||
ex->oq_open = (int)q;
|
||||
}
|
||||
|
||||
void vk_exec_oq_pass_end(VkExec *ex) { oq_close(ex); }
|
||||
|
||||
static void oq_segment(VkExec *ex, uint64_t addr, int reset);
|
||||
|
||||
// The queries a chunk closed: the completion thread adds them to the count before the next chunk's (vk_exec.c).
|
||||
void vk_exec_oq_flush(VkExec *ex) {
|
||||
if (ex->oq_nseg || ex->oq_lost) oq_segment(ex, 0, 0);
|
||||
}
|
||||
|
||||
void vk_exec_oq_forget(VkExec *ex) {
|
||||
ex->oq_counting = ex->oq_reported = ex->oq_keep = 0, ex->oq_open = -1;
|
||||
ex->oq_nseg = 0, ex->oq_lost = 0;
|
||||
}
|
||||
|
||||
void vk_exec_xfer_shutdown(void) {
|
||||
if (oq_state > 0) vkDestroyQueryPool(vk_ctx()->device, oq_pool, NULL);
|
||||
oq_pool = VK_NULL_HANDLE, oq_state = 0, oq_next = 0, oq_counting = oq_reported = oq_keep = 0, oq_open = -1;
|
||||
oq_nseg = 0, oq_lost = 0, oq_count = 0;
|
||||
oq_pool = VK_NULL_HANDLE, oq_state = 0, oq_next = 0, oq_count = 0;
|
||||
vk_exec_contexts(vk_exec_oq_forget);
|
||||
memset((void *)oq_busy, 0, sizeof oq_busy);
|
||||
}
|
||||
|
||||
@@ -240,30 +243,34 @@ static void oq_cb(void *arg) {
|
||||
// Hands the segment to the completion thread with a report (addr; 0: no CPU-visible memory) or a reset.
|
||||
static void oq_segment(VkExec *ex, uint64_t addr, int reset) {
|
||||
oq_close(ex);
|
||||
OqSegment *s = malloc(sizeof *s + sizeof *s->q * oq_nseg);
|
||||
OqSegment *s = malloc(sizeof *s + sizeof *s->q * ex->oq_nseg);
|
||||
if (!s) rt_fatal("vk: out of memory");
|
||||
s->addr = addr, s->reset = reset, s->lost = oq_lost, s->n = oq_nseg;
|
||||
memcpy(s->q, oq_seg, sizeof *s->q * oq_nseg);
|
||||
oq_nseg = 0, oq_lost = 0;
|
||||
s->addr = addr, s->reset = reset, s->lost = ex->oq_lost, s->n = ex->oq_nseg;
|
||||
memcpy(s->q, ex->oq_seg, sizeof *s->q * ex->oq_nseg);
|
||||
ex->oq_nseg = 0, ex->oq_lost = 0;
|
||||
vk_completion_add(ex->fence, oq_cb, s);
|
||||
}
|
||||
|
||||
static void reset_counter(VkExec *ex, const p_reset_counter *p) {
|
||||
if (p->type != NVN_COUNTER_TYPE_SAMPLES_PASSED || !oq_ready()) return;
|
||||
oq_segment(ex, 0, 1);
|
||||
oq_counting = 1, oq_reported = 0;
|
||||
ex->oq_counting = 1, ex->oq_reported = 0;
|
||||
}
|
||||
|
||||
// A samples-passed report: the segment goes with it (addr 0: no CPU-visible memory) and counting stops until the next
|
||||
// reset, unless reports came without resets before.
|
||||
static void oq_report_state(VkExec *ex) {
|
||||
if (ex->oq_reported && !ex->oq_keep) {
|
||||
ex->oq_keep = 1;
|
||||
LOG_ONCE("vk: samples passed reported twice without a reset: counting every draw from now on");
|
||||
}
|
||||
ex->oq_reported = 1;
|
||||
if (!ex->oq_keep) ex->oq_counting = 0;
|
||||
}
|
||||
|
||||
static void oq_report(VkExec *ex, uint64_t addr) {
|
||||
oq_segment(ex, addr, 0);
|
||||
if (oq_reported && !oq_keep) {
|
||||
oq_keep = 1;
|
||||
rt_log("vk: samples passed reported twice without a reset: counting every draw from now on");
|
||||
}
|
||||
oq_reported = 1;
|
||||
if (!oq_keep) oq_counting = 0;
|
||||
oq_report_state(ex);
|
||||
}
|
||||
|
||||
static void signal_event(VkExec *ex, const p_signal_event *p) {
|
||||
@@ -290,7 +297,7 @@ static void report_counter(VkExec *ex, const p_report_counter *p) {
|
||||
*a = p->addr;
|
||||
vk_completion_add(ex->fence, counter_cb, a);
|
||||
}
|
||||
pending[pending_head++ % PENDING] = (typeof(pending[0])){p->addr, ex->fence, samples};
|
||||
if (!ex->chunked) pending[pending_head++ % PENDING] = (typeof(pending[0])){p->addr, ex->fence, samples}; // else the scan's
|
||||
}
|
||||
|
||||
// ---- conditional rendering
|
||||
@@ -612,8 +619,77 @@ void vk_exec_op_xfer(VkExec *ex, uint32_t op, const void *payload) {
|
||||
case OP_RENDER_COND:
|
||||
ex->render_cond = *(const p_render_cond *)payload;
|
||||
ex->has_render_cond = 1;
|
||||
ex->render_disabled = !cond_enabled(payload);
|
||||
ex->render_disabled = !(ex->chunked ? ex->cond[ex->cond_i++] : cond_enabled(payload));
|
||||
break;
|
||||
default: rt_fatal("vk: op %s is not a transfer", nvn_op_name(op));
|
||||
}
|
||||
}
|
||||
|
||||
static int revive_new(const tex_t *t, uint32_t pos) { return t && t->mem && !vk_texture_revive(t, pos); }
|
||||
|
||||
int vk_exec_xfer_revive(uint32_t op, const void *payload, uint32_t pos) {
|
||||
if (op == OP_COPY_T2T) {
|
||||
int n = revive_new(((const p_copy_t2t *)payload)->src, pos);
|
||||
return revive_new(((const p_copy_t2t *)payload)->dst, pos) | n;
|
||||
}
|
||||
if (op == OP_COPY_B2T || op == OP_COPY_T2B) return revive_new(((const p_copy_b2t *)payload)->tex, pos);
|
||||
if (op == OP_CLEAR_TEX) return revive_new(((const p_clear_tex *)payload)->tex, pos);
|
||||
return 0;
|
||||
}
|
||||
|
||||
uint64_t vk_exec_xfer_written(const VkExec *ex, uint32_t op, const void *payload, uint64_t *addr) {
|
||||
VkMemSpan m;
|
||||
if (op == OP_COPY_B2B) {
|
||||
const p_copy_b2b *p = payload;
|
||||
*addr = p->dst;
|
||||
return vk_mem_resolve(p->src, 1, &m) && vk_mem_resolve(p->dst, 1, &m) ? p->size : 0;
|
||||
}
|
||||
if (op == OP_CLEAR_BUF) {
|
||||
const p_clear_buf *p = payload;
|
||||
*addr = p->addr;
|
||||
return (p->addr | p->size) & 3 || !vk_mem_resolve(p->addr, 1, &m) ? 0 : p->size;
|
||||
}
|
||||
// COPY_T2B: the buffer side of copy_buffer_image from the texture descriptor (sub_resolve, buffer_side)
|
||||
const p_copy_b2t *p = payload;
|
||||
const tex_t *t = p->tex;
|
||||
int bw, bh, bpb = t ? nvn_fmt_bpb(t->d.format, &bw, &bh) : 0;
|
||||
*addr = p->src;
|
||||
if (!bpb) return 0;
|
||||
uint32_t level = p->has_view && p->view.set_levels ? p->view.base_level : 0;
|
||||
uint32_t lw = max1((uint32_t)t->d.width >> level), lh = max1((uint32_t)t->d.height >> level);
|
||||
const NVNcopyRegion *r = &p->r;
|
||||
uint32_t w = r->width > 0 ? (uint32_t)r->width : r->xoffset < (int32_t)lw ? lw - (uint32_t)r->xoffset : 0;
|
||||
uint32_t h = r->height > 0 ? (uint32_t)r->height : r->yoffset < (int32_t)lh ? lh - (uint32_t)r->yoffset : 0;
|
||||
uint32_t nz = r->depth > 0 ? (uint32_t)r->depth : 1;
|
||||
uint32_t wb = (w + (uint32_t)bw - 1) / (uint32_t)bw, hb = (h + (uint32_t)bh - 1) / (uint32_t)bh;
|
||||
if (!wb || !hb) return 0;
|
||||
uint64_t row = ex->copy_stride > 0 ? (uint64_t)ex->copy_stride : (uint64_t)wb * (uint64_t)bpb;
|
||||
return row * hb * (nz - 1) + row * (hb - 1) + (uint64_t)wb * (uint64_t)bpb;
|
||||
}
|
||||
|
||||
int vk_exec_xfer_scan(VkExec *ex, uint32_t op, const void *payload) {
|
||||
switch (op) {
|
||||
case OP_RENDER_ENABLE: ex->render_disabled = !((const p_flag *)payload)->value; break;
|
||||
case OP_RENDER_COND: {
|
||||
int on = cond_enabled(payload);
|
||||
ex->render_cond = *(const p_render_cond *)payload;
|
||||
ex->has_render_cond = 1;
|
||||
ex->render_disabled = !on;
|
||||
return on;
|
||||
}
|
||||
case OP_RESET_COUNTER:
|
||||
if (((const p_reset_counter *)payload)->type == NVN_COUNTER_TYPE_SAMPLES_PASSED && oq_ready())
|
||||
ex->oq_counting = 1, ex->oq_reported = 0;
|
||||
break;
|
||||
case OP_REPORT_COUNTER: {
|
||||
const p_report_counter *p = payload;
|
||||
int samples = p->type == NVN_COUNTER_TYPE_SAMPLES_PASSED && oq_ready();
|
||||
VkMemSpan m;
|
||||
if (!nvn_gpu_only(p->addr) && !vk_mem_resolve(p->addr, sizeof(NVNcounterData), &m)) break; // the recording stops
|
||||
if (samples) oq_report_state(ex);
|
||||
if (!nvn_gpu_only(p->addr)) pending[pending_head++ % PENDING] = (typeof(pending[0])){p->addr, ex->fence, samples};
|
||||
break;
|
||||
}
|
||||
}
|
||||
return 1;
|
||||
}
|
||||
+5
-2
@@ -243,7 +243,9 @@ void vk_hdr_shader_swap(uint32_t **spirv, uint64_t *words) {
|
||||
int vk_hdr_track; // vk_exec_draw.c calls vk_hdr_draw: set at the first present with TOTK_HDR=1
|
||||
static volatile int32_t g_active; // the swapchain is HDR10 (gfx_present.c vk_hdr_set_active)
|
||||
static PlatMutex rec_lock = PLAT_MUTEX_INIT;
|
||||
static struct { VkImage win, src; VkFormat fmt; } recs[4]; // per presented window image: what the last draw copied
|
||||
// Per presented window image: what the last draw in replay order copied, and where that draw is. Chunks of a replay
|
||||
// record at once, so a draw of an earlier chunk can get here after one of a later chunk.
|
||||
static struct { VkImage win, src; VkFormat fmt; uint64_t fence; uint32_t pos; } recs[4];
|
||||
static int nrecs;
|
||||
|
||||
void vk_hdr_set_active(int on) { plat_xchg32(&g_active, on); }
|
||||
@@ -277,7 +279,8 @@ void vk_hdr_draw(VkExec *ex, const VkPipeInfo *info) {
|
||||
}
|
||||
plat_mutex_lock(&rec_lock);
|
||||
for (int i = 0; i < nrecs; i++)
|
||||
if (recs[i].win == t->image) recs[i].src = src, recs[i].fmt = fmt;
|
||||
if (recs[i].win == t->image && (ex->fence > recs[i].fence || (ex->fence == recs[i].fence && ex->pos >= recs[i].pos)))
|
||||
recs[i].src = src, recs[i].fmt = fmt, recs[i].fence = ex->fence, recs[i].pos = ex->pos;
|
||||
plat_mutex_unlock(&rec_lock);
|
||||
}
|
||||
|
||||
|
||||
+73
-20
@@ -490,12 +490,24 @@ int vk_mem_class_of(uint64_t addr) {
|
||||
// ---------------------------------------------------------------- census
|
||||
static volatile int64_t traffic[VK_MCLS_N][VK_MUSE_N];
|
||||
|
||||
static _Thread_local int64_t (*count_to)[VK_MUSE_N];
|
||||
|
||||
// Every caller is the replay (vk_exec*, under vk_sync_submit's lock), about ten adds per draw: one writer at a time, so
|
||||
// a plain add, not a locked one. An add racing the census's reset may be lost.
|
||||
// a plain add, not a locked one. An add racing the census's reset may be lost. Threads recording chunks in parallel
|
||||
// count into their own arrays (vk_mem_count_local), added with atomics once per chunk.
|
||||
void vk_mem_count(int cls, int use, uint64_t bytes) {
|
||||
if ((unsigned)cls < VK_MCLS_N && (unsigned)use < VK_MUSE_N)
|
||||
__atomic_store_n(&traffic[cls][use], __atomic_load_n(&traffic[cls][use], __ATOMIC_RELAXED) + (int64_t)bytes,
|
||||
__ATOMIC_RELAXED);
|
||||
if ((unsigned)cls >= VK_MCLS_N || (unsigned)use >= VK_MUSE_N) return;
|
||||
if (count_to) count_to[cls][use] += (int64_t)bytes;
|
||||
else __atomic_store_n(&traffic[cls][use], __atomic_load_n(&traffic[cls][use], __ATOMIC_RELAXED) + (int64_t)bytes,
|
||||
__ATOMIC_RELAXED);
|
||||
}
|
||||
|
||||
void vk_mem_count_local(int64_t (*acc)[VK_MUSE_N]) { count_to = acc; }
|
||||
|
||||
void vk_mem_count_flush(int64_t (*acc)[VK_MUSE_N]) {
|
||||
for (int c = 0; c < VK_MCLS_N; c++)
|
||||
for (int u = 0; u < VK_MUSE_N; u++)
|
||||
if (acc[c][u]) __atomic_fetch_add(&traffic[c][u], acc[c][u], __ATOMIC_RELAXED), acc[c][u] = 0;
|
||||
}
|
||||
|
||||
static void flag_names(uint32_t f, char *buf, size_t n) {
|
||||
@@ -1212,6 +1224,10 @@ static VkBufferCopy *mregions; // replay: vkCmdCopyBuffer regions
|
||||
static int cregions;
|
||||
static PlatMutex flush_lock = PLAT_MUTEX_INIT;
|
||||
static int flush_cap = FLUSH_RANGES; // ranges a pool keeps; 1 with TOTK_FLUSH_SPAN=1
|
||||
// The first uses of a submission's mirrors, the copies they queue, the GPU writes noted and their snapshots (pend, wbl, snl, mrep): threads
|
||||
// recording chunks of one submission take it. A mirror synced in this submission is read without it (vk_mem_use): its
|
||||
// acc_sub is stored last. Before pool_lock, gen_lock and flush_lock.
|
||||
static PlatMutex mirror_lock = PLAT_MUTEX_INIT;
|
||||
|
||||
typedef struct {
|
||||
uint64_t subs, syncs, upload_bytes, upload_copies, first_n, first_bytes, wb_bytes, wb_copies, wb_ranges;
|
||||
@@ -1336,16 +1352,21 @@ static void refuse(PoolRec *r, int why, const char *what) {
|
||||
static void mirror_create(PoolRec *r) {
|
||||
VkCtx *c = vk_ctx();
|
||||
uint64_t n = r->import_hi - r->import_lo;
|
||||
r->mstate = MS_REFUSED;
|
||||
__atomic_add_fetch(&mirror_gen, 1, __ATOMIC_RELEASE);
|
||||
if (r->kind != POOL_IMPORT || !r->buf || !n) return;
|
||||
if (r->kind != POOL_IMPORT || !r->buf || !n) {
|
||||
__atomic_store_n(&r->mstate, MS_REFUSED, __ATOMIC_RELEASE);
|
||||
return;
|
||||
}
|
||||
plat_mutex_lock_shared(&pool_lock);
|
||||
int i = pidx_upper(&pool_index, r->lo) - 1;
|
||||
while (i >= 0 && pool_index.e[i].buf != r) i--;
|
||||
int alone = i >= 0 && (i == 0 || pool_index.maxhi[i - 1] <= pool_index.e[i].lo) &&
|
||||
(i + 1 == pool_index.n || pool_index.e[i + 1].lo >= pool_index.e[i].hi);
|
||||
plat_mutex_unlock_shared(&pool_lock);
|
||||
if (i < 0) return; // finalized meanwhile
|
||||
if (i < 0) { // finalized meanwhile
|
||||
__atomic_store_n(&r->mstate, MS_REFUSED, __ATOMIC_RELEASE);
|
||||
return;
|
||||
}
|
||||
if (!alone) return refuse(r, MR_OVERLAP, "it overlaps another pool");
|
||||
uint64_t r0 = r->import_lo >> SNAP_SHIFT, r1 = (r->import_hi - 1) >> SNAP_SHIFT;
|
||||
if (r1 >= SNAP_REGIONS) return refuse(r, MR_UNTRACKED, "outside the write-watched address space");
|
||||
@@ -1383,9 +1404,9 @@ static void mirror_create(PoolRec *r) {
|
||||
mlist[at] = r;
|
||||
nm++;
|
||||
plat_mutex_unlock(&pool_lock);
|
||||
r->acc_sub = msub;
|
||||
r->mstate = MS_ON;
|
||||
range_add(&pend, &npend, &cpend, r, 0, n);
|
||||
__atomic_store_n(&r->acc_sub, msub, __ATOMIC_RELEASE);
|
||||
__atomic_store_n(&r->mstate, MS_ON, __ATOMIC_RELEASE);
|
||||
mrep.first_n++, mrep.first_bytes += n;
|
||||
plat_inc64(&m_live_n);
|
||||
plat_add64(&m_live_bytes, (int64_t)n);
|
||||
@@ -1426,7 +1447,6 @@ static void pool_retire(void *arg) {
|
||||
// Replay thread, the first access of this submission: queues the copies of the pages written since the last sync.
|
||||
static void mirror_sync(PoolRec *r) {
|
||||
uint64_t t0 = qpc();
|
||||
r->acc_sub = msub;
|
||||
uint64_t lo = r->import_lo, hi = r->import_hi;
|
||||
uint64_t r0 = lo >> SNAP_SHIFT, r1 = (hi - 1) >> SNAP_SHIFT;
|
||||
int excl = 0;
|
||||
@@ -1475,21 +1495,40 @@ static void mirror_sync(PoolRec *r) {
|
||||
}
|
||||
mrep.syncs++;
|
||||
mrep.sync_t += qpc() - t0;
|
||||
__atomic_store_n(&r->acc_sub, msub, __ATOMIC_RELEASE);
|
||||
}
|
||||
|
||||
// 1 when r is settled for this submission without mirror_lock: refused, or mirrored and synced (*on).
|
||||
static int mirror_settled(PoolRec *r, int *on) {
|
||||
int st = __atomic_load_n(&r->mstate, __ATOMIC_ACQUIRE);
|
||||
*on = st == MS_ON;
|
||||
return st == MS_REFUSED || (st == MS_ON && __atomic_load_n(&r->acc_sub, __ATOMIC_ACQUIRE) == msub);
|
||||
}
|
||||
|
||||
void vk_mem_use(VkMemSpan *sp) {
|
||||
PoolRec *r = sp->pool;
|
||||
if (r->mstate == MS_NONE) mirror_create(r);
|
||||
if (r->mstate != MS_ON) return;
|
||||
if (r->acc_sub != msub) mirror_sync(r);
|
||||
sp->buffer = r->mbuf;
|
||||
int on;
|
||||
if (!mirror_settled(r, &on)) {
|
||||
plat_mutex_lock(&mirror_lock);
|
||||
if (r->mstate == MS_NONE) mirror_create(r);
|
||||
on = r->mstate == MS_ON;
|
||||
if (on && r->acc_sub != msub) mirror_sync(r);
|
||||
plat_mutex_unlock(&mirror_lock);
|
||||
}
|
||||
if (on) sp->buffer = r->mbuf;
|
||||
}
|
||||
|
||||
void vk_mem_pick(VkMemSpan *sp) {
|
||||
PoolRec *r = sp->pool;
|
||||
if (!r || r->mstate != MS_ON) return;
|
||||
if (r->acc_sub != msub) mirror_sync(r);
|
||||
sp->buffer = r->mbuf;
|
||||
int on;
|
||||
if (!r) return;
|
||||
if (!mirror_settled(r, &on)) {
|
||||
plat_mutex_lock(&mirror_lock);
|
||||
on = r->mstate == MS_ON;
|
||||
if (on && r->acc_sub != msub) mirror_sync(r);
|
||||
plat_mutex_unlock(&mirror_lock);
|
||||
}
|
||||
if (on) sp->buffer = r->mbuf;
|
||||
}
|
||||
|
||||
int vk_mem_mirror_probe(uint64_t addr) {
|
||||
@@ -1753,10 +1792,14 @@ static void gpu_wrote(const VkMemSpan *sp, uint64_t size, int shader) {
|
||||
PoolRec *r = sp->pool;
|
||||
if (r && size) gpu_writes(r, sp->offset, size);
|
||||
if (!r || r->mstate != MS_ON || sp->buffer != r->mbuf || !size) return;
|
||||
plat_mutex_lock(&mirror_lock);
|
||||
// A binding to the end of its pool (no size) can span hundreds of MiB: those stay copies.
|
||||
if (shader && !(sp->offset & 3) && !(size & 3) && size <= MERGE_MAX && merging() && snap_add(r, sp->offset, size))
|
||||
if (shader && !(sp->offset & 3) && !(size & 3) && size <= MERGE_MAX && merging() && snap_add(r, sp->offset, size)) {
|
||||
plat_mutex_unlock(&mirror_lock);
|
||||
return;
|
||||
}
|
||||
range_add(&wbl, &nwbl, &cwbl, r, sp->offset, size);
|
||||
plat_mutex_unlock(&mirror_lock);
|
||||
}
|
||||
void vk_mem_gpu_wrote(const VkMemSpan *sp, uint64_t size) { gpu_wrote(sp, size, 0); }
|
||||
void vk_mem_gpu_wrote_shader(const VkMemSpan *sp, uint64_t size) { gpu_wrote(sp, size, 1); }
|
||||
@@ -1888,10 +1931,19 @@ void vk_mem_mirror_begin(void) {
|
||||
mrep.presnap_t += qpc() - t0;
|
||||
}
|
||||
|
||||
int vk_mem_mirror_pending(void) { return npend || snl_copied < nsnl; }
|
||||
int vk_mem_mirror_pending(void) {
|
||||
plat_mutex_lock(&mirror_lock);
|
||||
int n = npend || snl_copied < nsnl;
|
||||
plat_mutex_unlock(&mirror_lock);
|
||||
return n;
|
||||
}
|
||||
|
||||
void vk_mem_mirror_prologue(VkCommandBuffer cb) {
|
||||
if (!npend && snl_copied == nsnl) return;
|
||||
plat_mutex_lock(&mirror_lock);
|
||||
if (!npend && snl_copied == nsnl) {
|
||||
plat_mutex_unlock(&mirror_lock);
|
||||
return;
|
||||
}
|
||||
mbarrier(cb, VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT, VK_ACCESS_2_MEMORY_WRITE_BIT, VK_PIPELINE_STAGE_2_COPY_BIT,
|
||||
VK_ACCESS_2_TRANSFER_READ_BIT | VK_ACCESS_2_TRANSFER_WRITE_BIT);
|
||||
copy_ranges(cb, pend, npend, 1, &mrep.upload_bytes, &mrep.upload_copies);
|
||||
@@ -1904,6 +1956,7 @@ void vk_mem_mirror_prologue(VkCommandBuffer cb) {
|
||||
VK_ACCESS_2_MEMORY_READ_BIT | VK_ACCESS_2_MEMORY_WRITE_BIT);
|
||||
npend = 0;
|
||||
mrep.prologues++;
|
||||
plat_mutex_unlock(&mirror_lock);
|
||||
}
|
||||
|
||||
// ---- verify mode
|
||||
|
||||
@@ -157,10 +157,10 @@ void vk_mvsurvey_draw(VkExec *ex) {
|
||||
k->size = (uint32_t)u->size;
|
||||
uint32_t want = u->size < SNAP ? (uint32_t)u->size : SNAP;
|
||||
VkDescriptorBufferInfo bi;
|
||||
const uint8_t *src = vk_usnap_find(u->addr, want, 1, &bi) ? vk_usnap_map() + bi.offset : NULL;
|
||||
const uint8_t *src = vk_usnap_find(u->addr, want, UINT32_MAX, 1, &bi) ? vk_usnap_map() + bi.offset : NULL;
|
||||
k->have = src ? want : 0;
|
||||
if (src) memcpy(k->bytes, src, want);
|
||||
const uint8_t *all = vk_usnap_find(u->addr, u->size, 1, &bi) ? vk_usnap_map() + bi.offset : NULL;
|
||||
const uint8_t *all = vk_usnap_find(u->addr, u->size, UINT32_MAX, 1, &bi) ? vk_usnap_map() + bi.offset : NULL;
|
||||
k->mats = all ? lead_mats((const float *)all, (uint32_t)(u->size / 4)) : 0;
|
||||
if (all) {
|
||||
Stat *s = &S.st[b];
|
||||
|
||||
@@ -3223,10 +3223,11 @@ static void dyn_vals(const VkPipeState *s, DynVals *o) {
|
||||
}
|
||||
}
|
||||
|
||||
// What the replay's command buffer holds, as vk_pipeline_set_dynamic_delta last recorded it (the replay thread only):
|
||||
// the values, the inputs of the vertex input and the vertex bindings it reads. Invariant while the caller passes
|
||||
// valid 1: cb holds DC.v for every dynamic state of P.dyn, colour attachments [0, DC.v.n) included.
|
||||
static struct {
|
||||
// What the replay's command buffer holds, as vk_pipeline_set_dynamic_delta last recorded it: the values, the inputs of
|
||||
// the vertex input and the vertex bindings it reads. Invariant while the caller passes valid 1: cb holds DC.v for every
|
||||
// dynamic state of P.dyn, colour attachments [0, DC.v.n) included. Per thread: the threads of a chunked replay
|
||||
// (vk_exec.c) each record their own command buffers, and a new one starts with valid 0.
|
||||
static _Thread_local struct {
|
||||
DynVals v;
|
||||
const GfxShader *vs;
|
||||
uint32_t vattr[16];
|
||||
|
||||
@@ -454,6 +454,8 @@ void rec_check_log(void) {
|
||||
}
|
||||
|
||||
#define OWNED(cb) (rec_live && (cb) == rec_cb)
|
||||
|
||||
int rec_owns(VkCommandBuffer cb) { return OWNED(cb); }
|
||||
// A call the ring takes no packet for: everything queued first, then the call here.
|
||||
#define FALLBACK() (rec_drain(), RS_.fallbacks++)
|
||||
|
||||
|
||||
@@ -33,6 +33,7 @@ void rec_drain(void); // waits until the recorder made every cal
|
||||
// vk_queue_lock, before it takes the lock (not on the recorder, not when the thread holds the lock already): waits
|
||||
// until the recorder submitted every submission queued so far.
|
||||
void rec_queue_wait(void);
|
||||
int rec_owns(VkCommandBuffer cb); // the recorder makes cb's calls: a direct call into cb follows rec_drain
|
||||
void rec_stats_log(void); // TOTK_TRACE=perf: packets, bytes, drains and waits since the last call
|
||||
// TOTK_REC_CHECK=1 (debug, recorder on): vk_memory.c reports each object it defers (destroy 0, stamped tl) or destroys
|
||||
// (destroy 1) with the value the GPU reached (gpu); a packet of the part being recorded that still names it is logged
|
||||
|
||||
+86
-12
@@ -56,7 +56,7 @@ static VkQueryPool ts_pool;
|
||||
static int ts_state; // 0 not set up, 1 on, -1 off
|
||||
static double ts_period; // ns per tick
|
||||
static uint64_t ts_mask; // valid bits of a timestamp
|
||||
static uint32_t ts_next; // submitting thread (sub_lock): next pair
|
||||
static uint32_t ts_next; // next pair (atomic: parts recorded on other threads take pairs too)
|
||||
static volatile uint8_t ts_busy[TS_PAIRS]; // written until the completion thread read the pair
|
||||
static int ts_cur = -1; // the pair of the part being recorded, -1 untimed
|
||||
static uint64_t ts_last_end; // completion thread: end of the latest part read (ticks)
|
||||
@@ -175,6 +175,12 @@ static void ts_read_cb(void *arg) { // completion thread, in submission order
|
||||
free(t);
|
||||
}
|
||||
|
||||
static int ts_take(void) {
|
||||
uint32_t p = __atomic_fetch_add(&ts_next, 1, __ATOMIC_RELAXED) % TS_PAIRS;
|
||||
if (__atomic_exchange_n(&ts_busy[p], 1, __ATOMIC_ACQUIRE)) { plat_inc64(&ts_missed); return -1; }
|
||||
return (int)p;
|
||||
}
|
||||
|
||||
// At the start of a command buffer (sub_lock held): the pair it times, -1 untimed.
|
||||
static int ts_open(VkCommandBuffer cb) {
|
||||
if (ts_state != 1 || !perf_gpu_timing()) {
|
||||
@@ -182,10 +188,9 @@ static int ts_open(VkCommandBuffer cb) {
|
||||
ts_setup(vk_ctx()); // the overlay turned full: timestamps from now on
|
||||
if (ts_state != 1) return -1;
|
||||
}
|
||||
uint32_t p = ts_next % TS_PAIRS;
|
||||
if (__atomic_load_n(&ts_busy[p], __ATOMIC_ACQUIRE)) { plat_inc64(&ts_missed); return -1; }
|
||||
ts_next++;
|
||||
ts_busy[p] = 1;
|
||||
int t = ts_take();
|
||||
if (t < 0) return -1;
|
||||
uint32_t p = (uint32_t)t;
|
||||
vkCmdResetQueryPool(cb, ts_pool, 2 * p, 2);
|
||||
vkCmdWriteTimestamp(cb, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, ts_pool, 2 * p);
|
||||
return (int)p;
|
||||
@@ -203,6 +208,17 @@ static void ts_close(VkCommandBuffer cb, int pair, uint64_t fence) {
|
||||
vk_completion_add(fence, ts_read_cb, t);
|
||||
}
|
||||
|
||||
int vk_sync_ts_open(VkCommandBuffer cb) {
|
||||
if (__atomic_load_n(&ts_state, __ATOMIC_ACQUIRE) != 1 || !perf_gpu_timing()) return -1;
|
||||
int p = ts_take();
|
||||
if (p < 0) return -1;
|
||||
vkCmdResetQueryPool(cb, ts_pool, 2 * (uint32_t)p, 2);
|
||||
vkCmdWriteTimestamp(cb, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, ts_pool, 2 * (uint32_t)p);
|
||||
return p;
|
||||
}
|
||||
|
||||
void vk_sync_ts_close(VkCommandBuffer cb, int pair, uint64_t fence) { ts_close(cb, pair, fence); }
|
||||
|
||||
// The pair of the part being recorded.
|
||||
static void ts_begin(VkCommandBuffer cb) { ts_cur = ts_open(cb); }
|
||||
|
||||
@@ -349,26 +365,56 @@ static CmdSlot *take_slot(VkCtx *c, uint64_t fence) {
|
||||
// submitted right before the part that needs them and timed like a part; VK_NULL_HANDLE when there are none. After the
|
||||
// part was recorded, sub_lock held; may move slots[].
|
||||
// First brings the shadow storage up to date with the CPU's writes, which needs no command buffer.
|
||||
static VkCommandBuffer prologue(VkCtx *c, uint64_t fence) {
|
||||
// direct: the calling thread records, ends and submits it itself while the part being replayed stays the recorder's
|
||||
// (vk_sync_part, which the caller made the recorder idle for).
|
||||
static VkCommandBuffer prologue(VkCtx *c, uint64_t fence, int direct) {
|
||||
vk_mem_shadow_sync(); // shadow storage: the CPU's writes go in before the part is submitted
|
||||
if (!vk_mem_mirror_pending()) return VK_NULL_HANDLE;
|
||||
int i = claim_slot(c, fence); // before reading slots: the claim may move it
|
||||
VkCommandBuffer cb = slots[i].cb;
|
||||
rec_begin(cb, fence, 0, 0); // a command buffer of the part pool: the recorder's like the parts
|
||||
if (direct) {
|
||||
VkCommandBufferBeginInfo bi = {.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,
|
||||
.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
|
||||
vkBeginCommandBuffer(cb, &bi);
|
||||
} else {
|
||||
rec_begin(cb, fence, 0, 0); // a command buffer of the part pool: the recorder's like the parts
|
||||
}
|
||||
int pair = ts_open(cb);
|
||||
vk_mem_mirror_prologue(cb);
|
||||
ts_close(cb, pair, fence);
|
||||
if (!rec_end_cb(cb, 0, 0)) vkEndCommandBuffer(cb);
|
||||
if (direct || !rec_end_cb(cb, 0, 0)) vkEndCommandBuffer(cb);
|
||||
return cb;
|
||||
}
|
||||
|
||||
// A chunk of a chunked replay (vk_exec.c chunks_submit), while the submission's own part is open on the recorder. The
|
||||
// recorder begins, records, ends and submits the parts it owns (vk_rec.c rec_end_cb), so the prologue stays off it:
|
||||
// the recorder is made idle first, as the part pool allows one thread at a time, and the prologue is recorded, ended
|
||||
// and submitted here without taking the recorder's part away.
|
||||
void vk_sync_part(VkCommandBuffer cb) {
|
||||
VkCtx *c = vk_ctx();
|
||||
vk_upload_flush(0);
|
||||
rec_drain();
|
||||
VkCommandBuffer pro = prologue(c, cur_fence, 1);
|
||||
VkCommandBufferSubmitInfo cbi[2] = {{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO, .commandBuffer = pro},
|
||||
{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO, .commandBuffer = cb}};
|
||||
VkSubmitInfo2 si = {.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO_2, .commandBufferInfoCount = pro ? 2 : 1,
|
||||
.pCommandBufferInfos = pro ? cbi : cbi + 1};
|
||||
vk_queue_lock();
|
||||
VkResult r = vkQueueSubmit2(c->gfx_queue, 1, &si, VK_NULL_HANDLE);
|
||||
vk_queue_unlock();
|
||||
if (r != VK_SUCCESS) {
|
||||
vk_device_fault_report(r);
|
||||
rt_fatal("vk: vkQueueSubmit2 of a part of value %llu failed (%d)", (unsigned long long)cur_fence, (int)r);
|
||||
}
|
||||
}
|
||||
|
||||
VkCommandBuffer vk_sync_split(void) {
|
||||
VkCtx *c = vk_ctx();
|
||||
VkCommandBuffer cb = slots[cur_slot].cb;
|
||||
ts_end(cb, cur_fence);
|
||||
if (!rec_end_cb(cb, 0, 0)) vkEndCommandBuffer(cb); // every command of the part is in cb, ended by its recorder
|
||||
vk_upload_flush(0); // the uploads this part asked for precede it in queue order
|
||||
VkCommandBuffer pro = prologue(c, cur_fence);
|
||||
VkCommandBuffer pro = prologue(c, cur_fence, 0);
|
||||
VkCommandBufferSubmitInfo cbi[2] = {{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO, .commandBuffer = pro},
|
||||
{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO, .commandBuffer = cb}};
|
||||
VkSubmitInfo2 si = {.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO_2, .commandBufferInfoCount = pro ? 2 : 1,
|
||||
@@ -419,7 +465,7 @@ uint64_t vk_sync_submit(const uint64_t *handles, int n) {
|
||||
perf_replay(t0);
|
||||
// After the recording: the uploads it asked for (and every earlier one) precede it in queue order.
|
||||
vk_upload_flush(0);
|
||||
VkCommandBuffer pro = prologue(c, fence);
|
||||
VkCommandBuffer pro = prologue(c, fence, 0);
|
||||
s = &slots[cur_slot];
|
||||
VkCommandBufferSubmitInfo cbi[2] = {{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO, .commandBuffer = pro},
|
||||
{.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_SUBMIT_INFO, .commandBuffer = s->cb}};
|
||||
@@ -446,8 +492,11 @@ uint64_t vk_sync_submit(const uint64_t *handles, int n) {
|
||||
return fence;
|
||||
}
|
||||
|
||||
void vk_completion_add(uint64_t fence, void (*fn)(void *), void *arg) {
|
||||
plat_mutex_lock(&st_lock);
|
||||
static _Thread_local VkDeferred *deferred;
|
||||
|
||||
void vk_completion_defer(VkDeferred *d) { deferred = d; }
|
||||
|
||||
static void queue_put(uint64_t fence, void (*fn)(void *), void *arg) { // st_lock held
|
||||
if (qhead + qn == qcap) {
|
||||
if (qhead) {
|
||||
memmove(queue, queue + qhead, sizeof *queue * (size_t)qn);
|
||||
@@ -463,9 +512,34 @@ void vk_completion_add(uint64_t fence, void (*fn)(void *), void *arg) {
|
||||
// The completion thread sleeps only while nothing is issued beyond done: a callback for a later value (the replay's,
|
||||
// for the submission it records: counter reports, freed resources) becomes due through the submit, which wakes it.
|
||||
if (fence <= done) plat_cond_broadcast(&cv_work);
|
||||
}
|
||||
|
||||
void vk_completion_add(uint64_t fence, void (*fn)(void *), void *arg) {
|
||||
VkDeferred *d = deferred;
|
||||
if (d) {
|
||||
if (d->n == d->cap) {
|
||||
d->cap = d->cap ? d->cap * 2 : 64;
|
||||
d->items = realloc(d->items, sizeof(Pending) * (size_t)d->cap);
|
||||
if (!d->items) rt_fatal("vk: out of memory (completion queue)");
|
||||
}
|
||||
((Pending *)d->items)[d->n++] = (Pending){fence, fn, arg};
|
||||
return;
|
||||
}
|
||||
plat_mutex_lock(&st_lock);
|
||||
queue_put(fence, fn, arg);
|
||||
plat_mutex_unlock(&st_lock);
|
||||
}
|
||||
|
||||
void vk_completion_flush(VkDeferred *d) {
|
||||
plat_mutex_lock(&st_lock);
|
||||
for (int i = 0; i < d->n; i++) {
|
||||
const Pending *p = &((Pending *)d->items)[i];
|
||||
queue_put(p->fence, p->fn, p->arg);
|
||||
}
|
||||
plat_mutex_unlock(&st_lock);
|
||||
d->n = 0;
|
||||
}
|
||||
|
||||
static int fence_wait_impl(uint64_t value, uint64_t timeout_ns);
|
||||
|
||||
int vk_sync_fence_wait(uint64_t value, uint64_t timeout_ns) {
|
||||
|
||||
+77
-14
@@ -46,6 +46,7 @@ typedef struct Tex {
|
||||
uint32_t wgen; // page generation of the last upload check
|
||||
uint32_t checked; // submit_epoch of the last check from a draw binding
|
||||
int uploaded, released;
|
||||
uint32_t rev_epoch, rev_pos; // the state scan made it live in that submission at that op (vk_texture_revive)
|
||||
uint32_t n_up; // uploads of this entry (vk_texture_debug)
|
||||
int last_ch; // last vk_mem_changed result of make_fresh: 1 written, 0 clean, -1 not tracked
|
||||
struct Tex *next_at; // other entries at the same storage address
|
||||
@@ -122,10 +123,12 @@ static uint64_t n_checks; // draw-binding storage checks (make_fresh_snap) si
|
||||
// which emptied every cache keyed on the global generations.
|
||||
#define AGEN_N 4096
|
||||
static uint32_t agen[AGEN_N];
|
||||
static uint32_t arev[AGEN_N]; // per storage address (hashed): the submit_epoch of the state scan's last revival there
|
||||
// Per slot id, for vk_texture_slot_token without the lock: the registration serial (bits 16-47; 0: never registered)
|
||||
// and the address generation index (bits 0-15), one word so a reader gets a consistent pair. Fixed-size (slot ids are
|
||||
// 20 bits), unlike tslots, which moves when it grows.
|
||||
static uint64_t *sinfo;
|
||||
static uint32_t *sepoch; // per slot: the submit_epoch its entry was checked in (vk_texture_slot_checked), 0 none
|
||||
static uint32_t reg_serial;
|
||||
|
||||
static uint32_t agen_idx(uint64_t addr) { return (uint32_t)(addr * 0x9E3779B97F4A7C15ull >> 52); }
|
||||
@@ -1288,6 +1291,13 @@ static int alias_copy(Tex *e, const Tex *src) {
|
||||
return 1;
|
||||
}
|
||||
|
||||
static void entry_live(Tex *e) {
|
||||
if (!e->released) return;
|
||||
takeback_n[e->pub.gpu_owned != 0][age_bucket(coarse_ms - e->rel_ms)]++;
|
||||
e->use_seq = ++use_counter;
|
||||
e->released = 0, nreleased--, rel_unlink(e), tex_gen_bump(e->key.addr);
|
||||
}
|
||||
|
||||
static Tex *entry_of(const tex_t *t) {
|
||||
TexKey k;
|
||||
make_key(t, &k);
|
||||
@@ -1336,11 +1346,7 @@ static Tex *entry_of(const tex_t *t) {
|
||||
nvn_hm_put(&by_key, &k, e);
|
||||
tex_gen_bump(k.addr);
|
||||
}
|
||||
if (e->released) {
|
||||
takeback_n[e->pub.gpu_owned != 0][age_bucket(coarse_ms - e->rel_ms)]++;
|
||||
e->use_seq = ++use_counter;
|
||||
e->released = 0, nreleased--, rel_unlink(e), tex_gen_bump(k.addr);
|
||||
}
|
||||
entry_live(e);
|
||||
e->t = *t;
|
||||
return e;
|
||||
}
|
||||
@@ -1366,6 +1372,25 @@ const VkTexImage *vk_texture_get(const tex_t *t) {
|
||||
return e ? &e->pub : NULL;
|
||||
}
|
||||
|
||||
// The replay's state scan (vk_exec.c): the entry vk_texture_get would return, neither created nor made current; the
|
||||
// recording does that. It is revived here, in replay order, and remembers the op it happened at: a chunk may bind a
|
||||
// finalized target's slot before the chunk that renders it reached its SET_RT, or long after the scan went past a later
|
||||
// target at the same storage, and slot_resolve has to answer as the replay would at the chunk's own op (live_at).
|
||||
const VkTexImage *vk_texture_revive(const tex_t *t, uint32_t pos) {
|
||||
if (!t || !t->mem) return NULL;
|
||||
TexKey k;
|
||||
make_key(t, &k);
|
||||
lock();
|
||||
Tex *e = nvn_hm_get(&by_key, &k);
|
||||
if (e && e->released) {
|
||||
e->rev_epoch = submit_epoch, e->rev_pos = pos;
|
||||
__atomic_store_n(&arev[agen_idx(k.addr)], submit_epoch, __ATOMIC_RELEASE);
|
||||
entry_live(e);
|
||||
}
|
||||
unlock();
|
||||
return e ? &e->pub : NULL;
|
||||
}
|
||||
|
||||
// Render targets the CPU reads without NVN's invalidate (vk_texture_write_back_small); small_pending: one may be dirty.
|
||||
// wb_limit: targets up to this size (bytes) always go back when the CPU waits for a fence, TOTK_WB_FENCE_LIMIT, read
|
||||
// once. A larger one goes back unless the GPU wrote it in each of the last WB_RUN frames (wb_wanted). wb_trace:
|
||||
@@ -1468,7 +1493,9 @@ void vk_texture_register(int id, const tex_t *t, const view_t *v) {
|
||||
}
|
||||
if (!sinfo) {
|
||||
uint64_t *p = calloc(1 << 20, sizeof *p);
|
||||
if (!p) rt_fatal("vk: out of memory for texture slot tokens");
|
||||
uint32_t *q = calloc(1 << 20, sizeof *q);
|
||||
if (!p || !q) rt_fatal("vk: out of memory for texture slot tokens");
|
||||
__atomic_store_n(&sepoch, q, __ATOMIC_RELEASE);
|
||||
__atomic_store_n(&sinfo, p, __ATOMIC_RELEASE);
|
||||
}
|
||||
TexSlot *s = &tslots[id];
|
||||
@@ -1484,15 +1511,23 @@ void vk_texture_register(int id, const tex_t *t, const view_t *v) {
|
||||
unlock();
|
||||
}
|
||||
|
||||
// The entry slot s resolves to, and the view and has_view a caller gets with it (the slot's, with the descriptor's
|
||||
// swizzle when the entry is another descriptor at the same storage). NULL when the slot names no texture.
|
||||
static Tex *slot_resolve(TexSlot *s, view_t *view, int *has_view) {
|
||||
// Whether e is live at op pos of a chunked replay: the state scan is ahead of the chunks, and an entry it revived at a
|
||||
// later op is still released for this one. Two 4K targets at one address, RGBA8 and R8 with memory of their own, were
|
||||
// swapped whenever the scan got past the second's SET_RT before a chunk sampling the storage got its turn, and large
|
||||
// parts of the scene flickered. UINT32_MAX: the state now.
|
||||
static int live_at(const Tex *e, uint32_t pos) {
|
||||
return !e->released && !(e->rev_epoch == submit_epoch && e->rev_pos > pos);
|
||||
}
|
||||
|
||||
// The entry slot s resolves to at op pos, and the view and has_view a caller gets with it (the slot's, with the
|
||||
// descriptor's swizzle when the entry is another descriptor at the same storage). NULL when the slot names no texture.
|
||||
static Tex *slot_resolve(TexSlot *s, view_t *view, int *has_view, uint32_t pos) {
|
||||
*view = s->v;
|
||||
*has_view = s->has_view;
|
||||
TexKey k;
|
||||
make_key(&s->t, &k);
|
||||
Tex *e = nvn_hm_get(&by_key, &k);
|
||||
if (!e || e->released) { // MNG hle_nvn.c: a finalized slot follows the live texture at its storage
|
||||
if (!e || !live_at(e, pos)) { // MNG hle_nvn.c: a finalized slot follows the live texture at its storage
|
||||
// Prefer the live entry that differs from the slot descriptor only in its swizzle: transient render targets
|
||||
// share storage, and the newest one at the address is usually another size. The map screen samples its
|
||||
// 2250x1080 map target through an (R,G,B,1) descriptor and followed a 960x60 target at the same address.
|
||||
@@ -1505,7 +1540,7 @@ static Tex *slot_resolve(TexSlot *s, view_t *view, int *has_view) {
|
||||
Tex *own = e, *any = NULL;
|
||||
e = NULL;
|
||||
for (Tex *p = nvn_hm_get(&by_addr, &k.addr); p; p = p->next_at) {
|
||||
if (p->released || linear_format(p->key.d.format) != linear_format(k.d.format)) continue;
|
||||
if (!live_at(p, pos) || linear_format(p->key.d.format) != linear_format(k.d.format)) continue;
|
||||
if (!any) any = p;
|
||||
if (same_but_swizzle(&p->key, &k)) { e = p; break; }
|
||||
}
|
||||
@@ -1553,14 +1588,28 @@ const VkTexImage *vk_texture_by_id(int id, view_t *view, int *has_view) {
|
||||
const VkTexImage *vk_texture_by_id_locked(int id, view_t *view, int *has_view) {
|
||||
if (id < 0 || id >= ntslots || !tslots[id].used) return NULL;
|
||||
TexSlot *s = &tslots[id];
|
||||
uint32_t g = agen[sinfo[id] & 0xffff];
|
||||
uint32_t idx = sinfo[id] & 0xffff, g = agen[idx];
|
||||
uint32_t pos = arev[idx] == submit_epoch ? vk_exec_replay_pos() : UINT32_MAX;
|
||||
if (pos != UINT32_MAX) { // what the storage holds depends on the op: chunks at other ops share no answer in s
|
||||
view_t v;
|
||||
int hv;
|
||||
Tex *f = slot_resolve(s, &v, &hv, pos);
|
||||
if (f && (!f->uploaded || f->checked != submit_epoch)) {
|
||||
make_fresh_snap(f);
|
||||
f->checked = submit_epoch;
|
||||
}
|
||||
if (view) *view = has_view ? v : s->v;
|
||||
if (has_view) *has_view = view ? hv : s->has_view;
|
||||
return f ? &f->pub : NULL;
|
||||
}
|
||||
if (!s->ce || s->cgen != g) { // resolved again: an entry at its address was created, dropped or released, or the
|
||||
view_t v; // slot registered (s->ce cleared); an empty result is not kept: image creation may
|
||||
int hv; // succeed later
|
||||
s->ce = slot_resolve(s, &v, &hv); // entry_of may create an entry and bump g: g read before it
|
||||
s->ce = slot_resolve(s, &v, &hv, UINT32_MAX); // entry_of may create an entry and bump g: g read before it
|
||||
s->cview = v, s->chas_view = (uint8_t)hv;
|
||||
s->cgen = g;
|
||||
s->cepoch = 0;
|
||||
__atomic_store_n(&sepoch[id], 0, __ATOMIC_RELAXED);
|
||||
}
|
||||
Tex *e = s->ce;
|
||||
// Bindings of one submit share one check, answered from the submit's write-watch snapshot (guest writes during a
|
||||
@@ -1572,6 +1621,7 @@ const VkTexImage *vk_texture_by_id_locked(int id, view_t *view, int *has_view) {
|
||||
e->checked = submit_epoch;
|
||||
}
|
||||
s->cepoch = submit_epoch;
|
||||
__atomic_store_n(&sepoch[id], submit_epoch, __ATOMIC_RELEASE);
|
||||
}
|
||||
if (view) *view = has_view ? s->cview : s->v;
|
||||
if (has_view) *has_view = view ? s->chas_view : s->has_view;
|
||||
@@ -1593,6 +1643,19 @@ int vk_texture_slot_swz(int id, uint8_t swz[4]) {
|
||||
|
||||
uint32_t vk_texture_epoch(void) { return __atomic_load_n(&submit_epoch, __ATOMIC_RELAXED); }
|
||||
|
||||
int vk_texture_slot_moves(int id) {
|
||||
const uint64_t *si = __atomic_load_n(&sinfo, __ATOMIC_ACQUIRE);
|
||||
if (!si || id < 0 || id >= (1 << 20)) return 0;
|
||||
uint64_t v = __atomic_load_n(&si[id], __ATOMIC_RELAXED);
|
||||
return __atomic_load_n(&arev[v & 0xffff], __ATOMIC_ACQUIRE) == __atomic_load_n(&submit_epoch, __ATOMIC_RELAXED);
|
||||
}
|
||||
|
||||
int vk_texture_slot_checked(int id) {
|
||||
const uint32_t *e = __atomic_load_n(&sepoch, __ATOMIC_ACQUIRE);
|
||||
return e && id >= 0 && id < (1 << 20) &&
|
||||
__atomic_load_n(&e[id], __ATOMIC_ACQUIRE) == __atomic_load_n(&submit_epoch, __ATOMIC_RELAXED);
|
||||
}
|
||||
|
||||
void vk_texture_new_submit(void) {
|
||||
lock();
|
||||
if (!++submit_epoch) submit_epoch = 1;
|
||||
@@ -2419,6 +2482,6 @@ void vk_texture_shutdown(void) {
|
||||
free(tslots), free(sslots);
|
||||
tslots = NULL, sslots = NULL, ntslots = nsslots = 0, nreleased = 0;
|
||||
rel_head = rel_tail = NULL;
|
||||
if (sinfo) memset(sinfo, 0, sizeof *sinfo << 20); // no slot is registered: tokens 0
|
||||
if (sinfo) memset(sinfo, 0, sizeof *sinfo << 20), memset(sepoch, 0, sizeof *sepoch << 20); // no slot is registered: tokens 0
|
||||
unlock();
|
||||
}
|
||||
+24
-12
@@ -11,11 +11,13 @@
|
||||
// that submission's write-back, the mirror holds them until then.
|
||||
// - A GPU write within the submission (UpdateUniformBuffer, a storage buffer a shader writes, a buffer copy or fill,
|
||||
// an image-to-buffer copy: vk_usnap_drop) drops the copies it touches from then on, so later draws read the result.
|
||||
// A chunked replay (vk_exec.c) records the ops out of order: its scan drops at the position of the op
|
||||
// (vk_usnap_drop_at, scan_writes for the shaders' stores), and a chunk finds a copy only at a position before that.
|
||||
// - The arena goes back to a free list once the submission's fence is done (completion thread).
|
||||
// The design follows MacNativeGames' TOTK_UBO_SNAPSHOT (games/totk/third_party/d2r_gfx/gfx/nvn/nvn_mtl.m usnap_take,
|
||||
// usnap_drop; MIT, Copyright (c) 2026 jhaizhou); the code is ours.
|
||||
// The copy is taken where vk_exec_record starts: at the submit call when the replay runs on the submitting thread, later
|
||||
// with TOTK_REPLAY_ASYNC. TOTK_UBO_SNAPSHOT=0 binds live memory as before; =ab alternates (TOTK_AB).
|
||||
// with TOTK_REPLAY_ASYNC; a chunked replay takes it on a recording thread while its scan runs. TOTK_UBO_SNAPSHOT=0 binds live memory as before; =ab alternates (TOTK_AB).
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
@@ -42,7 +44,7 @@ typedef struct Arena {
|
||||
} Arena;
|
||||
|
||||
typedef struct { uint64_t lo, hi; } Rng;
|
||||
typedef struct { uint64_t lo, hi, off; int dropped; } Reg;
|
||||
typedef struct { uint64_t lo, hi, off, drop_at; } Reg; // drop_at: dropped from this position on; UINT64_MAX not
|
||||
typedef struct { uint64_t lo, hi, fence; } Wrote;
|
||||
|
||||
static struct {
|
||||
@@ -62,6 +64,7 @@ static struct {
|
||||
|
||||
static Arena *free_arenas;
|
||||
static PlatMutex free_lock = PLAT_MUTEX_INIT;
|
||||
static PlatMutex drop_lock = PLAT_MUTEX_INIT; // threads recording chunks of one submission drop concurrently
|
||||
|
||||
static void *grow(void *p, int *cap, size_t elem) {
|
||||
int n = *cap ? *cap * 2 : 1024;
|
||||
@@ -279,7 +282,7 @@ void vk_usnap_begin(const uint64_t *handles, int n, uint64_t fence) {
|
||||
continue;
|
||||
}
|
||||
if (U.nreg == U.creg) U.reg = grow(U.reg, &U.creg, sizeof *U.reg);
|
||||
U.reg[U.nreg++] = (Reg){lo, hi, total, 0};
|
||||
U.reg[U.nreg++] = (Reg){lo, hi, total, UINT64_MAX};
|
||||
total += (hi - lo + ALIGN - 1) & ~(uint64_t)(ALIGN - 1);
|
||||
}
|
||||
if (!U.nreg) return;
|
||||
@@ -313,29 +316,38 @@ static int first_above(uint64_t a) {
|
||||
return lo;
|
||||
}
|
||||
|
||||
int vk_usnap_find(uint64_t addr, uint64_t n, uint64_t align, VkDescriptorBufferInfo *out) {
|
||||
int vk_usnap_find(uint64_t addr, uint64_t n, uint64_t at, uint64_t align, VkDescriptorBufferInfo *out) {
|
||||
if (!U.on || !n) return 0;
|
||||
int i = first_above(addr);
|
||||
if (i >= U.nreg) return 0;
|
||||
const Reg *r = &U.reg[i];
|
||||
if (r->dropped || addr < r->lo || addr + n > r->hi) return 0;
|
||||
if (__atomic_load_n(&r->drop_at, __ATOMIC_RELAXED) <= at || addr < r->lo || addr + n > r->hi) return 0;
|
||||
uint64_t off = r->off + (addr - r->lo);
|
||||
if (off % align) return 0;
|
||||
*out = (VkDescriptorBufferInfo){U.arena->buf, off, n};
|
||||
U.hits++;
|
||||
__atomic_fetch_add(&U.hits, 1, __ATOMIC_RELAXED);
|
||||
return 1;
|
||||
}
|
||||
|
||||
void vk_usnap_drop(uint64_t addr, uint64_t n) {
|
||||
static void drop(uint64_t addr, uint64_t n, uint64_t at) {
|
||||
if (!U.mode || !n) return;
|
||||
plat_mutex_lock(&drop_lock);
|
||||
if (U.nwr == U.cwr) U.wr = grow(U.wr, &U.cwr, sizeof *U.wr);
|
||||
U.wr[U.nwr++] = (Wrote){addr, addr + n, U.fence};
|
||||
if (!U.on) return;
|
||||
for (int i = first_above(addr); i < U.nreg && U.reg[i].lo < addr + n; i++)
|
||||
if (!U.reg[i].dropped) U.reg[i].dropped = 1, U.drops++, U.gen++;
|
||||
if (U.on)
|
||||
for (int i = first_above(addr); i < U.nreg && U.reg[i].lo < addr + n; i++)
|
||||
if (at < U.reg[i].drop_at) {
|
||||
if (U.reg[i].drop_at == UINT64_MAX) U.drops++;
|
||||
__atomic_store_n(&U.reg[i].drop_at, at, __ATOMIC_RELAXED);
|
||||
__atomic_add_fetch(&U.gen, 1, __ATOMIC_RELAXED);
|
||||
}
|
||||
plat_mutex_unlock(&drop_lock);
|
||||
}
|
||||
|
||||
uint64_t vk_usnap_gen(void) { return U.gen; }
|
||||
void vk_usnap_drop(uint64_t addr, uint64_t n) { drop(addr, n, 0); }
|
||||
void vk_usnap_drop_at(uint64_t addr, uint64_t n, uint32_t at) { drop(addr, n, at); }
|
||||
|
||||
uint64_t vk_usnap_gen(void) { return __atomic_load_n(&U.gen, __ATOMIC_RELAXED); }
|
||||
|
||||
// the host copy where there is one (DLSS), else the device copy
|
||||
static uint8_t *host_copy(void) { return U.arena->cpu ? U.arena->cpu : U.arena->map; }
|
||||
@@ -344,7 +356,7 @@ const uint8_t *vk_usnap_map(void) { return U.arena ? host_copy() : NULL; }
|
||||
|
||||
const uint8_t *vk_usnap_bytes(uint64_t addr, uint64_t n) {
|
||||
VkDescriptorBufferInfo bi;
|
||||
return vk_usnap_find(addr, n, 1, &bi) ? host_copy() + bi.offset : NULL;
|
||||
return vk_usnap_find(addr, n, UINT32_MAX, 1, &bi) ? host_copy() + bi.offset : NULL;
|
||||
}
|
||||
|
||||
// The arenas back on the free list (vk_exec_draw_shutdown: the device is idle, every completion ran).
|
||||
|
||||
@@ -38,6 +38,11 @@
|
||||
# Replay NVN submissions on their own thread, at most one frame behind the game, 0 or 1. Default: off. Helps where the
|
||||
# game waits for the GPU; adds up to a frame of latency.
|
||||
# replay_async = 0
|
||||
# Record each large submission in chunks on this many threads (0 to 16; 0 = one thread, the default). The chunks go to
|
||||
# the GPU in order. In the town at 4K on an RTX 5070, 4 cut the replay from 8.2 to 4.5 ms a frame (71 to 79 fps); more
|
||||
# threads add little. Needs 8 or more logical processors to help. Contact shadows run in chunks too; DLSS (dlaa = 1) and
|
||||
# fxaa = 0 keep the replay on one thread. Read at start.
|
||||
# replay_workers = 0
|
||||
# EXPERIMENTAL, THE GAME CRASHES WITH 1. 1 lifts the virtual core gate: guest threads run without the console's limit
|
||||
# of one thread per core, and the game keeps per-core data without locks. With 1, 1.2.1 and 1.4.3 crash or hang,
|
||||
# usually at start (#212). Keep 0 to play.
|
||||
|
||||
Reference in new issue
Block a user