From 380e6ff30efcfdb95dfc560b51d30b66068b95e6 Mon Sep 17 00:00:00 2001 From: Valiantsin Dzerakh Date: Fri, 20 Nov 2026 14:30:51 +0300 Subject: [PATCH] Frame generation gets the per-object motion vectors DLSS uses, so Link running no longer shows a soft second outline and a trail in the generated pictures The motion image FG read held the camera's motion alone (fg_motion called mv_pass without its object pass). The camera follows Link, so his pixels carried the world's motion and the interpolation drew him twice, with a smeared strip where he had been; NPCs and enemies did the same. fg_motion now runs mv_pass with the object pass (vk_mvobj_execute), and its gates follow vk_objmv_wanted: DLSS once NGX came up (vk_dlaa_available, as before) or FG following the scene with its objects (vk_fg_objects_on). vk_mvprev_frame keeps last frame's block tables for either, vk_usnap_begin makes the host copy for either and vk_usnap_bytes reads the copy that submission wrote (FG turns on and off live), and vk_dlaa_prev_camera answers whenever the scene is followed. FG is off with DLSS, so every DLSS gate evaluates as before. fg_objects = 0 keeps the camera's motion alone for comparisons. Measured on A (RX 9070, 7900X on 4 cores, 1080p, field, 60 s): 85.6 drawn with fg_objects = 0, 80.0 with 1 (about 0.8 ms a drawn frame on the replay thread, half of it the snapshot's host copy), object pass 0.035 ms GPU. Still grass gets no false motion (SDK debug view: the game vector field stays within 0.2 px), and grass does not break up while the camera turns. Validation: 0 errors. Co-Authored-By: Claude Opus 5.5 --- docs/settings.md | 3 ++- runtime/cfg.c | 2 +- runtime/vk_dlaa.c | 36 ++++++++++++++++++++++++++---------- runtime/vk_dlaa.h | 4 ++++ runtime/vk_exec_draw.c | 2 +- runtime/vk_mvprev.c | 15 +++++++++------ runtime/vk_usnap.c | 15 +++++++++------ 7 files changed, 52 insertions(+), 25 deletions(-) diff --git a/docs/settings.md b/docs/settings.md index c9744ed7..7ab6ac2a 100644 --- a/docs/settings.md +++ b/docs/settings.md @@ -143,9 +143,10 @@ Many rows below are developer tools that only act together with another setting | `dyn_delta` (default `1`) | Records only the pipeline dynamic states (cull mode, depth and stencil tests, vertex input, blend enables, equations and write masks, ...) whose values changed since the last draw in the command buffer, also across pipeline changes; `0` records the full set at every change of pipeline state, as before; `ab` alternates with the `ab` blocks. | Fewer commands per pipeline change on the render and recorder threads; a driver that loses dynamic state across pipeline binds would draw with stale state. `TOTK_TRACE=perf` prints the calls and commands per submission. | Caution | | `feedback` | `0` samples a target that an earlier draw of the same pass wrote with no barrier; `restart` ends and restarts the pass behind a full barrier; otherwise the feedback-loop layout is used when the GPU supports it. | `0` brings back the old behaviour that the barrier fixed, so the picture can break in passes that read a target they draw into; `restart` costs extra passes. | Risky | | `flush_span` (default `0`) | `1` copies one span from the lowest to the highest flushed address of a pool, as an older version did. | Copies far more data to the GPU in crowded towns: an estimated 30-50 ms of GPU time per large copy over PCIe 3.0 x8. | Caution | -| `frame_gen` (default `off`) | The frame generator: `off`, or `fsr3`, which brings up AMD FSR 3 frame interpolation and optical flow (FidelityFX SDK 1.1.4) at start; with DLSS off the replay follows the scene and records the camera motion vectors, the picture before the HUD and the interpolation of a frame between each two the game draws (`runtime/vk_fg.cpp`). Each frame then goes out as two presents: the generated picture once the GPU has drawn the frame (a timeline wait, not before the pacer's time), the game's half the smoothed interval later (`runtime/vk_backend.c` `present_pair`); `fps_limit` counts the pictures shown and the game draws half, 30 at least (`runtime/frame_rate.c` `frame_step_cap_ns`). Off with DLSS, `replay_workers` above 0, `sync_ahead = 1` or HDR output, and while a menu background or a loading or event fade is open; the game then draws at the whole limit. The swapchain gets one image more. Applies at once: turned on in the menu, the contexts are made at the next frame (a hitch of about 0.1 s once); turned off, they stay until exit. | The controls answer at the drawn rate, about half a frame later. Costs about 1.5 ms of GPU and 0.6 ms of the replay thread per drawn frame at 1080p (RX 9070, 7900X on 4 cores: 94 drawn without it, 86 drawn and 169 shown with it, no limit). For displays of 120 Hz and more. | Caution | +| `frame_gen` (default `off`) | The frame generator: `off`, or `fsr3`, which brings up AMD FSR 3 frame interpolation and optical flow (FidelityFX SDK 1.1.4) at start; with DLSS off the replay follows the scene and records the motion vectors (the camera's and the moving objects', `fg_objects`), the picture before the HUD and the interpolation of a frame between each two the game draws (`runtime/vk_fg.cpp`). Each frame then goes out as two presents: the generated picture once the GPU has drawn the frame (a timeline wait, not before the pacer's time), the game's half the smoothed interval later (`runtime/vk_backend.c` `present_pair`); `fps_limit` counts the pictures shown and the game draws half, 30 at least (`runtime/frame_rate.c` `frame_step_cap_ns`). Off with DLSS, `replay_workers` above 0, `sync_ahead = 1` or HDR output, and while a menu background or a loading or event fade is open; the game then draws at the whole limit. The swapchain gets one image more. Applies at once: turned on in the menu, the contexts are made at the next frame (a hitch of about 0.1 s once); turned off, they stay until exit. | The controls answer at the drawn rate, about half a frame later. Costs about 1.5 ms of GPU and 0.6 ms of the replay thread per drawn frame at 1080p (RX 9070, 7900X on 4 cores: 94 drawn without it, 86 drawn and 169 shown with it, no limit). For displays of 120 Hz and more. | Caution | | `fg_test` | Frame generation checks: `1` presents the generated frame in place of the game's, `2` the picture before the HUD. Read at start. | Shows the frames out of their order: for checks only. | Caution | | `fg_debug` | `1`: the generated frames hold the SDK's debug view (motion fields, optical flow, disocclusion). Read at start. | For checks only. | Caution | +| `fg_objects` (default `1`) | Frame generation's motion vectors get the per-object pass DLSS uses over the camera's: the moving scene draws (characters, enemies, objects) are drawn again with last frame's blocks (`runtime/vk_exec_draw.c` `vk_mvobj_execute`, `runtime/vk_mvprev.c`). `0`: the camera's motion alone; Link, whom the camera follows, then carries the world's motion and the generated pictures show him twice. Read at start. `dlaa_objmv = 0` also turns the object pass off. | About 0.04 ms of GPU and 0.8 ms of the replay thread per drawn frame (RX 9070, 7900X on 4 cores, field, 1080p: 85.6 drawn with `0`, 80.0 with `1`), half of it the host copy of the uniform snapshot the object pass reads. | Safe | | `fg_shot` | Frame generation shots: a present number, or `w` of world time; the window texture, the generated frame and the picture before the HUD of three frames from there go to `fg_shot_dir` as `fg__.png`. | Writes PNGs. | Safe | | `fg_shot_dir` | Folder for the `fg_shot` PNGs (default the working folder). | Only moves the output. | Safe | | `fg_times` (default `1`) | `0` turns off the GPU timestamps of frame generation's steps; every 600 generated frames a log line gives their GPU times and the replay thread's recording time (median, p99). | Timestamps cost next to nothing. | Safe | diff --git a/runtime/cfg.c b/runtime/cfg.c index 7a4740c0..3dc8ea10 100644 --- a/runtime/cfg.c +++ b/runtime/cfg.c @@ -42,7 +42,7 @@ static const char *const known[] = { "BIND_ZL", "BIND_ZR", "BARRIER_MERGE", "BLOCK_VIEWS_3D", "CLIMB_FIX", "CLOTH_REALTIME", "COLOR_BRIGHTNESS", "COLOR_CONTRAST", "COLOR_GAMMA", "COLOR_PRESET", "COLOR_SATURATION", "CONTACT_DEBUG", "CONTACT_SHADOWS", "CORE_DIAG", "CRIT", "CRIT_OUT", "DESC_TEMPLATE", "DESWIZZLE", "DEVICE_PROFILE", "DIALOGUE_CLICK", "DISCORD_APP_ID", "DISCORD_IMAGE", "DISCORD_RPC", "DLAA", "DLAA_CAM_JITTER", "DLAA_EFFECTS_AFTER", "DLAA_JITTER", "DLAA_MV", "DLAA_MASKDEBUG", "DLAA_MVDEBUG", "DLAA_OBJMV", "DLAA_SURVEY", "DLAA_TIMES", "DLSS", "DLSS_CHAR_MASK", "DLSS_FIXED_EXPOSURE", "DLSS_PRESET", "DLSS_REACTIVE", "DUMP", "DUMP_BUF", "DUMP_DIR", - "DUMP_LOG", "DYNAMIC_RES", "DYN_DELTA", "EXEFS", "FEEDBACK", "FG_DEBUG", "FG_SHOT", "FG_SHOT_DIR", "FG_TEST", "FG_TIMES", "FIXED_RES_PATCH", "FIX_C0", "FIX_ZERO", "FLIGHT", "FLIGHT_AT", "FLIGHT_DIR", "FLIGHT_FRAMES", + "DUMP_LOG", "DYNAMIC_RES", "DYN_DELTA", "EXEFS", "FEEDBACK", "FG_DEBUG", "FG_OBJECTS", "FG_SHOT", "FG_SHOT_DIR", "FG_TEST", "FG_TIMES", "FIXED_RES_PATCH", "FIX_C0", "FIX_ZERO", "FLIGHT", "FLIGHT_AT", "FLIGHT_DIR", "FLIGHT_FRAMES", "FLIGHT_SPIKE_MS", "FLIGHT_THUMBS", "FLUSH_SPAN", "FOV", "FOV_PATCH", "FPS60", "FPS_LIMIT", "FRAME_GEN", "FRAME_QUEUE", "FREECAM_KEY", "FREECAM_PAD", "FREECAM_PATCH", "FREEZE_IDLE", "FREE_CORES", "FULLSCREEN", "FXAA", diff --git a/runtime/vk_dlaa.c b/runtime/vk_dlaa.c index 50ea7274..8ef57211 100644 --- a/runtime/vk_dlaa.c +++ b/runtime/vk_dlaa.c @@ -312,7 +312,7 @@ static struct { // generation ran (vk_dlaa_sub_end) VkExecTarget hud_t, win; // the output-sized target the scene is scaled into; the window texture float cam_dr[2]; // the viewport depth range of the draw cam_addr was noted at - int fg_mv_ok; // the frame's camera motion is in the motion image (fg_motion) + int fg_mv_ok; // the frame's motion is in the motion image (fg_motion) } D = {.mv_ok = -1, .cfg_seen = ~0u}; #define DLSS_SIZES (sizeof D.sizes / sizeof D.sizes[0]) @@ -324,6 +324,23 @@ static int dl_active(void) { return N.available && !D.failed && D.on; } static int track(void) { return dl_active() || D.fg; } int vk_dlaa_fg_on; // D.fg for vk_exec_draw.c's per-draw test (vk_dlaa_hud) +// TOTK_FG_OBJECTS (default 1, read once): frame generation's motion image gets the per-object pass over the camera's, +// as DLSS's does. The camera follows Link, so with the camera's motion alone his pixels carry the world's motion and +// the interpolation drew him twice (a soft second outline) and left a trail where he had been. 0 keeps the camera's +// motion alone (stage 2's vectors), for comparisons. +static int fg_objects(void) { + static int on = -1; + if (on < 0) on = rt_knob("FG_OBJECTS", "1"); + return on; +} + +int vk_fg_objects_on(void) { return D.fg && fg_objects(); } + +// The per-object motion's gates (vk_mvprev.c's tables, vk_usnap.c's host copy, vk_exec_draw.c's lock): DLSS once NGX +// came up with it, exactly as vk_dlaa_available, or frame generation following the scene with its objects. Frame +// generation is off with DLSS, so the second term is 0 whenever DLSS runs. +int vk_objmv_wanted(void) { return vk_dlaa_available() || vk_fg_objects_on(); } + static void dl_fail(const char *what, NgxResult r) { rt_log("dlaa: %s failed (%08x): DLAA off for this run", what, (unsigned)r); D.failed = 1; @@ -597,8 +614,8 @@ static int inv4(const double m[16], double o[16]) { // The pass, recorded through the recorder like the replay's own commands. 0: no camera this frame (the vectors stay // zero and DLSS starts a new history). objects: the per-object pass and the TOTK_MV_CHECK hooks after the camera's -// (DLSS only). bias > 0 with this frame's G-buffer classes (D.ids) also writes e->mask, the -// bias mask (dlaa_mv.comp); *masked says whether it did. +// (DLSS, and frame generation unless TOTK_FG_OBJECTS=0). bias > 0 with this frame's G-buffer classes (D.ids) also +// writes e->mask, the bias mask (dlaa_mv.comp); *masked says whether it did. static int mv_pass(VkExec *ex, struct Size *e, VkImageView zv, float jx, float jy, float bias, int *masked, int objects) { *masked = 0; VkCommandBuffer cb = ex->cb; @@ -713,7 +730,7 @@ static int mv_pass(VkExec *ex, struct Size *e, VkImageView zv, float jx, float j // Last frame's camera block for the per-object motion pass, recorded before this frame's mv_pass: that pass writes // this frame's into slot cam_slot and leaves the other slot, last frame's, as it is. 0: none (no history). int vk_dlaa_prev_camera(VkDescriptorBufferInfo *out, float jitter[2]) { - if (!dl_active() || !D.cam_prev_ok || !D.camb) return 0; + if (!track() || !D.cam_prev_ok || !D.camb) return 0; // DLSS, or frame generation (no jitter: D.jit_* stay 0) *out = (VkDescriptorBufferInfo){D.camb, (VkDeviceSize)(D.cam_slot ^ 1) * CAM_SLOT, CAM_BLOCK}; if (cam_mode()) jitter[0] = D.jit_used ? D.ux : 0.0f, jitter[1] = D.jit_used ? D.uy : 0.0f; else jitter[0] = D.jit_ok ? D.jx : 0.0f, jitter[1] = D.jit_ok ? D.jy : 0.0f; @@ -1570,11 +1587,10 @@ int vk_dlaa_plan(const VkExec *ex, VkExecJob *j) { // ---------------------------------------------------------------- frame generation (vk_fg.cpp), DLSS off // The scene is followed as for DLSS (vk_dlaa_plan: armed on the light target, ended at the bloom chain's first pass), -// without the jitter, the per-object motion, the reactive mask or NGX. Three points of the frame, in the sequential -// replay: -// 1. The scene's end (fg_motion, the VK_JOB_DLSS point): the camera motion vectors at the scene's size (mv_pass with -// the camera alone; Savage Wind's fg_mv.frag does the same reprojection, as dlaa_mv.comp already did here) and the -// SDK's prepare, which reads them with the scene depth while that still holds the scene. +// without the jitter, the reactive mask or NGX. Three points of the frame, in the sequential replay: +// 1. The scene's end (fg_motion, the VK_JOB_DLSS point): the motion vectors at the scene's size (mv_pass: the camera's +// reprojection, as Savage Wind's fg_mv.frag does, then the per-object pass over it unless TOTK_FG_OBJECTS=0) and +// the SDK's prepare, which reads them with the scene depth while that still holds the scene. // 2. The picture before the HUD (vk_dlaa_hud, before a draw): measured in a frame dump of 1.4.3 (PR #1089, a 1920x1080 // scene and output), after the bloom chain the game draws its bloom composite and FXAA into the light target with // the scene depth attached; then the HUD, in passes on the light target alone without depth, with the HUD's own @@ -1618,7 +1634,7 @@ static void fg_motion(VkExec *ex) { const float *cam = D.cam_addr ? (const float *)vk_usnap_bytes(D.cam_addr, 11 * 16) : NULL; int cut = cam ? vk_fg_camera(cam, nvn_depth_minus_w(), D.cam_dr, &s) : 1; vk_fg_mark(cb, FG_T_MV0); - int masked = 0, have = cam && mv_pass(ex, e, zv, 0.0f, 0.0f, 0.0f, &masked, 0); + int masked = 0, have = cam && mv_pass(ex, e, zv, 0.0f, 0.0f, 0.0f, &masked, fg_objects()); if (!have) { // no camera or no last frame: zero motion, and the SDK starts over VkClearColorValue zero = {{0, 0, 0, 0}}; VkImageSubresourceRange all = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1}; diff --git a/runtime/vk_dlaa.h b/runtime/vk_dlaa.h index c9cd320c..b6c7eb95 100644 --- a/runtime/vk_dlaa.h +++ b/runtime/vk_dlaa.h @@ -14,6 +14,10 @@ void vk_dlaa_device_exts(const VkExtensionProperties *have, uint32_t nhave, cons void vk_dlaa_init(void); // 1 when dlaa is on and NGX came up with DLSS (after vk_dlaa_init): a DLSS mode's smaller scene is scaled up by DLSS. int vk_dlaa_available(void); +// 1 when the per-object motion runs this frame: vk_dlaa_available, or frame generation following the scene with its +// objects (vk_fg_objects_on: TOTK_FG_OBJECTS, default 1). +int vk_objmv_wanted(void); +int vk_fg_objects_on(void); // vk_exec.c vk_exec_pass_begin, once the pass's targets are resolved and before its barrier: arms the frame on the // scene's light target or, at the first quarter-width pass after it, runs DLAA on that target. Each present ends the // frame (vk_dlaa_frame). diff --git a/runtime/vk_exec_draw.c b/runtime/vk_exec_draw.c index 42a2ff86..d65e9be5 100644 --- a/runtime/vk_exec_draw.c +++ b/runtime/vk_exec_draw.c @@ -3527,7 +3527,7 @@ static int prepare_draw(VkExec *ex, int prim, int32_t count, int restart) { } D->B.st_pm = D->PM.valid; // st is this draw's state, which PM holds while valid vk_dlaa_draw(ex); - int mo = ex->chunked && vk_dlaa_available(); + int mo = ex->chunked && vk_objmv_wanted(); if (mo) plat_mutex_lock(&mo_lock); vk_mvsurvey_draw(ex); vk_mvprev_draw(ex); diff --git a/runtime/vk_mvprev.c b/runtime/vk_mvprev.c index 386c48c9..86d43b42 100644 --- a/runtime/vk_mvprev.c +++ b/runtime/vk_mvprev.c @@ -1,6 +1,6 @@ -// Last frame's object and bone blocks, for per-object motion vectors (TOTK_DLAA_OBJMV, on with DLAA). The banks kept -// start as c5, c7 and c14 and grow with what the vertex shader variants read (vk_mvprev_want); c7 and c5 are described -// here, the others follow the same two-copy rule. +// Last frame's object and bone blocks, for per-object motion vectors (TOTK_DLAA_OBJMV, on with DLAA, and with frame +// generation unless TOTK_FG_OBJECTS=0). The banks kept start as c5, c7 and c14 and grow with what the vertex shader +// variants read (vk_mvprev_want); c7 and c5 are described here, the others follow the same two-copy rule. // // The scene's vertex shaders read an object's place from bank c7 (gsys_shape: one 3x4 matrix at the start of a // 0x100 block) and a skinned object's bones from c5 (gsys_skeleton: a run of 3x4 matrices), of this frame only. The @@ -172,8 +172,9 @@ static void mvprev_draw_(VkExec *ex) { } void vk_mvprev_frame(void) { - // without DLSS (no NGX, no RTX GPU, dlaa = 0) nothing reads last frame's blocks: P.on stays 0, no tables - if (!vk_dlaa_available()) return; + // without DLSS (no NGX, no RTX GPU, dlaa = 0) and without frame generation's object motion nothing reads last + // frame's blocks: P.on stays 0, no tables + if (!vk_objmv_wanted()) return; if (P.knob < 0) { P.knob = rt_knob("DLAA_OBJMV", "1"); if (P.knob) { @@ -186,7 +187,9 @@ void vk_mvprev_frame(void) { P.timed = rt_knob("MVOBJ_TIMES", "0"); } } - int on = P.knob && cfg_bool("TOTK_DLAA"); + // DLSS: dlaa on (as before frame generation came); frame generation: while it follows the scene with its objects + // (it is off with DLSS) + int on = P.knob && ((vk_dlaa_available() && cfg_bool("TOTK_DLAA")) || vk_fg_objects_on()); if (on != (P.on > 0)) { P.on = on ? 1 : -1; if (P.t[0]) memset(P.t[0], 0, TABLE * sizeof(Seen)), memset(P.t[1], 0, TABLE * sizeof(Seen)); diff --git a/runtime/vk_usnap.c b/runtime/vk_usnap.c index 43356141..fffa84b1 100644 --- a/runtime/vk_usnap.c +++ b/runtime/vk_usnap.c @@ -37,9 +37,10 @@ typedef struct Arena { VkBuffer buf; VmaAllocation alloc; uint8_t *map; - uint8_t *cpu; // with DLSS (vk_dlaa_available), the same bytes in host memory for the CPU's reads (vk_usnap_bytes): - // map is often device memory through the BAR, written fast but read back at about a microsecond a - // cache line. Without DLSS nothing reads them back, and the bytes go into map in one copy. + uint8_t *cpu; // with the per-object motion (vk_objmv_wanted: DLSS, or frame generation with its objects), + // the same bytes in host memory for the CPU's reads (vk_usnap_bytes): map is often device memory + // through the BAR, written fast but read back at about a microsecond a cache line. Without it + // nothing reads them back in bulk, and the bytes go into map in one copy. uint64_t size; } Arena; @@ -52,6 +53,7 @@ static struct { int on; // this submission has a snapshot uint64_t fence, gen; Arena *arena; + uint8_t *host; // the arena's copy the CPU reads: cpu or map (vk_usnap_begin) Rng *col; // collected ranges int ncol, ccol; Reg *reg; // merged, sorted by lo, disjoint @@ -289,10 +291,11 @@ void vk_usnap_begin(const uint64_t *handles, int n, uint64_t fence) { if (!U.nreg) return; if (!(U.arena = arena_get(total))) return; uint8_t *dst = U.arena->map; - if (vk_dlaa_available()) { + if (vk_objmv_wanted()) { if (!U.arena->cpu && !(U.arena->cpu = malloc(U.arena->size))) rt_fatal("vk_usnap: out of memory"); dst = U.arena->cpu; } + U.host = dst; // frame generation turns on and off live: an arena's old host copy may be stale for (int i = 0; i < U.nreg; i++) memcpy(dst + U.reg[i].off, (const void *)(uintptr_t)U.reg[i].lo, U.reg[i].hi - U.reg[i].lo); if (dst != U.arena->map) memcpy(U.arena->map, dst, total); // one sequential write into the device copy @@ -350,8 +353,8 @@ void vk_usnap_drop_at(uint64_t addr, uint64_t n, uint32_t at) { drop(addr, n, at uint64_t vk_usnap_gen(void) { return __atomic_load_n(&U.gen, __ATOMIC_RELAXED); } -// the host copy where there is one (DLSS), else the device copy -static uint8_t *host_copy(void) { return U.arena->cpu ? U.arena->cpu : U.arena->map; } +// the copy vk_usnap_begin wrote this time: the host copy with the per-object motion, else the device copy +static uint8_t *host_copy(void) { return U.host; } const uint8_t *vk_usnap_map(void) { return U.arena ? host_copy() : NULL; }