diff --git a/docs/agents/symptoms.md b/docs/agents/symptoms.md index 7c9f0bbb..c4769ad0 100644 --- a/docs/agents/symptoms.md +++ b/docs/agents/symptoms.md @@ -28,6 +28,9 @@ GPU semantics behind the graphics rows: [maxwell.md](../maxwell.md). The method | one flat ambient colour | a 1-layer 2D array sampled as 2D | the view type the shader expects | | upside-down text | lower-left window origin | origin mode in the log | | puddles and still water flicker: the reflection turns on and off in patches from frame to frame while the camera stands still, more with DLAA | the game's water volume mask: a volume's top face lies on the water surface and is tested LEQUAL against a depth rebuilt at quarter size, so rounding and the camera's sub-millimetre motion decide it (fixed: `vk_pipeline.c` `vk_pipeline_volume_bias` gives such draws a depth bias toward the viewer) | a `--capture` run: the puddle's brightness swing (`capture.py` frames, a region's mean) with `TOTK_VOLUME_MASK_BIAS=0` and without; in a frame dump, the quarter-size mask after the volume draw changes from frame to frame | +| with DLAA on, water seen at a low angle (ponds, streams) flickers in large patches with a period of about three frames, between the reflection of the trees and that of the sky; without DLAA it is still | the jitter moved the viewport of the scene's draws but not the game's camera, so its full-screen passes rebuilt positions half a pixel off the surface and its screen-space reflections flipped between hit and miss with the jitter's phase (fixed on 1.4.3: `vk_dlaa.c` puts the jitter into the camera blocks where the game copies its matrices, `vk_dlaa_camera_copied`) | `TOTK_DLAA_CAM_JITTER=0` brings the viewport jitter back; a `--capture` run: the region's brightness swing with DLAA, against `TOTK_DLAA_JITTER=0` | +| with DLAA on, the world behind the inventory smears into horizontal streaks on the right half, from the second time the inventory is opened (when Link's model is drawn there) | the camera jitter's read-back took the inventory camera's own off-centre projection (P[0][2] about -0.6) for a jitter and gave DLSS an offset of about 1152 pixels (fixed: `vk_dlaa.c` `cam_offset` takes only an offset it gave that block) | `TOTK_DLAA_CAM_JITTER=0` makes it go; a `--capture` run over the opened inventory: the background's horizontal gradient against its vertical one | +| with DLAA/DLSS on, large distant blocks behind the fog (the Depths) turn darker or lighter for a frame while the camera moves | DLSS created with AutoExposure estimated the exposure of the dark scene itself, and the estimate moved from frame to frame (fixed with `dlss_fixed_exposure = 1`: `vk_dlaa.c` evaluate creates the feature without it) | a flat fogged patch in `capture.py` frames: one value without DLAA, jumps with it | | flicker or a texture swapped with a similar one | the newest texture picked at shared storage instead of the one with the matching view | texture lookup by id in the dump | | with DLAA on, dust and small particles in fog (shrines) vanish, smear or behave oddly; the fog looks processed | DLSS ran at the bloom, after the fog and the particles, which have no motion vectors or jitter of their own (fixed: `vk_dlaa.c` effects_point runs DLSS at the first pass into the light target after the lighting, and the effects go over its picture) | `dlaa_effects_after = 0` (Screen tab) brings the old point back; the count "before the effects" in the `dlaa:` log line against the frames anti-aliased | | with DLAA on, the picture flickers from frame to frame: a little outdoors; the whole scene in a fogged shrine with an ability's targeting view (Ultrahand) open, where the fog also goes back under DLSS | DLSS ran before some effects in one frame and after them in the next: their order changes from frame to frame (the yard's half-size effects pass is in some frames only; the ability highlight, drawn into the light target with a D16 depth of its own, comes before or after the fog layer), and a rule that waited for one of them or took any D16 pass for the lighting followed it (fixed: `vk_dlaa.c` effects_point knows the lighting by its depth and runs DLSS right after it) | the `dlaa:` log line: "before the effects" must equal the frames anti-aliased in the world; a log of the passes into the light target per frame | diff --git a/docs/settings.md b/docs/settings.md index 6f083969..23983cf4 100644 --- a/docs/settings.md +++ b/docs/settings.md @@ -175,6 +175,7 @@ Many rows below are developer tools that only act together with another setting | Setting | What it does | Effect of changing it | Safety | | --- | --- | --- | --- | +| `dlaa_cam_jitter` (default `1`) | Game 1.4.3: `1` puts the jitter fed to DLSS into the game's camera (the projection's centre terms and the view-projection, where the game copies them into its camera blocks); `0` moves the viewport of the scene's draws instead, as on the other game versions. Needs `ubo_snapshot`. | With `0`, water seen at a low angle flickers with DLAA: the game's full-screen passes rebuild positions with a camera the scene was not drawn with; a test setting. | Caution | | `dlaa_jitter` (default `1`) | `0` turns off the camera jitter fed to DLSS. | DLSS gets no jitter between frames; a test setting. | Caution | | `dlaa_maskdebug` (default `0`) | `1` shows the reactive mask in place of the picture. | The frame shows a debug image while it is set. | Caution | | `dlaa_mv` (default `1`) | `0` sends zero motion vectors to DLSS. | DLSS sees no motion, so moving content can smear; a test setting. | Caution | diff --git a/runtime/cfg.c b/runtime/cfg.c index 3b0464e1..90951dfb 100644 --- a/runtime/cfg.c +++ b/runtime/cfg.c @@ -40,7 +40,7 @@ static const char *const known[] = { "BIND_INC", "BIND_SKIP", "BIND_STATS", "BIND_UP", "BIND_X", "BIND_Y", "BIND_ZL", "BIND_ZR", "BARRIER_MERGE", "BLOCK_VIEWS_3D", "CLIMB_FIX", "CLOTH_REALTIME", "COLOR_BRIGHTNESS", "COLOR_CONTRAST", "COLOR_GAMMA", "COLOR_PRESET", "COLOR_SATURATION", "CONTACT_DEBUG", "CONTACT_SHADOWS", "CORE_DIAG", "CRIT", "CRIT_OUT", - "DESC_TEMPLATE", "DESWIZZLE", "DEVICE_PROFILE", "DIALOGUE_CLICK", "DISCORD_APP_ID", "DISCORD_IMAGE", "DISCORD_RPC", "DLAA", "DLAA_EFFECTS_AFTER", "DLAA_JITTER", "DLAA_MV", "DLAA_MASKDEBUG", "DLAA_MVDEBUG", "DLAA_OBJMV", "DLAA_SURVEY", "DLAA_TIMES", "DLSS", "DLSS_CHAR_MASK", "DLSS_PRESET", "DLSS_REACTIVE", "DUMP", "DUMP_BUF", "DUMP_DIR", + "DESC_TEMPLATE", "DESWIZZLE", "DEVICE_PROFILE", "DIALOGUE_CLICK", "DISCORD_APP_ID", "DISCORD_IMAGE", "DISCORD_RPC", "DLAA", "DLAA_CAM_JITTER", "DLAA_EFFECTS_AFTER", "DLAA_JITTER", "DLAA_MV", "DLAA_MASKDEBUG", "DLAA_MVDEBUG", "DLAA_OBJMV", "DLAA_SURVEY", "DLAA_TIMES", "DLSS", "DLSS_CHAR_MASK", "DLSS_FIXED_EXPOSURE", "DLSS_PRESET", "DLSS_REACTIVE", "DUMP", "DUMP_BUF", "DUMP_DIR", "DUMP_LOG", "DYNAMIC_RES", "DYN_DELTA", "EXEFS", "FEEDBACK", "FIXED_RES_PATCH", "FIX_C0", "FIX_ZERO", "FLIGHT", "FLIGHT_AT", "FLIGHT_DIR", "FLIGHT_FRAMES", "FLIGHT_SPIKE_MS", "FLIGHT_THUMBS", "FLUSH_SPAN", "FOV", "FOV_PATCH", "FPS60", "FPS_LIMIT", "FRAME_QUEUE", "FREEZE_IDLE", "FREE_CORES", @@ -84,7 +84,7 @@ static const struct { const char *key, *section; } sections[] = { {"COLOR_BRIGHTNESS", "display"}, {"COLOR_CONTRAST", "display"}, {"COLOR_SATURATION", "display"}, {"COLOR_GAMMA", "display"}, {"HDR", "display"}, {"HDR_PEAK_NITS", "display"}, {"HDR_GAME_NITS", "display"}, {"HDR_UI_NITS", "display"}, {"HDR_HUE", "display"}, {"SHADOW_RES", "display"}, - {"DLAA", "display"}, {"DLSS", "display"}, {"DLSS_PRESET", "display"}, {"DLSS_CHAR_MASK", "display"}, {"DLAA_EFFECTS_AFTER", "display"}, {"REFLEX", "display"}, {"FXAA", "display"}, + {"DLAA", "display"}, {"DLSS", "display"}, {"DLSS_PRESET", "display"}, {"DLSS_CHAR_MASK", "display"}, {"DLSS_FIXED_EXPOSURE", "display"}, {"DLAA_EFFECTS_AFTER", "display"}, {"REFLEX", "display"}, {"FXAA", "display"}, {"DEVICE_PROFILE", "display"}, {"ASSUME_FOCUS", "display"}, {"DISCORD_RPC", "discord"}, {"DISCORD_APP_ID", "discord"}, {"DISCORD_IMAGE", "discord"}, {"SHADOW_SCALE", "display"}, {"SHADOW_RANGE", "display"}, {"GRASS_RANGE", "display"}, {"CONTACT_SHADOWS", "display"}, @@ -141,7 +141,7 @@ static const CfgMeta metas[] = { {"DYNAMIC_RES", "1", "0|1", LIVE}, // resolution.c res_dynres_live {"UPSCALER", "fsr", "fsr|bilinear", LIVE}, {"SHARPNESS", "80", "0..100", LIVE}, {"DLAA", "0", "0|1", RESTART}, {"DLSS", "dlaa", "dlaa|quality|balanced|performance|ultra_performance", LIVE}, - {"DLSS_PRESET", "default", "default|k|j", LIVE}, {"DLAA_EFFECTS_AFTER", "0", "0|1", LIVE}, + {"DLSS_PRESET", "default", "default|k|j", LIVE}, {"DLSS_FIXED_EXPOSURE", "0", "0|1", LIVE}, {"DLAA_EFFECTS_AFTER", "0", "0|1", LIVE}, {"DLSS_CHAR_MASK", "65", "0|35|65|100", LIVE}, {"REFLEX", "off", "off|on|boost", RESTART}, {"FXAA", "1", "0|1", LIVE}, {"LOD_STEADY", "0", "0|1", RESTART}, // sets steady {"ASPECT", "fit", "fit|stretch", LIVE}, {"MOVIE_ASPECT", "keep", "keep|stretch", LIVE}, {"FOV", "100", "70..140", LIVE}, diff --git a/runtime/nvn_api_core.c b/runtime/nvn_api_core.c index b1f92808..e61df50d 100644 --- a/runtime/nvn_api_core.c +++ b/runtime/nvn_api_core.c @@ -53,13 +53,19 @@ void nvn_guest_written(uint64_t addr, uint64_t size) { if (size) gfx_backend()->guest_written(addr, size); } +// The camera jitter of 1.4.3 (vk_dlaa.c); a weak default like the hooks below. +void vk_dlaa_camera_copied(uint64_t lr, uint64_t dst, uint64_t off); +__attribute__((weak)) void vk_dlaa_camera_copied(uint64_t lr, uint64_t dst, uint64_t off) { (void)lr, (void)dst, (void)off; } + // memcpy of pool storage the GPU rendered into must copy what the GPU wrote: TotK moves its generated grass maps // between pools with memcpy. Copies below 1 KiB skip the check; the copy itself is memmove, as overlapping -// arguments are undefined for memcpy anyway. +// arguments are undefined for memcpy anyway. TotK 1.4.3 copies its camera matrices with 64-byte memcpys +// (f_70000e31d4); x8 holds the member's offset in the block there (vk_dlaa.c vk_dlaa_camera_copied). static void ov_memcpy(Cpu *c) { uint64_t dst = arg_x(c, 0), src = arg_x(c, 1), n = arg_x(c, 2); if (n >= 1024) gfx_backend()->guest_read(src, n); memmove((void *)(uintptr_t)dst, (const void *)(uintptr_t)src, n); + if (n == 64) vk_dlaa_camera_copied(c->x[30], dst, c->x[8]); ret_x(c, dst); } HOST_OVERRIDE("memcpy", ov_memcpy); diff --git a/runtime/ui/menu.cpp b/runtime/ui/menu.cpp index b4f23c2f..fcab4708 100644 --- a/runtime/ui/menu.cpp +++ b/runtime/ui/menu.cpp @@ -750,6 +750,7 @@ static void tab_screen(float scale) { static const UiTextId chars[] = {UI_MENU_DLSS_CHARS_OFF, UI_MENU_DLSS_CHARS_LIGHT, UI_MENU_DLSS_CHARS_MEDIUM, UI_MENU_DLSS_CHARS_FULL}; opt_choice("TOTK_DLSS_CHAR_MASK", UI_MENU_DLSS_CHARS, chars, 4, UI_MENU_DLSS_CHARS_DESC); + opt_toggle("TOTK_DLSS_FIXED_EXPOSURE", UI_MENU_DLSS_FIXED_EXPOSURE, UI_MENU_DLSS_FIXED_EXPOSURE_DESC); opt_toggle("TOTK_DLAA_EFFECTS_AFTER", UI_MENU_DLAA_EFFECTS_AFTER, UI_MENU_DLAA_EFFECTS_AFTER_DESC); static const UiTextId reflex[] = {UI_MENU_REFLEX_OFF, UI_MENU_REFLEX_ON, UI_MENU_REFLEX_BOOST}; opt_choice("TOTK_REFLEX", UI_MENU_REFLEX, reflex, 3, UI_MENU_REFLEX_DESC); diff --git a/runtime/ui_text.c b/runtime/ui_text.c index 3933f8fb..ad05e2ab 100644 --- a/runtime/ui_text.c +++ b/runtime/ui_text.c @@ -522,6 +522,11 @@ static const char *const text_en[UI_TEXT_COUNT] = { "fog (shrines) no longer vanish or leave trails. Off: DLSS also processes " "these effects, as before. DLAA mode only; the upscaling DLSS modes always " "process them. Applies at once.", + [UI_MENU_DLSS_FIXED_EXPOSURE] = "Fixed DLSS exposure", + [UI_MENU_DLSS_FIXED_EXPOSURE_DESC] = "DLSS takes the brightness of the game's picture as it is instead of " + "estimating the exposure from each frame: in dark scenes (the Depths) large " + "fogged areas no longer turn darker or lighter for a frame while the camera " + "moves. Off: DLSS estimates the exposure itself, as before. Applies at once.", [UI_MENU_BIND_SKIP] = "Skip repeated binds", [UI_MENU_BIND_SKIP_DESC] = "Optimisation: skips bindings that did not change.", [UI_MENU_SHADER_MEMO] = "Shader memo", [UI_MENU_SHADER_MEMO_DESC] = "Optimisation: remembers shader lookups.", @@ -1128,6 +1133,11 @@ static const char *const text_ru[UI_TEXT_COUNT] = { "больше не пропадают и не оставляют шлейфов. Выкл.: DLSS обрабатывает и эти " "эффекты, как раньше. Только в режиме DLAA; режимы DLSS с повышением " "разрешения обрабатывают их всегда. Применяется сразу.", + [UI_MENU_DLSS_FIXED_EXPOSURE] = "Фиксированная экспозиция DLSS", + [UI_MENU_DLSS_FIXED_EXPOSURE_DESC] = "DLSS берёт яркость картинки игры как есть, а не оценивает экспозицию по " + "каждому кадру: в тёмных сценах (Глубины) большие участки в тумане больше не " + "темнеют и не светлеют на кадр при движении камеры. Выкл.: DLSS сам " + "оценивает экспозицию, как раньше. Применяется сразу.", [UI_MENU_BIND_SKIP] = "Пропуск повторных привязок", [UI_MENU_BIND_SKIP_DESC] = "Оптимизация: пропускает неизменившиеся привязки.", [UI_MENU_SHADER_MEMO] = "Память шейдеров", [UI_MENU_SHADER_MEMO_DESC] = "Оптимизация: запоминает поиск шейдеров.", @@ -1572,6 +1582,12 @@ static const char *const text_pt_br[UI_TEXT_COUNT] = { "pequenas na névoa (santuários) não somem mais nem deixam rastros. Desligado:" " o DLSS também processa esses efeitos, como antes. Só no modo DLAA; os modos" " de DLSS que aumentam a resolução sempre os processam. Aplica-se na hora.", + [UI_MENU_DLSS_FIXED_EXPOSURE] = "Exposição fixa do DLSS", + [UI_MENU_DLSS_FIXED_EXPOSURE_DESC] = "O DLSS usa o brilho da imagem do jogo como ele é, em vez de estimar a " + "exposição a cada quadro: em cenas escuras (as Profundezas), grandes áreas com " + "névoa não ficam mais escuras ou claras por um quadro enquanto a câmera se " + "move. Desligado: o DLSS estima a exposição sozinho, como antes. Aplica-se na " + "hora.", [UI_MENU_BIND_SKIP] = "Pular bindings repetidos", [UI_MENU_BIND_SKIP_DESC] = "Otimização: pula bindings que não mudaram.", [UI_MENU_SHADER_MEMO] = "Memória de shaders", [UI_MENU_SHADER_MEMO_DESC] = "Otimização: lembra as buscas de shaders.", @@ -2327,6 +2343,9 @@ static const char *const text_zh_hans[UI_TEXT_COUNT] = { [UI_MENU_DLAA_EFFECTS_AFTER_DESC] = "开启 DLAA 时,雾、灰尘和其他粒子绘制在 DLSS 已平滑的画面之上,与游戏不用 DLSS 时的" "绘制方式相同:雾中的细小粒子(神庙中)不再消失,也不再留下拖影。关闭:DLSS 也处理这些效果,与以" "前相同。仅用于 DLAA 模式;提升分辨率的 DLSS 模式始终处理它们。立即生效。", + [UI_MENU_DLSS_FIXED_EXPOSURE] = "固定 DLSS 曝光", + [UI_MENU_DLSS_FIXED_EXPOSURE_DESC] = "DLSS 直接使用游戏画面的亮度,而不是逐帧自行估算曝光:在暗场景(地底)中," + "镜头移动时大片雾中区域不再忽暗忽亮一帧。关闭:DLSS 自行估算曝光,与以前相同。立即生效。", [UI_MENU_BIND_SKIP] = "跳过重复绑定", [UI_MENU_BIND_SKIP_DESC] = "优化:跳过没有变化的绑定。", [UI_MENU_SHADER_MEMO] = "着色器查找缓存", diff --git a/runtime/ui_text.h b/runtime/ui_text.h index a2d67b75..401abee6 100644 --- a/runtime/ui_text.h +++ b/runtime/ui_text.h @@ -246,6 +246,7 @@ typedef enum { UI_MENU_LOD_GUARD, UI_MENU_LOD_GUARD_DESC, UI_MENU_FIX_C0, UI_MENU_FIX_C0_DESC, UI_MENU_FIX_ZERO, UI_MENU_FIX_ZERO_DESC, UI_MENU_DLAA_EFFECTS_AFTER, UI_MENU_DLAA_EFFECTS_AFTER_DESC, + UI_MENU_DLSS_FIXED_EXPOSURE, UI_MENU_DLSS_FIXED_EXPOSURE_DESC, UI_MENU_BIND_SKIP, UI_MENU_BIND_SKIP_DESC, UI_MENU_SHADER_MEMO, UI_MENU_SHADER_MEMO_DESC, UI_MENU_PIPE_MEMO, UI_MENU_PIPE_MEMO_DESC, diff --git a/runtime/vk_backend.c b/runtime/vk_backend.c index 09ec7e5e..497529a8 100644 --- a/runtime/vk_backend.c +++ b/runtime/vk_backend.c @@ -345,6 +345,7 @@ void gfx_exit_log(void) { static void vk_be_present(const tex_t *t, const int crop[4], uint64_t at_ns) { static _Thread_local int pcore; // the game's render thread: the one that presents if (!pcore) pcore = 1, perf_core_thread(PCORE_RENDER); + vk_dlaa_game_present(); // the camera jitter moves on with the game's frames (vk_dlaa.c) vk_sync_start(); if (!gfx_replay_async()) { uint64_t frame = __atomic_add_fetch(&gfx_counters.frames, 1, __ATOMIC_RELAXED); diff --git a/runtime/vk_dlaa.c b/runtime/vk_dlaa.c index 83e716bb..ecca3058 100644 --- a/runtime/vk_dlaa.c +++ b/runtime/vk_dlaa.c @@ -296,6 +296,7 @@ static struct { uint64_t why[3]; // history resets: scene size changed, no camera, scene target changed int on; // the dlaa setting, read at each present (the menu switches it live) int preset, preset_dirty; // the DLSS model features are created with (preset_value); a change pending + int fixed_exposure; // dlss_fixed_exposure the features are created with (preset_check) VkExecTarget ids; // this frame's G-buffer target 0 (object classes in green), valid with ids_ok int ids_ok; unsigned cfg_seen; // cfg_version when preset was read @@ -466,6 +467,9 @@ static void times_begin(VkCommandBuffer cb) { } // ---------------------------------------------------------------- motion vectors +static int cam_mode(void); // the jitter in the game's camera (below, "jitter in the game's camera") +static int cam_offset(uint64_t blk, float *sx, float *sy); +static int unj_put(VkCommandBuffer cb, const float *blk, float sx, float sy, VkBuffer buf, VkDeviceSize off); // The camera's motion first (runtime/shaders/dlaa_mv.comp): this frame's scene camera block is copied out of the // replay's uniform snapshot (vk_usnap.c, as of the submit call) into one slot of camb, last frame's stays in the other, // and the compute pass reprojects every pixel's depth through the two (inv4 and mv_pass below). Then the per-object @@ -579,8 +583,11 @@ static int mv_pass(VkExec *ex, struct Size *e, VkImageView zv, float jx, float j const float *cr = (const float *)vk_usnap_bytes(D.cam_addr + 48, 64); double cur_vp[16], m[16], ci[16]; int have_m = 0; + float osx = 0.0f, osy = 0.0f; + if (cam_mode()) cam_offset(D.cam_addr, &osx, &osy); if (cr) { for (int i = 0; i < 16; i++) cur_vp[i] = cr[i]; + for (int k = 0; k < 4; k++) cur_vp[k] -= osx * cr[12 + k], cur_vp[4 + k] -= osy * cr[12 + k]; if (D.cam_prev_ok && inv4(cur_vp, ci)) { for (int r = 0; r < 4; r++) for (int k = 0; k < 4; k++) { @@ -595,6 +602,8 @@ static int mv_pass(VkExec *ex, struct Size *e, VkImageView zv, float jx, float j VkBufferCopy bc = {cam.offset, (VkDeviceSize)s * CAM_SLOT, CAM_BLOCK}; vkCmdCopyBuffer(cb, cam.buffer, D.camb, 1, &bc); full_barrier(cb); + const float *cb_bytes = osx != 0.0f || osy != 0.0f ? (const float *)vk_usnap_bytes(D.cam_addr, 0xb0) : NULL; + if (cb_bytes && unj_put(cb, cb_bytes, osx, osy, D.camb, (VkDeviceSize)s * CAM_SLOT)) full_barrier(cb); D.cam_slot ^= 1, D.cam_prev_ok = cr != NULL; if (cr) memcpy(D.cam_prev, cur_vp, sizeof cur_vp); if (!had_prev || !have_m) { // nothing to reproject into: zero motion, new history @@ -670,7 +679,8 @@ static int mv_pass(VkExec *ex, struct Size *e, VkImageView zv, float jx, float j int vk_dlaa_prev_camera(VkDescriptorBufferInfo *out, float jitter[2]) { if (!dl_active() || !D.cam_prev_ok || !D.camb) return 0; *out = (VkDescriptorBufferInfo){D.camb, (VkDeviceSize)(D.cam_slot ^ 1) * CAM_SLOT, CAM_BLOCK}; - jitter[0] = D.jit_ok ? D.jx : 0.0f, jitter[1] = D.jit_ok ? D.jy : 0.0f; + if (cam_mode()) jitter[0] = D.jit_used ? D.ux : 0.0f, jitter[1] = D.jit_used ? D.uy : 0.0f; + else jitter[0] = D.jit_ok ? D.jx : 0.0f, jitter[1] = D.jit_ok ? D.jy : 0.0f; return 1; } @@ -830,7 +840,8 @@ static struct Snap *mask_pass(VkCommandBuffer cb, VkImageView cv, uint32_t w, ui } // ---------------------------------------------------------------- jitter -// The scene's draws (every pass drawing with the scene depth, until the DLAA point) have their viewport moved by a +// On 1.4.3 the game's camera carries the jitter instead (below, "jitter in the game's camera"); on the other versions +// the scene's draws (every pass drawing with the scene depth, until the DLAA point) have their viewport moved by a // sub-pixel Halton(2,3) offset, which shifts what they rasterise as a jittered projection would; full-screen passes // that shade by window position are unaffected. Jitter only follows a frame DLSS ran on: a jittered frame that DLSS // does not resolve shakes on screen, so a missed DLAA point stops the jitter for a second. @@ -849,6 +860,7 @@ static int ended_at(const VkExec *ex) { } void vk_dlaa_viewport(VkExec *ex, VkViewport *v) { + if (cam_mode()) return; // the camera carries the jitter if (!D.jit_ok || !ex->depth.image || ex->depth.image != D.jdepth || ended_at(ex)) return; v->x += D.jx, v->y += D.jy; D.ux = D.jx, D.uy = D.jy; @@ -861,8 +873,15 @@ static void note_camera(const VkExec *ex) { const p_bind_buf *b = &ex->ubo[0][1]; // constant bank c4: the replay keeps user banks from c3 on const VkExecTarget *z = &ex->depth; if (b->size == 0x940 && b->addr && z->image && z->width >= DLAA_SCENE_MIN_W && z->format == VK_FORMAT_D32_SFLOAT && - (!D.jdepth || z->image == D.jdepth)) + (!D.jdepth || z->image == D.jdepth)) { + if (cam_mode() && b->addr != D.cam_addr) { + float sx, sy; + int j = cam_offset(b->addr, &sx, &sy); + D.ux = sx * (float)z->width * 0.5f, D.uy = -sy * (float)z->height * 0.5f; + __atomic_store_n(&D.jit_used, j, __ATOMIC_RELAXED); + } D.cam_addr = b->addr; + } } void vk_dlaa_draw(VkExec *ex) { @@ -921,6 +940,204 @@ static void next_jitter(uint32_t w, uint32_t ow) { D.jit_ok = !D.hold && !nj; } +// ---------------------------------------------------------------- jitter in the game's camera (1.4.3) +// Moving the viewport jitters what the scene draws rasterise, but not the camera the game reconstructs positions with: +// its full-screen passes rebuild each pixel's view ray from the camera block (the half-size lighting's vertex shaders +// read the projection's centre terms, P[0][2] and P[1][2], with its tangents) and its depth, so with the depth half a +// pixel off they put a surface's points off the surface by half a pixel of its depth slope. On water seen at a low +// angle that decides whether a screen-space reflection ray hits or misses, and whole patches of a pond flipped between +// the trees' reflection and the sky's with the jitter's phase (a period of three frames), more than DLSS could resolve; +// measured without DLSS with the same jitter, the pond's brightness swung 2.3-3.0 levels with the viewport moved and +// 0.03-0.04 with the camera moved, the edges shimmering alike. So on 1.4.3 the jitter goes into the camera itself, where +// the game copies its matrices into each camera's uniform block (f_70000e31d4: the projection, rows 7-10, then the +// view-projection, rows 3-6, each a 64-byte memcpy, which ov_memcpy hands over): a perspective camera with the scene's +// aspect, the main one and its half-size copy among them, gets the offset in its projection's centre terms and the same +// in its view-projection, x_clip += sx w and y_clip += sy w. Its shadow cascades are orthographic and stay as they are. +// The offset moves on at the first projection copied after each of the game's presents (vk_dlaa_game_present): the game +// fills two sets of blocks in turn, one per frame, so a block written again is not a new frame. +// The replay reads the offset back from the snapshot of the scene camera's block (cam_offset, the snapshot is the block +// as of the submit) for DLSS, the motion vectors and the per-object pass, which reproject with the camera unjittered; +// after the DLSS point the snapshots of the jittered blocks are written back unjittered (cam_unjitter), so the effects +// drawn over DLSS's picture stay unjittered as before. A block's centre terms count as an offset only when they are one +// this code gave that block (CJ.given): the game has off-centre projections of its own, such as the inventory's camera, +// which draws Link right of the screen's centre with P[0][2] about -0.6, and read as a jitter that sent DLSS an offset +// of 1152 pixels, smearing the world behind the inventory into horizontal streaks. TOTK_DLAA_CAM_JITTER=0 (test) moves +// the viewport as on the other game versions. +#define CAMJ_P_LR 0x70000e3688ull // the return address of the projection's copy (rows 7-10) +#define CAMJ_VP_LR 0x70000e36f4ull // and of the view-projection's (rows 3-6) +#define CAMJ_BLOCKS 16 +#define CAMJ_GIVEN 256 // offsets given lately, by block: about 40 frames' worth, the replay is a few behind +#define UNJ_SLOTS 4096u // 64-byte patches in flight (cam_unjitter, mv_pass): about 60 frames' worth + +static struct { + PlatMutex lk; + uint64_t presents, moved; // the game's presents, and their count when the offset last moved on + struct { uint64_t blk; float sx, sy; } cur[CAMJ_BLOCKS]; // this frame's blocks and the offset each projection took + int ncur; + struct { uint64_t blk; float sx, sy; } given[CAMJ_GIVEN]; // each offset given and its block, a ring + uint32_t ngiven; + uint32_t idx; // phases handed out + float sx, sy; // this frame's offset, clip units (0: none) + VkBuffer unj; // UNJ_SLOTS host-visible patches the replay copies from + VmaAllocation unj_mem; + uint8_t *unj_map; + uint32_t unj_slot; +} CJ = {.lk = PLAT_MUTEX_INIT}; + +static int cam_mode(void) { + static int on = -1; + if (on < 0) on = rt_game_version == 143 && rt_knob("DLAA_CAM_JITTER", "1") && rt_knob("UBO_SNAPSHOT", "1"); + return on; +} + +static int in_list(const uint64_t *l, int n, uint64_t a) { + for (int i = 0; i < n; i++) + if (l[i] == a) return 1; + return 0; +} + +void vk_dlaa_game_present(void) { __atomic_add_fetch(&CJ.presents, 1, __ATOMIC_RELEASE); } + +// The game's thread, after a 64-byte memcpy: lr is the copy's return address into the game, dst where it copied to, off +// the member's offset in the block. f_70000e31d4 takes each offset from the shader's layout of the block (x8 at both +// copies, dst = the block's base + x8), so a block takes the offset only with its matrices where the rest of this +// code reads and writes them, the projection at 0x70 and the view-projection at 0x30, as in the scene camera's block. +void vk_dlaa_camera_copied(uint64_t lr, uint64_t dst, uint64_t off) { + if ((lr != CAMJ_P_LR && lr != CAMJ_VP_LR) || !cam_mode() || !dl_active()) return; + if (lr == CAMJ_P_LR && off != 0x70) return; + float *m = (float *)(uintptr_t)dst; + uint64_t blk = dst - off; + plat_mutex_lock(&CJ.lk); + if (lr == CAMJ_P_LR) { + unsigned o[2], s[2]; + res_sizes(o, s); + float asp = s[1] ? (float)s[0] / (float)s[1] : 0.0f; + // perspective (w = -z), symmetric, with the scene's aspect + if (m[14] != -1.0f || m[15] != 0.0f || m[2] != 0.0f || m[6] != 0.0f || !(m[0] > 0.0f) || + fabsf(m[5] / m[0] - asp) > 0.01f * asp) { + plat_mutex_unlock(&CJ.lk); + return; + } + uint64_t pf = __atomic_load_n(&CJ.presents, __ATOMIC_ACQUIRE); + if (pf != CJ.moved) { // the first projection of a new frame + static int nj = -1; + if (nj < 0) nj = !rt_knob("DLAA_JITTER", "1"); + CJ.moved = pf, CJ.ncur = 0, CJ.sx = CJ.sy = 0.0f; + if (dl_active() && !__atomic_load_n(&D.hold, __ATOMIC_RELAXED) && !nj && s[0] && s[1]) { + double r = (double)o[0] / s[0]; + uint32_t ph = (uint32_t)ceil(8.0 * r * r - 1e-6); + ph = ph < 16 ? 16 : ph > 72 ? 72 : ph; + uint32_t i = CJ.idx++ % ph + 1; + CJ.sx = 2.0f * (halton(i, 2) - 0.5f) / (float)s[0]; // content right by the offset in pixels + CJ.sy = -2.0f * (halton(i, 3) - 0.5f) / (float)s[1]; // and down: NDC +y is the top row + } + } + int i = 0; + while (i < CJ.ncur && CJ.cur[i].blk != blk) i++; + if ((CJ.sx != 0.0f || CJ.sy != 0.0f) && i < CAMJ_BLOCKS) { // no room: no offset, its view-projection takes none + m[2] -= CJ.sx, m[6] -= CJ.sy; + CJ.cur[i].blk = blk, CJ.cur[i].sx = CJ.sx, CJ.cur[i].sy = CJ.sy, CJ.ncur += i == CJ.ncur; + } + } else { // the view-projection takes the offset its block's projection took this frame + for (int i = 0; i < CJ.ncur; i++) + if (CJ.cur[i].blk == blk) { + if (off != 0x30) { // elsewhere in this block: the projection goes back to its centre terms of 0 + float *pm = (float *)(uintptr_t)(blk + 0x70); + pm[2] += CJ.cur[i].sx, pm[6] += CJ.cur[i].sy; + CJ.cur[i] = CJ.cur[--CJ.ncur]; + break; + } + for (int k = 0; k < 4; k++) m[k] += CJ.cur[i].sx * m[12 + k], m[4 + k] += CJ.cur[i].sy * m[12 + k]; + // Given only now that both matrices carry it: a block whose view-projection did not take it is never + // read back as jittered, nor written back by cam_unjitter. + uint32_t g = CJ.ngiven++ % CAMJ_GIVEN; + CJ.given[g].blk = blk, CJ.given[g].sx = CJ.cur[i].sx, CJ.given[g].sy = CJ.cur[i].sy; + break; + } + } + plat_mutex_unlock(&CJ.lk); +} + +// The offset a camera block took (clip units), from its snapshot, else from guest memory: a chunked replay's scan can +// come before the snapshot, and the block holds this frame's bytes until the game's next frame. Centre terms this code +// did not give the block are the game's own and read as no offset. +static int cam_offset(uint64_t blk, float *sx, float *sy) { + *sx = *sy = 0.0f; + if (!blk) return 0; + const float *p = (const float *)vk_usnap_bytes(blk + 0x70, 32); + if (!p) p = (const float *)(uintptr_t)(blk + 0x70); + float x = -p[2], y = -p[6]; + if (x == 0.0f && y == 0.0f) return 0; + int ours = 0; + plat_mutex_lock(&CJ.lk); + for (uint32_t i = 0, n = CJ.ngiven < CAMJ_GIVEN ? CJ.ngiven : CAMJ_GIVEN; i < n && !ours; i++) + ours = CJ.given[i].blk == blk && CJ.given[i].sx == x && CJ.given[i].sy == y; + plat_mutex_unlock(&CJ.lk); + if (ours) *sx = x, *sy = y; + return ours; +} + +// Records copies that write a block's view-projection rows 3-4 and projection rows 7-8 back without the offset (blk: +// the block's bytes as the snapshot holds them) into buf at off, the block's place there. 0: no room. +static int unj_put(VkCommandBuffer cb, const float *blk, float sx, float sy, VkBuffer buf, VkDeviceSize off) { + if (!CJ.unj) { + VkBufferCreateInfo bi = {.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO, .size = UNJ_SLOTS * 64u, + .usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT}; + VmaAllocationCreateInfo ai = {.usage = VMA_MEMORY_USAGE_AUTO_PREFER_HOST, + .flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | + VMA_ALLOCATION_CREATE_MAPPED_BIT}; + VmaAllocationInfo inf; + if (vmaCreateBuffer(vk_ctx()->vma, &bi, &ai, &CJ.unj, &CJ.unj_mem, &inf) != VK_SUCCESS) return CJ.unj = 0, 0; + CJ.unj_map = inf.pMappedData; + } + uint32_t sl = __atomic_fetch_add(&CJ.unj_slot, 1, __ATOMIC_RELAXED) % UNJ_SLOTS; + float v[16]; + for (int k = 0; k < 4; k++) v[k] = blk[12 + k] - sx * blk[24 + k], v[4 + k] = blk[16 + k] - sy * blk[24 + k]; + memcpy(v + 8, blk + 28, 32); + v[10] = 0.0f, v[14] = 0.0f; + memcpy(CJ.unj_map + (size_t)sl * 64, v, sizeof v); + vmaFlushAllocation(vk_ctx()->vma, CJ.unj_mem, (VkDeviceSize)sl * 64, 64); + VkBufferCopy r[2] = {{(VkDeviceSize)sl * 64, off + 0x30, 32}, {(VkDeviceSize)sl * 64 + 32, off + 0x70, 32}}; + vkCmdCopyBuffer(cb, CJ.unj, buf, 2, r); + return 1; +} + +// After the DLSS point, once in each command buffer of the frame: the snapshots of the blocks that took an offset are +// written back without it, so the effects drawn over DLSS's picture use the camera unjittered, as before the move. +static void cam_unjitter(VkExec *ex) { + if (!cam_mode() || !dl_active() || !ended_at(ex)) return; + static PlatMutex lk = PLAT_MUTEX_INIT; + static VkCommandBuffer done[32]; + static int nd; + static uint64_t done_frame = UINT64_MAX; + uint64_t f = __atomic_load_n(&gfx_counters.frames, __ATOMIC_RELAXED); + plat_mutex_lock(&lk); + if (f != done_frame) done_frame = f, nd = 0; + int dup = 0; + for (int i = 0; i < nd && !dup; i++) dup = done[i] == ex->cb; + if (!dup && nd < 32) done[nd++] = ex->cb; + plat_mutex_unlock(&lk); + if (dup) return; + uint64_t blocks[CAMJ_BLOCKS]; // the blocks given an offset lately, newest first + int n = 0; + plat_mutex_lock(&CJ.lk); + for (uint32_t k = 0, m = CJ.ngiven < CAMJ_GIVEN ? CJ.ngiven : CAMJ_GIVEN; k < m && n < CAMJ_BLOCKS; k++) { + uint64_t blk = CJ.given[(CJ.ngiven - 1 - k) % CAMJ_GIVEN].blk; + if (!in_list(blocks, n, blk)) blocks[n++] = blk; + } + plat_mutex_unlock(&CJ.lk); + int any = 0; + for (int i = 0; i < n; i++) { + VkDescriptorBufferInfo bi; + float sx, sy; + const float *b = (const float *)vk_usnap_bytes(blocks[i], 0xb0); + if (!b || !cam_offset(blocks[i], &sx, &sy) || !vk_usnap_find(blocks[i], 0xb0, UINT32_MAX, 4, &bi)) continue; + if (!any) full_barrier(ex->cb); // the scene's draws read the jittered rows until here + any |= unj_put(ex->cb, b, sx, sy, bi.buffer, bi.offset); + } + if (any) full_barrier(ex->cb), ex->recorded += 3; +} + // The DLSS model (TOTK_DLSS_PRESET, live): default takes NVIDIA's model per mode (evaluate: K for DLAA, Quality and // Balanced, M for Performance, L for Ultra Performance), 0 here. A letter a..m takes that model in every mode: k pins // K, j is the other transformer model, which NVIDIA describes as slightly less ghosting at the cost of extra flicker, e @@ -1007,8 +1224,13 @@ static void evaluate(VkExec *ex, const VkExecTarget *dst) { p_setu(p, "OutWidth", ow); p_setu(p, "OutHeight", oh); p_seti(p, "PerfQualityValue", perf_quality(w, ow)); - // IsHDR | MVLowRes | AutoExposure: linear scene light, motion vectors at the render size, no exposure texture - p_seti(p, "DLSS.Feature.Create.Flags", (1 << 0) | (1 << 1) | (1 << 6)); + // IsHDR | MVLowRes: linear scene light, motion vectors at the render size, no exposure texture; AutoExposure + // (1 << 6) unless dlss_fixed_exposure. With AutoExposure DLSS estimates the exposure from each frame itself, and + // in the Depths, where the whole scene is dark, the estimate moved with the camera; distant blocks under uniform + // fog came out up to 0.7 levels (of 255) darker or brighter for a frame, where the game's own picture held one + // value (a 64x64 patch at 30.79 without DLAA read 30.09-30.79 frame to frame with AutoExposure and 30.4-30.7, + // steady, without it). Without it DLSS takes the exposure as 1. + p_seti(p, "DLSS.Feature.Create.Flags", (1 << 0) | (1 << 1) | (D.fixed_exposure ? 0 : 1 << 6)); p_seti(p, "DLSS.Enable.Output.Subrects", 0); // The model per mode (NVSDK_NGX_DLSS_Hint_Render_Preset: 1 A ... 11 K, 12 L, 13 M), as NVIDIA recommends for // DLSS 4.5 (310.x): K (transformer) for DLAA, Quality and Balanced, M for Performance, L for Ultra Performance. @@ -1090,6 +1312,7 @@ static void evaluate(VkExec *ex, const VkExecTarget *dst) { vk_exec_draw_new_cb(ex); // NGX bound pipelines and descriptor sets of its own: the next draw binds everything again D.jdepth = z->image; next_jitter(w, ow); + cam_unjitter(ex); if (++D.evals == 1 || D.evals == 300 || D.evals % 3600 == 0) // 300: one early line with the frames counted rt_log("dlaa: %llu frames anti-aliased (scene %ux%u, output %ux%u; %llu before the effects, %llu frames without the " "DLAA point, %llu with camera motion, %llu with the characters' mask, %llu without a camera, %llu jitter " @@ -1309,6 +1532,7 @@ void vk_dlaa_exec(VkExec *ex, const VkExecJob *j) { } void vk_dlaa_pass(VkExec *ex) { + cam_unjitter(ex); VkExecJob j; if (!ex->chunked && vk_dlaa_plan(ex, &j)) vk_dlaa_exec(ex, &j); } @@ -1329,18 +1553,21 @@ static void dlss_mode(void) { } -// The preset, read again when a setting changed (the menu); a different one recreates the features at the next DLAA run. +// The preset and dlss_fixed_exposure, read again when a setting changed (the menu); a different one recreates the +// features at the next DLAA run. static void preset_check(void) { unsigned v = cfg_version(); if (v == D.cfg_seen) return; D.cfg_seen = v; - int p = preset_value(); - if (p == D.preset) return; - if (D.evals) { + int p = preset_value(), x = cfg_bool("TOTK_DLSS_FIXED_EXPOSURE"); + if (p == D.preset && x == D.fixed_exposure) return; + if (D.evals && p != D.preset) { if (p) rt_log("dlaa: DLSS preset %c in every mode; the features are created again", 'A' + p - 1); else rt_log("dlaa: DLSS presets per mode; the features are created again"); } - D.preset = p, D.preset_dirty = 1, D.reset = 1; + if (D.evals && x != D.fixed_exposure) + rt_log("dlaa: DLSS %s; the features are created again", x ? "exposure fixed at 1" : "auto exposure"); + D.preset = p, D.fixed_exposure = x, D.preset_dirty = 1, D.reset = 1; } void vk_dlaa_frame(void) { diff --git a/runtime/vk_dlaa.h b/runtime/vk_dlaa.h index cce8877d..1275c27e 100644 --- a/runtime/vk_dlaa.h +++ b/runtime/vk_dlaa.h @@ -42,6 +42,9 @@ void vk_mvprev_draw(VkExec *ex); // vk_mvprev.c: notes the scene draw's object int vk_mvobj_execute(VkExec *ex, uint64_t cam, VkImageView mv, VkImageView depth, VkFormat depth_fmt, uint32_t w, uint32_t h); int vk_dlaa_prev_camera(VkDescriptorBufferInfo *out, float jitter[2]); +// nvn_api_core.c ov_memcpy, after a 64-byte copy on the game's thread: the camera jitter of 1.4.3 (vk_dlaa.c). +void vk_dlaa_camera_copied(uint64_t lr, uint64_t dst, uint64_t off); +void vk_dlaa_game_present(void); // vk_backend.c, each of the game's presents, on its thread // vk_mvcheck.c (TOTK_MV_CHECK, test): mv_pass hands it the motion image after the camera pass and, after the object // pass, the motion image again and the scene light target it ran on; it measures how well the vectors fit the picture. void vk_mvcheck_camera(VkCommandBuffer cb, VkImage mv, uint32_t w, uint32_t h, float jx, float jy); diff --git a/runtime/vk_usnap.c b/runtime/vk_usnap.c index 660aa724..43356141 100644 --- a/runtime/vk_usnap.c +++ b/runtime/vk_usnap.c @@ -89,7 +89,8 @@ static Arena *arena_get(uint64_t need) { a->size = need < ARENA_MIN ? ARENA_MIN : (need + (1u << 20) - 1) & ~(uint64_t)((1u << 20) - 1); VkBufferCreateInfo bi = {.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO, .size = a->size, // transfer source: DLAA copies the camera block out (vk_dlaa.c mv_pass) - .usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT}; + .usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT | + VK_BUFFER_USAGE_TRANSFER_DST_BIT}; // Device-local through the BAR when the device has it (the GPU reads constants on every draw), else host memory. VmaAllocationCreateInfo ai = {.usage = VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE, .flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | diff --git a/totk.ini.example b/totk.ini.example index 4960e697..bb17f2bc 100644 --- a/totk.ini.example +++ b/totk.ini.example @@ -304,6 +304,11 @@ # frame before, DLSS is told to favour the current frame by this percentage (0, 35, 65 or 100; 0 off). Higher removes # more of the trails and leaves those edges a little less smooth. Changes while the game runs take effect at once. # dlss_char_mask = 65 +# 1: DLSS takes the brightness of the game's picture as it is (exposure 1) instead of estimating the exposure from each +# frame. In dark scenes (the Depths) the estimate moves with the camera, and large fogged areas turn darker or lighter +# for a frame. 0, the default: DLSS estimates the exposure itself, as before. Every DLSS mode. Changes while the game +# runs take effect at once. +# dlss_fixed_exposure = 0 # With DLAA (dlaa = 1, dlss = dlaa), fog, dust and other particles are drawn over the picture DLSS has already smoothed, # as the game draws them without DLSS, so small particles in fog (shrines) no longer vanish or leave trails. 0: DLSS # processes these effects too, as before (the default until the inventory preview, the scope and water edges are