Commit Graph
71 Commits
Author SHA1 Message Date
Valiantsin DzerakhandClaude Opus 5.5 b34ca6ea4a Merge ASTC decode in a compute shader with native ASTC upload where the device samples it, behind TOTK_ASTC
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:36:43 +03:00
Valiantsin DzerakhandClaude Opus 5.5 21cc2f2962 Merge the descriptor build skip for draws with repeated inputs and descriptor writes through update templates
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:34:21 +03:00
Valiantsin DzerakhandClaude Opus 5.5 291267e65d Decode ASTC a block per lane group through shared memory, in scratch chunks of at most 32 MiB of texels, and time the dispatches with TOTK_ASTC_TIME
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:28:00 +03:00
Valiantsin DzerakhandClaude Opus 5.5 00d7c6689f Write descriptor sets through update templates made with each set layout, with TOTK_DESC_TEMPLATE=0 keeping the write structs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:03:30 +03:00
Valiantsin DzerakhandClaude Opus 5.5 d36034b1ec Skip building a graphics draw's descriptor sets when its inputs equal the previous draw's in the command buffer
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 17:42:54 +03:00
Valiantsin DzerakhandClaude Opus 5.5 0140dc1672 Merge the per-draw shader memo and push skip, the page-indexed pool resolve cache, region-only ReadTexels and WriteTexels, GOB-wide swizzle, the Intel legacy driver vertex-input fallback and all-thread profile chains
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 17:42:50 +03:00
Valiantsin DzerakhandClaude Opus 5.5 9a859dfb2c Decode ASTC textures with a compute shader in the upload batch, or upload them natively where the device samples ASTC, behind TOTK_ASTC
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 14:42:51 +03:00
Valiantsin DzerakhandClaude Opus 5.5 dc75826d71 Turn dynamic vertex input off on Intel's older Windows driver, which draws nothing with it
Intel's frozen driver for UHD 630 and Ice Lake (Vulkan 1.3.215 and older) leaves the picture black with
VK_EXT_vertex_input_dynamic_state (LAS Recomp 61fe44ff, its issues #8 and #357); the GTX 1050 Ti laptop has a
UHD 630. TOTK_VK_DYN_VERTEX=0/1 forces it. The fallback path, checked on the RX 9070 with validation: field picture
as with it, no new validation messages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 14:17:54 +03:00
Valiantsin DzerakhandClaude Opus 5.5 476715e0c2 Swizzle a texture row a GOB at a time instead of computing the block-linear address of every 16 bytes
Within a row, consecutive GOB columns are one block apart and a GOB row's four 16-byte segments sit at fixed
offsets, so the address is computed once per row. Non-ASTC texture upload time in the monsters2 load frames falls
by 28-38% (90 to 65 ms at the world load). probe-vkmem compares it with its reference swizzle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 14:14:46 +03:00
Valiantsin DzerakhandClaude Opus 5.5 f178be9eb2 Copy only the region of a ReadTexels or WriteTexels call instead of swizzling the whole texture level
A guest WriteTexels of a small region deswizzled and reswizzled the whole level while its thread held a virtual core
and the texture lock: 80 ms preemption delays and 50-160 ms frames in the monsters2 replay (frames 765-790,
1225-1235), with the render thread waiting on the texture lock. nvn_texels_box copies the box alone; probe-vkmem
compares it with the whole-level path on random boxes of five block sizes, mip levels, array layers and 3D slices.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 14:12:43 +03:00
Valiantsin DzerakhandClaude Opus 5.5 8a41e3a7ef Keep call chains of every thread in the sampling profile with TOTK_PROFILE_CHAINS=1
Render-thread chains could not show which guest thread held a virtual core during a stall; with every thread's
chains over a short frame range the running samples name it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 14:12:43 +03:00
Valiantsin DzerakhandClaude Opus 5.5 faa246df38 Cache resolved pools per thread by the page of the address instead of the last four, so resolving stays one compare
The replay resolves every vertex stream and buffer binding of each draw; four entries thrashed over the pools a
frame touches, and each miss took the pool lock and searched the index. vk_mem_resolve falls from 5.3-6.1% to
1.6% of the RX 9070 render thread, field replay from 9.2 to 8.9 ms. TOTK_RESOLVE_CHECK=1 compares every hit with
the locked lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 14:00:58 +03:00
Valiantsin DzerakhandClaude Opus 5.5 b7c59adcd9 Reuse the previous draw's shaders while its program, bound state and textures repeat, and skip pushes equal to the last one
About 65% of field draws repeat the previous draw's program, state and texture handles; shader_for falls from
10.8% to 6.4% of the RX 9070 render thread and field replay from 9.7 to 9.2 ms. answers_for looks up only the
slots whose handle changed. TOTK_SHADER_MEMO=check verifies both against full lookups; TOTK_BIND_STATS counts
repeats per draw.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 13:40:37 +03:00
Valiantsin DzerakhandClaude Opus 5.5 7596212938 Merge the render-thread call chains in the sampling profile and the set-associative shader cache that stops per-frame SHA-256 of shader code
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 13:04:57 +03:00
Valiantsin DzerakhandClaude Opus 5.5 bf41b9cfef Make the program-to-shader cache 4-way set-associative so conflict misses stop hashing whole shaders with SHA-256 every frame
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 13:02:21 +03:00
Valiantsin DzerakhandClaude Opus 5.5 2790f7d0e2 Keep render-thread call chains in the sampling profile and report inclusive time and driver callers from them
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 12:40:49 +03:00
Valiantsin DzerakhandClaude Opus 5.5 78957c71ce Merge the profile write on window close and guest exit, which skip atexit handlers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 12:02:52 +03:00
Valiantsin DzerakhandClaude Opus 5.5 5d7958cbb1 Write the sampling profile when the window closes or the game exits, which skip atexit handlers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 11:55:32 +03:00
Valiantsin DzerakhandClaude Opus 5.5 6b1d15534e Merge the device-local mirror of buffer-bound pools on discrete GPUs and the frozen idle regions that cut write-watch queries
On the GTX 1050 Ti the GPU time per frame in the world drops from 110-120 ms to 10-11 ms. TOTK_MEMORY=verify on the
same machine compared 4.45 million mirror pages over a 74-second run with 0 mismatches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 09:43:34 +03:00
Valiantsin DzerakhandClaude Opus 5.5 51567ac32a Freeze idle pool regions read-only so write-watch queries skip them
The mirror and the texture checks queried about 930 MiB of write-watched memory per frame (0.96 ms on the RX 9070
machine, about 3 ms on an 8th-gen i7), at about 1 us per MiB, and 770 MiB of it lay in 2 MiB regions nothing had
written for two seconds. In mirror mode a region left unwritten for TOTK_FREEZE_IDLE submissions (300; 0 turns it off)
and lying wholly in imported pools is now made read-only (memory.c mem_freeze), and the snapshot and change queries
skip it. The first write faults and a vectored handler thaws the region before the write retries; file reads into
guest memory, which the kernel would refuse on a read-only page, thaw their buffer first and log a refusal, as do pool
init and finalize, and commits, decommits and protection changes thaw under the same lock. A region stays thawed for
5 s after a file read. Field: 66 queries over 380 MiB, 0.56 ms per frame; 0.38 ms of it are the queries that find
writes. The perf line `mem: write-watch` gives queries, bytes, time, frozen MiB, freezes and thaws. The gfx-abi probe
covers freeze, thaw by a write and the hold after mem_thaw. TOTK_MEMORY, TOTK_MIRROR_GAP and TOTK_FREEZE_IDLE are known
settings now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 08:39:47 +03:00
Valiantsin DzerakhandClaude Opus 5.5 1fe552ae0a Mirror buffer-bound pools in device-local memory on discrete GPUs
Imported pools put vertices, indices, constants and storage buffers in system memory, so discrete GPUs read them over
PCIe on every draw. A pool now gets a device-local mirror on its first use as a vertex, index, uniform, storage or
indirect buffer; bindings, buffer copies, fills and uniform updates use it at the same offsets. The first access of a
submission syncs the pages written since (write-watch snapshots shared with the texture checks, plus
nvnMemoryPoolFlushMappedRange ranges) in a prologue command buffer before the part; what the GPU writes into a mirror
goes back to guest memory at the end of the submission. TOTK_MEMORY=import|mirror|verify overrides the device class;
verify compares sampled mirror pages with guest memory at the fence. The vkexec probe gains a mirror part.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 00:45:56 +03:00
Valiantsin DzerakhandClaude Opus 5.5 c8a7b889d2 Merge the perf-gated count of what a VRAM mirror of buffer-bound pools would upload, the device class line and the census at frame 600
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 00:22:32 +03:00
Valiantsin DzerakhandClaude Opus 5.5 b8ecf1d937 Document the mem: class and mem: dirty lines in the agent playbook
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 00:17:27 +03:00
Valiantsin DzerakhandClaude Opus 5.5 04314ef476 Count what a VRAM mirror of buffer-bound pools would upload under TOTK_TRACE=perf
At the start of every submit the dirty census queries write watch over the imported pools the replay bound as vertex,
index, uniform, storage or indirect buffers, stamps the shared page generations and logs eager, lazy and first-use
bytes per frame and per submit by pool class as mem: dirty every 60 frames, with FlushMappedRange calls and bytes and
the live and cumulative mirror size. The device class line (uma or discrete, heaps and memory types) is logged once at
device selection and the pool census also at frame 600. perf_summary.py summarises the new lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 00:12:47 +03:00
Valiantsin DzerakhandClaude Opus 5.5 0eb0ba99b3 Merge the UTF-8 code page manifests, quoted paths and damaged file messages that let the installer and the game run from non-ASCII and spaced paths
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 20:36:37 +03:00
Valiantsin DzerakhandClaude Opus 5.5 02158f3f46 Quote paths with spaces on the Ninja command lines of build_game.py
An install folder with a space in its name (or a user name with one) made the compile step fail: clang split
-I temp/Игра TotK-<hash>/gen at the space. Include, library and tool paths and the link map path are now quoted when
they hold a space.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:59:02 +03:00
Valiantsin DzerakhandClaude Opus 5.5 7880790b91 Merge main into task/unicode
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:42:42 +03:00
Valiantsin DzerakhandClaude Opus 5.5 a4bb2d5355 Document the UTF-8 manifests of hactool and totk.exe and the damaged file message
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:35:41 +03:00
Valiantsin DzerakhandClaude Opus 5.5 1820a83f30 Say which game file is damaged when its hashes fail
The ExeFS and RomFS extractors report a failed block hash or a short read as an opaque script failure, and a failed
hactool merge as a generic error. extract.DamagedFile now names the file; a merge whose hash checks fail runs hactool
on the base Program NCA alone to tell the base game from the update. The installer shows check.input_damaged: the file
is damaged or incomplete and should be copied again. hactool output is decoded and its log written as UTF-8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:35:40 +03:00
Valiantsin DzerakhandClaude Opus 5.5 c50f31164d Embed a manifest in totk.exe that makes UTF-8 its code page
GetModuleFileNameA, fopen, CreateFileA and getenv returned and took ANSI code page bytes while platform/win32.c,
fs_host.c and romfs.c convert paths as UTF-8, so an install folder under a Cyrillic user name lost its RomFS, profile,
RomFS patches and caches. runtime/totk.rc is compiled with the runtime objects (windres, or llvm-windres with
--toolchain), so it is also a member of libtotkrt.a. build_game.py writes build.ninja and modules.c as UTF-8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:35:33 +03:00
Valiantsin DzerakhandClaude Opus 5.5 55739d1b05 Merge the perf line that reports GPU time, VRAM, process RAM and what bounds the frame, and the pool census
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:21:14 +03:00
Valiantsin DzerakhandClaude Opus 5.5 94d5367040 Log GPU time per frame, VRAM, process RAM and a bound verdict in the perf line, and a pool census
Each submission part writes a timestamp pair under TOTK_TRACE=perf; the completion thread reads it and sums the busy
time without double-counting overlap. A second 60-frame perf line carries GPU ms per frame, fence waits of the render
thread and of other threads, VRAM usage and budget (VK_EXT_memory_budget, else the allocator's own blocks against the
heap size), working set and private bytes, and bound: gpu, cpu or pace. Devices without timestamp bits or a timestamp
period log one line and show gpu n/a.

vk_mem_resolve now reports the pool class of a span. The replay counts bytes bound or copied per class and use, and
vk_mem_census logs the live pools by kind and flags with that traffic at frame 1800 (every 1800 frames with perf).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:17:24 +03:00
Valiantsin DzerakhandClaude Opus 5.5 cb570e3e35 Merge the missing vertex attribute fix that puts the Brightbloom bulb back in its cup
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 13:55:03 +03:00
Valiantsin DzerakhandClaude Opus 5.5 a70b171aa8 Read a vertex attribute without a format as (0, 0, 0, 1), as Maxwell's fixed attribute does
The zero binding used four-component formats, so shader inputs the game left unset read w = 0. TOTK's foliage
shaders scale wind sway by 1 - colour.w; the Brightbloom stem and leaves have no vertex colour and swayed fully,
while the bulb (which has one) stayed put and floated beside the cup. One-component R32 formats expand to
(0, 0, 0, 1), matching deko3d's IsFixed 1x32 float for unused attributes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 13:53:38 +03:00
Valiantsin DzerakhandClaude Opus 5.5 56e242553a Merge the GPU write-back that brings back grass cutting
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 10:44:40 +03:00
Valiantsin DzerakhandClaude Opus 5.5 48335d0c35 Add the Brightbloom cave save to the run-game save table
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 10:41:55 +03:00
Valiantsin DzerakhandClaude Opus 5.5 0beafeb066 Write GPU-rendered textures back when the CPU invalidates a pool range or waits for a fence
nvn_pool_invalidate was a weak no-op, so nvnMemoryPoolInvalidateMappedRange never wrote anything back. TotK
invalidates its 260x260 R16 grass maps and 1x1 targets before reading them; they now go through guest_read.

The grass system also renders a 128x128 RGBA8 cut map into a CPU-cached pool every frame and reads it with no
invalidate. Without its texels the CPU saw no cut: swings left the grass standing, with no blades and no insects.
After nvnSyncWait or nvnQueueFinish, dirty mirrored targets up to 64 KiB rendered in a submission up to the waited
fence go back to storage (cpu_synced, vk_texture_write_back_small), unless the CPU rewrote it since.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 10:41:54 +03:00
Valiantsin DzerakhandClaude Opus 5.5 167490e708 Merge the installer window test fixture fix
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 10:35:51 +03:00
Valiantsin DzerakhandClaude Opus 5.5 0ec06312d2 Recompute the end-to-end fixture's step keys for the checkout under test, because they fingerprint recomp/, tools/ and runtime/ and install-test was recorded by an older tree
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 10:05:49 +03:00
Valiantsin DzerakhandClaude Opus 5.5 3505258a06 Merge the installer: setup steps, window, launcher and release bundle
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 10:04:19 +03:00
Valiantsin DzerakhandClaude Opus 5.5 2d8a93f823 Re-render the window log from the kept events on a language switch, show language, step and input names instead of codes, reset the model on a new run and feed the selftest's ok screen a complete run
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 09:27:39 +03:00
Valiantsin DzerakhandClaude Opus 5.5 f1f8600294 Run the window's end-to-end test on hard links to install-test and skip it when the RomFS hash reads wrong three times on the machine with the faulty RAM
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 09:24:08 +03:00
Valiantsin DzerakhandClaude Opus 5.5 5c1626e7ac Build TotK-Setup.exe, write VERSION, copy installer/ and totk.ini.example into the release and run a check-only installer pass and the window selftest on the bundle
clean_env compared the upper-cased os.environ keys with mixed-case names and dropped SystemRoot, which crashed Tk in the bundled python.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 09:19:02 +03:00
Valiantsin DzerakhandClaude Opus 5.5 e15f037f65 Make Pillow optional in the window selftest: only --shots needs it and the release has none
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 09:19:02 +03:00
Valiantsin DzerakhandClaude Opus 5.5 b0e354ed11 Add the TotK-Setup.exe launcher: a no-console C program that starts the bundled pythonw with installer.gui and shows a message when python is missing
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 09:15:27 +03:00
Valiantsin DzerakhandClaude Opus 5.5 094bccfb24 Add the Russian text catalog for the installer window and the CLI messages
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 09:13:47 +03:00
Valiantsin DzerakhandClaude Opus 5.5 826448e5e3 Add the installer window: a tkinter front end over the CLI with steps, progress, space, log, cancel, update mode and a self-closing selftest
The check step now emits a space event with the bytes the install needs and the free bytes. The window logic (event model, form, CLI subprocess, Ctrl+Break) is in installer/guilogic.py and tested without Tk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 09:13:42 +03:00
Valiantsin DzerakhandClaude Opus 5.5 f7b846d3ab Merge the gamepad and key remapping
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 08:59:10 +03:00
Valiantsin DzerakhandClaude Opus 5.5 e8697659ac Remap gamepad buttons and keys with bind_<button> lines in totk.ini, one data-driven table for the pad and one for keys
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 08:50:48 +03:00
Valiantsin DzerakhandClaude Opus 5.5 a56a0e6fda Extract the RomFS again when a kept image fails its full hash instead of trusting its index.json
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 08:50:46 +03:00