mirror of
https://github.com/valentderah/totk-pc-recomp.git
synced 2026-11-20 04:35:57 +00:00
Give CPU-visible pools shadow storage on Linux, so the game renders and plays.
On Linux, write-protecting memory the GPU imported costs milliseconds per submit and the import's pinning clears the write watch, so each CPU-visible pool gets a host-visible copy instead: the pages the CPU wrote since the last sync go in right before each submit (skipping bytes the GPU still has to write back), and the replay's GPU writes are copied back into guest memory at their fence (vk_gpu_written). In 1.2.1 play it costs about 0.2 ms a frame. TOTK_SHADOW=1 turns it on under Windows too. Also: build_game.py fetches the VMA header the Windows release pins when the system has none; plat_localtime and plat_gmtime reject the times Windows rejects; the SDL text entry honours TOTK_SWKBD_TEST=cancel; probe-vkmem follows the replay's protocol for GPU writes and syncs.
This commit is contained in:
1 parent
1d51343ac1
commit
5a84b19476
10 files changed
+279
-21
No files matched your search
+14
-8
@@ -120,9 +120,8 @@ folder. The folder holds code and data derived from your game: for your own mach
|
||||
## Linux (in progress)
|
||||
|
||||
The runtime builds and runs on x86-64 Linux with a window, input and menu on SDL3 (`*_sdl.c`) in place of the Win32
|
||||
files (`*_win.c`); `runtime/platform/linux.c` implements `platform.h`. Status: the selftest and the probes that need no
|
||||
game files pass on RADV. `--probe-vkmem` still fails, because watched guest memory cannot be imported on Linux (below);
|
||||
playing the game is not verified yet.
|
||||
files (`*_win.c`); `runtime/platform/linux.c` implements `platform.h`. Status: the selftest and the probes pass, and
|
||||
1.2.1 plays at 60 fps on RADV (RX 7800 XT); 1.4.3 and long play sessions are not verified yet.
|
||||
|
||||
Requirements:
|
||||
|
||||
@@ -132,17 +131,24 @@ Requirements:
|
||||
|
||||
```sh
|
||||
pacman -S --needed clang lld ninja make sdl3 opus libvpx zstd vulkan-headers vulkan-icd-loader \
|
||||
vulkan-memory-allocator vulkan-validation-layers shaderc spirv-tools dotnet-sdk uv git
|
||||
vulkan-validation-layers shaderc spirv-tools dotnet-sdk uv git
|
||||
```
|
||||
|
||||
Vulkan Memory Allocator is a single header (AUR `vulkan-memory-allocator`). Without it, `build_game.py` fetches the
|
||||
version the Windows release pins (3.4.0) into `toolchain/vma/` on first use and checks its hash.
|
||||
|
||||
The steps are the Windows ones with POSIX paths: `.venv/bin/python` in place of `.venv\Scripts\python`, and `totk` in
|
||||
place of `totk.exe`. `build_game.py` takes clang, lld and Ninja from PATH (or the ones in `<dir>/bin` with `--toolchain <dir>`) and links against
|
||||
the system libraries, with `libstdc++` static and an `$ORIGIN` rpath; `build_hactool.py` builds `hactool` with make.
|
||||
place of `totk.exe`. `build_game.py` takes clang, lld and Ninja from PATH (or the ones in `<dir>/bin` with
|
||||
`--toolchain <dir>`) and links against the system libraries, with `libstdc++` static and an `$ORIGIN` rpath;
|
||||
`build_hactool.py` builds `hactool` with make. `tools/prepare_shaders.py` reads the shader list from
|
||||
`refs/MacNativeGames` at `origin/HEAD`, which must be the pinned commit with its history (the list format changed later).
|
||||
|
||||
Memory: on Windows the GPU imports guest memory and `GetWriteWatch` tracks the CPU's writes to it. On Linux,
|
||||
re-protecting imported memory invalidates the driver's userptr buffer on every submit (about 1.3 ms per 16 MiB on
|
||||
amdgpu), and importing pins the pages, which drops their write protection. Watched guest memory is therefore copied to
|
||||
host-visible buffers instead of being imported; that part of `runtime/vk_memory.c` is the next step.
|
||||
amdgpu), and importing pins the pages, which drops their write protection. Each CPU-visible pool therefore gets a
|
||||
host-visible copy (shadow storage, `runtime/vk_memory.c`): the pages the CPU wrote go in before each submit, and the GPU's
|
||||
writes come back at its fence. In play that costs about 0.2 ms a frame and doubles the memory of those pools (about
|
||||
1.1 GB). `TOTK_SHADOW=1` turns it on under Windows too.
|
||||
|
||||
## Checks
|
||||
|
||||
|
||||
@@ -1248,14 +1248,18 @@ void plat_local_time(struct tm *out) {
|
||||
localtime_r(&t, out);
|
||||
}
|
||||
|
||||
// The range of the Windows CRT's localtime_s and gmtime_s (1970 to 3000), which glibc exceeds: the game's time
|
||||
// service reports times outside it as invalid on both hosts.
|
||||
#define PLAT_TIME_MAX 32535215999ll
|
||||
|
||||
int plat_localtime(int64_t unix_s, struct tm *out) {
|
||||
time_t t = (time_t)unix_s;
|
||||
return localtime_r(&t, out) != NULL;
|
||||
return unix_s >= 0 && unix_s <= PLAT_TIME_MAX && localtime_r(&t, out) != NULL;
|
||||
}
|
||||
|
||||
int plat_gmtime(int64_t unix_s, struct tm *out) {
|
||||
time_t t = (time_t)unix_s;
|
||||
return gmtime_r(&t, out) != NULL;
|
||||
return unix_s >= 0 && unix_s <= PLAT_TIME_MAX && gmtime_r(&t, out) != NULL;
|
||||
}
|
||||
|
||||
long plat_tz_west(void) {
|
||||
|
||||
+5
-1
@@ -1,13 +1,17 @@
|
||||
// The software keyboard's front end for SDL windows (swkbd.h). Until the in-game text field exists (an ImGui entry in
|
||||
// the menu overlay, which SDL_StartTextInput can pair with the Steam Deck's on-screen keyboard), the text entry takes
|
||||
// the starting text at once, as TOTK_SWKBD_ANSWER=<initial> does: the game's initial text, else the default name from
|
||||
// the catalog. TotK asks only for a horse's name.
|
||||
// the catalog. TotK asks only for a horse's name. TOTK_SWKBD_TEST=cancel stands for Cancel pressed, as in the Win32
|
||||
// window.
|
||||
#include <string.h>
|
||||
|
||||
#include "cfg.h"
|
||||
#include "host.h"
|
||||
#include "swkbd.h"
|
||||
|
||||
int text_entry_run(const SwkbdRequest *req, uint16_t *out, size_t cap, size_t *n) {
|
||||
const char *test = cfg_get("TOTK_SWKBD_TEST");
|
||||
if (test && !strcmp(test, "cancel")) { *n = 0; return 0; }
|
||||
size_t k = 0;
|
||||
while (k < cap && req->initial[k]) k++;
|
||||
memcpy(out, req->initial, k * 2);
|
||||
|
||||
+7
-2
@@ -318,7 +318,9 @@ static void copy_wait(VkBuffer src, VkDeviceSize so, VkBuffer dst, VkDeviceSize
|
||||
}
|
||||
|
||||
// A CPU-visible pool over the guest heap: GPU writes land in guest memory, CPU writes are read by the GPU, write watch
|
||||
// reports the CPU writes and the explicit ones.
|
||||
// reports the CPU writes and the explicit ones. GPU writes are noted and reported as the replay does
|
||||
// (vk_mem_gpu_pending, vk_gpu_written at the fence), and a GPU read follows vk_mem_shadow_sync as a submit does: with
|
||||
// shadow storage that is what moves the bytes; with an import they are already in place.
|
||||
static int test_import(pool_t *pool) {
|
||||
VkCtx *c = vk_ctx();
|
||||
uint8_t *m = pool->mem;
|
||||
@@ -338,7 +340,9 @@ static int test_import(pool_t *pool) {
|
||||
vmaFlushAllocation(c->vma, sa, 0, VK_WHOLE_SIZE);
|
||||
vk_mem_resolve(a + 0x1000, 0x20000, &sp);
|
||||
mem_written_pages(a, pool->size, pages, 64);
|
||||
vk_mem_gpu_pending(a + 0x1000, 0x20000);
|
||||
copy_wait(sb, 0, sp.buffer, sp.offset, 0x20000);
|
||||
vk_gpu_written(a + 0x1000, 0x20000);
|
||||
GCHECK(!memcmp(m + 0x1000, s, 0x20000), "GPU copy into the imported heap range differs from the source");
|
||||
n = mem_written_pages(a, pool->size, pages, 64);
|
||||
g_gpu_seen = n;
|
||||
@@ -347,10 +351,11 @@ static int test_import(pool_t *pool) {
|
||||
|
||||
// CPU writes: reported exactly, and the GPU reads them through the same buffer
|
||||
for (int i = 0; i < 0x4000; i++) m[0x30000 + i] = rnd8();
|
||||
n = mem_written_pages(a, pool->size, pages, 64);
|
||||
n = mem_written_peek(a, pool->size, pages, 64);
|
||||
GCHECK(n == 4 && pages[0] == a + 0x30000 && pages[3] == a + 0x33000, "CPU writes to 4 pages: %llu reported",
|
||||
(unsigned long long)n);
|
||||
vk_mem_resolve(a + 0x30000, 0x4000, &sp);
|
||||
vk_mem_shadow_sync();
|
||||
copy_wait(sp.buffer, sp.offset, sb, 0, 0x4000);
|
||||
vmaInvalidateAllocation(c->vma, sa, 0, VK_WHOLE_SIZE);
|
||||
GCHECK(!memcmp(s, m + 0x30000, 0x4000), "GPU read of CPU-written heap differs");
|
||||
|
||||
@@ -173,6 +173,12 @@ void vk_mem_mirror_begin(void);
|
||||
int vk_mem_mirror_end(VkCommandBuffer cb, uint64_t fence);
|
||||
int vk_mem_mirror_pending(void);
|
||||
void vk_mem_mirror_prologue(VkCommandBuffer cb);
|
||||
// Shadow storage (vk_memory.c): vk_sync.c calls vk_mem_shadow_sync right before each submit; the replay notes every GPU
|
||||
// write into CPU-visible memory at record time (vk_mem_gpu_pending) and reports it at the fence with vk_gpu_written,
|
||||
// which copies the bytes back into guest memory before telling the texture cache (vk_guest_written).
|
||||
void vk_mem_shadow_sync(void);
|
||||
void vk_mem_gpu_pending(uint64_t addr, uint64_t size);
|
||||
void vk_gpu_written(uint64_t addr, uint64_t size);
|
||||
int vk_mem_mirror_probe(uint64_t addr); // --probe-vkexec: the pool at addr gets its mirror as on a first buffer use;
|
||||
// 1 when it has one
|
||||
void vk_mem_class_log(const VkPhysicalDeviceProperties *p, const VkPhysicalDeviceMemoryProperties *m); // once
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
//
|
||||
// Per command buffer the replay keeps transient memory (support buffers, copied uniform ranges, generated indices),
|
||||
// descriptor pools, image views and the storage-buffer ranges the GPU may write; all of it is freed, and the ranges are
|
||||
// reported with vk_guest_written, from one completion callback once the submission's fence is done (the write watch
|
||||
// reported with vk_gpu_written, from one completion callback once the submission's fence is done (the write watch
|
||||
// does not see GPU writes). Nothing here locks: it runs on the submitting thread under vk_sync_submit's lock.
|
||||
//
|
||||
// Provenance: the op semantics (bind ops, program and slot conventions, draw and dispatch flow, vertex streams with a
|
||||
@@ -175,7 +175,7 @@ static uint64_t bind_gen = 1; // bumped by every bind op that writes
|
||||
static void cb_done(void *arg) { // completion thread
|
||||
CbRes *r = arg;
|
||||
VkCtx *c = vk_ctx();
|
||||
for (int i = 0; i < r->nwritten; i++) vk_guest_written(r->written[i].addr, r->written[i].size);
|
||||
for (int i = 0; i < r->nwritten; i++) vk_gpu_written(r->written[i].addr, r->written[i].size);
|
||||
for (int i = 0; i < r->npools; i++) {
|
||||
vkResetDescriptorPool(c->device, r->pools[i], 0);
|
||||
plat_mutex_lock(&spare_lock);
|
||||
@@ -253,6 +253,7 @@ static void note_written(VkExec *ex, uint64_t addr, uint64_t size) {
|
||||
if (r->written[i].addr == addr && r->written[i].size == size) return;
|
||||
if (r->nwritten == r->cwritten) r->written = grow(r->written, &r->cwritten, sizeof *r->written);
|
||||
r->written[r->nwritten++] = (Range){addr, size};
|
||||
vk_mem_gpu_pending(addr, size);
|
||||
}
|
||||
|
||||
// ---- stand-ins for what a binding names but the game did not bind (zeros, a 1x1 image, the default sampler)
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
// (linear texel layout, GENERAL layout; block-linear storage is only converted when a mirrored texture is uploaded). So a
|
||||
// copy is one Vulkan transfer command and the pitch of a buffer side comes from the payload (OP_COPY_STRIDE or the
|
||||
// tight row). Every copy ends the open pass and starts with a full barrier. Destinations: an image is marked
|
||||
// GPU-written at record time; a CPU-visible buffer range is reported to vk_guest_written once the GPU is done (write
|
||||
// GPU-written at record time; a CPU-visible buffer range is reported to vk_gpu_written once the GPU is done (write
|
||||
// watch does not see GPU writes, textures stored there re-upload).
|
||||
//
|
||||
// Events and counters are CPU-visible side effects. Their values go into guest memory on the completion thread through
|
||||
@@ -66,7 +66,7 @@ typedef struct { uint64_t addr, size; } Range;
|
||||
|
||||
static void guest_written_cb(void *arg) {
|
||||
Range *r = arg;
|
||||
vk_guest_written(r->addr, r->size);
|
||||
vk_gpu_written(r->addr, r->size);
|
||||
free(r);
|
||||
}
|
||||
|
||||
@@ -77,6 +77,7 @@ static void note_gpu_write(VkExec *ex, uint64_t addr, uint64_t size) {
|
||||
Range *r = malloc(sizeof *r);
|
||||
if (!r) rt_fatal("vk: out of memory");
|
||||
*r = (Range){addr, size};
|
||||
vk_mem_gpu_pending(addr, size);
|
||||
vk_completion_add(ex->fence, guest_written_cb, r);
|
||||
}
|
||||
|
||||
|
||||
+204
-2
@@ -59,9 +59,18 @@ typedef struct PoolRec {
|
||||
uint32_t mgen; // gen_lock: every page write stamped up to this generation is in the mirror
|
||||
// (or in a copy queued before its next use)
|
||||
uint64_t fl_lo, fl_hi; // flush_lock: nvnMemoryPoolFlushMappedRange bytes not copied yet (fl_hi 0: none)
|
||||
// Shadow storage (POOL_IMPORT in shadow mode, "shadow storage" below): buf is a host-visible copy of the guest pages.
|
||||
VmaAllocation salloc;
|
||||
uint8_t *smap; // its mapping; offsets as in buf (from import_lo)
|
||||
uint32_t sgen; // gen_lock: every page write stamped up to this generation is in the shadow
|
||||
volatile uint8_t stouched; // on the touched list: the GPU may use it in the submission being recorded
|
||||
} PoolRec;
|
||||
|
||||
static void mirror_forget(PoolRec *r); // pool_lock exclusive: off the mirrored list
|
||||
static int shadow_on; // shadow mode: imported pools get shadow storage instead (vk_mem_class_log)
|
||||
static int shadow_pool(VkCtx *c, PoolRec *r, uint64_t addr, uint64_t size);
|
||||
static void shadow_touch(PoolRec *r);
|
||||
static void shadow_forget(PoolRec *r);
|
||||
static void mirror_retiring(PoolRec *r); // vk_pool_fini: kept until mirror_retire (or vk_mem_shutdown)
|
||||
static void mirror_retire(void *arg); // completion thread: destroys a finalized mirrored pool's record
|
||||
static void grow(void **arr, int *cap, size_t elem);
|
||||
@@ -101,6 +110,7 @@ static int memory_type(const VkCtx *c, uint32_t bits, VkMemoryPropertyFlags want
|
||||
|
||||
// CPU-visible storage: the guest pages themselves become the buffer's memory (VK_EXT_external_memory_host).
|
||||
static int import_pool(VkCtx *c, PoolRec *r, uint64_t addr, uint64_t size) {
|
||||
if (shadow_on) return shadow_pool(c, r, addr, size);
|
||||
if (!p_host_props)
|
||||
p_host_props = (PFN_vkGetMemoryHostPointerPropertiesEXT)vkGetDeviceProcAddr(c->device, "vkGetMemoryHostPointerPropertiesEXT");
|
||||
uint64_t al = c->min_imported_host_alignment ? c->min_imported_host_alignment : 0x1000;
|
||||
@@ -220,6 +230,7 @@ static PoolRec *phys_find(const pool_t *key) {
|
||||
|
||||
static void destroy_rec(VkCtx *c, PoolRec *r) {
|
||||
if (r->mbuf) vmaDestroyBuffer(c->vma, r->mbuf, r->malloc_);
|
||||
if (r->salloc) vmaDestroyBuffer(c->vma, r->buf, r->salloc), r->buf = VK_NULL_HANDLE;
|
||||
if (r->buf) vkDestroyBuffer(c->device, r->buf, NULL);
|
||||
if (r->mem) vkFreeMemory(c->device, r->mem, NULL);
|
||||
if (r->alloc) vmaFreeMemory(c->vma, r->alloc);
|
||||
@@ -298,6 +309,7 @@ void vk_pool_fini(pool_t *p) {
|
||||
}
|
||||
plat_mutex_unlock(&pool_lock);
|
||||
if (!r) return;
|
||||
if (r->smap) shadow_forget(r);
|
||||
if (r->kind == POOL_IMPORT) mem_thaw(r->lo, r->size); // the storage goes back to the guest writable
|
||||
// Images cut from this range go first; a used buffer or memory waits until the GPU is done with it, since the
|
||||
// guest gets the storage back when this returns.
|
||||
@@ -381,6 +393,7 @@ int vk_mem_resolve(uint64_t addr, uint64_t size, VkMemSpan *out) {
|
||||
out->cls = h->cls;
|
||||
out->pool = h->pool;
|
||||
if (h->alloc) out->alloc_offset = out->offset;
|
||||
if (shadow_on && h->pool && !((PoolRec *)h->pool)->stouched) shadow_touch(h->pool);
|
||||
if (resolve_check_on()) {
|
||||
VkMemSpan s;
|
||||
if (!resolve_locked(addr, size, &s) || memcmp(&s, out, sizeof s))
|
||||
@@ -414,6 +427,7 @@ static int resolve_locked(uint64_t addr, uint64_t size, VkMemSpan *out) {
|
||||
out->buffer = r->buf;
|
||||
out->offset = addr - r->import_lo;
|
||||
out->avail = r->import_hi - addr;
|
||||
if (r->smap && !r->stouched) shadow_touch(r);
|
||||
ok = 1;
|
||||
} else if (r && r->kind == POOL_PHYS) {
|
||||
out->buffer = r->buf;
|
||||
@@ -479,6 +493,7 @@ static void flag_names(uint32_t f, char *buf, size_t n) {
|
||||
static void dirty_frame(uint64_t frame);
|
||||
|
||||
static void mirror_log(uint64_t frame);
|
||||
static void shadow_log(uint64_t frame);
|
||||
static int dead_pending(void);
|
||||
|
||||
static void vram_log(uint64_t frame);
|
||||
@@ -487,8 +502,8 @@ void vk_mem_census(uint64_t frame) {
|
||||
static uint64_t window_start;
|
||||
if ((rt_trace & RT_TRACE_PERF) && frame % 600 == 0) vram_log(frame);
|
||||
if ((rt_trace & RT_TRACE_PERF) && !vk_mirror_on) dirty_frame(frame);
|
||||
if ((rt_trace & RT_TRACE_PERF) ? frame % 60 == 0 : frame == 600 || frame == 1800) mirror_log(frame); // and the
|
||||
// write-watch line
|
||||
if ((rt_trace & RT_TRACE_PERF) ? frame % 60 == 0 : frame == 600 || frame == 1800)
|
||||
mirror_log(frame), shadow_log(frame); // mirror_log also logs the write-watch line
|
||||
if (frame == 1200) { // loading is over: the first window starts here
|
||||
for (int c = 0; c < VK_MCLS_N; c++)
|
||||
for (int u = 0; u < VK_MUSE_N; u++) plat_xchg64(&traffic[c][u], 0);
|
||||
@@ -803,6 +818,181 @@ static int query_stamp(uint64_t lo, uint64_t hi, uint32_t g) {
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- shadow storage
|
||||
// Where importing guest memory costs too much (Linux: re-protecting pages that amdgpu imported as a userptr
|
||||
// invalidates the buffer, about 1.3 ms per 16 MiB at the next submit, and the import's write pinning clears the
|
||||
// write watch), a CPU-visible pool gets a host-visible buffer of its own instead, at the offsets the import would have:
|
||||
// - CPU -> shadow: right before each submit (vk_mem_shadow_sync), the pools the replay resolved since the last one are
|
||||
// queried for written pages; pages stamped after a pool's sgen are copied in by the CPU.
|
||||
// - GPU -> guest: the replay notes each GPU write into CPU-visible memory at record time (vk_mem_gpu_pending) and
|
||||
// reports it at the fence (vk_gpu_written), which copies those bytes back into guest memory. Until then the CPU
|
||||
// copies skip them: a page the CPU and the GPU both write keeps the GPU's bytes.
|
||||
// TOTK_SHADOW=1/0 forces the mode on or off; the default is on for Linux.
|
||||
typedef struct { uint64_t lo, hi; } SRange;
|
||||
static PlatMutex shadow_lock = PLAT_MUTEX_INIT; // after pool_lock; before gen_lock
|
||||
static PoolRec **stouch; // shadow_lock: pools resolved since the last sync
|
||||
static int nstouch, cstouch;
|
||||
static SRange *spend; // shadow_lock: GPU writes recorded and not copied back yet
|
||||
static int nspend, cspend;
|
||||
static int64_t s_live_n, s_live_bytes; // shadow_lock
|
||||
static uint64_t s_syncs, s_copied, s_queries, s_untracked, s_wb_bytes, s_ticks; // shadow_lock: since the last log
|
||||
|
||||
static int shadow_pool(VkCtx *c, PoolRec *r, uint64_t addr, uint64_t size) {
|
||||
r->import_lo = addr & ~0xfffull;
|
||||
r->import_hi = (addr + size + 0xfff) & ~0xfffull;
|
||||
uint64_t n = r->import_hi - r->import_lo;
|
||||
VkBufferCreateInfo bi = {.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO, .size = n, .usage = pool_usage(c),
|
||||
.sharingMode = VK_SHARING_MODE_EXCLUSIVE};
|
||||
// Cached where the device has it: the fence copies read it back on the CPU.
|
||||
VmaAllocationCreateInfo ai = {.flags = VMA_ALLOCATION_CREATE_MAPPED_BIT,
|
||||
.requiredFlags = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT,
|
||||
.preferredFlags = VK_MEMORY_PROPERTY_HOST_CACHED_BIT};
|
||||
VmaAllocationInfo info;
|
||||
VkResult res = vmaCreateBuffer(c->vma, &bi, &ai, &r->buf, &r->salloc, &info);
|
||||
if (res != VK_SUCCESS) {
|
||||
rt_log("vk: pool %#llx+%#llx: shadow allocation failed (%d)", (unsigned long long)addr, (unsigned long long)size,
|
||||
(int)res);
|
||||
r->buf = VK_NULL_HANDLE, r->salloc = NULL;
|
||||
return 0;
|
||||
}
|
||||
r->smap = info.pMappedData;
|
||||
plat_mutex_lock(&gen_lock);
|
||||
r->sgen = gen_now; // the copy below holds every write stamped so far; later stamps are copied again
|
||||
plat_mutex_unlock(&gen_lock);
|
||||
memcpy(r->smap, (const void *)(uintptr_t)r->import_lo, n);
|
||||
plat_mutex_lock(&shadow_lock);
|
||||
s_live_n++, s_live_bytes += (int64_t)n;
|
||||
plat_mutex_unlock(&shadow_lock);
|
||||
plat_inc32(&n_imports);
|
||||
return 1;
|
||||
}
|
||||
|
||||
static void shadow_touch(PoolRec *r) {
|
||||
plat_mutex_lock(&shadow_lock);
|
||||
if (!r->stouched) {
|
||||
if (nstouch == cstouch) grow((void **)&stouch, &cstouch, sizeof *stouch);
|
||||
stouch[nstouch++] = r;
|
||||
r->stouched = 1;
|
||||
}
|
||||
plat_mutex_unlock(&shadow_lock);
|
||||
}
|
||||
|
||||
// vk_pool_fini, after the pool left the index: off the touched list before its record goes.
|
||||
static void shadow_forget(PoolRec *r) {
|
||||
plat_mutex_lock(&shadow_lock);
|
||||
for (int i = 0; i < nstouch; i++)
|
||||
if (stouch[i] == r) { stouch[i] = stouch[--nstouch]; break; }
|
||||
s_live_n--, s_live_bytes -= (int64_t)(r->import_hi - r->import_lo);
|
||||
plat_mutex_unlock(&shadow_lock);
|
||||
}
|
||||
|
||||
// shadow_lock held: copies guest [a, b) into the shadow, leaving out the bytes of GPU writes not copied back yet
|
||||
// (spend from index i on).
|
||||
static void shadow_copy(PoolRec *r, uint64_t a, uint64_t b, int i) {
|
||||
for (; i < nspend; i++)
|
||||
if (spend[i].lo < b && spend[i].hi > a) {
|
||||
if (spend[i].lo > a) shadow_copy(r, a, spend[i].lo, i + 1);
|
||||
if (spend[i].hi < b) shadow_copy(r, spend[i].hi, b, i + 1);
|
||||
return;
|
||||
}
|
||||
memcpy(r->smap + (a - r->import_lo), (const void *)(uintptr_t)a, b - a);
|
||||
s_copied += b - a;
|
||||
}
|
||||
|
||||
// shadow_lock and gen_lock held: stamps the pool's written pages and copies those newer than its shadow.
|
||||
static void shadow_sync_pool(PoolRec *r) {
|
||||
uint64_t lo = r->import_lo, hi = r->import_hi;
|
||||
uint32_t g = ++gen_now;
|
||||
int tracked = 1;
|
||||
for (uint64_t a = lo; a < hi;) { // one query per run of regions that are not frozen
|
||||
uint64_t b = (a | ((1ull << REG_SHIFT) - 1)) + 1;
|
||||
if (b > hi) b = hi;
|
||||
if (mem_frozen(a)) { a = b; continue; }
|
||||
while (b < hi && !mem_frozen(b)) b = b + (1ull << REG_SHIFT) < hi ? b + (1ull << REG_SHIFT) : hi;
|
||||
s_queries++;
|
||||
if (!query_stamp(a, b, g)) tracked = 0;
|
||||
a = b;
|
||||
}
|
||||
uint32_t since = r->sgen;
|
||||
r->sgen = gen_now;
|
||||
if (!tracked) { // nothing tells which pages changed: all of them
|
||||
s_untracked++;
|
||||
shadow_copy(r, lo, hi, 0);
|
||||
return;
|
||||
}
|
||||
uint64_t run = 0;
|
||||
for (uint64_t k = lo >> REG_SHIFT; k <= (hi - 1) >> REG_SHIFT; k++) {
|
||||
uint64_t a = k << REG_SHIFT, b = a + (1ull << REG_SHIFT);
|
||||
if (a < lo) a = lo;
|
||||
if (b > hi) b = hi;
|
||||
const uint32_t *ch = gen_chunk[a >> GEN_CHUNK_SHIFT]; // a 2 MiB region lies in one chunk
|
||||
if (reg_max[k] <= since || !ch) {
|
||||
if (run) shadow_copy(r, run, a, 0), run = 0;
|
||||
continue;
|
||||
}
|
||||
for (uint64_t x = a; x < b; x += 1ull << GEN_PAGE_SHIFT) {
|
||||
int dirty = ch[(x >> GEN_PAGE_SHIFT) & (GEN_PER_CHUNK - 1)] > since;
|
||||
if (dirty && !run) run = x;
|
||||
else if (!dirty && run) shadow_copy(r, run, x, 0), run = 0;
|
||||
}
|
||||
}
|
||||
if (run) shadow_copy(r, run, hi, 0);
|
||||
}
|
||||
|
||||
void vk_mem_shadow_sync(void) {
|
||||
if (!shadow_on) return;
|
||||
uint64_t t0 = qpc();
|
||||
plat_mutex_lock(&shadow_lock);
|
||||
if (nstouch) {
|
||||
plat_mutex_lock(&gen_lock);
|
||||
for (int i = 0; i < nstouch; i++) shadow_sync_pool(stouch[i]), stouch[i]->stouched = 0;
|
||||
plat_mutex_unlock(&gen_lock);
|
||||
s_syncs += (uint64_t)nstouch;
|
||||
nstouch = 0;
|
||||
}
|
||||
s_ticks += qpc() - t0;
|
||||
plat_mutex_unlock(&shadow_lock);
|
||||
}
|
||||
|
||||
void vk_mem_gpu_pending(uint64_t addr, uint64_t size) {
|
||||
if (!shadow_on || !size) return;
|
||||
plat_mutex_lock(&shadow_lock);
|
||||
if (nspend == cspend) grow((void **)&spend, &cspend, sizeof *spend);
|
||||
spend[nspend++] = (SRange){addr, addr + size};
|
||||
plat_mutex_unlock(&shadow_lock);
|
||||
}
|
||||
|
||||
void vk_gpu_written(uint64_t addr, uint64_t size) {
|
||||
if (shadow_on && size) {
|
||||
plat_mutex_lock_shared(&pool_lock);
|
||||
const pidx_ent *e = pidx_find(&pool_index, addr, size);
|
||||
PoolRec *r = e ? e->buf : NULL;
|
||||
plat_mutex_lock(&shadow_lock);
|
||||
if (r && r->smap && addr >= r->import_lo && addr + size <= r->import_hi) {
|
||||
mem_thaw(addr, size);
|
||||
memcpy((void *)(uintptr_t)addr, r->smap + (addr - r->import_lo), size);
|
||||
s_wb_bytes += size;
|
||||
}
|
||||
for (int i = 0; i < nspend; i++)
|
||||
if (spend[i].lo == addr && spend[i].hi == addr + size) { spend[i] = spend[--nspend]; break; }
|
||||
plat_mutex_unlock(&shadow_lock);
|
||||
plat_mutex_unlock_shared(&pool_lock);
|
||||
}
|
||||
vk_guest_written(addr, size);
|
||||
}
|
||||
|
||||
static void shadow_log(uint64_t frame) {
|
||||
if (!shadow_on) return;
|
||||
plat_mutex_lock(&shadow_lock);
|
||||
rt_log("mem: shadow at frame %llu: %lld pools %.1f MiB; since the last: %llu pool syncs, %llu queries, %.1f MiB "
|
||||
"copied in (%llu untracked), %.1f MiB written back, %.2f ms, %d GPU writes pending",
|
||||
(unsigned long long)frame, (long long)s_live_n, s_live_bytes / 1048576.0, (unsigned long long)s_syncs,
|
||||
(unsigned long long)s_queries, s_copied / 1048576.0, (unsigned long long)s_untracked, s_wb_bytes / 1048576.0,
|
||||
qpc_ms(s_ticks), nspend);
|
||||
s_syncs = s_queries = s_copied = s_untracked = s_wb_bytes = s_ticks = 0;
|
||||
plat_mutex_unlock(&shadow_lock);
|
||||
}
|
||||
|
||||
void vk_mem_dirty_submit(void) {
|
||||
if (!(rt_trace & RT_TRACE_PERF) || vk_mirror_on) return; // the mirror's own line replaces the census
|
||||
uint64_t t0 = kern_now_ns();
|
||||
@@ -995,6 +1185,14 @@ static void mirror_mode(int discrete) {
|
||||
const char *fi = cfg_get("TOTK_FREEZE_IDLE");
|
||||
freeze_idle = fi && *fi ? atoi(fi) : 60;
|
||||
if (freeze_idle < 0) freeze_idle = 0;
|
||||
const char *sh = cfg_get("TOTK_SHADOW");
|
||||
#ifdef _WIN32
|
||||
shadow_on = sh && *sh && strcmp(sh, "0");
|
||||
#else
|
||||
shadow_on = !sh || !*sh || strcmp(sh, "0");
|
||||
#endif
|
||||
if (shadow_on) rt_log("mem: shadow storage: CPU-visible pools are host-visible copies of guest memory, not imports%s",
|
||||
sh && *sh ? " (TOTK_SHADOW)" : "");
|
||||
rt_log("mem: mode %s (%s): %s", mode == 2 ? "verify" : mode ? "mirror" : "import", why,
|
||||
mode ? "pools used as buffers get a device-local mirror" : "every CPU-visible pool is imported host memory");
|
||||
}
|
||||
@@ -1810,6 +2008,10 @@ void vk_mem_shutdown(void) {
|
||||
for (int i = 0; i < pool_index.n; i++) destroy_rec(c, pool_index.e[i].buf);
|
||||
pool_index.n = 0;
|
||||
nm = npend = nwbl = 0; // mirrors went with their pools
|
||||
plat_mutex_lock(&shadow_lock);
|
||||
nstouch = nspend = 0; // shadows too
|
||||
s_live_n = s_live_bytes = 0;
|
||||
plat_mutex_unlock(&shadow_lock);
|
||||
for (int i = 0; i < nretiring; i++) destroy_rec(c, retiring[i]); // finalized ones whose callback never ran
|
||||
nretiring = 0;
|
||||
__atomic_add_fetch(&pool_gen, 1, __ATOMIC_RELEASE);
|
||||
|
||||
@@ -273,7 +273,9 @@ static CmdSlot *take_slot(VkCtx *c, uint64_t fence) {
|
||||
// The copies the replay's first accesses to VRAM mirrors queued (vk_memory.c), in a command buffer of their own that is
|
||||
// submitted right before the part that needs them and timed like a part; VK_NULL_HANDLE when there are none. After the
|
||||
// part was recorded, sub_lock held; may move slots[].
|
||||
// First brings the shadow storage up to date with the CPU's writes, which needs no command buffer.
|
||||
static VkCommandBuffer prologue(VkCtx *c, uint64_t fence) {
|
||||
vk_mem_shadow_sync(); // shadow storage: the CPU's writes go in before the part is submitted
|
||||
if (!vk_mem_mirror_pending()) return VK_NULL_HANDLE;
|
||||
int i = claim_slot(c, fence); // before reading slots: the claim may move it
|
||||
VkCommandBuffer cb = slots[i].cb;
|
||||
|
||||
+29
-2
@@ -104,7 +104,34 @@ else:
|
||||
LINK_FLAGS = ['-fuse-ld=lld', '-no-pie', '-Wl,-z,noexecstack', "'-Wl,-rpath,$$ORIGIN'",
|
||||
'"-Wl,-Map=$builddir/totk.map"']
|
||||
LINK_LIBS = ['-lvulkan', '-lSDL3', '-lopus', '-lvpx', '-lzstd', '-l:libstdc++.a', '-lm', '-lpthread', '-ldl']
|
||||
VMA_INCLUDE = Path('/usr/include') # vk_mem_alloc.h; Arch package vulkan-memory-allocator, Debian libvulkan-memory-allocator-dev
|
||||
VMA_INCLUDE = Path('/usr/include') # vk_mem_alloc.h; Arch (AUR) vulkan-memory-allocator, Debian libvulkan-memory-allocator-dev
|
||||
# Without a system vk_mem_alloc.h, the header the Windows release pins (tools/make_release.py, VMA 3.4.0) is fetched once
|
||||
# into toolchain/vma: one MIT header (THIRD_PARTY_NOTICES.md), checked against its hash.
|
||||
VMA_FETCH = ('https://raw.githubusercontent.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator/v3.4.0/include/vk_mem_alloc.h',
|
||||
'8487b7995ad3b263eb73bc5b9a77d71aa69b6bef5d58a715c02d2663afd81f1a')
|
||||
VMA_LOCAL = ROOT / 'toolchain' / 'vma'
|
||||
|
||||
|
||||
def vma_include():
|
||||
"""The directory holding vk_mem_alloc.h: the system's, else toolchain/vma, fetched there on first use."""
|
||||
if (VMA_INCLUDE / 'vk_mem_alloc.h').is_file():
|
||||
return VMA_INCLUDE
|
||||
dst = VMA_LOCAL / 'vk_mem_alloc.h'
|
||||
if not dst.is_file():
|
||||
import urllib.request
|
||||
url, sha = VMA_FETCH
|
||||
print(f'build_game: no {VMA_INCLUDE / "vk_mem_alloc.h"}; fetching VMA 3.4.0 into {VMA_LOCAL}')
|
||||
try:
|
||||
data = urllib.request.urlopen(url, timeout=60).read()
|
||||
except OSError as e:
|
||||
sys.exit(f'build_game: fetching {url} failed ({e}); install VMA or put vk_mem_alloc.h in {VMA_LOCAL}')
|
||||
if hashlib.sha256(data).hexdigest() != sha:
|
||||
sys.exit(f'build_game: {url} does not match the pinned hash')
|
||||
VMA_LOCAL.mkdir(parents=True, exist_ok=True)
|
||||
dst.write_bytes(data)
|
||||
return VMA_LOCAL
|
||||
|
||||
|
||||
# --toolchain: an llvm-mingw tree. The libraries, headers and DLLs MSYS2 supplies come from the deps directory instead.
|
||||
LLVM_LINK_LIBS = ['-lpsapi', '-lbcrypt', '-luser32', '-lgdi32', '-ldwmapi', '-lwinmm', '-lvulkan-1', '-lSDL3', '-lopus',
|
||||
'-l:libvpx.a', '-l:libzstd.a', '-l:libc++.a', '-l:libc++abi.a', '-l:libunwind.a', '-l:libwinpthread.a']
|
||||
@@ -282,7 +309,7 @@ def ninja_file(gen, out, mods, pool_depth, skip=(), prebuilt=False, runtime_lib=
|
||||
out/obj and are link inputs without a compile edge. runtime_lib: a prebuilt library replaces the runtime objects
|
||||
except INSTALL_SIDE. make_lib: build only that library, from the runtime objects except INSTALL_SIDE."""
|
||||
inc = f'-I {cmd_path(RUNTIME)} -I {cmd_path(gen)}'
|
||||
vma = f'{cmd_path(DEPS / "include")} -I {cmd_path(DEPS / "include" / "vma")}' if DEPS else cmd_path(VMA_INCLUDE)
|
||||
vma = f'{cmd_path(DEPS / "include")} -I {cmd_path(DEPS / "include" / "vma")}' if DEPS else cmd_path(vma_include())
|
||||
imgui = cmd_path(RUNTIME / 'third_party' / 'imgui')
|
||||
rt_inc = f'{inc} -I {cmd_path(out)} -I {vma} -I {imgui}' # romfs_config.h, vk_mem_alloc.h (+ deps headers), imgui.h
|
||||
libs = [f'-L {cmd_path(DEPS / "lib")}'] + (LLVM_LINK_LIBS if WIN else LINK_LIBS) if DEPS else LINK_LIBS
|
||||
|
||||
Reference in new issue
Block a user