Merge ASTC decode in a compute shader with native ASTC upload where the device samples it, behind TOTK_ASTC

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Valiantsin DzerakhandClaude Opus 5.5 committed 2026-10-25 18:36:43 +03:00
commit b34ca6ea4a
17 files changed
+3172 -26

No files matched your search

+1 -1
View File
@@ -6,7 +6,7 @@ Code copied or adapted from another project is listed here with its source, the
|---|---|---|---|
| MacNativeGames extraction scripts and verification hashes | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11`, Copyright (c) 2026 MacNativeGames contributors | MIT | `tools/vendor/macnativegames/` (unmodified), `tools/data/expected-v*.json` |
| LAS Recomp lifter, code generator, FP/SIMD lowering, CPU headers and differential tests | https://github.com/peiikai/LASRecomp @ `da35b83a9611d046a9a681aae668d7085954d37f`, Copyright (c) 2026 Peiikai | MIT | `recomp/lifter.py`, `recomp/simd_c.py`, `recomp/nso.py`, `recomp/gen.py`, `recomp/module.py`, `tests/jumptables_check.py`, `runtime/cpu.h`, `runtime/simd.h`, `runtime/simd_fp.h`, `tests/diff/` |
| LAS Recomp host runtime: NSO loader, host-function table, guest-space reservation and re-execution, guest thread start | https://github.com/peiikai/LASRecomp @ `da35b83a9611d046a9a681aae668d7085954d37f`, Copyright (c) 2026 Peiikai | MIT | `runtime/loader.c`, `runtime/hostfn.c`, `runtime/memory.c`; M3 tasks T2 and T6: `runtime/kernel.c`, `runtime/boot.c`; M4 T2: `runtime/vk_device.c`, `runtime/gfx_window.c`, M4 T5a: `runtime/tex_astc.c` (`runtime/nvn/astc.c`), `runtime/vk_formats.c` (format table of `runtime/nvn/vulkan/textures.c`), M4 T5b: `runtime/vk_memory.c` (page write generations of `runtime/nvn/vulkan/exec.c`, deferred destruction of `textures.c`), `runtime/vk_texture.c` (image creation, uploads and samplers of `runtime/nvn/vulkan/textures.c`), M4 T6a: `runtime/vk_sync.c` (timeline submissions and waits of `runtime/nvn/vulkan/exec.c`), M4 T6b: `runtime/vk_pipeline.c` (blend, compare and stencil mappings, vertex format table with its 3-component fallback and pipeline cache file of `runtime/nvn/vulkan/exec.c` and `device.c`), M4 T6c1: `runtime/vk_exec.c` (pass begin and end, scissor clip and attachment clears of `runtime/nvn/vulkan/exec.c`), M4 T6c3: `runtime/vk_exec_xfer.c` (copies of `runtime/nvn/vulkan/exec.c`), M4 T6c2: `runtime/vk_exec_draw.c` (uniform buffer spans, texture and sampler descriptors and dispatch flow of `runtime/nvn/vulkan/exec.c`) |
| LAS Recomp host runtime: NSO loader, host-function table, guest-space reservation and re-execution, guest thread start | https://github.com/peiikai/LASRecomp @ `da35b83a9611d046a9a681aae668d7085954d37f`, Copyright (c) 2026 Peiikai | MIT | `runtime/loader.c`, `runtime/hostfn.c`, `runtime/memory.c`; M3 tasks T2 and T6: `runtime/kernel.c`, `runtime/boot.c`; M4 T2: `runtime/vk_device.c`, `runtime/gfx_window.c`, M4 T5a: `runtime/tex_astc.c` (`runtime/nvn/astc.c`; its decode ported to GLSL in `runtime/shaders/astc_decode.comp`, compiled into `runtime/astc_decode_spv.h`, and its tables copied into `runtime/vk_astc.c`), `runtime/vk_formats.c` (format table of `runtime/nvn/vulkan/textures.c`), M4 T5b: `runtime/vk_memory.c` (page write generations of `runtime/nvn/vulkan/exec.c`, deferred destruction of `textures.c`), `runtime/vk_texture.c` (image creation, uploads and samplers of `runtime/nvn/vulkan/textures.c`), M4 T6a: `runtime/vk_sync.c` (timeline submissions and waits of `runtime/nvn/vulkan/exec.c`), M4 T6b: `runtime/vk_pipeline.c` (blend, compare and stencil mappings, vertex format table with its 3-component fallback and pipeline cache file of `runtime/nvn/vulkan/exec.c` and `device.c`), M4 T6c1: `runtime/vk_exec.c` (pass begin and end, scissor clip and attachment clears of `runtime/nvn/vulkan/exec.c`), M4 T6c3: `runtime/vk_exec_xfer.c` (copies of `runtime/nvn/vulkan/exec.c`), M4 T6c2: `runtime/vk_exec_draw.c` (uniform buffer spans, texture and sampler descriptors and dispatch flow of `runtime/nvn/vulkan/exec.c`) |
| MacNativeGames TotK host runtime: dynamic linking, identity check, module permissions, guest memory and SVCs, threads, IPC, audio renderer, system services, RomFS and save storage, boot sequence | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/runtime/`, `games/totk/scripts/prepare_runtime.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `runtime/loader.c`, `runtime/memory.c`; M3 tasks T2-T6: `runtime/kernel.c`, `runtime/svc.c`, `runtime/ipc.c`, `runtime/ipc.h`, `runtime/audren.c`, `runtime/audren.h`, `runtime/audio_mixer.c`, `runtime/audio_effects.c`, `runtime/audio_effects.h`, `runtime/audio_reverb.c`, `runtime/audio_reverb.h` (M5; SDL3 output replaces CoreAudio), `runtime/ov_system.c` (also `shared_font.c`: the BFTTF layout behind nn::pl), `runtime/ov_input.c` (also `hid_feedback.c`: vibration and six-axis handles and calls), `runtime/motion_fusion.c` (six-axis fusion of `motion_fusion.c`), `runtime/ov_media.c` (M5: hardware Opus decoder over libopus), `runtime/swkbd_win.c` (`swkbd_host.c`: ShowKeyboardArg offsets, UTF-8/UTF-16 conversion, disabled-key filter, result codes; Win32 window replaces the macOS text bar), `runtime/ov_gfx.c`, `runtime/romfs.c`, `runtime/romfs_path.h`, `runtime/fs_host.c`, `runtime/fs.h`, `runtime/boot.c`; M4 T2: `runtime/gfx_window.c` |
| MacNativeGames TotK frame-rate patches: 60 fps present-interval controller, real-time frame step and its present sampling, hair and cloth clock, trampoline slow-motion check (instruction words and 1.2.1 sites), player position chain; quick-menu capture background RomFS overlay | https://github.com/ly05010419/MacNativeGames @ `ebe621de99bfc48f3c42f2d2aa73e04fbcc6eae2` (`games/totk/runtime/game_patches.c`, `game_version.h`, `world_survey.c`, `games/totk/scripts/make_romfs_patches.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `runtime/frame_rate.c`, `recomp/patches-v121.txt`, `tools/make_romfs_patches.py` |
| MacNativeGames RomFS index builder (`TOTKRFS1` layout) | https://github.com/ly05010419/MacNativeGames @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (`games/totk/scripts/prepare_romfs_backend.py`), Copyright (c) 2026 MacNativeGames contributors | MIT | `tools/romfs_index.py` |
File diff suppressed because it is too large. Load diff
+260
View File
@@ -0,0 +1,260 @@
// --probe-astc: the compute ASTC decoder (vk_astc.c, shaders/astc_decode.comp) against astc_decode_ldr, bit for bit,
// for every block footprint, linear and sRGB. Synthetic blocks only (no game files): valid void extents, invalid and
// HDR void extents, random blocks kept by what they exercise (1-4 partitions, dual plane with 1-3 partitions, every
// weight grid and quantization the random modes reach), then plain random blocks (mostly error blocks, HDR endpoint
// modes among them). Each case decodes two regions into two mip levels of an RGBA8 image with cropped edges, and
// compares both the decoder's texels (readback) and what the image holds afterwards. Also times one 2048x2048 texture
// on the CPU (astc_decode_ldr_mt) and on the GPU (wall time of the upload batch).
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <windows.h>
#include "host.h"
#include "tex.h"
#include "vk.h"
static uint64_t g_rng = 0x2545F4914F6CDD1Dull;
static uint64_t rnd64(void) {
g_rng ^= g_rng << 13; g_rng ^= g_rng >> 7; g_rng ^= g_rng << 17;
return g_rng;
}
static const uint8_t dims[14][2] = {{4, 4}, {5, 4}, {5, 5}, {6, 5}, {6, 6}, {8, 5}, {8, 6},
{8, 8}, {10, 5}, {10, 6}, {10, 8}, {10, 10}, {12, 10}, {12, 12}};
// Buckets of valid blocks: partitions 1-4 single plane, then dual plane with 1-3 partitions.
enum { NB = 7, PER_BUCKET = 48 };
static const char *const bucket_name[NB] = {"1p", "2p", "3p", "4p", "dual 1p", "dual 2p", "dual 3p"};
static uint64_t g_bucket[NB], g_void, g_err, g_blocks, g_texels;
static int dual_bit(uint32_t m) {
if (!(m & 3) && (m >> 7 & 3) == 2) return 0; // the 6..10 x 6..10 modes have no dual plane
return (int)(m >> 10 & 1);
}
static void void_block(uint8_t b[16], int kind) {
// kind 0: valid LDR extent (random coordinates s0 < s1, t0 < t1), 1: all-ones extent, 2: s0 >= s1 (error),
// 3: HDR (error), 4: reserved bits not 11 (error)
uint64_t lo = 0x1FC | 3ull << 10, s0 = rnd64() & 0xFFF, s1 = s0 + 1 + (rnd64() & 0xFFF), t0 = rnd64() & 0xFFF,
t1 = t0 + 1 + (rnd64() & 0xFFF);
if (s1 > 0x1FFF) s1 = 0x1FFF;
if (t1 > 0x1FFF) t1 = 0x1FFF;
if (kind == 1) s0 = s1 = t0 = t1 = 0x1FFF;
if (kind == 2) { uint64_t x = s0; s0 = s1; s1 = x; }
lo |= s0 << 12 | s1 << 25 | t0 << 38 | t1 << 51;
if (kind == 3) lo |= 0x200;
if (kind == 4) lo &= ~(1ull << 11);
uint64_t hi = rnd64();
memcpy(b, &lo, 8), memcpy(b + 8, &hi, 8);
}
// n blocks for footprint bw x bh: the buckets first, void extents, then random.
static void make_blocks(uint8_t *blk, size_t n, uint32_t bw, uint32_t bh) {
size_t k = 0;
uint8_t tmp[12 * 12 * 4];
int have[NB] = {0};
for (int tries = 0; tries < 400000 && k < n; tries++) {
uint64_t v[2] = {rnd64(), rnd64()};
uint8_t *b = (uint8_t *)v;
uint32_t m = (uint32_t)(v[0] & 0x7FF);
if ((m & 0x1FF) == 0x1FC || !astc_decode_block(b, bw, bh, tries & 1, tmp)) continue;
int p = (int)(v[0] >> 11 & 3), bk = dual_bit(m) ? 4 + p : p;
if (bk >= NB || have[bk] >= PER_BUCKET) continue;
have[bk]++, g_bucket[bk]++;
memcpy(blk + 16 * k++, b, 16);
}
for (int i = 0; i < 40 && k < n; i++, g_void++) void_block(blk + 16 * k++, i % 5);
for (; k < n; k++) {
uint64_t v[2] = {rnd64(), rnd64()};
memcpy(blk + 16 * k, v, 16);
if (!astc_decode_block(blk + 16 * k, bw, bh, false, tmp)) g_err++;
}
}
typedef struct {
VkImage img;
VmaAllocation alloc;
} Img;
static int make_image(VkCtx *c, uint32_t w, uint32_t h, uint32_t levels, int srgb, Img *o) {
VkImageCreateInfo ci = {.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO, .imageType = VK_IMAGE_TYPE_2D,
.format = srgb ? VK_FORMAT_R8G8B8A8_SRGB : VK_FORMAT_R8G8B8A8_UNORM, .extent = {w, h, 1},
.mipLevels = levels, .arrayLayers = 1, .samples = VK_SAMPLE_COUNT_1_BIT,
.tiling = VK_IMAGE_TILING_OPTIMAL,
.usage = VK_IMAGE_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT |
VK_IMAGE_USAGE_SAMPLED_BIT,
.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED};
VmaAllocationCreateInfo ai = {.usage = VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE};
return vmaCreateImage(c->vma, &ci, &ai, &o->img, &o->alloc, NULL) == VK_SUCCESS;
}
static void barrier(VkCommandBuffer cb, VkPipelineStageFlags2 src, VkAccessFlags2 sa, VkPipelineStageFlags2 dst,
VkAccessFlags2 da) {
VkMemoryBarrier2 mb = {.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER_2, .srcStageMask = src, .srcAccessMask = sa,
.dstStageMask = dst, .dstAccessMask = da};
VkDependencyInfo di = {.sType = VK_STRUCTURE_TYPE_DEPENDENCY_INFO, .memoryBarrierCount = 1, .pMemoryBarriers = &mb};
vkCmdPipelineBarrier2(cb, &di);
}
#define ALL_RW VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT, VK_ACCESS_2_MEMORY_READ_BIT | VK_ACCESS_2_MEMORY_WRITE_BIT
static void to_general(VkCommandBuffer cb, VkImage img) {
VkImageMemoryBarrier2 ib = {.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER_2, .srcStageMask = VK_PIPELINE_STAGE_2_NONE,
.dstStageMask = VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT,
.dstAccessMask = VK_ACCESS_2_MEMORY_READ_BIT | VK_ACCESS_2_MEMORY_WRITE_BIT,
.oldLayout = VK_IMAGE_LAYOUT_UNDEFINED, .newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED, .dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = img, .subresourceRange = {VK_IMAGE_ASPECT_COLOR_BIT, 0, VK_REMAINING_MIP_LEVELS, 0, 1}};
VkDependencyInfo di = {.sType = VK_STRUCTURE_TYPE_DEPENDENCY_INFO, .imageMemoryBarrierCount = 1, .pImageMemoryBarriers = &ib};
vkCmdPipelineBarrier2(cb, &di);
}
static double now_ms(void) {
LARGE_INTEGER t, f;
QueryPerformanceCounter(&t), QueryPerformanceFrequency(&f);
return (double)t.QuadPart * 1e3 / (double)f.QuadPart;
}
// Decodes regions (w0 x h0 at level 0, w1 x h1 at level 1) of blk on the GPU; out_rb: the decoder's texels, out_img:
// the image's two levels read back. 0 on a Vulkan failure.
static int gpu_decode(VkCtx *c, const uint8_t *blk, size_t n0, size_t n1, uint32_t w0, uint32_t h0, uint32_t w1,
uint32_t h1, uint32_t bw, uint32_t bh, int srgb, uint8_t *out_rb, uint8_t *out_img, double *ms) {
Img im;
if (!make_image(c, w0, h0, 2, srgb, &im)) return 0;
size_t t0 = (size_t)w0 * h0 * 4, t1 = (size_t)w1 * h1 * 4;
VkCommandBuffer cb = vk_upload_begin();
VkBuffer sb, rb, ib;
VmaAllocation ra, ia;
uint8_t *stg = vk_staging((n0 + n1) * 16, &sb, NULL);
uint8_t *rmap = vk_staging(t0 + t1, &rb, &ra), *imap = vk_staging(t0 + t1, &ib, &ia);
memcpy(stg, blk, (n0 + n1) * 16);
to_general(cb, im.img);
VkAstcRegion ar[2] = {{.src = 0, .w = w0, .h = h0, .slices = 1, .level = 0},
{.src = n0 * 16, .w = w1, .h = h1, .slices = 1, .level = 1}};
barrier(cb, ALL_RW, VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_WRITE_BIT);
int ok = vk_astc_record(cb, sb, (n0 + n1) * 16, ar, 2, bw, bh, srgb, im.img, VK_IMAGE_ASPECT_COLOR_BIT, rb);
barrier(cb, VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_WRITE_BIT, ALL_RW);
VkBufferImageCopy cp[2] = {{.bufferOffset = 0, .imageSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 0, 1}, .imageExtent = {w0, h0, 1}},
{.bufferOffset = t0, .imageSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, 1, 0, 1}, .imageExtent = {w1, h1, 1}}};
vkCmdCopyImageToBuffer(cb, im.img, VK_IMAGE_LAYOUT_GENERAL, ib, 2, cp);
barrier(cb, VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_WRITE_BIT, VK_PIPELINE_STAGE_2_HOST_BIT, VK_ACCESS_2_HOST_READ_BIT);
vk_upload_end();
double a = now_ms();
vk_upload_flush(1);
if (ms) *ms = now_ms() - a;
vmaInvalidateAllocation(c->vma, ra, 0, VK_WHOLE_SIZE);
vmaInvalidateAllocation(c->vma, ia, 0, VK_WHOLE_SIZE);
if (out_rb) memcpy(out_rb, rmap, t0 + t1);
if (out_img) memcpy(out_img, imap, t0 + t1);
vmaDestroyBuffer(c->vma, rb, ra), vmaDestroyBuffer(c->vma, ib, ia);
vk_defer_image(im.img, im.alloc);
vk_upload_flush(1);
return ok;
}
static int first_diff(const uint8_t *a, const uint8_t *b, size_t texels, size_t *at) {
for (size_t i = 0; i < texels; i++)
if (memcmp(a + 4 * i, b + 4 * i, 4)) { *at = i; return 1; }
return 0;
}
static int test_case(VkCtx *c, uint32_t bw, uint32_t bh, int srgb) {
// level 0: 37 x 23 blocks, cropped by 1-3 texels; level 1 half of that, other blocks
uint32_t w0 = 37 * bw - 1 - (bw > 5), h0 = 23 * bh - 2, w1 = w0 > 1 ? w0 >> 1 : 1, h1 = h0 > 1 ? h0 >> 1 : 1;
size_t n0 = (size_t)((w0 + bw - 1) / bw) * ((h0 + bh - 1) / bh), n1 = (size_t)((w1 + bw - 1) / bw) * ((h1 + bh - 1) / bh);
uint8_t *blk = malloc((n0 + n1) * 16);
make_blocks(blk, n0, bw, bh);
make_blocks(blk + n0 * 16, n1, bw, bh);
size_t t0 = (size_t)w0 * h0, t1 = (size_t)w1 * h1;
uint8_t *want = malloc((t0 + t1) * 4), *rb = malloc((t0 + t1) * 4), *im = malloc((t0 + t1) * 4);
memset(rb, 0x5A, (t0 + t1) * 4), memset(im, 0xA5, (t0 + t1) * 4);
astc_decode_ldr(blk, w0, h0, bw, bh, srgb, want);
astc_decode_ldr(blk + n0 * 16, w1, h1, bw, bh, srgb, want + t0 * 4);
int bad = 0;
if (!gpu_decode(c, blk, n0, n1, w0, h0, w1, h1, bw, bh, srgb, rb, im, NULL)) {
printf("probe-astc: FAIL %ux%u %s: the GPU decode was not recorded\n", bw, bh, srgb ? "sRGB" : "linear");
bad = 1;
}
for (int lv = 0; lv < 2 && !bad; lv++) {
size_t off = lv ? t0 * 4 : 0, n = lv ? t1 : t0, at;
uint32_t w = lv ? w1 : w0;
for (int which = 0; which < 2 && !bad; which++) {
const uint8_t *got = (which ? im : rb) + off;
if (first_diff(want + off, got, n, &at)) {
const uint8_t *e = want + off + 4 * at, *g = got + 4 * at;
printf("probe-astc: FAIL %ux%u %s level %d (%s): texel (%zu,%zu) cpu %02x%02x%02x%02x gpu %02x%02x%02x%02x\n",
bw, bh, srgb ? "sRGB" : "linear", lv, which ? "image" : "decoder output", at % w, at / w,
e[0], e[1], e[2], e[3], g[0], g[1], g[2], g[3]);
bad = 1;
}
}
}
g_blocks += n0 + n1, g_texels += t0 + t1;
free(blk), free(want), free(rb), free(im);
return bad;
}
// One 2048 x 2048 texture of valid blocks: the CPU worker-pool decode against the GPU time of the dispatches alone
// (vk_astc.c timestamps, TOTK_ASTC_TIME) and the wall time of the upload batch.
static void timing(VkCtx *c, uint32_t bw, uint32_t bh) {
uint32_t w = 2048, h = 2048;
size_t n = (size_t)((w + bw - 1) / bw) * ((h + bh - 1) / bh);
uint8_t *blk = malloc(n * 16), *out = malloc((size_t)w * h * 4), tmp[12 * 12 * 4];
for (size_t k = 0; k < 4096;) { // valid, non-constant blocks: error blocks would flatter the GPU
uint64_t v[2] = {rnd64(), rnd64()};
if ((v[0] & 0x1FF) != 0x1FC && astc_decode_block((uint8_t *)v, bw, bh, false, tmp)) memcpy(blk + 16 * k++, v, 16);
}
for (size_t k = 4096; k < n; k++) memcpy(blk + 16 * k, blk + 16 * (k % 4096), 16);
double a = now_ms();
astc_decode_ldr_mt(blk, w, h, bw, bh, false, out);
double cpu = now_ms() - a, wall = 0;
vk_astc_time_enable(1);
uint64_t t0, c0, t1, c1;
double u0, u1;
vk_astc_time_get(bw, bh, &t0, &c0, &u0);
for (int i = 0; i < 4; i++) gpu_decode(c, blk, n, 1, w, h, 1, 1, bw, bh, 0, NULL, NULL, i ? &wall : NULL);
vk_astc_time_get(bw, bh, &t1, &c1, &u1);
vk_astc_time_enable(0);
double mtex = (double)(t1 - t0) / 1e6;
printf("probe-astc: 2048x2048 %ux%u: CPU (worker pool) %.1f ms; GPU dispatches %.0f us per M texel (%llu decodes "
"timed, %.2f M texels), batch wall %.1f ms\n", bw, bh, cpu, mtex > 0 ? (u1 - u0) / mtex : 0.0,
(unsigned long long)(c1 - c0), mtex, wall);
free(blk), free(out);
}
int probe_astc(void) {
if (!vk_init()) { printf("probe-astc: FAIL vk_init\n"); return 1; }
VkCtx *c = vk_ctx();
printf("probe-astc: GPU %s, textureCompressionASTC_LDR %d, mode %d\n", c->props.deviceName, c->has_astc_ldr,
vk_astc_mode());
int bad = 0;
if (!vk_astc_gpu()) {
printf("probe-astc: FAIL the compute decoder is not available (TOTK_ASTC=cpu or pipeline creation failed)\n");
bad = 1;
}
for (int i = 0; i < 14 && !bad; i++)
for (int s = 0; s < 2 && !bad; s++) bad = test_case(c, dims[i][0], dims[i][1], s);
for (int b = 0; b < NB && !bad; b++)
if (!g_bucket[b]) { printf("probe-astc: FAIL no valid %s block was generated\n", bucket_name[b]); bad = 1; }
if (!bad) {
timing(c, 4, 4);
timing(c, 6, 6);
timing(c, 8, 8);
timing(c, 12, 12);
}
vk_mem_shutdown();
vk_shutdown();
long errors = vk_validation_errors_total();
if (bad || errors) {
printf("probe-astc: FAILED (%s, %ld validation errors)\n", bad ? "a check failed" : "checks passed", errors);
return 1;
}
printf("probe-astc: passed (14 footprints x linear/sRGB, %llu blocks, %llu texels bit-exact with astc_decode_ldr; "
"valid blocks by kind: 1p %llu, 2p %llu, 3p %llu, 4p %llu, dual 1p %llu, dual 2p %llu, dual 3p %llu; "
"%llu void extents, %llu random error blocks; 0 validation errors)\n",
(unsigned long long)g_blocks, (unsigned long long)g_texels, (unsigned long long)g_bucket[0],
(unsigned long long)g_bucket[1], (unsigned long long)g_bucket[2], (unsigned long long)g_bucket[3],
(unsigned long long)g_bucket[4], (unsigned long long)g_bucket[5], (unsigned long long)g_bucket[6],
(unsigned long long)g_void, (unsigned long long)g_err);
return 0;
}
+1 -1
View File
@@ -23,7 +23,7 @@ static int n_entries;
// Every name the runtime reads (and the patch sets in generated/), without the prefix. A file key outside this list is
// logged as unknown. A new variable goes here too, or its ini key logs a harmless "unknown key" line.
static const char *const known[] = {
"APP_POOL", "ASSUME_FOCUS", "AUDIO", "AUDIO_OFF", "AUDIO_PREFILL_MS", "BACKGROUND_WINDOW", "BIND_A", "BIND_B", "BIND_DOWN", "BIND_L", "BIND_LCLICK", "BIND_LEFT",
"APP_POOL", "ASSUME_FOCUS", "ASTC", "ASTC_TIME", "AUDIO", "AUDIO_OFF", "AUDIO_PREFILL_MS", "BACKGROUND_WINDOW", "BIND_A", "BIND_B", "BIND_DOWN", "BIND_L", "BIND_LCLICK", "BIND_LEFT",
"BIND_MINUS", "BIND_PLUS", "BIND_R", "BIND_RCLICK", "BIND_RIGHT", "BIND_SKIP", "BIND_STATS", "BIND_UP", "BIND_X", "BIND_Y", "BIND_ZL",
"BIND_ZR",
"CLOTH_REALTIME", "DESC_TEMPLATE", "DUMP", "DUMP_BUF", "DUMP_DIR", "DUMP_LOG", "EXEFS", "FEEDBACK", "FIX_C0", "FIX_ZERO", "FLIGHT",
+1
View File
@@ -17,3 +17,4 @@ WEAK const GfxBackend gfx_null_backend = {.name = "unimplemented"};
#define STUB_PROBE(fn, name) WEAK int fn(void) { printf(name ": not built yet\n"); return 2; }
STUB_PROBE(probe_nvn_tex, "probe-nvn-tex")
STUB_PROBE(probe_vkmem, "probe-vkmem")
STUB_PROBE(probe_astc, "probe-astc")
+1 -1
View File
@@ -235,7 +235,7 @@ int probe_vkdevice(void); // M4 T2: vk_device.c
int probe_nvn(void), probe_nvn_tex(void); // M4 T3: nvn_probe.c
int probe_window(void); // M4 T2
int probe_shader(void); // M4 T4
int probe_vkmem(void), probe_vkexec(void); // M4 T5, T6
int probe_vkmem(void), probe_vkexec(void), probe_astc(void); // M4 T5, T6
int probe_movie(void); // movie_probe.c: host VP9 movie::Decoder on one RomFS movie
int vk_pipeline_prepare_shaders(void); // vk_pipeline.c: --prepare-shaders (tools/prepare_shaders.py)
// gfx_stubs.c defines each of these weakly: prints "probe-x: not built yet", returns 2.
+3 -1
View File
@@ -6,6 +6,7 @@
// --probe-kernel, --probe-ipc, --probe-services, --probe-fs task probes (T2-T5)
// --probe-gfx-abi NVN object layouts against nvn.h's reserved sizes (M4 T1); needs no game files
// --probe-nvn, --probe-nvn-tex, --probe-window, --probe-shader, --probe-vkmem, --probe-vkexec M4 probes
// --probe-astc the compute ASTC decoder against astc_decode_ldr; needs no game files
// "not built yet", exit 2, until their task lands
// --probe-movie host VP9 movie::Decoder on one RomFS movie; skips without game data
// --boot run the game to the first NVN call (T6)
@@ -387,7 +388,7 @@ static int usage(const char *argv0) {
fprintf(stderr, "usage: %s [--exefs DIR] [--romfs DIR] [--profile DIR] [--selftest | --probe-loader | "
"--probe-cfg | --probe-kernel | --probe-ipc | --probe-services | --probe-fs | --probe-gfx-abi | "
"--probe-nvn | --probe-nvn-tex | --probe-window | --probe-shader | --probe-vkmem | "
"--probe-vkexec | --probe-movie | --prepare-shaders | --boot |--frames N [--shot PNG]]\n", argv0);
"--probe-vkexec | --probe-astc | --probe-movie | --prepare-shaders | --boot |--frames N [--shot PNG]]\n", argv0);
return 2;
}
@@ -484,6 +485,7 @@ int main(int argc, char **argv) {
if (!strcmp(mode, "--probe-shader")) return probe_shader();
if (!strcmp(mode, "--probe-vkmem")) return probe_vkmem();
if (!strcmp(mode, "--probe-vkexec")) return probe_vkexec();
if (!strcmp(mode, "--probe-astc")) return probe_astc();
if (!strcmp(mode, "--probe-vkdevice")) return probe_vkdevice();
if (!strcmp(mode, "--probe-movie")) return probe_movie();
if (!strcmp(mode, "--prepare-shaders")) return vk_pipeline_prepare_shaders();
+429
View File
@@ -0,0 +1,429 @@
// ASTC LDR 2D decode to RGBA8 on the GPU, one invocation per texel. Ported from runtime/tex_astc.c (itself from LAS
// Recomp runtime/nvn/astc.c @ da35b83a, MIT, Copyright (c) 2026 Peiikai), so the output is the same bytes as
// astc_decode_ldr and astcenc's 8-bit decode: endpoints expand to 16 bits (linear v * 257, sRGB v << 8 | 0x80),
// texel = (e0 * (64 - w) + e1 * w + 32) >> 6, top 8 bits. Error blocks are magenta; HDR endpoint modes give magenta
// endpoints for their partition.
//
// A workgroup of 64 lanes takes up to four blocks of one block row (main below): every lane decodes its block's header,
// the block's lanes share the integer-sequence decode of its weights and colour values through shared memory, and
// each lane then interpolates its own texels.
// Compiled to runtime/astc_decode_spv.h by tools/build_shaders.py; vk_astc.c records the dispatch.
#version 450
layout(local_size_x = 64, local_size_y = 1, local_size_z = 1) in;
layout(std430, set = 0, binding = 0) readonly buffer Blocks { uvec4 blk[]; };
layout(std430, set = 0, binding = 1) writeonly buffer Texels { uint px[]; };
// uint 0..11: byte offsets of the weight unquantization rows (QUANT_2..QUANT_32), 12..27: of the colour rows
// (QUANT_6..QUANT_256 minus one); the bytes follow (vk_astc.c astc_tables).
layout(std430, set = 0, binding = 2) readonly buffer Tables { uint tab[]; };
layout(push_constant) uniform Params {
uint in_base; // first block of slice 0 (uvec4 index into blk)
uint out_base; // first texel of slice 0 (index into px)
uint width, height;
uint bw, bh; // block footprint
uint bx; // blocks per row
uint bslice; // blocks per slice
uint srgb;
} pc;
uint tab_byte(uint i) { return (tab[i >> 2] >> ((i & 3u) * 8u)) & 0xFFu; }
// n (<= 32) bits at pos; bits past 127 read as zero.
uint rbits(uvec4 b, int pos, int n) {
if (n == 0 || pos >= 128) return 0u;
int wi = pos >> 5, sh = pos & 31;
uint v = b[wi] >> uint(sh);
if (sh != 0 && wi < 3) v |= b[wi + 1] << uint(32 - sh);
return n >= 32 ? v : v & ((1u << uint(n)) - 1u);
}
// Quantization levels QUANT_2 .. QUANT_256 (0..20): bits, trits, quints of tex_astc.c q_bits / q_trits / q_quints.
int q_bits(int q) {
if (q < 2) return q == 0 ? 1 : 0;
int g = (q - 2) / 3, r = (q - 2) - 3 * g;
return r == 0 ? g + 2 : r == 1 ? g : g + 1;
}
bool q_trit(int q) { return q == 1 || (q >= 2 && (q - 2) % 3 == 2); }
bool q_quint(int q) { return q >= 2 && (q - 2) % 3 == 1; }
int ise_bitcount(int count, int q) {
int n = count * q_bits(q);
if (q_trit(q)) n += (8 * count + 4) / 5;
if (q_quint(q)) n += (7 * count + 2) / 3;
return n;
}
int trit_at(uint T, int k) {
uint C;
int t0, t1, t2, t3, t4;
if (((T >> 2) & 7u) == 7u) {
C = ((T >> 5) & 7u) << 2 | (T & 3u);
t4 = 2; t3 = 2;
} else {
C = T & 0x1Fu;
if (((T >> 5) & 3u) == 3u) { t4 = 2; t3 = int((T >> 7) & 1u); }
else { t4 = int((T >> 7) & 1u); t3 = int((T >> 5) & 3u); }
}
if ((C & 3u) == 3u) {
t2 = 2; t1 = int((C >> 4) & 1u);
t0 = int(((C >> 3) & 1u) << 1 | (((C >> 2) & 1u) & ~(C >> 3) & 1u));
} else if (((C >> 2) & 3u) == 3u) {
t2 = 2; t1 = 2; t0 = int(C & 3u);
} else {
t2 = int((C >> 4) & 1u); t1 = int((C >> 2) & 3u);
t0 = int(((C >> 1) & 1u) << 1 | ((C & 1u) & ~(C >> 1) & 1u));
}
return k == 0 ? t0 : k == 1 ? t1 : k == 2 ? t2 : k == 3 ? t3 : t4;
}
int quint_at(uint Q, int k) {
int q0, q1, q2;
if (((Q >> 1) & 3u) == 3u && ((Q >> 5) & 3u) == 0u) {
uint b0 = Q & 1u;
q2 = int(b0 << 2 | (((Q >> 4) & 1u) & ~b0 & 1u) << 1 | (((Q >> 3) & 1u) & ~b0 & 1u));
q1 = 4; q0 = 4;
} else {
uint C;
if (((Q >> 1) & 3u) == 3u) { q2 = 4; C = ((Q >> 3) & 3u) << 3 | (~(Q >> 5) & 3u) << 1 | (Q & 1u); }
else { q2 = int((Q >> 5) & 3u); C = Q & 0x1Fu; }
if ((C & 7u) == 5u) { q1 = 4; q0 = int((C >> 3) & 3u); }
else { q1 = int((C >> 3) & 3u); q0 = int(C & 7u); }
}
return k == 0 ? q0 : k == 1 ? q1 : q2;
}
// Raw ISE value idx (trit/quint << bits | bits) of a sequence of count values at bit start; tex_astc.c ise_decode
// reads the sequence front to back, this reads one value's group only. Bits of values past count are not read.
uint ise_at(uvec4 b, int start, int count, int q, int idx) {
int nb = q_bits(q);
if (q_trit(q)) {
// value k of a group: nb bits at k * nb + ts[k], then tb[k] trit bits; ts = {0, 2, 4, 5, 7}, tb = {2, 2, 1, 2, 1}
int g = idx / 5, k = idx - 5 * g, gs = start + g * (5 * nb + 8);
uint m = rbits(b, gs + k * nb + (k * 7 + 2) / 4, nb);
uint T = 0u;
for (int j = 0; j < 5; j++) {
int ts = (j * 7 + 2) / 4, tb = j == 2 || j == 4 ? 1 : 2;
if (5 * g + j < count) T |= rbits(b, gs + (j + 1) * nb + ts, tb) << uint(ts);
}
return uint(trit_at(T, k)) << uint(nb) | m;
}
if (q_quint(q)) {
// qs = {0, 3, 5}, qb = {3, 2, 2}
int g = idx / 3, k = idx - 3 * g, gs = start + g * (3 * nb + 7);
uint m = rbits(b, gs + k * nb + (k == 0 ? 0 : k == 1 ? 3 : 5), nb);
uint Q = 0u;
for (int j = 0; j < 3; j++) {
int qs = j == 0 ? 0 : j == 1 ? 3 : 5, qb = j == 0 ? 3 : 2;
if (3 * g + j < count) Q |= rbits(b, gs + (j + 1) * nb + qs, qb) << uint(qs);
}
return uint(quint_at(Q, k)) << uint(nb) | m;
}
return rbits(b, start + idx * nb, nb);
}
uint hash52(uint p) {
p ^= p >> 15; p -= p << 17; p += p << 7; p += p << 4;
p ^= p >> 5; p += p << 16; p ^= p >> 7; p ^= p >> 3;
p ^= p << 6; p ^= p >> 17;
return p;
}
int part_of(int seed, int count, int x, int y) {
seed += (count - 1) * 1024;
uint rnum = hash52(uint(seed));
int sh1, sh2;
if ((seed & 1) != 0) { sh1 = (seed & 2) != 0 ? 4 : 5; sh2 = count == 3 ? 6 : 5; }
else { sh1 = count == 3 ? 6 : 5; sh2 = (seed & 2) != 0 ? 4 : 5; }
int s[8];
for (int i = 0; i < 8; i++) {
int v = int((rnum >> uint(4 * i)) & 0xFu);
s[i] = (v * v) >> ((i & 1) != 0 ? sh2 : sh1);
}
int a = int((uint(s[0] * x + s[1] * y) + (rnum >> 14)) & 0x3Fu);
int b = int((uint(s[2] * x + s[3] * y) + (rnum >> 10)) & 0x3Fu);
int c = count >= 3 ? int((uint(s[4] * x + s[5] * y) + (rnum >> 6)) & 0x3Fu) : 0;
int d = count >= 4 ? int((uint(s[6] * x + s[7] * y) + (rnum >> 2)) & 0x3Fu) : 0;
if (a >= b && a >= c && a >= d) return 0;
if (b >= c && b >= d) return 1;
if (c >= d) return 2;
return 3;
}
int clamp255(int v) { return clamp(v, 0, 255); }
// v: the partition's unquantized colour values. False for an HDR mode.
bool unpack_endpoints(int cem, int v[8], out ivec4 e0, out ivec4 e1) {
e0 = ivec4(0); e1 = ivec4(0);
switch (cem) {
case 0:
e0 = ivec4(v[0], v[0], v[0], 255); e1 = ivec4(v[1], v[1], v[1], 255);
return true;
case 1: {
int l0 = v[0] >> 2 | (v[1] & 0xC0);
int l1 = min(l0 + (v[1] & 0x3F), 255);
e0 = ivec4(l0, l0, l0, 255); e1 = ivec4(l1, l1, l1, 255);
return true;
}
case 4:
e0 = ivec4(v[0], v[0], v[0], v[2]); e1 = ivec4(v[1], v[1], v[1], v[3]);
return true;
case 5: {
int l0 = v[0], l1 = v[1], a0 = v[2], a1 = v[3];
l0 |= (l1 & 0x80) << 1; a0 |= (a1 & 0x80) << 1;
l1 &= 0x7F; a1 &= 0x7F;
if ((l1 & 0x40) != 0) l1 -= 0x80;
if ((a1 & 0x40) != 0) a1 -= 0x80;
l0 >>= 1; a0 >>= 1; l1 >>= 1; a1 >>= 1;
l1 = clamp255(l1 + l0); a1 = clamp255(a1 + a0);
e0 = ivec4(l0, l0, l0, a0); e1 = ivec4(l1, l1, l1, a1);
return true;
}
case 6:
case 10: {
int s = v[3];
e1 = ivec4(v[0], v[1], v[2], 255);
e0 = ivec4(v[0] * s >> 8, v[1] * s >> 8, v[2] * s >> 8, 255);
if (cem == 10) { e0.w = v[4]; e1.w = v[5]; }
return true;
}
case 8:
case 12: {
ivec4 a = ivec4(v[0], v[2], v[4], cem == 12 ? v[6] : 0);
ivec4 b = ivec4(v[1], v[3], v[5], cem == 12 ? v[7] : 0);
if (a.x + a.y + a.z > b.x + b.y + b.z) {
e0 = b; e1 = a;
e0.x = (e0.x + e0.z) >> 1; e0.y = (e0.y + e0.z) >> 1;
e1.x = (e1.x + e1.z) >> 1; e1.y = (e1.y + e1.z) >> 1;
} else {
e0 = a; e1 = b;
}
if (cem == 8) { e0.w = 255; e1.w = 255; }
return true;
}
case 9:
case 13: {
ivec4 a = ivec4(0), b = ivec4(0);
for (int c = 0; c < 4; c++) {
if (c == 3 && cem != 13) break;
int base = v[2 * c], delta = v[2 * c + 1];
a[c] = base >> 1 | (delta & 0x80);
int d = (delta >> 1) & 0x3F;
if ((d & 0x20) != 0) d -= 0x40;
b[c] = d;
}
int sum = b.x + b.y + b.z;
b += a;
if (sum < 0) {
e0 = b; e1 = a;
e0.x = (e0.x + e0.z) >> 1; e0.y = (e0.y + e0.z) >> 1;
e1.x = (e1.x + e1.z) >> 1; e1.y = (e1.y + e1.z) >> 1;
} else {
e0 = a; e1 = b;
}
e0 = clamp(e0, ivec4(0), ivec4(255)); e1 = clamp(e1, ivec4(0), ivec4(255));
if (cem == 9) { e0.w = 255; e1.w = 255; }
return true;
}
default:
return false;
}
}
const uint MAGENTA = 0xFFFF00FFu;
// What every texel of a block needs from its header (tex_astc.c astc_decode_block up to the weight infill).
struct Hdr {
uint konst; // the colour of a void-extent or error block
bool solid; // konst is the whole block
int wx, wy, dual, quant, wtotal, pcount, cem0, cem1, cem2, cem3, n0, n1, n2, ccount, cq, ccs;
};
Hdr header(uvec4 b, int bw, int bh) {
Hdr H;
H.solid = true; H.konst = MAGENTA;
H.wx = 0; H.wy = 0; H.dual = 0; H.quant = 0; H.wtotal = 0; H.pcount = 1;
H.cem0 = 0; H.cem1 = 0; H.cem2 = 0; H.cem3 = 0; H.n0 = 0; H.n1 = 0; H.n2 = 0; H.ccount = 0; H.cq = 20; H.ccs = -1;
uint mode = b.x & 0x7FFu;
if ((mode & 0x1FFu) == 0x1FCu) { // void extent
if ((mode & 0x200u) != 0u || ((b.x >> 10) & 3u) != 3u) return H;
uint s0 = rbits(b, 12, 13), s1 = rbits(b, 25, 13), t0 = rbits(b, 38, 13), t1 = rbits(b, 51, 13);
bool all_ones = s0 == 0x1FFFu && s1 == 0x1FFFu && t0 == 0x1FFFu && t1 == 0x1FFFu;
if ((s0 >= s1 || t0 >= t1) && !all_ones) return H;
H.konst = (b.z >> 8 & 0xFFu) | (b.z >> 24) << 8 | (b.w >> 8 & 0xFFu) << 16 | (b.w >> 24) << 24;
return H;
}
// block mode (tex_astc.c block_mode_2d)
uint m = mode;
uint r = (m >> 4) & 1u, h = (m >> 9) & 1u, d = (m >> 10) & 1u, a = (m >> 5) & 3u;
int wx, wy;
if ((m & 3u) != 0u) {
r |= (m & 3u) << 1;
uint bb = (m >> 7) & 3u;
uint sel = (m >> 2) & 3u;
if (sel == 0u) { wx = int(bb) + 4; wy = int(a) + 2; }
else if (sel == 1u) { wx = int(bb) + 8; wy = int(a) + 2; }
else if (sel == 2u) { wx = int(a) + 2; wy = int(bb) + 8; }
else {
bb &= 1u;
if ((m & 0x100u) != 0u) { wx = int(bb) + 2; wy = int(a) + 2; }
else { wx = int(a) + 2; wy = int(bb) + 6; }
}
} else {
r |= ((m >> 2) & 3u) << 1;
if (((m >> 2) & 3u) == 0u) return H;
uint bb = (m >> 9) & 3u;
uint sel = (m >> 7) & 3u;
if (sel == 0u) { wx = 12; wy = int(a) + 2; }
else if (sel == 1u) { wx = int(a) + 2; wy = 12; }
else if (sel == 2u) { wx = int(a) + 6; wy = int(bb) + 6; d = 0u; h = 0u; }
else {
if (a == 0u) { wx = 6; wy = 10; }
else if (a == 1u) { wx = 10; wy = 6; }
else return H;
}
}
int dual = int(d);
int wtotal = (wx * wy) << dual;
int quant = int(r) - 2 + 6 * int(h);
int wbits = ise_bitcount(wtotal, quant);
if (wtotal > 64 || wbits < 24 || wbits > 96 || wx > bw || wy > bh) return H;
int pcount = int((b.x >> 11) & 3u) + 1;
if (dual != 0 && pcount == 4) return H;
// colour endpoint modes
int cem0, cem1 = 0, cem2 = 0, cem3 = 0;
int below = 128 - wbits, high = 0;
if (pcount == 1) {
cem0 = int((b.x >> 13) & 0xFu);
} else {
high = 3 * pcount - 4;
below -= high;
uint enc = ((b.x >> 23) & 0x3Fu) | rbits(b, below, high) << 6;
uint base = enc & 3u;
if (base == 0u) {
cem0 = cem1 = cem2 = cem3 = int((enc >> 2) & 0xFu);
below += high;
high = 0;
} else {
base--;
cem0 = int((((enc >> 2) & 1u) + base) << 2 | ((enc >> uint(2 + pcount)) & 3u));
cem1 = int((((enc >> 3) & 1u) + base) << 2 | ((enc >> uint(4 + pcount)) & 3u));
cem2 = int((((enc >> 4) & 1u) + base) << 2 | ((enc >> uint(6 + pcount)) & 3u));
cem3 = int((((enc >> 5) & 1u) + base) << 2 | ((enc >> uint(8 + pcount)) & 3u));
}
}
int n0 = ((cem0 >> 2) + 1) * 2, n1 = pcount > 1 ? ((cem1 >> 2) + 1) * 2 : 0;
int n2 = pcount > 2 ? ((cem2 >> 2) + 1) * 2 : 0, n3 = pcount > 3 ? ((cem3 >> 2) + 1) * 2 : 0;
int ccount = n0 + n1 + n2 + n3;
if (ccount > 18) return H;
int cbits = max((pcount == 1 ? 111 : 99) - wbits - high - (dual != 0 ? 2 : 0), 0);
int cq = -1;
for (int q = 20; q >= 0; q--) {
if (ise_bitcount(ccount, q) <= cbits) { cq = q; break; }
}
if (cq < 4) return H; // QUANT_6
H.solid = false;
H.wx = wx; H.wy = wy; H.dual = dual; H.quant = quant; H.wtotal = wtotal; H.pcount = pcount;
H.cem0 = cem0; H.cem1 = cem1; H.cem2 = cem2; H.cem3 = cem3; H.n0 = n0; H.n1 = n1; H.n2 = n2;
H.ccount = ccount; H.cq = cq;
H.ccs = dual != 0 ? int(rbits(b, below - 2, 2)) : -1;
return H;
}
// A workgroup decodes BPW blocks of one block row when a block has 64 texels or fewer (64 / texels of them), else one
// block: S lanes per block (its texels, at most 64). The lanes of a block decode its weights and colour values into
// shared memory, the first pcount lanes then unpack the endpoints, and each lane writes its texels.
const int MAXB = 4; // 4x4: 4 blocks of 16 lanes
shared uint s_w[MAXB][64]; // unquantized weights 0..64 (dual plane: interleaved as in the bit stream)
shared int s_cv[MAXB][18]; // unquantized colour values
shared uvec2 s_e0[MAXB][4], s_e1[MAXB][4]; // 16-bit endpoints, two channels per uint
uvec2 pack16(ivec4 e) { return uvec2(uint(e.x) | uint(e.y) << 16, uint(e.z) | uint(e.w) << 16); }
int ch16(uvec2 p, int c) { return int(((c < 2 ? p.x : p.y) >> (16 * (c & 1))) & 0xFFFFu); }
void main() {
int bw = int(pc.bw), bh = int(pc.bh), G = bw * bh;
int bpw = G <= 64 ? 64 / G : 1, S = G <= 64 ? G : 64;
int l = int(gl_LocalInvocationID.x), lb = l / S, lane = l - lb * S;
bool live = lb < bpw;
int bxi = int(gl_WorkGroupID.x) * bpw + lb, byi = int(gl_WorkGroupID.y);
live = live && bxi < int(pc.bx);
lb = min(lb, MAXB - 1);
uvec4 b = live ? blk[pc.in_base + gl_WorkGroupID.z * pc.bslice + uint(byi) * pc.bx + uint(bxi)] : uvec4(0u);
Hdr H = header(b, bw, bh);
bool work = live && !H.solid;
if (work) {
uvec4 rev = uvec4(bitfieldReverse(b.w), bitfieldReverse(b.z), bitfieldReverse(b.y), bitfieldReverse(b.x));
uint wrow = tab[H.quant];
for (int j = lane; j < H.wtotal; j += S) s_w[lb][j] = tab_byte(wrow + ise_at(rev, 0, H.wtotal, H.quant, j));
uint crow = H.cq == 20 ? 0u : tab[12 + H.cq - 4];
int cstart = H.pcount == 1 ? 17 : 29;
for (int j = lane; j < H.ccount; j += S) {
uint raw = ise_at(b, cstart, H.ccount, H.cq, j);
s_cv[lb][j] = H.cq == 20 ? int(raw) : int(tab_byte(crow + raw));
}
}
barrier();
if (work && lane < H.pcount) {
int p = lane;
int cem = p == 0 ? H.cem0 : p == 1 ? H.cem1 : p == 2 ? H.cem2 : H.cem3;
int k = p == 0 ? 0 : p == 1 ? H.n0 : p == 2 ? H.n0 + H.n1 : H.n0 + H.n1 + H.n2;
int n = ((cem >> 2) + 1) * 2;
int v[8];
for (int j = 0; j < 8; j++) {
v[j] = 0;
if (j < n) v[j] = s_cv[lb][k + j];
}
ivec4 e0, e1;
if (!unpack_endpoints(cem, v, e0, e1)) { e0 = ivec4(0xFF, 0, 0xFF, 0xFF); e1 = e0; }
if (pc.srgb != 0u) { e0 = e0 << 8 | 0x80; e1 = e1 << 8 | 0x80; }
else { e0 *= 257; e1 *= 257; }
s_e0[lb][p] = pack16(e0);
s_e1[lb][p] = pack16(e1);
}
barrier();
if (!live) return;
bool small = G < 32;
int seed = int((b.x >> 13) & 0x3FFu);
int sx = int((1024 + bw / 2) / (bw - 1)), sy = int((1024 + bh / 2) / (bh - 1));
for (int t = lane; t < G; t += S) {
int x = t % bw, y = t / bw;
uint gx = uint(bxi * bw + x), gy = uint(byi * bh + y);
if (gx >= pc.width || gy >= pc.height) continue;
uint o = H.konst;
if (!H.solid) {
int p = H.pcount > 1 ? part_of(seed, H.pcount, small ? x << 1 : x, small ? y << 1 : y) : 0;
int vx = (sx * x * (H.wx - 1) + 32) >> 6, vy = (sy * y * (H.wy - 1) + 32) >> 6;
int fs = vx & 0xF, ft = vy & 0xF;
int i0 = (vy >> 4) * H.wx + (vx >> 4);
int w11 = (fs * ft + 8) >> 4, w10 = fs - w11, w01 = ft - w11, w00 = 16 - fs - ft + w11;
int wcount = H.wx * H.wy;
int w[2];
for (int pl = 0; pl < 2; pl++) {
w[pl] = 0;
if (pl > H.dual) continue;
int acc = 0;
for (int c = 0; c < 4; c++) {
int idx = i0 + (c == 0 ? 0 : c == 1 ? 1 : c == 2 ? H.wx : H.wx + 1);
int f = c == 0 ? w00 : c == 1 ? w10 : c == 2 ? w01 : w11;
if (f == 0 || idx >= wcount) continue;
acc += int(s_w[lb][H.dual != 0 ? 2 * idx + pl : idx]) * f;
}
w[pl] = (acc + 8) >> 4;
}
uvec2 q0 = s_e0[lb][p], q1 = s_e1[lb][p];
o = 0u;
for (int c = 0; c < 4; c++) {
int wc = c == H.ccs ? w[1] : w[0];
o |= uint((ch16(q0, c) * (64 - wc) + ch16(q1, c) * wc + 32) >> 14) << uint(8 * c);
}
}
px[pc.out_base + (gl_WorkGroupID.z * pc.height + gy) * pc.width + gx] = o;
}
}
+4 -3
View File
@@ -49,7 +49,8 @@ typedef struct TexVkFormat {
VkFormatFeatureFlags need; // optimal-tiling features vk must report for sampling and upload
uint8_t expand_rgb; // 3-component format: the upload expands each texel to 4 components (alpha / integer 1)
uint8_t alpha_one; // "X" format: sample alpha as 1 (view swizzle)
uint8_t astc_bw, astc_bh; // nonzero: ASTC, decoded on the CPU by astc_decode_ldr into vk (RGBA8, same colour space)
uint8_t astc_bw, astc_bh; // nonzero: ASTC decoded into vk (RGBA8, same colour space) by vk_astc.c or tex_astc.c;
// zero for native ASTC (vk == ideal, raw blocks as for BC)
uint8_t astc_srgb;
uint8_t depth, stencil; // aspect flags of depth / stencil formats
const char *note; // why a fallback or an expansion was chosen; NULL when the mapping is direct
@@ -57,8 +58,8 @@ typedef struct TexVkFormat {
} TexVkFormat;
// 1 when nvn_format (an NVNformat value) has a Vulkan mapping, 0 otherwise (out->reason says why). The mapping is
// fixed: ASTC always goes through the CPU decoder, D24S8 is D32S8, DEPTH24 is D32. Whether the GPU reports the
// features is a separate question (tex_vk_format_supported).
// fixed per device: ASTC decodes to RGBA8 unless vk_astc_native (TOTK_ASTC) takes the format as it is, D24S8 is
// D32S8, DEPTH24 is D32. Whether the GPU reports the features is a separate question (tex_vk_format_supported).
int tex_vk_format(int nvn_format, TexVkFormat *out);
// 1 when vk_ctx()->phys reports every bit of f->need for f->vk in optimal tiling. Needs vk_init().
int tex_vk_format_supported(const TexVkFormat *f);
+23 -1
View File
@@ -37,7 +37,8 @@ typedef struct VkCtx {
int validation; // layer active
int has_custom_border_color, has_eds, has_eds2, has_eds3, has_vertex_input_dynamic_state,
has_conditional_rendering, has_transform_feedback, has_provoking_vertex, has_device_fault, has_memory_budget,
has_robustness2, has_buffer_device_address, has_fragment_shader_interlock;
has_robustness2, has_buffer_device_address, has_fragment_shader_interlock,
has_astc_ldr; // textureCompressionASTC_LDR (enabled when reported)
// Feedback loops (maxwell.md 1.9, vk_exec_draw.c feedback_prepare): a draw samples a target of its open pass that
// an earlier draw of the pass wrote. TOTK_FEEDBACK=0: VK_FEEDBACK_OFF (sampled in GENERAL without a barrier, the
// old behaviour); VK_FEEDBACK_LAYOUT with VK_EXT_attachment_feedback_loop_layout and dynamicRenderingLocalRead
@@ -198,6 +199,27 @@ uint64_t vk_upload_flush(int wait);
void vk_defer_image(VkImage img, VmaAllocation alloc); // alloc NULL: vkDestroyImage only
void vk_defer_view(VkImageView view);
void vk_defer_buffer(VkBuffer buf, VmaAllocation alloc);
// ---- ASTC (vk_astc.c). TOTK_ASTC = auto | gpu | native | cpu | check; see vk_astc.c.
enum { VK_ASTC_CPU, VK_ASTC_GPU, VK_ASTC_NATIVE, VK_ASTC_CHECK };
int vk_astc_mode(void); // decided once per device (logged); VK_ASTC_CPU before vk_init
int vk_astc_native(VkFormat astc_fmt); // 1: images of this ASTC format are created as such and get the raw blocks
int vk_astc_gpu(void); // 1: ASTC images decoded to RGBA8 by the compute decoder (vk_astc_record)
// One image region decoded from raw blocks: tight rows of 16-byte blocks at byte src of the source buffer (slices one
// after the other), w x h texels and slices deep, written at offset of level / layer.
typedef struct { VkDeviceSize src; uint32_t w, h, slices, level, layer; VkOffset3D offset; } VkAstcRegion;
// Records into cb, the open upload batch (between vk_upload_begin and vk_upload_end): src (TRANSFER_SRC, src_size
// bytes of blocks) goes to a device-local scratch buffer, a dispatch per region decodes it, and the texels are copied
// into img (VK_IMAGE_LAYOUT_GENERAL). The caller records its usual barriers around an image copy before and after the
// call. readback (optional, host-visible, TRANSFER_DST): also gets the decoded texels, the regions one after the other.
// 0 when the decoder is unavailable or the scratch buffer cannot be allocated (nothing recorded).
int vk_astc_record(VkCommandBuffer cb, VkBuffer src, VkDeviceSize src_size, const VkAstcRegion *rg, int n, uint32_t bw,
uint32_t bh, int srgb, VkImage img, VkImageAspectFlags aspect, VkBuffer readback);
void vk_astc_stats(uint64_t *dispatches, uint64_t *texels); // since start
// TOTK_ASTC_TIME=1: GPU timestamps around each vk_astc_record's dispatches (logged per footprint and the largest one)
void vk_astc_time_enable(int on);
int vk_astc_time_get(uint32_t bw, uint32_t bh, uint64_t *texels, uint64_t *calls, double *us); // totals so far
void vk_astc_time_summary(void); // logs the totals
void vk_astc_release(void); // vk_device.c, before the device goes: frees the decoder's objects
void vk_mem_note_submit(uint64_t timeline_value); // T6: after each submit that signals ctx->timeline with this value
void vk_mem_collect(void); // frees what has completed
typedef struct { uint64_t imported_pools, physical_pools, virtual_pools, import_failures, upload_batches; } VkMemStats;
+541
View File
@@ -0,0 +1,541 @@
// ASTC textures on the GPU: the TOTK_ASTC knob, a compute decoder that turns raw blocks into the RGBA8 image the CPU
// decoder (tex_astc.c) would have written, and the native ASTC formats where the device samples them itself.
//
// TOTK_ASTC = auto (default): native when the device reports textureCompressionASTC_LDR and the format's features,
// else gpu. gpu: compute decode into RGBA8 (runtime/shaders/astc_decode.comp). native: like auto, but logs when the
// device cannot. cpu: astc_decode_ldr_mt on the uploading thread, as before. check: gpu, and every upload is also
// decoded on the CPU and compared (waits for the GPU; for testing only).
//
// The decode is recorded into the upload batch (vk_memory.c) in chunks of at most 32 MiB of texels: a chunk's raw
// blocks go from staging to a device-local scratch buffer, its dispatches write RGBA8 texels behind them, and a
// buffer-to-image copy puts them into the image; the next chunk reuses the scratch after a barrier. The scratch
// buffer is freed once the batch completes (vk_defer_buffer).
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <windows.h>
#include "astc_decode_spv.h"
#include "cfg.h"
#include "host.h"
#include "tex.h"
#include "vk.h"
// tex_astc.c's unquantization tables, as the shader reads them (Tables in astc_decode.comp): 12 + 16 row offsets in
// bytes, then the bytes. Kept in step with tex_astc.c by the oracle (--probe-astc), which compares every mode.
static const uint8_t wunq_data[] = {
0,64,
0,32,64,
0,21,43,64,
0,16,32,48,64,
0,64,12,52,25,39,
0,9,18,27,37,46,55,64,
0,64,7,57,14,50,21,43,28,36,
0,64,17,47,5,59,23,41,11,53,28,36,
0,4,8,12,17,21,25,29,35,39,43,47,52,56,60,64,
0,64,16,48,3,61,19,45,6,58,23,41,9,55,26,38,13,51,29,35,
0,64,8,56,16,48,24,40,2,62,11,53,19,45,27,37,5,59,13,51,22,42,30,34,
0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,
};
static const uint16_t wunq_off[12] = {0,2,5,9,14,20,28,38,50,66,86,110};
static const uint8_t cunq_data[] = {
0,255,51,204,102,153,
0,36,73,109,146,182,219,255,
0,255,28,227,56,199,84,171,113,142,
0,255,69,186,23,232,92,163,46,209,116,139,
0,17,34,51,68,85,102,119,136,153,170,187,204,221,238,255,
0,255,67,188,13,242,80,175,27,228,94,161,40,215,107,148,54,201,121,134,
0,255,33,222,66,189,99,156,11,244,44,211,77,178,110,145,22,233,55,200,88,167,121,134,
0,8,16,24,33,41,49,57,66,74,82,90,99,107,115,123,132,140,148,156,165,173,181,189,198,206,214,222,
231,239,247,255,
0,255,32,223,65,190,97,158,6,249,39,216,71,184,104,151,13,242,45,210,78,177,110,145,19,236,52,203,
84,171,117,138,26,229,58,197,91,164,123,132,
0,255,16,239,32,223,48,207,65,190,81,174,97,158,113,142,5,250,21,234,38,217,54,201,70,185,86,169,
103,152,119,136,11,244,27,228,43,212,59,196,76,179,92,163,108,147,124,131,
0,4,8,12,16,20,24,28,32,36,40,44,48,52,56,60,65,69,73,77,81,85,89,93,97,101,105,109,113,117,121,125,
130,134,138,142,146,150,154,158,162,166,170,174,178,182,186,190,195,199,203,207,211,215,219,223,227,
231,235,239,243,247,251,255,
0,255,16,239,32,223,48,207,64,191,80,175,96,159,112,143,3,252,19,236,35,220,51,204,67,188,83,172,
100,155,116,139,6,249,22,233,38,217,54,201,71,184,87,168,103,152,119,136,9,246,25,230,42,213,58,197,
74,181,90,165,106,149,122,133,13,242,29,226,45,210,61,194,77,178,93,162,109,146,125,130,
0,255,8,247,16,239,24,231,32,223,40,215,48,207,56,199,64,191,72,183,80,175,88,167,96,159,104,151,
112,143,120,135,2,253,10,245,18,237,26,229,35,220,43,212,51,204,59,196,67,188,75,180,83,172,91,164,
99,156,107,148,115,140,123,132,5,250,13,242,21,234,29,226,37,218,45,210,53,202,61,194,70,185,78,177,
86,169,94,161,102,153,110,145,118,137,126,129,
0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,
70,72,74,76,78,80,82,84,86,88,90,92,94,96,98,100,102,104,106,108,110,112,114,116,118,120,122,124,
126,129,131,133,135,137,139,141,143,145,147,149,151,153,155,157,159,161,163,165,167,169,171,173,175,
177,179,181,183,185,187,189,191,193,195,197,199,201,203,205,207,209,211,213,215,217,219,221,223,225,
227,229,231,233,235,237,239,241,243,245,247,249,251,253,255,
0,255,8,247,16,239,24,231,32,223,40,215,48,207,56,199,64,191,72,183,80,175,88,167,96,159,104,151,
112,143,120,135,1,254,9,246,17,238,25,230,33,222,41,214,49,206,57,198,65,190,73,182,81,174,89,166,
97,158,105,150,113,142,121,134,3,252,11,244,19,236,27,228,35,220,43,212,51,204,59,196,67,188,75,180,
83,172,91,164,99,156,107,148,115,140,123,132,4,251,12,243,20,235,28,227,36,219,44,211,52,203,60,195,
68,187,76,179,84,171,92,163,100,155,108,147,116,139,124,131,6,249,14,241,22,233,30,225,38,217,46,
209,54,201,62,193,70,185,78,177,86,169,94,161,102,153,110,145,118,137,126,129,
0,255,4,251,8,247,12,243,16,239,20,235,24,231,28,227,32,223,36,219,40,215,44,211,48,207,52,203,56,
199,60,195,64,191,68,187,72,183,76,179,80,175,84,171,88,167,92,163,96,159,100,155,104,151,108,147,
112,143,116,139,120,135,124,131,1,254,5,250,9,246,13,242,17,238,21,234,25,230,29,226,33,222,37,218,
41,214,45,210,49,206,53,202,57,198,61,194,65,190,69,186,73,182,77,178,81,174,85,170,89,166,93,162,
97,158,101,154,105,150,109,146,113,142,117,138,121,134,125,130,2,253,6,249,10,245,14,241,18,237,22,
233,26,229,30,225,34,221,38,217,42,213,46,209,50,205,54,201,58,197,62,193,66,189,70,185,74,181,78,
177,82,173,86,169,90,165,94,161,98,157,102,153,106,149,110,145,114,141,118,137,122,133,126,129,
};
static const uint16_t cunq_off[16] = {0,6,14,24,36,52,72,96,128,168,216,280,360,456,584,744};
_Static_assert(sizeof wunq_data == 142 && sizeof cunq_data == 936, "unquantization table size");
enum { TAB_BYTES = 28 * 4, TAB_WUNQ = TAB_BYTES, TAB_CUNQ = TAB_BYTES + 144, TAB_SIZE = TAB_CUNQ + 936 };
typedef struct {
uint32_t in_base, out_base, width, height, bw, bh, bx, bslice, srgb;
} Push;
static SRWLOCK g_lock = SRWLOCK_INIT;
static struct {
VkDevice dev; // the device the objects below belong to (probes recreate it)
int mode; // VK_ASTC_*; -1 before the first vk_astc_mode with a device
int failed; // pipeline creation failed on dev: gpu falls back to cpu
VkDescriptorSetLayout dsl;
VkPipelineLayout layout;
VkPipeline pipe;
PFN_vkCmdPushDescriptorSetKHR push;
VkBuffer tab; // device-local copy of the tables
VmaAllocation tab_alloc;
int tab_uploaded;
} G = {.mode = -1};
static uint64_t n_dispatch, n_texels;
// TOTK_ASTC_TIME=1 (or vk_astc_time_enable): timestamps around the dispatches of each scratch chunk (the decode
// alone) and around a whole vk_astc_record call (one texture upload or WriteTexels: copies and barriers included),
// read back by later calls. Per footprint: texels and dispatch time; a new largest call is logged with both times.
// Needs hostQueryReset (a slot is reset on the host after its read).
#define TQ 256
static SRWLOCK t_lock = SRWLOCK_INIT;
static int t_on = -1; // -1: knob not read yet
static VkQueryPool t_pool;
static VkDevice t_dev;
static double t_ns_per_tick;
static uint64_t t_mask;
static struct { uint64_t texels; uint32_t id; uint8_t fp, busy, whole; } t_slot[TQ]; // whole: the call, else a chunk
static int t_next;
static struct { uint64_t texels, chunks; double us; } t_fp[13 * 13]; // [bw * 13 + bh]: dispatch time of the chunks
#define TA 64 // calls in flight, by id % TA
typedef struct { uint32_t id, w, h, regions, chunks, done; int complete; uint64_t texels; double us, total; uint8_t fp; } TCall;
static TCall t_call[TA];
static uint32_t t_id;
static uint64_t t_max_texels;
static double t_max_us, t_max_total;
static uint8_t t_max_fp;
static void time_init(VkCtx *c) { // t_lock held
if (t_on < 0) {
const char *e = cfg_get("TOTK_ASTC_TIME");
t_on = e && *e && *e != '0';
}
if (t_on <= 0 || (t_pool && t_dev == c->device)) return;
t_pool = VK_NULL_HANDLE;
memset(t_slot, 0, sizeof t_slot);
VkPhysicalDeviceVulkan12Features f12 = {.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_2_FEATURES};
VkPhysicalDeviceFeatures2 f2 = {.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2, .pNext = &f12};
vkGetPhysicalDeviceFeatures2(c->phys, &f2);
uint32_t nq = 0;
vkGetPhysicalDeviceQueueFamilyProperties(c->phys, &nq, NULL);
VkQueueFamilyProperties *qp = calloc(nq ? nq : 1, sizeof *qp);
vkGetPhysicalDeviceQueueFamilyProperties(c->phys, &nq, qp);
uint32_t bits = c->gfx_family < nq ? qp[c->gfx_family].timestampValidBits : 0;
free(qp);
VkQueryPoolCreateInfo qi = {.sType = VK_STRUCTURE_TYPE_QUERY_POOL_CREATE_INFO, .queryType = VK_QUERY_TYPE_TIMESTAMP,
.queryCount = 2 * TQ};
if (!f12.hostQueryReset || !bits || vkCreateQueryPool(c->device, &qi, NULL, &t_pool) != VK_SUCCESS) {
rt_log("vk: TOTK_ASTC_TIME: no hostQueryReset or no timestamps on the queue; ASTC decodes not timed");
t_pool = VK_NULL_HANDLE;
t_on = 0;
return;
}
vkResetQueryPool(c->device, t_pool, 0, 2 * TQ);
t_dev = c->device;
t_ns_per_tick = c->props.limits.timestampPeriod;
t_mask = bits >= 64 ? ~0ull : (1ull << bits) - 1;
}
// Takes in the slots whose timestamps are available; t_lock held.
static void time_poll_locked(VkCtx *c) {
if (!t_pool || t_dev != c->device) return;
for (int i = 0; i < TQ; i++) {
if (!t_slot[i].busy) continue;
uint64_t r[4];
if (vkGetQueryPoolResults(c->device, t_pool, 2 * (uint32_t)i, 2, sizeof r, r, 16,
VK_QUERY_RESULT_64_BIT | VK_QUERY_RESULT_WITH_AVAILABILITY_BIT) != VK_SUCCESS ||
!r[1] || !r[3])
continue;
double us = (double)((r[2] - r[0]) & t_mask) * t_ns_per_tick / 1e3;
int fp = t_slot[i].fp;
if (!t_slot[i].whole) t_fp[fp].texels += t_slot[i].texels, t_fp[fp].chunks++, t_fp[fp].us += us;
TCall *k = &t_call[t_slot[i].id % TA];
if (k->id == t_slot[i].id) {
if (t_slot[i].whole) k->total = us;
else k->us += us, k->texels += t_slot[i].texels;
if (++k->done == k->chunks + 1 && k->complete && k->texels > t_max_texels) {
t_max_texels = k->texels, t_max_us = k->us, t_max_total = k->total, t_max_fp = k->fp;
rt_log("vk: ASTC time: largest decode so far %.2f M texels (%dx%d; %u regions, the first %ux%u; %u scratch "
"chunks): dispatches %.0f us of GPU time (%.0f us per M texel), %.0f us with the copies",
(double)k->texels / 1e6, k->fp / 13, k->fp % 13, k->regions, k->w, k->h, k->chunks, k->us,
k->us * 1e6 / (double)k->texels, k->total);
}
}
t_slot[i].busy = 0;
vkResetQueryPool(c->device, t_pool, 2 * (uint32_t)i, 2);
}
}
// A free slot for a call's id (t_lock held); -1 when the ring is full.
static int time_slot(uint32_t id, uint8_t fp, uint64_t texels, int whole) {
for (int i = 0; i < TQ; i++, t_next = (t_next + 1) % TQ)
if (!t_slot[t_next].busy) {
int k = t_next;
t_slot[k].busy = 1, t_slot[k].id = id, t_slot[k].fp = fp, t_slot[k].texels = texels;
t_slot[k].whole = (uint8_t)whole;
t_next = (k + 1) % TQ;
return k;
}
return -1;
}
void vk_astc_time_enable(int on) {
AcquireSRWLockExclusive(&t_lock);
t_on = on;
ReleaseSRWLockExclusive(&t_lock);
}
int vk_astc_time_get(uint32_t bw, uint32_t bh, uint64_t *texels, uint64_t *calls, double *us) {
VkCtx *c = vk_ctx();
if (!c || bw > 12 || bh > 12) return 0;
AcquireSRWLockExclusive(&t_lock);
time_poll_locked(c);
int fp = (int)(bw * 13 + bh);
*texels = t_fp[fp].texels, *calls = t_fp[fp].chunks, *us = t_fp[fp].us;
ReleaseSRWLockExclusive(&t_lock);
return 1;
}
void vk_astc_time_summary(void) {
VkCtx *c = vk_ctx();
if (!c) return;
AcquireSRWLockExclusive(&t_lock);
if (t_on > 0) {
time_poll_locked(c);
for (int fp = 0; fp < 13 * 13; fp++)
if (t_fp[fp].chunks)
rt_log("vk: ASTC time: %dx%d: %llu dispatch batches, %.2f M texels, %.0f us GPU (%.0f us per M texel)",
fp / 13, fp % 13, (unsigned long long)t_fp[fp].chunks, (double)t_fp[fp].texels / 1e6, t_fp[fp].us,
t_fp[fp].us * 1e6 / (double)t_fp[fp].texels);
if (t_max_texels)
rt_log("vk: ASTC time: largest decode %.2f M texels (%dx%d): dispatches %.0f us GPU, %.0f us with the copies",
(double)t_max_texels / 1e6, t_max_fp / 13, t_max_fp % 13, t_max_us, t_max_total);
}
ReleaseSRWLockExclusive(&t_lock);
}
static void fill_tables(uint8_t *p) {
uint32_t off[28];
for (int i = 0; i < 12; i++) off[i] = TAB_WUNQ + wunq_off[i];
for (int i = 0; i < 16; i++) off[12 + i] = TAB_CUNQ + cunq_off[i];
memset(p, 0, TAB_SIZE);
memcpy(p, off, sizeof off);
memcpy(p + TAB_WUNQ, wunq_data, sizeof wunq_data);
memcpy(p + TAB_CUNQ, cunq_data, sizeof cunq_data);
}
// Objects of the current device; under g_lock (exclusive).
static void reset_for(VkCtx *c) {
if (G.dev == c->device) return;
memset(&G.dsl, 0, sizeof G - offsetof(__typeof__(G), dsl)); // the old device's objects went with it
G.dev = c->device;
G.failed = 0;
G.mode = -1;
}
static int create_pipeline(VkCtx *c) {
G.push = (PFN_vkCmdPushDescriptorSetKHR)vkGetDeviceProcAddr(c->device, "vkCmdPushDescriptorSetKHR");
VkDescriptorSetLayoutBinding b[3];
for (int i = 0; i < 3; i++)
b[i] = (VkDescriptorSetLayoutBinding){.binding = (uint32_t)i, .descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.descriptorCount = 1, .stageFlags = VK_SHADER_STAGE_COMPUTE_BIT};
VkDescriptorSetLayoutCreateInfo dci = {.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO,
.flags = VK_DESCRIPTOR_SET_LAYOUT_CREATE_PUSH_DESCRIPTOR_BIT_KHR,
.bindingCount = 3, .pBindings = b};
VkPushConstantRange pr = {VK_SHADER_STAGE_COMPUTE_BIT, 0, sizeof(Push)};
VkShaderModuleCreateInfo mi = {.sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO, .codeSize = sizeof astc_decode_spv,
.pCode = astc_decode_spv};
VkShaderModule m = VK_NULL_HANDLE;
if (!G.push || vkCreateDescriptorSetLayout(c->device, &dci, NULL, &G.dsl) != VK_SUCCESS) return 0;
VkPipelineLayoutCreateInfo li = {.sType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO, .setLayoutCount = 1,
.pSetLayouts = &G.dsl, .pushConstantRangeCount = 1, .pPushConstantRanges = &pr};
if (vkCreatePipelineLayout(c->device, &li, NULL, &G.layout) != VK_SUCCESS) return 0;
if (vkCreateShaderModule(c->device, &mi, NULL, &m) != VK_SUCCESS) return 0;
LARGE_INTEGER t0, t1, fq;
QueryPerformanceCounter(&t0);
VkComputePipelineCreateInfo ci = {.sType = VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO,
.stage = {.sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO,
.stage = VK_SHADER_STAGE_COMPUTE_BIT, .module = m, .pName = "main"},
.layout = G.layout};
VkResult r = vkCreateComputePipelines(c->device, VK_NULL_HANDLE, 1, &ci, NULL, &G.pipe);
vkDestroyShaderModule(c->device, m, NULL);
if (r != VK_SUCCESS) return 0;
VkBufferCreateInfo bi = {.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO, .size = TAB_SIZE,
.usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT};
VmaAllocationCreateInfo aci = {.usage = VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE};
if (vmaCreateBuffer(c->vma, &bi, &aci, &G.tab, &G.tab_alloc, NULL) != VK_SUCCESS) return 0;
QueryPerformanceCounter(&t1);
QueryPerformanceFrequency(&fq);
rt_log("vk: ASTC compute decoder ready (pipeline built in %.1f ms)", (double)(t1.QuadPart - t0.QuadPart) * 1e3 / (double)fq.QuadPart);
return 1;
}
static int native_device(VkCtx *c) { return c->has_astc_ldr; }
static const char *mode_name(int m) {
return m == VK_ASTC_CPU ? "cpu" : m == VK_ASTC_GPU ? "gpu" : m == VK_ASTC_NATIVE ? "native" : "check";
}
int vk_astc_mode(void) {
VkCtx *c = vk_ctx();
if (!c) return VK_ASTC_CPU;
AcquireSRWLockShared(&g_lock);
int m = G.dev == c->device ? G.mode : -1;
ReleaseSRWLockShared(&g_lock);
if (m >= 0) return m;
AcquireSRWLockExclusive(&g_lock);
reset_for(c);
if (G.mode < 0) {
const char *e = cfg_get("TOTK_ASTC");
int want = !e || !*e || !strcmp(e, "auto") ? -1 : !strcmp(e, "cpu") ? VK_ASTC_CPU : !strcmp(e, "gpu") ? VK_ASTC_GPU
: !strcmp(e, "native") ? VK_ASTC_NATIVE : !strcmp(e, "check") ? VK_ASTC_CHECK : -2;
if (want == -2) rt_log("vk: TOTK_ASTC=%s is not cpu, gpu, native, check or auto: auto", e), want = -1;
if ((want == -1 || want == VK_ASTC_NATIVE) && native_device(c)) m = VK_ASTC_NATIVE;
else {
if (want == VK_ASTC_NATIVE) rt_log("vk: TOTK_ASTC=native: the device lacks textureCompressionASTC_LDR; gpu");
m = want == VK_ASTC_CPU ? VK_ASTC_CPU : want == VK_ASTC_CHECK ? VK_ASTC_CHECK : VK_ASTC_GPU;
}
if ((m == VK_ASTC_GPU || m == VK_ASTC_CHECK || m == VK_ASTC_NATIVE) && !G.pipe && !G.failed &&
!create_pipeline(c)) {
G.failed = 1;
rt_log("vk: ASTC compute decoder unavailable on %s: ASTC decodes on the CPU", c->props.deviceName);
}
if (G.failed && m != VK_ASTC_NATIVE) m = VK_ASTC_CPU;
G.mode = m;
rt_log("vk: ASTC textures: %s (textureCompressionASTC_LDR %d)", mode_name(m), native_device(c));
}
m = G.mode;
ReleaseSRWLockExclusive(&g_lock);
return m;
}
int vk_astc_native(VkFormat astc_format) {
VkCtx *c = vk_ctx();
if (!c || vk_astc_mode() != VK_ASTC_NATIVE) return 0;
VkFormatProperties p;
vkGetPhysicalDeviceFormatProperties(c->phys, astc_format, &p);
VkFormatFeatureFlags need = VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT | VK_FORMAT_FEATURE_TRANSFER_DST_BIT |
VK_FORMAT_FEATURE_TRANSFER_SRC_BIT;
return (p.optimalTilingFeatures & need) == need;
}
int vk_astc_gpu(void) {
int m = vk_astc_mode();
if (m == VK_ASTC_GPU || m == VK_ASTC_CHECK) return 1;
if (m != VK_ASTC_NATIVE) return 0;
AcquireSRWLockShared(&g_lock); // native mode, a format the device lacks: the decoder when it exists
int ok = G.pipe != VK_NULL_HANDLE;
ReleaseSRWLockShared(&g_lock);
return ok;
}
static void mem_barrier(VkCommandBuffer cb, VkPipelineStageFlags2 src, VkAccessFlags2 sa, VkPipelineStageFlags2 dst,
VkAccessFlags2 da) {
VkMemoryBarrier2 mb = {.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER_2, .srcStageMask = src, .srcAccessMask = sa,
.dstStageMask = dst, .dstAccessMask = da};
VkDependencyInfo di = {.sType = VK_STRUCTURE_TYPE_DEPENDENCY_INFO, .memoryBarrierCount = 1, .pMemoryBarriers = &mb};
vkCmdPipelineBarrier2(cb, &di);
}
// Scratch per vk_astc_record: texels of at most this many bytes at a time (or one region, when larger), so a large
// texture array (169 M texels, 677 MB as RGBA8, on monsters2) does not need its whole RGBA8 copy as scratch.
#define CHUNK_OUT (32ull << 20)
static VkDeviceSize region_out(const VkAstcRegion *r) { return (VkDeviceSize)r->w * r->h * r->slices * 4; }
static VkDeviceSize region_in(const VkAstcRegion *r, uint32_t bw, uint32_t bh) {
return (VkDeviceSize)((r->w + bw - 1) / bw) * ((r->h + bh - 1) / bh) * r->slices * 16;
}
int vk_astc_record(VkCommandBuffer cb, VkBuffer src, VkDeviceSize src_size, const VkAstcRegion *rg, int n, uint32_t bw,
uint32_t bh, int srgb, VkImage img, VkImageAspectFlags aspect, VkBuffer readback) {
static volatile long n_fail;
(void)src_size;
VkCtx *c = vk_ctx();
if (!c || !vk_astc_gpu() || bw < 4 || bh < 4 || bw > 12 || bh > 12) return 0;
AcquireSRWLockShared(&g_lock);
VkPipeline pipe = G.dev == c->device ? G.pipe : VK_NULL_HANDLE;
ReleaseSRWLockShared(&g_lock);
if (!pipe) return 0;
// chunks of consecutive regions; scratch = the largest chunk's blocks, then its texels
VkDeviceSize out_total = 0, cap_in = 0, cap_out = 0, ci = 0, co = 0;
for (int i = 0; i < n; i++) {
VkDeviceSize o = region_out(&rg[i]), b = region_in(&rg[i], bw, bh);
if (co && co + o > CHUNK_OUT) ci = co = 0;
ci += b, co += o, out_total += o;
if (ci > cap_in) cap_in = ci;
if (co > cap_out) cap_out = co;
}
if (!out_total) return 1;
VkDeviceSize al = c->props.limits.minStorageBufferOffsetAlignment; // the texels bind at cap_in
if (al > 1) cap_in = (cap_in + al - 1) / al * al;
VkBufferCreateInfo bi = {.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO, .size = cap_in + cap_out,
.usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT |
VK_BUFFER_USAGE_TRANSFER_DST_BIT};
VmaAllocationCreateInfo aci = {.usage = VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE};
VkBuffer sb;
VmaAllocation sa;
if (vmaCreateBuffer(c->vma, &bi, &aci, &sb, &sa, NULL) != VK_SUCCESS) {
if (InterlockedIncrement(&n_fail) <= 16)
rt_log("vk: ASTC scratch buffer of %llu bytes failed: texture left undecoded",
(unsigned long long)(cap_in + cap_out));
return 0;
}
AcquireSRWLockExclusive(&g_lock);
int need_tab = !G.tab_uploaded;
G.tab_uploaded = 1;
VkBuffer tab = G.tab;
ReleaseSRWLockExclusive(&g_lock);
if (need_tab) { // first use on this device: the tables go up in this batch, before the dispatch
VkBuffer tb;
uint8_t *p = vk_staging(TAB_SIZE, &tb, NULL);
fill_tables(p);
VkBufferCopy cp = {0, 0, TAB_SIZE};
vkCmdCopyBuffer(cb, tb, tab, 1, &cp);
}
int timed = 0, whole = -1;
uint32_t id = 0;
uint8_t fp = (uint8_t)(bw * 13 + bh);
AcquireSRWLockExclusive(&t_lock);
time_init(c);
if (t_on > 0 && t_pool) {
time_poll_locked(c);
timed = 1;
id = ++t_id;
t_call[id % TA] = (TCall){.id = id, .w = rg[0].w, .h = rg[0].h, .regions = (uint32_t)n, .fp = fp, .complete = 1};
whole = time_slot(id, fp, out_total / 4, 1);
if (whole < 0) t_call[id % TA].complete = 0;
}
VkQueryPool tpool = t_pool;
ReleaseSRWLockExclusive(&t_lock);
if (whole >= 0) vkCmdWriteTimestamp2(cb, VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT, tpool, 2 * (uint32_t)whole);
VkDescriptorBufferInfo dbi[3] = {{sb, 0, cap_in}, {sb, cap_in, cap_out}, {tab, 0, TAB_SIZE}};
VkWriteDescriptorSet w[3];
for (int i = 0; i < 3; i++)
w[i] = (VkWriteDescriptorSet){.sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET, .dstBinding = (uint32_t)i,
.descriptorCount = 1, .descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.pBufferInfo = &dbi[i]};
VkBufferImageCopy *copies = calloc((size_t)n, sizeof *copies);
VkBufferCopy *bcp = calloc((size_t)n, sizeof *bcp);
if (!copies || !bcp) rt_fatal("vk: out of memory (ASTC regions)");
VkDeviceSize rb_off = 0;
int chunks = 0;
for (int i0 = 0; i0 < n; chunks++) {
int i1 = i0;
VkDeviceSize in = 0, out = 0;
for (; i1 < n && (i1 == i0 || out + region_out(&rg[i1]) <= CHUNK_OUT); i1++) {
bcp[i1 - i0] = (VkBufferCopy){rg[i1].src, in, region_in(&rg[i1], bw, bh)};
in += region_in(&rg[i1], bw, bh), out += region_out(&rg[i1]);
}
if (chunks) // the previous chunk's dispatches and image copy are done with the scratch
mem_barrier(cb, VK_PIPELINE_STAGE_2_COPY_BIT | VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT,
VK_ACCESS_2_TRANSFER_READ_BIT | VK_ACCESS_2_SHADER_STORAGE_READ_BIT,
VK_PIPELINE_STAGE_2_COPY_BIT | VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT,
VK_ACCESS_2_TRANSFER_WRITE_BIT | VK_ACCESS_2_SHADER_STORAGE_WRITE_BIT);
vkCmdCopyBuffer(cb, src, sb, (uint32_t)(i1 - i0), bcp);
mem_barrier(cb, VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_WRITE_BIT, VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT,
VK_ACCESS_2_SHADER_STORAGE_READ_BIT);
int slot = -1; // the timestamps bracket this chunk's dispatches: the first waits for the blocks copy
if (timed) {
AcquireSRWLockExclusive(&t_lock);
slot = time_slot(id, fp, out / 4, 0);
if (slot < 0) t_call[id % TA].complete = 0;
ReleaseSRWLockExclusive(&t_lock);
}
if (slot >= 0) vkCmdWriteTimestamp2(cb, VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT, tpool, 2 * (uint32_t)slot);
vkCmdBindPipeline(cb, VK_PIPELINE_BIND_POINT_COMPUTE, pipe);
G.push(cb, VK_PIPELINE_BIND_POINT_COMPUTE, G.layout, 0, 3, w);
VkDeviceSize ib = 0, ob = 0;
for (int i = i0; i < i1; i++) {
const VkAstcRegion *r = &rg[i];
uint32_t bx = (r->w + bw - 1) / bw, by = (r->h + bh - 1) / bh;
Push pc = {.in_base = (uint32_t)(ib / 16), .out_base = (uint32_t)(ob / 4), .width = r->w, .height = r->h,
.bw = bw, .bh = bh, .bx = bx, .bslice = bx * by, .srgb = (uint32_t)(srgb != 0)};
vkCmdPushConstants(cb, G.layout, VK_SHADER_STAGE_COMPUTE_BIT, 0, sizeof pc, &pc);
uint32_t bpw = bw * bh <= 64 ? 64 / (bw * bh) : 1; // blocks per workgroup, as astc_decode.comp main
vkCmdDispatch(cb, (bx + bpw - 1) / bpw, by, r->slices);
copies[i - i0] = (VkBufferImageCopy){.bufferOffset = cap_in + ob, .bufferRowLength = r->w,
.bufferImageHeight = r->h, .imageSubresource = {aspect, r->level, r->layer, 1},
.imageOffset = r->offset, .imageExtent = {r->w, r->h, r->slices}};
ib += region_in(r, bw, bh), ob += region_out(r);
n_texels += (uint64_t)r->w * r->h * r->slices;
}
if (slot >= 0)
vkCmdWriteTimestamp2(cb, VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT, tpool, 2 * (uint32_t)slot + 1);
mem_barrier(cb, VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT, VK_ACCESS_2_SHADER_STORAGE_WRITE_BIT,
VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_READ_BIT);
vkCmdCopyBufferToImage(cb, sb, img, VK_IMAGE_LAYOUT_GENERAL, (uint32_t)(i1 - i0), copies);
if (readback) {
VkBufferCopy rc = {cap_in, rb_off, out};
vkCmdCopyBuffer(cb, sb, readback, 1, &rc);
rb_off += out;
}
i0 = i1;
}
if (readback)
mem_barrier(cb, VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_WRITE_BIT, VK_PIPELINE_STAGE_2_HOST_BIT,
VK_ACCESS_2_HOST_READ_BIT);
if (whole >= 0) vkCmdWriteTimestamp2(cb, VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT, tpool, 2 * (uint32_t)whole + 1);
if (timed) {
AcquireSRWLockExclusive(&t_lock);
t_call[id % TA].chunks = (uint32_t)chunks;
ReleaseSRWLockExclusive(&t_lock);
}
if ((n_dispatch + (uint64_t)n) / 1024 != n_dispatch / 1024 || !n_dispatch) {
rt_log("vk: ASTC: %llu regions, %.1f M texels decoded on the GPU so far",
(unsigned long long)(n_dispatch + (uint64_t)n), (double)n_texels / 1e6);
vk_astc_time_summary();
}
n_dispatch += (uint64_t)n;
free(copies), free(bcp);
vk_defer_buffer(sb, sa);
return 1;
}
void vk_astc_release(void) {
VkCtx *c = vk_ctx();
vk_astc_time_summary();
AcquireSRWLockExclusive(&t_lock);
if (c && t_pool && t_dev == c->device) vkDestroyQueryPool(c->device, t_pool, NULL);
t_pool = VK_NULL_HANDLE, t_dev = VK_NULL_HANDLE;
ReleaseSRWLockExclusive(&t_lock);
AcquireSRWLockExclusive(&g_lock);
if (c && G.dev == c->device) {
if (G.pipe) vkDestroyPipeline(c->device, G.pipe, NULL);
if (G.layout) vkDestroyPipelineLayout(c->device, G.layout, NULL);
if (G.dsl) vkDestroyDescriptorSetLayout(c->device, G.dsl, NULL);
if (G.tab) vmaDestroyBuffer(c->vma, G.tab, G.tab_alloc);
}
memset(&G.dsl, 0, sizeof G - offsetof(__typeof__(G), dsl));
G.dev = VK_NULL_HANDLE;
G.mode = -1;
G.failed = 0;
ReleaseSRWLockExclusive(&g_lock);
}
void vk_astc_stats(uint64_t *dispatches, uint64_t *texels) {
*dispatches = n_dispatch;
*texels = n_texels;
}
+2
View File
@@ -269,6 +269,7 @@ static int create_device(void) {
else if (!h->textureCompressionBC) miss = "textureCompressionBC";
if (miss) { rt_log("vk: required feature missing on %s: %s", g_vk.props.deviceName, miss); free(c); return 0; }
g_vk.has_buffer_device_address = c->f12.bufferDeviceAddress;
g_vk.has_astc_ldr = h->textureCompressionASTC_LDR;
{
const char *e = cfg_get("TOTK_FEEDBACK");
int layout_ok = has_fbl && has_lr && c->fbl.attachmentFeedbackLoopLayout && c->lr.dynamicRenderingLocalRead;
@@ -326,6 +327,7 @@ static long g_errors_total; // outlives the device so the probe can print it
static void destroy_all(void) {
gfx_window_release_vk();
if (g_vk.device) vkDeviceWaitIdle(g_vk.device);
if (g_vk.device) vk_astc_release();
if (g_vk.timeline) vkDestroySemaphore(g_vk.device, g_vk.timeline, NULL);
if (g_vk.vma) vmaDestroyAllocator(g_vk.vma);
if (g_vk.device) vkDestroyDevice(g_vk.device, NULL);
+9 -4
View File
@@ -1,8 +1,8 @@
// NVN format -> VkFormat for the textures of the M4 graphics layer.
// Ported from the format table of LAS Recomp runtime/nvn/vulkan/textures.c @ da35b83a (MIT, Copyright (c) 2026 Peiikai).
// Differences from LAS: the table is indexed by the NVNformat values of nvn.h and covers all of them, ASTC is always
// decoded on the CPU (tex_astc.c) into RGBA8, D24S8 is always D32S8 (no probing: the RX 9070 lacks it), 3-component
// formats expand to 4 components, and formats with no Vulkan equivalent are reported with a reason.
// Differences from LAS: the table is indexed by the NVNformat values of nvn.h and covers all of them, ASTC decodes to
// RGBA8 unless the device samples it (vk_astc.c, TOTK_ASTC), D24S8 is always D32S8 (no probing: the RX 9070 lacks
// it), 3-component formats expand to 4 components, and formats with no Vulkan equivalent are reported with a reason.
#include <string.h>
#include "tex.h"
@@ -99,11 +99,16 @@ int tex_vk_format(int nvn_format, TexVkFormat *o) {
unsigned i = (f - 0x79) % 14;
int srgb = f >= 0x87;
o->ideal = srgb ? astc[i].srgb : astc[i].unorm;
if (vk_astc_native(o->ideal)) { // TOTK_ASTC auto / native on a device that samples ASTC: raw blocks, like BC
o->vk = o->ideal;
o->note = "native ASTC";
return 1;
}
o->vk = srgb ? VK_FORMAT_R8G8B8A8_SRGB : VK_FORMAT_R8G8B8A8_UNORM;
o->astc_bw = astc[i].bw;
o->astc_bh = astc[i].bh;
o->astc_srgb = (uint8_t)srgb;
o->note = "the RX 9070 has no ASTC: decoded on the CPU to RGBA8";
o->note = "no native ASTC: decoded to RGBA8 (vk_astc.c compute decoder, or tex_astc.c with TOTK_ASTC=cpu)";
return 1;
}
if ((f >= 0x51 && f <= 0x68) || f == 0x6B || f == 0x6C) {
+1
View File
@@ -229,6 +229,7 @@ void vk_sync_start(void) {
completion_thread = CreateThread(NULL, 0, completion_main, c, 0, NULL);
if (!completion_thread) rt_fatal("vk: cannot start the completion thread (error %lu)", GetLastError());
__atomic_store_n(&started, 1, __ATOMIC_RELEASE);
vk_astc_mode(); // decides TOTK_ASTC and builds the ASTC decoder now, not at the first ASTC upload
}
ReleaseSRWLockExclusive(&start_lock);
}
+99 -14
View File
@@ -1,5 +1,5 @@
// Vulkan textures of the M4 graphics layer: a VkImage cache keyed by storage
// GPU address plus descriptor, uploads through staging buffers with block-linear deswizzle and CPU ASTC decode,
// GPU address plus descriptor, uploads through staging buffers with block-linear deswizzle and ASTC decode (vk_astc.c),
// re-uploads when page write generations (vk_memory.c) show the storage written, ReadTexels / WriteTexels, the texture
// and sampler descriptor slots, and VkSamplers.
//
@@ -467,9 +467,49 @@ static void shrink_rgb(int format, const uint8_t *src, uint8_t *dst, size_t n) {
for (size_t i = 0; i < n; i++, src += 4 * e, dst += 3 * e) memcpy(dst, src, 3 * e);
}
// TOTK_ASTC=check: the compute decoder's texels of n regions (gpu: the regions one after the other) against
// astc_decode_ldr of the same raw blocks (raw: the staging bytes the regions point into).
static void astc_check(const Tex *e, const uint8_t *raw, const VkAstcRegion *ar, int n, const uint8_t *gpu) {
static volatile long n_bad;
static uint64_t n_tex, n_texels, n_diff_tex;
uint32_t bw = e->f.astc_bw, bh = e->f.astc_bh;
uint64_t diff = 0, first = UINT64_MAX;
uint32_t want = 0, got = 0;
int first_k = 0;
for (int k = 0; k < n; k++) {
const VkAstcRegion *r = &ar[k];
size_t sb = (size_t)((r->w + bw - 1) / bw) * ((r->h + bh - 1) / bh) * 16, st = (size_t)r->w * r->h * 4;
uint8_t *cpu = malloc(st);
for (uint32_t z = 0; z < r->slices; z++, gpu += st) {
astc_decode_ldr(raw + r->src + z * sb, r->w, r->h, bw, bh, e->f.astc_srgb, cpu);
if (!memcmp(cpu, gpu, st)) continue;
for (size_t i = 0; i < st / 4; i++)
if (memcmp(cpu + 4 * i, gpu + 4 * i, 4)) {
if (first == UINT64_MAX) {
first = i, first_k = k;
memcpy(&want, cpu + 4 * i, 4), memcpy(&got, gpu + 4 * i, 4);
}
diff++;
}
}
free(cpu);
n_texels += (uint64_t)r->w * r->h * r->slices;
}
n_tex++;
if (diff) {
n_diff_tex++;
log_limited(&n_bad, "ASTC check: texture %#llx fmt %#x %ux%u: %llu texels differ, first region %d texel %llu "
"cpu %08x gpu %08x", (unsigned long long)e->pub.addr, e->t.d.format, bw, bh,
(unsigned long long)diff, first_k, (unsigned long long)first, want, got);
}
if (n_tex == 1 || !(n_tex & 63) || diff)
rt_log("vk: ASTC check: %llu textures, %llu texels compared, %llu textures differ", (unsigned long long)n_tex,
(unsigned long long)n_texels, (unsigned long long)n_diff_tex);
}
// Copies the whole storage into the image (every level and layer the image has).
static void upload(Tex *e) {
static volatile long n_bad;
static volatile long n_bad, n_undecoded;
const tex_t *t = &e->t;
const nvn_texdesc *d = &t->d;
int bw, bh, bpb = nvn_fmt_bpb(d->format, &bw, &bh);
@@ -478,34 +518,40 @@ static void upload(Tex *e) {
int linear = nvn_tex_is_linear(d), is3d = d->target == NVN_TEXTURE_TARGET_3D, one_d = e->pub.type == VK_IMAGE_TYPE_1D;
uint32_t levels = linear ? 1 : e->pub.levels, layers = linear ? 1 : e->pub.layers;
int astc = e->f.astc_bw != 0, rgb = e->f.expand_rgb;
int agpu = astc && vk_astc_gpu(), acheck = agpu && vk_astc_mode() == VK_ASTC_CHECK; // raw blocks to the GPU
uint32_t n = levels * layers, k = 0;
VkBufferImageCopy *rg = calloc(n, sizeof *rg);
VkDeviceSize total = 0;
VkAstcRegion *ar = agpu ? calloc(n, sizeof *ar) : NULL;
VkDeviceSize total = 0, decoded = 0;
for (uint32_t lv = 0; lv < levels; lv++)
for (uint32_t ly = 0; ly < layers; ly++) {
uint32_t w = mip(d->width, lv), h = one_d ? 1 : mip(d->height, lv), sl = is3d ? mip(d->depth, lv) : 1;
uint64_t texels = (uint64_t)w * h * sl, raw = (uint64_t)(w + bw - 1) / bw * bpb * ((h + bh - 1) / bh) * sl;
uint64_t out = astc ? texels * 4 : rgb ? texels * (uint64_t)bpb / 3 * 4 : raw;
uint64_t out = agpu ? raw : astc ? texels * 4 : rgb ? texels * (uint64_t)bpb / 3 * 4 : raw;
rg[k++] = (VkBufferImageCopy){.bufferOffset = total, .imageSubresource = {e->pub.aspect, lv, ly, 1},
.imageExtent = {w, h, sl}};
total += (out + 15) & ~15ull;
decoded += texels * 4;
}
VkCommandBuffer cb = vk_upload_begin();
VkBuffer sb;
uint8_t *stg = vk_staging(total, &sb, NULL);
VkBuffer sb, rbuf = VK_NULL_HANDLE;
VmaAllocation ralloc = NULL;
uint8_t *stg = vk_staging(total, &sb, NULL), *rmap = acheck ? vk_staging(decoded, &rbuf, &ralloc) : NULL;
k = 0;
for (uint32_t lv = 0; lv < levels; lv++)
for (uint32_t ly = 0; ly < layers; ly++, k++) {
uint32_t w = rg[k].imageExtent.width, h = rg[k].imageExtent.height, sl = rg[k].imageExtent.depth;
uint32_t rb = (w + bw - 1) / bw * (uint32_t)bpb, rows = (h + bh - 1) / bh;
size_t raw = (size_t)rb * rows * sl;
uint8_t *dst = stg + rg[k].bufferOffset, *tmp = astc || rgb ? malloc(raw) : NULL;
uint8_t *dst = stg + rg[k].bufferOffset, *tmp = (astc && !agpu) || rgb ? malloc(raw) : NULL;
if (nvn_deswizzle(d, lv, ly, t->mem, tmp ? tmp : dst, rb)) {
log_limited(&n_bad, "texture %#llx fmt %#x target %d: level %u layer %u has no CPU layout; uploaded as zeros",
(unsigned long long)e->pub.addr, d->format, d->target, lv, ly);
memset(tmp ? tmp : dst, 0, raw);
}
if (astc) {
if (agpu) {
ar[k] = (VkAstcRegion){.src = rg[k].bufferOffset, .w = w, .h = h, .slices = sl, .level = lv, .layer = ly};
} else if (astc) {
uint64_t a0 = perf_begin();
for (uint32_t z = 0; z < sl; z++)
astc_decode_ldr_mt(tmp + (size_t)z * rb * rows, w, h, e->f.astc_bw, e->f.astc_bh, e->f.astc_srgb,
@@ -520,10 +566,33 @@ static void upload(Tex *e) {
}
free(tmp);
}
uint8_t *raw_copy = acheck ? malloc(total) : NULL;
if (raw_copy) memcpy(raw_copy, stg, total);
barrier(cb, ALL_RW, VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_WRITE_BIT);
vkCmdCopyBufferToImage(cb, sb, e->pub.image, VK_IMAGE_LAYOUT_GENERAL, n, rg);
int checked = 0;
if (agpu) {
uint64_t a0 = perf_begin();
checked = vk_astc_record(cb, sb, total, ar, (int)n, e->f.astc_bw, e->f.astc_bh, e->f.astc_srgb, e->pub.image,
e->pub.aspect, rbuf);
perf_end(PERF_ASTC, a0);
if (!checked)
log_limited(&n_undecoded, "ASTC texture %#llx (fmt %#x %ux%u) not decoded on the GPU: left as it was",
(unsigned long long)e->pub.addr, d->format, d->width, d->height);
} else {
vkCmdCopyBufferToImage(cb, sb, e->pub.image, VK_IMAGE_LAYOUT_GENERAL, n, rg);
}
barrier(cb, VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_WRITE_BIT, ALL_RW);
vk_upload_end();
if (acheck) { // waits for the GPU
vk_upload_flush(1);
if (checked) {
vmaInvalidateAllocation(vk_ctx()->vma, ralloc, 0, VK_WHOLE_SIZE);
astc_check(e, raw_copy, ar, (int)n, rmap);
}
vmaDestroyBuffer(vk_ctx()->vma, rbuf, ralloc);
}
free(raw_copy);
free(ar);
free(rg);
if (e->uploaded) n_reuploads++, perf_count(PERF_CNT_TEX_REUP);
else n_uploads++, perf_count(PERF_CNT_TEX_UP);
@@ -973,9 +1042,9 @@ static int image_texels(Tex *e, const Region *g, const NVNcopyRegion *r, uint8_t
}
if (g->level >= e->pub.levels) return 0;
uint32_t w = (uint32_t)r->width, h = (uint32_t)r->height;
int rgb = e->f.expand_rgb, astc = e->f.astc_bw != 0;
size_t texels = (size_t)w * h, ib = astc ? 4 : rgb ? (size_t)g->bpb / 3 * 4 : 0;
size_t slice = ib ? texels * ib : g->wb * g->hb;
int rgb = e->f.expand_rgb, astc = e->f.astc_bw != 0, agpu = astc && write && vk_astc_gpu();
size_t texels = (size_t)w * h, ib = agpu ? 0 : astc ? 4 : rgb ? (size_t)g->bpb / 3 * 4 : 0;
size_t slice = ib ? texels * ib : g->wb * g->hb; // agpu: the raw blocks, decoded by vk_astc_record
VkBuffer sb;
VmaAllocation sa;
uint8_t *stg = vk_staging(slice * (size_t)g->nz, &sb, &sa);
@@ -984,7 +1053,9 @@ static int image_texels(Tex *e, const Region *g, const NVNcopyRegion *r, uint8_t
uint8_t *s = stg + slice * (size_t)z;
const uint8_t *m = mem + (size_t)z * g->img;
if (write) {
if (astc) {
if (agpu) {
for (size_t y = 0; y < g->hb; y++) memcpy(s + y * g->wb, m + y * g->row, g->wb);
} else if (astc) {
uint8_t *tight = malloc(g->wb * g->hb);
for (size_t y = 0; y < g->hb; y++) memcpy(tight + y * g->wb, m + y * g->row, g->wb);
uint64_t a0 = perf_begin();
@@ -1006,7 +1077,21 @@ static int image_texels(Tex *e, const Region *g, const NVNcopyRegion *r, uint8_t
if (write) {
vmaFlushAllocation(c->vma, sa, 0, VK_WHOLE_SIZE);
barrier(cb, ALL_RW, VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_WRITE_BIT);
vkCmdCopyBufferToImage(cb, sb, e->pub.image, VK_IMAGE_LAYOUT_GENERAL, (uint32_t)g->nz, rg);
int done = 0;
if (agpu) {
VkAstcRegion *ar = calloc((size_t)g->nz, sizeof *ar);
for (int z = 0; z < g->nz; z++)
ar[z] = (VkAstcRegion){.src = slice * (size_t)z, .w = w, .h = h, .slices = 1, .level = rg[z].imageSubresource.mipLevel,
.layer = rg[z].imageSubresource.baseArrayLayer, .offset = rg[z].imageOffset};
uint64_t a0 = perf_begin();
done = vk_astc_record(cb, sb, slice * (size_t)g->nz, ar, g->nz, e->f.astc_bw, e->f.astc_bh, e->f.astc_srgb,
e->pub.image, aspect, VK_NULL_HANDLE);
perf_end(PERF_ASTC, a0);
free(ar);
}
if (!agpu) vkCmdCopyBufferToImage(cb, sb, e->pub.image, VK_IMAGE_LAYOUT_GENERAL, (uint32_t)g->nz, rg);
else if (!done) log_limited(&n_astc, "WriteTexels of ASTC texture %#llx not decoded on the GPU: left as it was",
(unsigned long long)e->pub.addr);
barrier(cb, VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_WRITE_BIT, ALL_RW);
} else {
barrier(cb, ALL_RW, VK_PIPELINE_STAGE_2_COPY_BIT, VK_ACCESS_2_TRANSFER_READ_BIT);
+4
View File
@@ -61,6 +61,7 @@ import time
from pathlib import Path
import romfs_index
from build_shaders import stale as stale_shaders
from check_content import GENERATED_MARKER
ROOT = Path(__file__).resolve().parent.parent
@@ -647,6 +648,9 @@ def main():
print(f'build_game: warning: no {paths["romfs"] / "index.json"}; totk.exe will have no RomFS', file=sys.stderr)
global RUNTIME
RUNTIME = (args.runtime or RUNTIME).resolve()
stale = stale_shaders(RUNTIME)
if stale:
sys.exit(f'{stale[0]}: its SPIR-V header no longer matches; run python tools/build_shaders.py')
skip = {s for s in args.skip_runtime.split(',') if s}
main_out = (ROOT / 'build' / args.version).resolve()
pairs = None
+82
View File
@@ -0,0 +1,82 @@
"""Compile the runtime's own GLSL shaders (runtime/shaders/*.comp) into C headers of SPIR-V words.
Each runtime/shaders/<name>.comp becomes runtime/<name>_spv.h, which declares `static const uint32_t <name>_spv[]`
and records the SHA-256 of the source it came from. The headers are committed, so building totk.exe needs no shader
compiler; tools/build_game.py refuses to build when a header no longer matches its source. Run this after editing a
shader. Needs glslc and spirv-val from MSYS2 UCRT64 (package mingw-w64-ucrt-x86_64-shaderc, spirv-tools).
python tools/build_shaders.py [--check]
"""
import argparse
import hashlib
import subprocess
import sys
import tempfile
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
BIN = Path(r'C:\msys64\ucrt64\bin')
TARGET = 'vulkan1.1'
def source_hash(src):
return hashlib.sha256(src.read_bytes().replace(b'\r\n', b'\n')).hexdigest() # the same on a CRLF checkout
def header_of(src, words):
name = src.stem
lines = [f'// Generated by tools/build_shaders.py from runtime/shaders/{src.name}; do not edit.',
f'// source sha256 {source_hash(src)}',
'#pragma once',
'#include <stdint.h>',
f'static const uint32_t {name}_spv[] = {{']
for i in range(0, len(words), 8):
lines.append(' ' + ', '.join(f'0x{w:08x}' for w in words[i:i + 8]) + ',')
lines.append('};')
return '\n'.join(lines) + '\n'
def header_path(src):
return src.parent.parent / f'{src.stem}_spv.h'
def stale(runtime):
"""Shader sources under runtime/shaders whose header is missing or records another source hash."""
out = []
for src in sorted((runtime / 'shaders').glob('*.comp')):
h = header_path(src)
if not h.exists() or f'// source sha256 {source_hash(src)}' not in h.read_text(encoding='utf-8'):
out.append(src)
return out
def compile_one(src):
with tempfile.TemporaryDirectory() as tmp:
spv = Path(tmp) / 'out.spv'
subprocess.run([str(BIN / 'glslc.exe'), '-O', f'--target-env={TARGET}', '-fshader-stage=compute',
'-o', str(spv), str(src)], check=True)
subprocess.run([str(BIN / 'spirv-val.exe'), f'--target-env', TARGET, str(spv)], check=True)
data = spv.read_bytes()
words = [int.from_bytes(data[i:i + 4], 'little') for i in range(0, len(data), 4)]
header_path(src).write_text(header_of(src, words), encoding='utf-8', newline='\n')
print(f'{header_path(src).relative_to(ROOT)}: {len(words)} words')
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument('--check', action='store_true', help='only report headers that do not match their source')
ap.add_argument('--runtime', type=Path, default=ROOT / 'runtime')
args = ap.parse_args()
if args.check:
bad = stale(args.runtime)
for s in bad:
print(f'{header_path(s)}: stale, run tools/build_shaders.py')
return 1 if bad else 0
for src in sorted((args.runtime / 'shaders').glob('*.comp')):
compile_one(src)
return 0
if __name__ == '__main__':
sys.exit(main())