From 9035dd1dc230a696e408745eef20e28328bdf8e2 Mon Sep 17 00:00:00 2001 From: The412Banner Date: Mon, 12 Oct 2026 23:41:52 -0400 Subject: [PATCH] Mali (Venus): David Cimo's Venus route sources and recipes From DavidCimo/droiddeck-mali, branch mali-venus @ 4f838891 (prototype 13; his 12 commits on top of DroidDeck 0.3.1, GPL-3.0), by David Cimo: - tools/venus: Mesa Venus vtest patch (incl. the per-queue submit_mutex fix and the cached-memory frame readback), virglrenderer Android patch, the Mali compatibility layer (the six features DXVK needs: fill modes, multi-viewport, robustness2, clip/cull distance, BC1-BC7) and his tools. - tools/linuxfs/preload/venus.c: keeps Zink's kopper under Venus. - tools/gamescope/patches/0127 (his 0114): shm buffers uploaded on every commit. - docs/development/mali-venus.md: his design notes. - tools/venus/ci-build.sh and build-venus.yml: his build recipes as one CI job that packs the component (vtest server, ICD, layer) as venus.tzst for the apk. Co-authored-by: Dersim Abbas --- .github/workflows/build-venus.yml | 31 + docs/development/mali-venus.md | 340 +++++++ tools/gamescope/PATCHES.md | 6 + ...0127-shm-buffers-upload-every-commit.patch | 44 + tools/linuxfs/preload/venus.c | 18 + tools/venus/addr2line.sh | 14 + tools/venus/ci-build.sh | 144 +++ tools/venus/guest-vulkaninfo.sh | 15 + .../layer/VkLayer_droiddeck_mali_compat.json | 17 + tools/venus/layer/bc.c | 869 ++++++++++++++++++ tools/venus/layer/layer.c | 617 +++++++++++++ tools/venus/layer/mali_compat.h | 246 +++++ tools/venus/layer/pipeline.c | 281 ++++++ tools/venus/layer/spirv.c | 833 +++++++++++++++++ tools/venus/measure-present.patch | 153 +++ tools/venus/memtypes.c | 40 + tools/venus/mesa-venus-vtest.patch | 541 +++++++++++ tools/venus/virglrenderer-android.patch | 322 +++++++ tools/venus/watch-maps.sh | 19 + 19 files changed, 4550 insertions(+) create mode 100644 .github/workflows/build-venus.yml create mode 100644 docs/development/mali-venus.md create mode 100644 tools/gamescope/patches/0127-shm-buffers-upload-every-commit.patch create mode 100644 tools/linuxfs/preload/venus.c create mode 100644 tools/venus/addr2line.sh create mode 100755 tools/venus/ci-build.sh create mode 100644 tools/venus/guest-vulkaninfo.sh create mode 100644 tools/venus/layer/VkLayer_droiddeck_mali_compat.json create mode 100644 tools/venus/layer/bc.c create mode 100644 tools/venus/layer/layer.c create mode 100644 tools/venus/layer/mali_compat.h create mode 100644 tools/venus/layer/pipeline.c create mode 100644 tools/venus/layer/spirv.c create mode 100644 tools/venus/measure-present.patch create mode 100644 tools/venus/memtypes.c create mode 100644 tools/venus/mesa-venus-vtest.patch create mode 100644 tools/venus/virglrenderer-android.patch create mode 100644 tools/venus/watch-maps.sh diff --git a/.github/workflows/build-venus.yml b/.github/workflows/build-venus.yml new file mode 100644 index 000000000..d355bdcd7 --- /dev/null +++ b/.github/workflows/build-venus.yml @@ -0,0 +1,31 @@ +name: Build Venus (Mali) +# The Mali (Venus) route's binaries from tools/venus (David Cimo's recipes): Mesa's Venus ICD and the +# Mali compatibility layer (glibc, for the session) and virglrenderer's vtest server (bionic, run by +# the app over the phone's own Vulkan driver), packed as venus.tzst rooted at app/src/main. The apk +# build fetches it by tools/venus/release.env (tag + sha256); nothing compiled is kept in this repo. +on: + workflow_dispatch: + push: + branches: [test/non-adreno-all] + paths: ['tools/venus/**', '.github/workflows/build-venus.yml'] +permissions: + contents: read +jobs: + build: + runs-on: ubuntu-24.04 + timeout-minutes: 120 + steps: + - uses: actions/checkout@v4 + - name: Tools + run: | + sudo apt-get update -qq + sudo apt-get install -y -qq ninja-build pkg-config zstd python3-pip + pip3 install --break-system-packages 'meson>=1.5' + - name: Build + run: ANDROID_NDK_HOME="$ANDROID_NDK_LATEST_HOME" WORK=$PWD/venus-work tools/venus/ci-build.sh + - uses: actions/upload-artifact@v4 + with: + name: venus + path: | + venus-work/venus.tzst + venus-work/venus.tzst.sha256 diff --git a/docs/development/mali-venus.md b/docs/development/mali-venus.md new file mode 100644 index 000000000..9cd3054e0 --- /dev/null +++ b/docs/development/mali-venus.md @@ -0,0 +1,340 @@ +# DroidDeck on Mali (Pixel 9 Pro) through Venus + +Handoff notes for the next session. +Upstream DroidDeck 0.3.1 runs on Adreno only. +This branch makes it run on the user's Pixel 9 Pro (Tensor G4, Mali-G715, Android 17). + +## Status (2026-10-08, night) + +Prototype 13 is the APK on the phone, a release build (not debuggable). +Prototypes 12 and 13 are published as GitHub prereleases. +Steam Big Picture shows a clean, live picture, and Slay the Spire 2 runs at about 40 fps. + +Direct3D 9-11 games run through DXVK. +Test game: Hollow Knight Silksong (Unity, D3D11), app id 1030300. +- Prototype 12: 35-40 fps in game, stable, with good audio. The phone was not throttling. +- Prototype 13 (cached frame readback, see Performance): 45-52 fps, averaging 47-52. + +Done and verified on the phone: +- The Mali compatibility layer (`tools/venus/layer`) loads and reports the six features DXVK requires. +- DXVK accepts the GPU and creates its device. D3D11 runs at feature level 11_0. +- Silksong loads and plays. +- The APK carries the layer and the Venus ICD; nothing on the phone is installed by hand any more. + +Fixed: a crash in the Venus guest driver, about 20 s after launch, with a black screen until then. +- Where: `vtest_vcmd_submit_cmd2` (`src/virtio/vulkan/vn_renderer_vtest.c`), on the `dxvk-submit` thread inside `vkQueueSubmit2`. +- What: it reads `batch->syncs[1]->sync_id` and `syncs[1]` is the pointer `0x1`. +- Cause: two threads submit to the same queue at once. + - Venus's async present thread (`vn_wsi`) makes the WSI submits. On vtest, `vn_wsi_fence_wait` drops the queue lock while it waits on a fence. + - It assumes one submit per swapchain. But `wsi_queue_submit2_unordered` makes a second, empty submit for DXVK's present fence. + - DXVK's thread passes `vn_wsi_flush` in that window and submits at the same time. + - Both use the per-queue scratch buffer `queue->storage`. The empty submit's `sync_vals[0] = 1` lands on DXVK's `syncs[1]`. + - gamescope and Zink use no present fences, so Steam and Slay the Spire 2 never hit it. +- Fix: `vn_queue_submit` holds a new per-queue `submit_mutex`. Verified with Silksong. + +Other open problems: +- Steam's UI blocks 30 s at a time on `IClientFriends::GetVoiceMicrophoneVolume`. A launch looks stuck on "Launching" even after the game died. +- The render server aborts ("pthread_mutex_lock called on a destroyed mutex") when a context dies after an error. Not fixed. +- The phone runs short of memory: about 1-2 GB free during a session. BC decoding makes textures 4-8x bigger. +- Prototype 13 is not debuggable, so `adb run-as` fails and the hand-install loop below does not work. + - To debug again: add `debuggable true` to the `release` build type in `app/build.gradle`, rebuild, reinstall. Keep it out of commits. + - A debuggable build shows a harmless "not 16 KB-compatible" dialog. + +## Performance + +How a game frame reaches the screen (Silksong, 1280x720): +1. The game draws through Venus. gamescope's WSI layer turns its X11 window into a Wayland surface on gamescope. +2. Venus's present thread (`vn_wsi[0,0]`) runs WSI's software path (`MESA_VK_WSI_DEBUG=sw`). + - It blits the image into a host-visible buffer, waits for the GPU, and `memcpy`s the buffer into a `wl_shm` buffer (3.6 MB). +3. gamescope uploads that `wl_shm` buffer into a texture (patch 0114), composites, and hands a dma-buf to the app. +4. The app's compositor draws it at 60 Hz. Its own work is about 2-3 ms a frame (`wayland.log`, `perf` lines). + +Measured with `tools/venus/measure-present.patch` (see "Measuring a frame" below): + +| Per frame | Prototype 12 | Prototype 13 | +|---|---|---| +| Frame interval | 26-29 ms | ~20 ms | +| `memcpy` out of the readback buffer | 7-10 ms (~400 MB/s) | ~1.3 ms, cache sync included | +| Present thread total | 17-20 ms | ~12 ms | +| GPU fence waits (two per present) | ~4 ms | ~4.5 ms | +| Game blocked in `vn_wsi_flush` | ~0.4 ms | ~0.01 ms | + +Why the copy was slow: Venus hid Mali's cached memory type. +- Mali's native types (`memtypes` probe): 0 = device-local, host-visible, coherent (uncached); 1 = device-local, host-visible, cached (not coherent). +- `vn_physical_device_init_memory_properties` strips host-visible from cached types that are not coherent, and labels type 0 "cached" for app compatibility. +- So WSI's readback buffer landed on uncached memory, and the CPU read it at ~400 MB/s. + +The fix (prototype 13): +- Venus allocates WSI's readback buffer in the hidden cached type (`vn_wsi_memory_type_index`). Apps still never see that type. +- The vtest renderer keeps that buffer's dma-buf fd and implements flush and invalidate as `DMA_BUF_IOCTL_SYNC`. +- WSI calls `vkInvalidateMappedMemoryRanges` before every CPU read of a frame (Wayland and X11 software paths). +- The render server allocates cached memory types from the cached dma-buf heap (`/dev/dma_heap/system`). + +What is left on the present thread (~12 ms): +- ~4.5 ms in two GPU fence waits. WSI makes a second, empty submit for DXVK's present fence, and Venus waits on each. +- ~6 ms in those two submits and their Venus round trips. +- Thread sample (`top -H`, prototype 12): no thread at 100%. Game main thread 48%, `vn_wsi` 30%, `dxvk-frame` 30%, Unity render thread 25%. + - `dxvk-frame` mostly waits for presents; 30% CPU looks like Venus polling in its semaphore waits. + +## State outside the repo and the APK + +On the phone, changed by hand (read this before testing anything): +- `/sdcard/Download/droiddeck-env` holds `PROTON_LOG=1`. Every game writes a full Proton log to `/root/steam-.log`. + - Delete it when debugging is done: `adb shell rm /sdcard/Download/droiddeck-env`. +- `/data/local/tmp` holds copies of the pushed files and `watch-maps.sh`. + +Published: prototypes 12 and 13 as the prereleases `mali-venus-proto12` and `mali-venus-proto13`. Prototypes 7 to 11 were never published. + +## Where things are + +- **Source:** `C:\Users\derab\source\repos\DroidDeck`, branch `mali-venus` (off tag `0.3.1`). +- **Remote:** `mine` = https://github.com/DavidCimo/droiddeck-mali. It's public; the user made it public to download on the phone. + - The gh account is `DavidCimo`. + - Releases `mali-venus-proto1`…`proto6`, `proto12` and `proto13` hold the APKs. Proto 7 to 11 were installed over ADB only. + - Publish with `gh release create mali-venus-proto -R DavidCimo/droiddeck-mali --prerelease --target mali-venus --title "0.3.1 + Venus prototype " --notes-file … `, after pushing the branch. +- **Build outputs:** `C:\Users\derab\source\repos\droiddeck-mali\` + - `DroidDeck-0.3.1.apk`: the official APK. Prebuilt assets and proot come from it. + - `venus-out/`: the Mesa Venus ICD. + - `layer/`: the Mali compatibility layer and its manifest. + - `libblsession.so`: the session preload with `venus.c`. + - `gamescope`: gamescope 3.16.29 with the app's patches plus 0114. + - `venus.patch`: the source diff the APK build applies. + - `DroidDeck-0.3.1-mali-venus-.apk`: the builds. +- **WSL Ubuntu** (`wsl -d Ubuntu`, user `delo`; use `-u root` for apt, there's no passwordless sudo): + - `/home/delo/android-sdk`: SDK, NDK 27.3.13750724, CMake 3.22.1, build-tools 34. + - `/home/delo/mali`: virglrenderer 1.3.0 (patched), Mesa 26.2.4 checkout (patched), libepoxy, Vulkan-Headers, `android-out/`. + - `/home/delo/DroidDeck`: the clean LF clone the APK is built in. + - The Edit and Read tools open WSL files at `\\wsl.localhost\Ubuntu\home\delo\...`. +- **ADB:** `C:\Users\derab\tools\platform-tools\adb.exe`, paired with the Pixel over wireless debugging. +- **DXVK source** for reference: `git clone https://github.com/doitsujin/dxvk` at commit `25ca63f17f34` (v3.1.1-27), the one in Proton Experimental ARM64 11.0-20260924. + - Its shader compiler is the `subprojects/dxbc-spirv` submodule. + +## How it works + +The session's Linux processes (glibc) cannot load Mali's Android (bionic) Vulkan driver. +Venus bridges them: + +1. The guest loads Mesa's Venus ICD (`libvulkan_virtio.so`) with `VN_DEBUG=vtest`. +2. The ICD serializes Vulkan calls over a unix socket: `files/venus/vtest.sock`. +3. virglrenderer's `virgl_test_server --venus --no-virgl` runs on the Android side as an app child process. +4. Per client, it forks `virgl_render_server`, which replays the calls on Mali's system Vulkan driver. + +For Wine games, the Mali compatibility layer sits between DXVK (via winevulkan) and the Venus ICD. + +The app's own compositor uses Mali's system driver directly. + +## Mali compatibility layer (`tools/venus/layer`) + +Why: DXVK 3.1.1 refuses any GPU without six features, and Mali-G715 has none of them. +The list comes from `dxvk/dxvk_device_info.cpp`, `getFeatureList`, compared with `vulkaninfo` through Venus. +Everything else DXVK requires is present: Vulkan 1.4, geometry and tessellation shaders, maxPushConstantsSize 256. + +What it does, per feature: +- `fillModeNonSolid`: LINE and POINT polygon modes draw as FILL. +- `multiViewport`: reports maxViewports 16. Pipelines and `vkCmdSet{Viewport,Scissor}[WithCount]` keep only the first. +- `robustBufferAccess2`: the device gets `robustBufferAccess` (version 1) instead. +- `shaderClipDistance`: emulated per pipeline. + - The last pre-rasterization stage writes the distances to a free varying location. + - The fragment shader reads them and runs `OpKill` when one is negative. + - Only when that stage is the only one using distances and the fragment shader's code is known. + - Otherwise the distances are dropped. Depth-only passes lose clipping too. + - DXVK passes shader code inline (`VkShaderModuleCreateInfo` in the stage), so its pipelines qualify. + - Mali has no `graphicsPipelineLibraryIndependentInterpolationDecoration`, so DXVK compiles whole pipelines and the layer sees both stages. +- `shaderCullDistance`: dropped. +- `textureCompressionBC`: BC1-BC7 images are created as RGBA8, R8, RG8 or RGBA16F. + - Every upload into one is decoded on the CPU with `bcdec` while the copy is recorded. + - The source must be memory the application has mapped; DXVK's staging memory is. + - Decoded texels go through the layer's own staging buffers, recycled when the command buffer is reset. + - Copies from a BC image into a buffer, and between BC and uncompressed images, are skipped with a warning. + +Files: +- `layer.c`: loader plumbing, feature and limit reporting, device creation, the hook table. +- `pipeline.c`: pipelines, shader modules, viewports. +- `spirv.c`: the SPIR-V rewriter for clip and cull distances. +- `bc.c`: BC formats, images and uploads. +- `VkLayer_droiddeck_mali_compat.json`: implicit layer manifest. + +When it is active: +- Only in Wine processes (`WINEPREFIX` set), and only for features the GPU lacks. +- `DROIDDECK_MALI_COMPAT=1` forces it on, `0` off. `DROIDDECK_MALI_COMPAT_DISABLE=1` keeps the loader from loading it. +- Wine's own `explorer.exe` uses Zink, so it gets the layer too. +- It logs one line per device: `droiddeck-mali-compat: emulating …`. + +How it ships: +- `build-apk.sh` copies `layer/` into the APK's `assets/linuxfs/usr/local/…`. +- `SessionFiles.kt` stages both files into the guest at every session start (its `optional` list). + - Guest paths: `usr/local/lib/droiddeck-mali/` and `usr/local/share/vulkan/implicit_layer.d/`. + +Not done yet: +- Unit tests for the SPIR-V rewriter. It is untested beyond DXVK's shaders, and unknown whether Silksong's shaders used clip distances at all. +- Performance: BC decoding on the CPU makes loads slow. A compute-shader decoder would be the next step. + +## Changes on the branch + +App (Kotlin/C): +- `session/VenusComponent.kt` (new): on a non-Adreno GPU, starts the vtest server and stages the ICD to `files/venus/`. + - It sets the guest env: `VK_ICD_FILENAMES`, `VN_DEBUG=vtest`, `VTEST_SOCKET_NAME`, `MESA_VK_WSI_DEBUG=sw`. +- `session/SessionService.kt`: uses the Venus env instead of the Turnip ICD, and adds the component. +- `gpu/TurnipDriver.java`: the compositor uses the system Vulkan driver on non-Adreno. +- `cpp/waylandcomp/src/vk_present.c`: + - Enables only the dma-buf extensions the driver has. + - Logs the driver's full extension list. + - Gives a zero row pitch a packed 64-byte row (gamescope's 1x1 root buffer). +- `gpu/GpuInfo.kt`, `MainActivity.kt`, `core/DeviceReport.kt`: wording, plus a "Venus" line in `device.txt`. +- `session/SessionFiles.kt`: stages the Mali compatibility layer when the APK carries it. +- `tools/linuxfs/preload/venus.c` (new, part of `libblsession`): unsets `LIBGL_KOPPER_DISABLE` when `VN_DEBUG` has `vtest`. + - Steam's `steamwebhelper.sh` sets it unconditionally. + - Without kopper and without DRI3, Zink gets no GL context, and the Steam UI crash-loops. + - Steam restores that script when it verifies its install, so it cannot be edited. + +`tools/venus/mesa-venus-vtest.patch` (Mesa 26.2.4, guest side): +- No `VK_EXT_physical_device_drm` under vtest. gamescope otherwise demanded a primary node. +- Sync-fd export for fences and semaphores under vtest: a CPU wait that returns -1 (already signaled). gamescope requires `VK_KHR_external_semaphore_fd`. +- WSI skips attaching a -1 sync file to a dma-buf. +- `vn_queue_submission_init_syncs` submits only the sync slots it filled. +- Timeline semaphores free all `queue_count + 1` renderer syncs. Upstream freed one too few and leaked a vtest sync each. +- `vn_queue_submit` runs under a per-queue `submit_mutex`. The async present thread raced the app's submits; see Status. +- WSI's CPU readback buffer goes to Mali's hidden cached memory type; see Performance. + - `vn_physical_device`: keeps the renderer's real memory flags, and picks the hidden type under vtest. + - `vn_renderer_vtest.c`: flush and invalidate of incoherent memory are `DMA_BUF_IOCTL_SYNC` on the bo's dma-buf. + - WSI common code: `vkInvalidateMappedMemoryRanges` before every CPU read of a frame. + +`tools/venus/measure-present.patch`: per-frame present timings on stderr, applied on top for `MEASURE=1` builds only. + +`tools/venus/virglrenderer-android.patch` (virglrenderer 1.3.0, host side): +- Mali on Android can import dma-bufs but cannot export any memory. + - Buffers report dma_buf features 0x4 (importable only); opaque fd reports nothing. + - virglrenderer's minigbm fallback does not exist on Android. +- On Android the server allocates the dma-buf itself and imports it. + - Source for coherent types: `/dev/dma_heap/system-uncached`, then `/dev/dma_heap/system`, then an `AHardwareBuffer` fd. + - Uncached because Mali here is not cache-coherent with the CPU, and nobody calls `DMA_BUF_IOCTL_SYNC` on coherent memory. + - The guest maps coherent memory without syncs, and on the cached heap frames tore at cache-line granularity: the streaks. + - Cached types (not coherent) come from `/dev/dma_heap/system`. Whoever maps them syncs through the dma-buf. + - The fd rides the existing udmabuf plumbing. +- Image and buffer queries report "importable dma-buf" as exportable too. gamescope keeps only exportable modifiers. +- Exports of any memory type go through that allocator. +- Logs what the driver reports, and failed allocations. + +`tools/gamescope/patches/0114-shm-buffers-upload-every-commit.patch`: +- Xwayland has no glamor here, so it reuses about three wl_shm buffers with new pixels. +- gamescope cached one texture per buffer, copied at first import. +- The screen cycled through three stale frames: the flicker. Input and audio still worked. +- The patch caches dma-buf textures only, and uploads shm buffers on every commit. +- See `tools/gamescope/PATCHES.md`. +- The session stages gamescope from the APK assets every start, so it needs an APK rebuild. + +## Build + +Run each step after its inputs change: + +1. Android side (WSL): `bash /mnt/c/Users/derab/source/repos/DroidDeck/tools/venus/build-virgl-android.sh` + - It does a full clean rebuild; `ninja -C /home/delo/mali/virgl-build install` is enough for code changes. + - After editing `/home/delo/mali/virglrenderer`, refresh the patch: `git -C /home/delo/mali/virglrenderer diff > .../tools/venus/virglrenderer-android.patch`. +2. Guest ICD (Git Bash, Docker Desktop running): `MSYS_NO_PATHCONV=1 bash tools/venus/build-venus-icd.sh` + - It clones Mesa fresh and applies `tools/venus/mesa-venus-vtest.patch`. About 5 minutes. + - To change Venus: edit `/home/delo/mali/mesa`, then refresh the patch from Git Bash: + `MSYS_NO_PATHCONV=1 wsl -d Ubuntu -u delo -- git -C /home/delo/mali/mesa diff > tools/venus/mesa-venus-vtest.patch` +3. Mali compatibility layer (Git Bash, Docker): `MSYS_NO_PATHCONV=1 bash tools/venus/build-layer.sh` + - Fetches Vulkan-Headers and SPIRV-Headers (vulkan-sdk-1.4.357.0) and `bcdec.h` at a pinned commit. Under a minute. +4. Preload (Git Bash, Docker): `MSYS_NO_PATHCONV=1 bash tools/venus/build-libblsession.sh` +5. gamescope (Git Bash, Docker): `MSYS_NO_PATHCONV=1 bash tools/venus/build-gamescope.sh` + - Emulated arm64, so it takes most of an hour. + - Prototype 11's binary came from the same steps run by hand; the script itself has not run yet. +6. Source patch (Git Bash): `git diff 0.3.1 > ../droiddeck-mali/venus.patch` + - Run `git add -N` on new files first. +7. APK (WSL): `bash /mnt/c/Users/derab/source/repos/DroidDeck/tools/venus/build-apk.sh ` + - It packs the outputs of steps 1-5. Rebuild those first if their sources changed. + +Gotchas: +- Run Docker from Git Bash, not WSL: Docker Desktop's WSL integration is off. +- Prefix Git Bash commands with `MSYS_NO_PATHCONV=1`, or paths given to `wsl`, `docker` and `adb` get mangled. +- `wsl -- bash -c '…'` loses `$VARIABLES`; put the commands in a script file and run `wsl -- bash /mnt/c/…/script.sh`. +- `adb pull` needs a Windows path as its target. +- The APK is signed with the project's public test key, so it installs over earlier prototypes but not over the official app. +- Use `git -c core.autocrlf=false -c core.eol=lf archive` to export files for a Linux build; plain `git archive` writes CRLF. + +## Test loop over ADB + +```sh +A=/c/Users/derab/tools/platform-tools/adb.exe; export MSYS_NO_PATHCONV=1 +$A mdns services # find the phone's connect port if it dropped +timeout 300 $A install -r "C:/Users/derab/source/repos/droiddeck-mali/DroidDeck-0.3.1-mali-venus-.apk" +$A shell am force-stop com.droiddeck.launcher +$A shell monkey -p com.droiddeck.launcher -c android.intent.category.LAUNCHER 1 +$A shell input tap 897 1064 # "Play Steam" on the main page +$A exec-out screencap -p > shot.png +``` + +- The phone must be awake and unlocked, or `adb install` hangs. +- Tap "Play Steam" only once MainActivity is the top resumed activity; an earlier tap is lost. +- Wrap a remote `run-as ... sh -c` command in double quotes as one argument, or adb splits it. Longer scripts: push a file and run it with `run-as … sh /data/local/tmp/x.sh`. +- Session logs: `/sdcard/Download/DroidDeck//` + - `session.log`: gamescope, Steam and game stdout and stderr. + - `venus.log`: the vtest and render servers. + - `wayland.log`: the compositor; frames on screen and windows open. + - `events.jsonl`: `frame.first` means the loading overlay lifted. + - `crash.log`. +- Steam's own logs, live: `$A shell run-as com.droiddeck.launcher sh -c 'cd files/linuxfs/root/.local/share/Steam/logs; ...'` + - The useful ones are `webhelper.txt`, `webhelper_gpu.txt`, `cef_log.txt`, `console_log.txt` and `gameprocess_log.txt`. +- The session script copies those logs into `/steam/` only on a clean exit. + - Clean exit: "Stop session" on the DroidDeck notification. Leaving the app only suspends the session. +- `Download/droiddeck-env` (KEY=VALUE lines) adds guest env without a rebuild. It is read at session start only. +- First Steam start downloads and unpacks about 670 MB. On later starts Steam "verifies" and re-extracts changed files. + +Faster iteration on GPU code, without an APK (needs a debuggable build, see Status): +- Venus ICD: push it, then replace `files/venus/libvulkan_virtio.so` with `cp` to a new name and `mv` over it. + - `mv` keeps the running gamescope and Steam on their old copy; overwriting in place could crash them. +- Layer: copy the two files to the guest paths above. New processes load it. The next session start restages the APK's copy. +- What the guest sees: `MSYS_NO_PATHCONV=1 bash tools/venus/guest-vulkaninfo.sh [0|1]` with a session running. + +Debugging a game: +- Proton log: `PROTON_LOG=1` in `droiddeck-env`, restart the session, launch. The log is `files/linuxfs/root/steam-.log`. + - DXVK's lines start with `info:`, `warn:`, `err:`. Proton otherwise turns DXVK logging off. +- Unity games: `Player.log` in the prefix, `pfx/drive_c/users/steamuser/AppData/LocalLow///`. +- A crash in Linux code shows in the Proton log as `handle_syscall_fault … pc=0x…`, then `Exception 0xc0000005 in Unix call`. + 1. Before the launch, run `tools/venus/watch-maps.sh '.exe'` on the phone (see its header). + 2. After the crash, find the library whose range holds the pc in `cache/game-maps.txt`. Offset = pc - range start + file offset. + 3. Resolve it in WSL: `bash /mnt/c/…/tools/venus/addr2line.sh `. + - The thread name is in the log: `Thread renamed to "dxvk-submit"`. +- Untested idea for launching a game without the user: write `steam://rungameid/` into `files/linuxfs/root/.steam/steam.pipe`. + +Measuring a frame (needs a debuggable build): +1. `MEASURE=1 MSYS_NO_PATHCONV=1 bash tools/venus/build-venus-icd.sh` builds the ICD with `measure-present.patch` into `droiddeck-mali/venus-out-measure/`. +2. Install it by hand (see "Faster iteration"), with `PROTON_LOG=1` in `droiddeck-env`. +3. Every 120 presents the game's Proton log gets two lines: + - `MEASURE wl`: frame interval, the `memcpy` out of the readback buffer, the whole Wayland present. Average/max in ms. + - `MEASURE vn`: the present thread, its GPU fence waits, and how long the game blocked in `vn_wsi_flush`. +4. Per-thread CPU while playing: `adb shell "top -H -b -n 2 -d 8 -o TID,PID,%CPU,S,CMD,NAME"`, sorted by the third column. +5. `wayland.log` in the session folder has the compositor's fps and per-frame costs every 10 s, without any special build. + +## Failure history (prototype → cause → fix) + +1. Official APK installed → Turnip ICD, no GPU → install the prototype. +2. gamescope: "physical device has no primary node" → Mesa: drop `EXT_physical_device_drm` under vtest. +3. gamescope: missing `VK_KHR_external_semaphore_fd` → Mesa: CPU-wait sync-fd export. +4. Hang on loading: server "minigbm_allocation is not enabled" → virgl: relax the export check. Not enough: Mali exports nothing. +5. Same hang → virgl: allocate a dma-buf (dma-heap or AHB) and import it. +6. gamescope assert `modifiers.size() > 0` → virgl: report importable dma-buf as exportable; route exports through the allocator. +7. Updater "zink: could not create swapchain"; Xwayland has no glamor, so no DRI3 → `MESA_VK_WSI_DEBUG=sw`. +8. steamwebhelper restart loop, "failed to acquire a gl context" → `venus.c` preload unsets `LIBGL_KOPPER_DISABLE`. +9. Frames reach the compositor but "0 windows open" and the overlay stays → compositor: fix stride 0 on gamescope's 1x1 root buffer. +10. Sign-in screen shows, with dash streaks across it → virgl: allocate from the uncached dma-heap. +11. Picture looks frozen or flickers between old frames, though clicks play sounds → gamescope patch 0114. +12. Steam sign-in and Slay the Spire 2 at about 40 fps. Silksong: "d3d11: failed to create factory"; DXVK skips the GPU for `fillModeNonSolid` → the Mali compatibility layer. +13. Silksong: D3D11 starts, black screen, then a crash in Venus's `vtest_vcmd_submit_cmd2` → Mesa: serialize queue submits against the async present thread. Silksong plays at 35-40 fps. +14. Silksong held at 35-40 fps; Venus's present thread spent half its time copying frames from uncached memory → cached readback buffer (prototype 13), 47-52 fps. + +## Next steps + +1. Performance, measured first with `MEASURE=1` (see Performance): + - The two GPU fence waits and two submits per present (~10 ms on the present thread). + - The `dxvk-frame` thread at 30% CPU, probably Venus polling in semaphore waits. +2. Silksong: check BC textures look right and loading time is bearable. +3. Test more games on different engines: GameMaker, FNA (Celeste, Stardew Valley), Unity D3D9, Godot. Fix shared gaps in the layer, never per game. +4. Direct3D 12 games use VKD3D-Proton, which has its own requirement list. Check it the same way. +5. Clean up: delete `droiddeck-env` when debugging is done. + +Rules from the user: +- Never enter credentials for the user. They type the password and handle Steam Guard themselves. +- Test every build on the phone over wireless debugging. +- Fixes go in shared layers (Venus, the compatibility layer), not per game. diff --git a/tools/gamescope/PATCHES.md b/tools/gamescope/PATCHES.md index 8f8dfb40e..9dd697b1b 100644 --- a/tools/gamescope/PATCHES.md +++ b/tools/gamescope/PATCHES.md @@ -68,3 +68,9 @@ app's Wayland-hosted gamescope never reaches, or need a newer gamescope than the not an Adreno when the kernel is older than 6.6 (no DRM_IOCTL_SYNCOBJ_EVENTFD: an Exynos 2400e on 6.1 logged hundreds of "timeline: DRM_IOCTL_SYNCOBJ_EVENTFD failed" and a game hung at a black screen) or when the render node is Mali's stand-in (/dev/mali0 has no DRM syncobjs). +- `0127-shm-buffers-upload-every-commit.patch` (David Cimo, for Mali through Venus; his `0114`): with + no DRM node, Xwayland has no glamor and commits a few wl_shm buffers over and over with new pixels. + gamescope memoizes one texture per `wlr_buffer`, and a shm buffer's texture is a copy taken at + import, so the screen cycled through the first few frames forever while Steam ran on behind them. + Only dma-buf textures are memoized now; a shm buffer is uploaded on every commit. Adreno never + takes this path, since its Xwayland hands over dma-bufs. Also what XWAYLAND_NO_GLAMOR sessions need. diff --git a/tools/gamescope/patches/0127-shm-buffers-upload-every-commit.patch b/tools/gamescope/patches/0127-shm-buffers-upload-every-commit.patch new file mode 100644 index 000000000..5f382c15b --- /dev/null +++ b/tools/gamescope/patches/0127-shm-buffers-upload-every-commit.patch @@ -0,0 +1,44 @@ +diff --git a/src/steamcompmgr.cpp b/src/steamcompmgr.cpp +index 18b799e..b803ee0 100644 +--- a/src/steamcompmgr.cpp ++++ b/src/steamcompmgr.cpp +@@ -1850,16 +1850,23 @@ import_commit ( + commit->fifo = false; + } + +- if ( gamescope::OwningRc pTexture = s_BufferMemos.LookupVulkanTexture( buf ) ) ++ struct wlr_dmabuf_attributes dmabuf = {0}; ++ const bool bDmabuf = wlr_buffer_get_dmabuf( buf, &dmabuf ); ++ ++ // Only a dma-buf's texture shows the buffer itself. A shm buffer's texture is a copy taken at ++ // import, and Xwayland without glamor commits the same shm buffers again with new pixels. ++ if ( bDmabuf ) + { +- // Going from OwningRc -> Rc now. +- commit->vulkanTex = pTexture; +- return commit; ++ if ( gamescope::OwningRc pTexture = s_BufferMemos.LookupVulkanTexture( buf ) ) ++ { ++ // Going from OwningRc -> Rc now. ++ commit->vulkanTex = pTexture; ++ return commit; ++ } + } + +- struct wlr_dmabuf_attributes dmabuf = {0}; + gamescope::OwningRc pBackendFb; +- if ( wlr_buffer_get_dmabuf( buf, &dmabuf ) ) ++ if ( bDmabuf ) + { + pBackendFb = GetBackend()->ImportDmabufToBackend( &dmabuf ); + } +@@ -1873,7 +1880,8 @@ import_commit ( + + commit->vulkanTex = pOwnedTexture; + +- s_BufferMemos.MemoizeBuffer( buf, std::move( pOwnedTexture ) ); ++ if ( bDmabuf ) ++ s_BufferMemos.MemoizeBuffer( buf, std::move( pOwnedTexture ) ); + + return commit; + } diff --git a/tools/linuxfs/preload/venus.c b/tools/linuxfs/preload/venus.c new file mode 100644 index 000000000..4feb9ee03 --- /dev/null +++ b/tools/linuxfs/preload/venus.c @@ -0,0 +1,18 @@ +/* + * Zink keeps kopper when the session draws through Venus. + * + * On Mali the session reaches the GPU through Venus over vtest, and Xwayland has no DRM node to + * build glamor on, so it offers no DRI3. Without DRI3, Zink can only present through kopper (its + * Vulkan WSI path). Steam's steamwebhelper.sh exports LIBGL_KOPPER_DISABLE=true for the client + * interface unconditionally, so on Mali every GL context the interface asked for failed and the + * webhelper restarted until Steam gave up on GPU acceleration. Steam restores that script when + * it verifies its install, so the variable is dropped here instead, before Mesa reads it. + */ +#define _GNU_SOURCE 1 +#include +#include + +__attribute__((constructor)) static void bl_venus_keep_kopper(void) { + const char *debug = getenv("VN_DEBUG"); + if (debug && strstr(debug, "vtest")) unsetenv("LIBGL_KOPPER_DISABLE"); +} diff --git a/tools/venus/addr2line.sh b/tools/venus/addr2line.sh new file mode 100644 index 000000000..e9f6e5f52 --- /dev/null +++ b/tools/venus/addr2line.sh @@ -0,0 +1,14 @@ +#!/bin/bash +# Function, source line and disassembly at offsets into an aarch64 library, with the NDK's tools. +# Run in WSL: bash addr2line.sh ... +# The offset is the crash pc minus the library's start in the process's maps (tools/venus/watch-maps.sh). +LIB=${1:?library} +shift +B=/home/delo/android-sdk/ndk/27.3.13750724/toolchains/llvm/prebuilt/linux-x86_64/bin +md5sum "$LIB" +$B/llvm-addr2line -f -C -i -e "$LIB" "$@" +for a in "$@"; do + start=$(printf "0x%x" $((a - 48))) + stop=$(printf "0x%x" $((a + 16))) + $B/llvm-objdump -d --no-show-raw-insn --start-address=$start --stop-address=$stop "$LIB" | tail -n +7 +done diff --git a/tools/venus/ci-build.sh b/tools/venus/ci-build.sh new file mode 100755 index 000000000..4825557e4 --- /dev/null +++ b/tools/venus/ci-build.sh @@ -0,0 +1,144 @@ +#!/bin/bash +# Builds the Mali (Venus) component in CI, from David Cimo's tools/venus recipes (build-venus-icd.sh, +# build-virgl-android.sh, build-layer.sh), into venus.tzst rooted at app/src/main: +# jniLibs/arm64-v8a/ libvirgl_test_server.so, libvirgl_render_server.so, libvirglrenderer.so, +# libepoxy.so (bionic: the vtest server the app runs over the phone's Vulkan) +# assets/venus/ libvulkan_virtio.so (glibc: Mesa's Venus ICD for the session) +# assets/linuxfs/usr/local/... the Mali compatibility layer and its manifest +# Needs: docker (Debian trixie for the glibc parts), the NDK in $ANDROID_NDK_HOME, meson, ninja. +set -euo pipefail +REPO=$(cd "$(dirname "$0")/../.." && pwd) +W=${WORK:-$PWD/venus-work} +OUT=$W/out +rm -rf "$OUT" && mkdir -p "$OUT/jniLibs/arm64-v8a" "$OUT/assets/venus" \ + "$OUT/assets/linuxfs/usr/local/lib/droiddeck-mali" "$OUT/assets/linuxfs/usr/local/share/vulkan/implicit_layer.d" + +# --- Linux side: Mesa 26.2.4's Venus ICD and the Mali compatibility layer (aarch64 glibc) --- +docker run --rm --platform linux/amd64 -v "$W":/w -v "$REPO/tools/venus":/venus:ro -w /tmp debian:trixie bash -c ' +set -euo pipefail +export DEBIAN_FRONTEND=noninteractive +dpkg --add-architecture arm64 +apt-get update -qq +apt-get install -y -qq --no-install-recommends git ca-certificates curl crossbuild-essential-arm64 meson ninja-build \ + pkg-config python3-mako python3-yaml python3-packaging bison flex glslang-tools libwayland-bin wayland-protocols \ + libdrm-dev:arm64 libwayland-dev:arm64 libx11-dev:arm64 libx11-xcb-dev:arm64 libxcb1-dev:arm64 \ + libxcb-dri3-dev:arm64 libxcb-present-dev:arm64 libxcb-randr0-dev:arm64 libxcb-shm0-dev:arm64 \ + libxcb-sync-dev:arm64 libxcb-xfixes0-dev:arm64 libxshmfence-dev:arm64 libxrandr-dev:arm64 \ + libexpat1-dev:arm64 zlib1g-dev:arm64 libzstd-dev:arm64 >/dev/null +cat > /tmp/arm64.ini </dev/null +ninja -C venus-build src/virtio/vulkan/libvulkan_virtio.so +aarch64-linux-gnu-strip --strip-unneeded -o /w/out/assets/venus/libvulkan_virtio.so venus-build/src/virtio/vulkan/libvulkan_virtio.so +echo "venus icd min glibc: $(aarch64-linux-gnu-objdump -T /w/out/assets/venus/libvulkan_virtio.so | grep -o "GLIBC_[0-9.]*" | sort -Vu | tail -1)" +SDK=vulkan-sdk-1.4.357.0 +git -c advice.detachedHead=false clone -q --depth 1 -b $SDK https://github.com/KhronosGroup/Vulkan-Headers.git /tmp/vulkan +git -c advice.detachedHead=false clone -q --depth 1 -b $SDK https://github.com/KhronosGroup/SPIRV-Headers.git /tmp/spirv +mkdir -p /tmp/bcdec +curl -sSfL -o /tmp/bcdec/bcdec.h https://raw.githubusercontent.com/iOrange/bcdec/80859ed3b7afb1c527a2a99d70c61457bea72d0c/bcdec.h +L=/w/out/assets/linuxfs/usr/local +aarch64-linux-gnu-gcc -shared -fPIC -O2 -Wall -Wextra -Wno-unused-parameter -fvisibility=hidden -pthread \ + -I/tmp/vulkan/include -I/tmp/spirv/include -I/tmp/bcdec \ + -o $L/lib/droiddeck-mali/libVkLayer_droiddeck_mali_compat.so /venus/layer/*.c +aarch64-linux-gnu-strip --strip-unneeded $L/lib/droiddeck-mali/libVkLayer_droiddeck_mali_compat.so +cp /venus/layer/VkLayer_droiddeck_mali_compat.json $L/share/vulkan/implicit_layer.d/ +chmod -R a+rwX /w/out +' + +# --- Android side: libepoxy + virglrenderer 1.3.0 (venus, vtest) for arm64 with the NDK --- +NDK=${ANDROID_NDK_HOME:?} +TC=$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin +API=29 +A=$W/android +AOUT=$A/out +mkdir -p "$A/cross" "$A/compat/log" "$A/compat/cutils" "$AOUT/lib/pkgconfig" +cd "$A" +git clone -q --depth 1 -b 1.5.10 https://github.com/anholt/libepoxy.git +git clone -q --depth 1 -b v1.4.365 https://github.com/KhronosGroup/Vulkan-Headers.git +git clone -q --depth 1 -b virglrenderer-1.3.0 https://gitlab.freedesktop.org/virgl/virglrenderer.git +git -C virglrenderer apply "$REPO/tools/venus/virglrenderer-android.patch" +cat > compat/log/log.h <<'EOC' +#pragma once +#include +#define LOG_PRI(prio, tag, ...) __android_log_print(prio, tag, __VA_ARGS__) +EOC +cat > compat/cutils/properties.h <<'EOC' +#pragma once +#include +#include +#define PROPERTY_VALUE_MAX PROP_VALUE_MAX +#define PROPERTY_KEY_MAX 32 +static inline int property_get(const char *key, char *value, const char *def) +{ + int len = __system_property_get(key, value); + if (len <= 0 && def) { strcpy(value, def); len = (int)strlen(def); } + return len; +} +EOC +cat > "$AOUT/lib/pkgconfig/vulkan.pc" < cross/android.ini </dev/null +ninja -C epoxy-build install >/dev/null +meson setup virgl-build virglrenderer --cross-file cross/android.ini --prefix="$AOUT" --libdir=lib \ + --buildtype=release -Dvenus=true -Dplatforms=auto -Drender-server-worker=thread \ + -Dvulkan-dload=true -Dcheck-gl-errors=false -Dtests=false >/dev/null +ninja -C virgl-build install >/dev/null +J=$OUT/jniLibs/arm64-v8a +cp "$AOUT/bin/virgl_test_server" "$J/libvirgl_test_server.so" +cp "$AOUT/libexec/virgl_render_server" "$J/libvirgl_render_server.so" +cp "$AOUT/lib/libvirglrenderer.so" "$AOUT/lib/libepoxy.so" "$J/" +for f in "$J"/*.so; do "$TC/llvm-strip" --strip-unneeded "$f"; done + +cd "$OUT" +find . -type f | sort +tar --owner=0 --group=0 --numeric-owner --sort=name -cf - . | zstd -19 -T0 -q -o "$W/venus.tzst" +sha256sum "$W/venus.tzst" | tee "$W/venus.tzst.sha256" diff --git a/tools/venus/guest-vulkaninfo.sh b/tools/venus/guest-vulkaninfo.sh new file mode 100644 index 000000000..124c5fb26 --- /dev/null +++ b/tools/venus/guest-vulkaninfo.sh @@ -0,0 +1,15 @@ +#!/bin/bash +# vulkaninfo as the session's Linux processes see the GPU: through Venus, and through the Mali +# compatibility layer when DROIDDECK_MALI_COMPAT is 1. Needs a running session (the vtest server). +# Run from Git Bash: MSYS_NO_PATHCONV=1 bash tools/venus/guest-vulkaninfo.sh [0|1] > vulkaninfo.txt +set -euo pipefail +COMPAT=${1:-1} +A=/c/Users/derab/tools/platform-tools/adb.exe +PKG=com.droiddeck.launcher +F=/data/user/0/$PKG/files +APK=$($A shell pm path $PKG | head -1 | tr -d '\r' | sed 's/^package://') +L=${APK%/base.apk}/lib/arm64 +$A exec-out "run-as $PKG sh -c 'PROOT_TMP_DIR=/data/user/0/$PKG/cache PROOT_LOADER=$L/libproot-loader.so \ + $L/libproot.so -r $F/linuxfs -w /root -b /dev -b /proc -b /sys -b $F /usr/bin/env -i HOME=/root \ + PATH=/usr/bin:/bin DROIDDECK_MALI_COMPAT=$COMPAT VK_ICD_FILENAMES=$F/venus/virtio_icd.json VN_DEBUG=vtest \ + VTEST_SOCKET_NAME=$F/venus/vtest.sock vulkaninfo 2>&1'" diff --git a/tools/venus/layer/VkLayer_droiddeck_mali_compat.json b/tools/venus/layer/VkLayer_droiddeck_mali_compat.json new file mode 100644 index 000000000..e1efa4c15 --- /dev/null +++ b/tools/venus/layer/VkLayer_droiddeck_mali_compat.json @@ -0,0 +1,17 @@ +{ + "file_format_version": "1.2.0", + "layer": { + "name": "VK_LAYER_DROIDDECK_mali_compat", + "type": "GLOBAL", + "library_path": "/usr/local/lib/droiddeck-mali/libVkLayer_droiddeck_mali_compat.so", + "api_version": "1.4.357", + "implementation_version": "1", + "description": "Features DXVK requires and Mali lacks, made up for", + "functions": { + "vkNegotiateLoaderLayerInterfaceVersion": "vkNegotiateLoaderLayerInterfaceVersion" + }, + "disable_environment": { + "DROIDDECK_MALI_COMPAT_DISABLE": "1" + } + } +} diff --git a/tools/venus/layer/bc.c b/tools/venus/layer/bc.c new file mode 100644 index 000000000..328b68169 --- /dev/null +++ b/tools/venus/layer/bc.c @@ -0,0 +1,869 @@ +/* + * BC1-BC7 textures on a GPU without them: each BC image is created in an uncompressed format, and + * every upload into one is decoded on the CPU while the copy is recorded. The decoded texels go + * through a staging buffer of the layer's own, which lives until the command buffer is reset. + * Uploads must come from memory the application has mapped, as DXVK's staging memory always is. + */ +#include "mali_compat.h" + +#include +#include + +#define BCDEC_IMPLEMENTATION +#define BCDEC_STATIC +#define BCDEC_BC4BC5_PRECISE +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wunused-function" +#include "bcdec.h" +#pragma GCC diagnostic pop + +#define HANDLE(h) ((uint64_t)(uintptr_t)(h)) +#define STAGING_BLOCK_SIZE (16u << 20) +#define STAGING_KEPT (64u << 20) + +enum bc_kind { BC1, BC2, BC3, BC4, BC5, BC6H, BC7 }; + +struct bc_format { + VkFormat format; + VkFormat substitute; + enum bc_kind kind; + uint32_t block_bytes; + uint32_t texel_bytes; + bool is_signed; + /* BC1's RGB formats have no alpha; its 1-bit transparency must read as opaque. */ + bool opaque; +}; + +static const struct bc_format bc_formats[] = { + {VK_FORMAT_BC1_RGB_UNORM_BLOCK, VK_FORMAT_R8G8B8A8_UNORM, BC1, 8, 4, false, true}, + {VK_FORMAT_BC1_RGB_SRGB_BLOCK, VK_FORMAT_R8G8B8A8_SRGB, BC1, 8, 4, false, true}, + {VK_FORMAT_BC1_RGBA_UNORM_BLOCK, VK_FORMAT_R8G8B8A8_UNORM, BC1, 8, 4, false, false}, + {VK_FORMAT_BC1_RGBA_SRGB_BLOCK, VK_FORMAT_R8G8B8A8_SRGB, BC1, 8, 4, false, false}, + {VK_FORMAT_BC2_UNORM_BLOCK, VK_FORMAT_R8G8B8A8_UNORM, BC2, 16, 4, false, false}, + {VK_FORMAT_BC2_SRGB_BLOCK, VK_FORMAT_R8G8B8A8_SRGB, BC2, 16, 4, false, false}, + {VK_FORMAT_BC3_UNORM_BLOCK, VK_FORMAT_R8G8B8A8_UNORM, BC3, 16, 4, false, false}, + {VK_FORMAT_BC3_SRGB_BLOCK, VK_FORMAT_R8G8B8A8_SRGB, BC3, 16, 4, false, false}, + {VK_FORMAT_BC4_UNORM_BLOCK, VK_FORMAT_R8_UNORM, BC4, 8, 1, false, false}, + {VK_FORMAT_BC4_SNORM_BLOCK, VK_FORMAT_R8_SNORM, BC4, 8, 1, true, false}, + {VK_FORMAT_BC5_UNORM_BLOCK, VK_FORMAT_R8G8_UNORM, BC5, 16, 2, false, false}, + {VK_FORMAT_BC5_SNORM_BLOCK, VK_FORMAT_R8G8_SNORM, BC5, 16, 2, true, false}, + {VK_FORMAT_BC6H_UFLOAT_BLOCK, VK_FORMAT_R16G16B16A16_SFLOAT, BC6H, 16, 8, false, false}, + {VK_FORMAT_BC6H_SFLOAT_BLOCK, VK_FORMAT_R16G16B16A16_SFLOAT, BC6H, 16, 8, true, false}, + {VK_FORMAT_BC7_UNORM_BLOCK, VK_FORMAT_R8G8B8A8_UNORM, BC7, 16, 4, false, false}, + {VK_FORMAT_BC7_SRGB_BLOCK, VK_FORMAT_R8G8B8A8_SRGB, BC7, 16, 4, false, false}, +}; + +static const struct bc_format *find_bc(VkFormat format) +{ + for (size_t i = 0; i < sizeof(bc_formats) / sizeof(bc_formats[0]); i++) { + if (bc_formats[i].format == format) + return &bc_formats[i]; + } + return NULL; +} + +VkFormat bc_substitute(VkFormat format) +{ + const struct bc_format *bc = find_bc(format); + return bc ? bc->substitute : VK_FORMAT_UNDEFINED; +} + +/* What a real BC format allows: sampling and copies. */ +#define BC_FEATURES \ + (VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT | VK_FORMAT_FEATURE_SAMPLED_IMAGE_FILTER_LINEAR_BIT | \ + VK_FORMAT_FEATURE_SAMPLED_IMAGE_FILTER_MINMAX_BIT | VK_FORMAT_FEATURE_TRANSFER_SRC_BIT | \ + VK_FORMAT_FEATURE_TRANSFER_DST_BIT | VK_FORMAT_FEATURE_BLIT_SRC_BIT) +#define BC_USAGE (VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT) + +static const struct bc_format *physical_bc(VkPhysicalDevice physical_device, VkFormat format, struct instance **inst) +{ + *inst = instance_from_handle(physical_device); + if (!(physical_device_gaps(*inst, physical_device) & GAP_BC)) + return NULL; + return find_bc(format); +} + +VKAPI_ATTR void VKAPI_CALL GetPhysicalDeviceFormatProperties(VkPhysicalDevice physical_device, VkFormat format, + VkFormatProperties *properties) +{ + struct instance *inst; + const struct bc_format *bc = physical_bc(physical_device, format, &inst); + inst->GetPhysicalDeviceFormatProperties(physical_device, bc ? bc->substitute : format, properties); + if (bc) { + properties->optimalTilingFeatures &= BC_FEATURES; + properties->linearTilingFeatures = 0; + properties->bufferFeatures = 0; + } +} + +VKAPI_ATTR void VKAPI_CALL GetPhysicalDeviceFormatProperties2(VkPhysicalDevice physical_device, VkFormat format, + VkFormatProperties2 *properties) +{ + struct instance *inst; + const struct bc_format *bc = physical_bc(physical_device, format, &inst); + inst->GetPhysicalDeviceFormatProperties2(physical_device, bc ? bc->substitute : format, properties); + if (!bc) + return; + properties->formatProperties.optimalTilingFeatures &= BC_FEATURES; + properties->formatProperties.linearTilingFeatures = 0; + properties->formatProperties.bufferFeatures = 0; + for (VkBaseOutStructure *s = properties->pNext; s; s = s->pNext) { + if (s->sType == VK_STRUCTURE_TYPE_FORMAT_PROPERTIES_3) { + VkFormatProperties3 *p3 = (VkFormatProperties3 *)s; + p3->optimalTilingFeatures &= (VkFormatFeatureFlags2)BC_FEATURES; + p3->linearTilingFeatures = 0; + p3->bufferFeatures = 0; + } else if (s->sType == VK_STRUCTURE_TYPE_DRM_FORMAT_MODIFIER_PROPERTIES_LIST_EXT) { + ((VkDrmFormatModifierPropertiesListEXT *)s)->drmFormatModifierCount = 0; + } else if (s->sType == VK_STRUCTURE_TYPE_DRM_FORMAT_MODIFIER_PROPERTIES_LIST_2_EXT) { + ((VkDrmFormatModifierPropertiesList2EXT *)s)->drmFormatModifierCount = 0; + } + } +} + +VKAPI_ATTR VkResult VKAPI_CALL GetPhysicalDeviceImageFormatProperties( + VkPhysicalDevice physical_device, VkFormat format, VkImageType type, VkImageTiling tiling, + VkImageUsageFlags usage, VkImageCreateFlags flags, VkImageFormatProperties *properties) +{ + struct instance *inst; + const struct bc_format *bc = physical_bc(physical_device, format, &inst); + if (!bc) + return inst->GetPhysicalDeviceImageFormatProperties(physical_device, format, type, tiling, usage, flags, + properties); + if (tiling != VK_IMAGE_TILING_OPTIMAL || (usage & ~BC_USAGE)) + return VK_ERROR_FORMAT_NOT_SUPPORTED; + return inst->GetPhysicalDeviceImageFormatProperties(physical_device, bc->substitute, type, tiling, usage, + flags & ~VK_IMAGE_CREATE_BLOCK_TEXEL_VIEW_COMPATIBLE_BIT, + properties); +} + +/* A view format list naming BC formats, as one naming their substitutes. */ +struct format_list { + VkImageFormatListCreateInfo *list; + const VkFormat *original; + VkFormat formats[16]; +}; + +static void substitute_format_list(const void *chain, VkFormat substitute, struct format_list *saved) +{ + saved->list = NULL; + for (const VkBaseInStructure *s = chain; s; s = s->pNext) { + if (s->sType != VK_STRUCTURE_TYPE_IMAGE_FORMAT_LIST_CREATE_INFO) + continue; + VkImageFormatListCreateInfo *list = (VkImageFormatListCreateInfo *)s; + uint32_t count = 0; + saved->formats[count++] = substitute; + for (uint32_t i = 0; i < list->viewFormatCount && count < 16; i++) { + VkFormat format = bc_substitute(list->pViewFormats[i]); + /* Uncompressed views of the compressed blocks have nothing to alias any more. */ + if (format == VK_FORMAT_UNDEFINED) + continue; + bool seen = false; + for (uint32_t k = 0; k < count; k++) + seen |= saved->formats[k] == format; + if (!seen) + saved->formats[count++] = format; + } + saved->list = list; + saved->original = list->pViewFormats; + list->pViewFormats = saved->formats; + list->viewFormatCount = count; + return; + } +} + +static void restore_format_list(struct format_list *saved, uint32_t original_count) +{ + if (saved->list) { + saved->list->pViewFormats = saved->original; + saved->list->viewFormatCount = original_count; + } +} + +static uint32_t format_list_count(const void *chain) +{ + for (const VkBaseInStructure *s = chain; s; s = s->pNext) { + if (s->sType == VK_STRUCTURE_TYPE_IMAGE_FORMAT_LIST_CREATE_INFO) + return ((const VkImageFormatListCreateInfo *)s)->viewFormatCount; + } + return 0; +} + +VKAPI_ATTR VkResult VKAPI_CALL GetPhysicalDeviceImageFormatProperties2( + VkPhysicalDevice physical_device, const VkPhysicalDeviceImageFormatInfo2 *info, + VkImageFormatProperties2 *properties) +{ + struct instance *inst; + const struct bc_format *bc = physical_bc(physical_device, info->format, &inst); + if (!bc) + return inst->GetPhysicalDeviceImageFormatProperties2(physical_device, info, properties); + if (info->tiling != VK_IMAGE_TILING_OPTIMAL || (info->usage & ~BC_USAGE)) + return VK_ERROR_FORMAT_NOT_SUPPORTED; + VkPhysicalDeviceImageFormatInfo2 copy = *info; + copy.format = bc->substitute; + copy.flags &= ~VK_IMAGE_CREATE_BLOCK_TEXEL_VIEW_COMPATIBLE_BIT; + uint32_t list_count = format_list_count(info->pNext); + struct format_list saved; + substitute_format_list(info->pNext, bc->substitute, &saved); + VkResult result = inst->GetPhysicalDeviceImageFormatProperties2(physical_device, ©, properties); + restore_format_list(&saved, list_count); + return result; +} + +struct bc_image { + const struct bc_format *format; + uint32_t array_layers; +}; + +static void substitute_image_info(const struct bc_format *bc, VkImageCreateInfo *info) +{ + info->format = bc->substitute; + info->flags &= ~VK_IMAGE_CREATE_BLOCK_TEXEL_VIEW_COMPATIBLE_BIT; + info->usage &= BC_USAGE; +} + +VKAPI_ATTR VkResult VKAPI_CALL CreateImage(VkDevice device, const VkImageCreateInfo *info, + const VkAllocationCallbacks *allocator, VkImage *image) +{ + struct device *dev = device_from_handle(device); + const struct bc_format *bc = find_bc(info->format); + if (!bc) + return dev->CreateImage(device, info, allocator, image); + + VkImageCreateInfo copy = *info; + substitute_image_info(bc, ©); + uint32_t list_count = format_list_count(info->pNext); + struct format_list saved; + substitute_format_list(info->pNext, bc->substitute, &saved); + VkResult result = dev->CreateImage(device, ©, allocator, image); + restore_format_list(&saved, list_count); + if (result != VK_SUCCESS) + return result; + + struct bc_image *tracked = malloc(sizeof(*tracked)); + tracked->format = bc; + tracked->array_layers = info->arrayLayers; + pthread_mutex_lock(&dev->lock); + map_put(&dev->images, HANDLE(*image), tracked); + pthread_mutex_unlock(&dev->lock); + return VK_SUCCESS; +} + +VKAPI_ATTR void VKAPI_CALL DestroyImage(VkDevice device, VkImage image, const VkAllocationCallbacks *allocator) +{ + struct device *dev = device_from_handle(device); + pthread_mutex_lock(&dev->lock); + free(map_remove(&dev->images, HANDLE(image))); + pthread_mutex_unlock(&dev->lock); + dev->DestroyImage(device, image, allocator); +} + +static const struct bc_image *tracked_image(struct device *dev, VkImage image) +{ + pthread_mutex_lock(&dev->lock); + const struct bc_image *tracked = map_get(&dev->images, HANDLE(image)); + pthread_mutex_unlock(&dev->lock); + return tracked; +} + +VKAPI_ATTR VkResult VKAPI_CALL CreateImageView(VkDevice device, const VkImageViewCreateInfo *info, + const VkAllocationCallbacks *allocator, VkImageView *view) +{ + struct device *dev = device_from_handle(device); + const struct bc_image *tracked = tracked_image(dev, info->image); + if (!tracked) + return dev->CreateImageView(device, info, allocator, view); + VkImageViewCreateInfo copy = *info; + VkFormat format = bc_substitute(info->format); + copy.format = format != VK_FORMAT_UNDEFINED ? format : tracked->format->substitute; + return dev->CreateImageView(device, ©, allocator, view); +} + +VKAPI_ATTR void VKAPI_CALL GetDeviceImageMemoryRequirements(VkDevice device, + const VkDeviceImageMemoryRequirements *info, + VkMemoryRequirements2 *requirements) +{ + struct device *dev = device_from_handle(device); + const struct bc_format *bc = find_bc(info->pCreateInfo->format); + if (!bc) { + dev->GetDeviceImageMemoryRequirements(device, info, requirements); + return; + } + VkImageCreateInfo image = *info->pCreateInfo; + substitute_image_info(bc, &image); + VkDeviceImageMemoryRequirements copy = *info; + copy.pCreateInfo = ℑ + dev->GetDeviceImageMemoryRequirements(device, ©, requirements); +} + +VKAPI_ATTR void VKAPI_CALL GetDeviceImageSparseMemoryRequirements( + VkDevice device, const VkDeviceImageMemoryRequirements *info, uint32_t *count, + VkSparseImageMemoryRequirements2 *requirements) +{ + struct device *dev = device_from_handle(device); + const struct bc_format *bc = find_bc(info->pCreateInfo->format); + if (!bc) { + dev->GetDeviceImageSparseMemoryRequirements(device, info, count, requirements); + return; + } + VkImageCreateInfo image = *info->pCreateInfo; + substitute_image_info(bc, &image); + VkDeviceImageMemoryRequirements copy = *info; + copy.pCreateInfo = ℑ + dev->GetDeviceImageSparseMemoryRequirements(device, ©, count, requirements); +} + +VKAPI_ATTR void VKAPI_CALL GetDeviceImageSubresourceLayout(VkDevice device, + const VkDeviceImageSubresourceInfo *info, + VkSubresourceLayout2 *layout) +{ + struct device *dev = device_from_handle(device); + const struct bc_format *bc = find_bc(info->pCreateInfo->format); + if (!bc) { + dev->GetDeviceImageSubresourceLayout(device, info, layout); + return; + } + VkImageCreateInfo image = *info->pCreateInfo; + substitute_image_info(bc, &image); + VkDeviceImageSubresourceInfo copy = *info; + copy.pCreateInfo = ℑ + dev->GetDeviceImageSubresourceLayout(device, ©, layout); +} + +struct bound_buffer { + VkDeviceMemory memory; + VkDeviceSize offset; +}; + +static void track_binding(struct device *dev, VkBuffer buffer, VkDeviceMemory memory, VkDeviceSize offset) +{ + struct bound_buffer *bound = malloc(sizeof(*bound)); + bound->memory = memory; + bound->offset = offset; + pthread_mutex_lock(&dev->lock); + free(map_remove(&dev->buffers, HANDLE(buffer))); + map_put(&dev->buffers, HANDLE(buffer), bound); + pthread_mutex_unlock(&dev->lock); +} + +VKAPI_ATTR VkResult VKAPI_CALL BindBufferMemory(VkDevice device, VkBuffer buffer, VkDeviceMemory memory, + VkDeviceSize offset) +{ + struct device *dev = device_from_handle(device); + VkResult result = dev->BindBufferMemory(device, buffer, memory, offset); + if (result == VK_SUCCESS) + track_binding(dev, buffer, memory, offset); + return result; +} + +VKAPI_ATTR VkResult VKAPI_CALL BindBufferMemory2(VkDevice device, uint32_t count, const VkBindBufferMemoryInfo *infos) +{ + struct device *dev = device_from_handle(device); + VkResult result = dev->BindBufferMemory2(device, count, infos); + if (result == VK_SUCCESS) { + for (uint32_t i = 0; i < count; i++) + track_binding(dev, infos[i].buffer, infos[i].memory, infos[i].memoryOffset); + } + return result; +} + +VKAPI_ATTR void VKAPI_CALL DestroyBuffer(VkDevice device, VkBuffer buffer, const VkAllocationCallbacks *allocator) +{ + struct device *dev = device_from_handle(device); + pthread_mutex_lock(&dev->lock); + free(map_remove(&dev->buffers, HANDLE(buffer))); + pthread_mutex_unlock(&dev->lock); + dev->DestroyBuffer(device, buffer, allocator); +} + +/* The memory's start as the application sees it mapped. */ +static void track_mapping(struct device *dev, VkDeviceMemory memory, VkDeviceSize offset, void *data) +{ + pthread_mutex_lock(&dev->lock); + map_put(&dev->memories, HANDLE(memory), (uint8_t *)data - offset); + pthread_mutex_unlock(&dev->lock); +} + +static void forget_mapping(struct device *dev, VkDeviceMemory memory) +{ + pthread_mutex_lock(&dev->lock); + map_remove(&dev->memories, HANDLE(memory)); + pthread_mutex_unlock(&dev->lock); +} + +VKAPI_ATTR VkResult VKAPI_CALL MapMemory(VkDevice device, VkDeviceMemory memory, VkDeviceSize offset, + VkDeviceSize size, VkMemoryMapFlags flags, void **data) +{ + struct device *dev = device_from_handle(device); + VkResult result = dev->MapMemory(device, memory, offset, size, flags, data); + if (result == VK_SUCCESS) + track_mapping(dev, memory, offset, *data); + return result; +} + +VKAPI_ATTR VkResult VKAPI_CALL MapMemory2(VkDevice device, const VkMemoryMapInfo *info, void **data) +{ + struct device *dev = device_from_handle(device); + VkResult result = dev->MapMemory2(device, info, data); + if (result == VK_SUCCESS) + track_mapping(dev, info->memory, info->offset, *data); + return result; +} + +VKAPI_ATTR void VKAPI_CALL UnmapMemory(VkDevice device, VkDeviceMemory memory) +{ + struct device *dev = device_from_handle(device); + forget_mapping(dev, memory); + dev->UnmapMemory(device, memory); +} + +VKAPI_ATTR VkResult VKAPI_CALL UnmapMemory2(VkDevice device, const VkMemoryUnmapInfo *info) +{ + struct device *dev = device_from_handle(device); + forget_mapping(dev, info->memory); + return dev->UnmapMemory2(device, info); +} + +VKAPI_ATTR void VKAPI_CALL FreeMemory(VkDevice device, VkDeviceMemory memory, const VkAllocationCallbacks *allocator) +{ + struct device *dev = device_from_handle(device); + forget_mapping(dev, memory); + dev->FreeMemory(device, memory, allocator); +} + +struct staging_block { + VkBuffer buffer; + VkDeviceMemory memory; + uint8_t *data; + VkDeviceSize size; + VkDeviceSize used; + struct staging_block *next; +}; + +struct command_buffer { + VkCommandPool pool; + struct staging_block *staging; +}; + +static void destroy_block(struct device *dev, struct staging_block *block) +{ + dev->DestroyBuffer(dev->handle, block->buffer, NULL); + dev->FreeMemory(dev->handle, block->memory, NULL); + free(block); +} + +static struct staging_block *create_block(struct device *dev, VkDeviceSize size) +{ + struct staging_block *block = calloc(1, sizeof(*block)); + block->size = size; + VkBufferCreateInfo buffer_info = { + .sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO, + .size = size, + .usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT, + .sharingMode = VK_SHARING_MODE_EXCLUSIVE, + }; + if (dev->CreateBuffer(dev->handle, &buffer_info, NULL, &block->buffer) != VK_SUCCESS) { + free(block); + return NULL; + } + VkMemoryRequirements requirements; + dev->GetBufferMemoryRequirements(dev->handle, block->buffer, &requirements); + const VkMemoryPropertyFlags wanted = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT; + uint32_t type = UINT32_MAX; + for (uint32_t i = 0; i < dev->memory_properties.memoryTypeCount && type == UINT32_MAX; i++) { + if ((requirements.memoryTypeBits & (1u << i)) && + (dev->memory_properties.memoryTypes[i].propertyFlags & wanted) == wanted) + type = i; + } + VkMemoryAllocateInfo allocate_info = { + .sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO, + .allocationSize = requirements.size, + .memoryTypeIndex = type, + }; + void *data = NULL; + if (type == UINT32_MAX || dev->AllocateMemory(dev->handle, &allocate_info, NULL, &block->memory) != VK_SUCCESS) { + dev->DestroyBuffer(dev->handle, block->buffer, NULL); + free(block); + return NULL; + } + if (dev->BindBufferMemory(dev->handle, block->buffer, block->memory, 0) != VK_SUCCESS || + dev->MapMemory(dev->handle, block->memory, 0, VK_WHOLE_SIZE, 0, &data) != VK_SUCCESS) { + destroy_block(dev, block); + return NULL; + } + block->data = data; + return block; +} + +/* Called with the device lock held: the command buffer is done with its staging. */ +static void recycle_staging(struct device *dev, struct command_buffer *state) +{ + VkDeviceSize kept = 0; + for (struct staging_block *b = dev->free_staging; b; b = b->next) + kept += b->size; + while (state->staging) { + struct staging_block *block = state->staging; + state->staging = block->next; + if (kept + block->size > STAGING_KEPT) { + destroy_block(dev, block); + continue; + } + block->used = 0; + block->next = dev->free_staging; + dev->free_staging = block; + kept += block->size; + } +} + +/* Room for `size` bytes in a staging buffer that lives as long as the command buffer's recording. */ +static struct staging_block *staging_for(struct device *dev, VkCommandBuffer cmd, VkDeviceSize size, + VkDeviceSize *offset) +{ + pthread_mutex_lock(&dev->lock); + struct command_buffer *state = map_get(&dev->command_buffers, HANDLE(cmd)); + if (!state) { + state = calloc(1, sizeof(*state)); + map_put(&dev->command_buffers, HANDLE(cmd), state); + } + struct staging_block *block = state->staging; + VkDeviceSize start = block ? (block->used + 63) & ~(VkDeviceSize)63 : 0; + if (!block || start + size > block->size) { + struct staging_block **link = &dev->free_staging; + while (*link && (*link)->size < size) + link = &(*link)->next; + block = *link; + if (block) + *link = block->next; + else + block = create_block(dev, size > STAGING_BLOCK_SIZE ? size : STAGING_BLOCK_SIZE); + if (block) { + block->next = state->staging; + state->staging = block; + } + start = 0; + } + if (block) + block->used = start + size; + pthread_mutex_unlock(&dev->lock); + *offset = start; + return block; +} + +/* Where the application has the buffer's contents mapped, or NULL. */ +static const uint8_t *mapped_buffer(struct device *dev, VkBuffer buffer) +{ + pthread_mutex_lock(&dev->lock); + const struct bound_buffer *bound = map_get(&dev->buffers, HANDLE(buffer)); + const uint8_t *base = bound ? map_get(&dev->memories, HANDLE(bound->memory)) : NULL; + const uint8_t *data = base ? base + bound->offset : NULL; + pthread_mutex_unlock(&dev->lock); + return data; +} + +static void decode_block(const struct bc_format *bc, const uint8_t *block, uint8_t *texels) +{ + switch (bc->kind) { + case BC1: + bcdec_bc1(block, texels, 16); + if (bc->opaque) { + for (int i = 0; i < 16; i++) + texels[i * 4 + 3] = 0xff; + } + break; + case BC2: + bcdec_bc2(block, texels, 16); + break; + case BC3: + bcdec_bc3(block, texels, 16); + break; + case BC4: + bcdec_bc4(block, texels, 4, bc->is_signed); + break; + case BC5: + bcdec_bc5(block, texels, 8, bc->is_signed); + break; + case BC6H: { + uint16_t rgb[16 * 3]; + bcdec_bc6h_half(block, rgb, 12, bc->is_signed); + uint16_t *rgba = (uint16_t *)texels; + for (int i = 0; i < 16; i++) { + rgba[i * 4 + 0] = rgb[i * 3 + 0]; + rgba[i * 4 + 1] = rgb[i * 3 + 1]; + rgba[i * 4 + 2] = rgb[i * 3 + 2]; + rgba[i * 4 + 3] = 0x3c00; + } + break; + } + case BC7: + bcdec_bc7(block, texels, 16); + break; + } +} + +struct upload { + VkDeviceSize offset; + uint32_t row_length; + uint32_t image_height; + VkImageSubresourceLayers subresource; + VkOffset3D image_offset; + VkExtent3D extent; +}; + +static uint32_t warned; + +static void warn_once(uint32_t bit, const char *message) +{ + if (!(__atomic_fetch_or(&warned, bit, __ATOMIC_RELAXED) & bit)) + layer_log("%s", message); +} + +static void upload_region(struct device *dev, VkCommandBuffer cmd, const struct bc_image *image, + const uint8_t *source, VkImage dst, VkImageLayout layout, const struct upload *region) +{ + const struct bc_format *bc = image->format; + uint32_t width = region->extent.width, height = region->extent.height; + uint32_t layers = region->subresource.layerCount; + if (layers == VK_REMAINING_ARRAY_LAYERS) + layers = image->array_layers - region->subresource.baseArrayLayer; + uint32_t slices = region->extent.depth * layers; + uint32_t row_texels = region->row_length ? region->row_length : width; + uint32_t height_texels = region->image_height ? region->image_height : height; + size_t source_row = (size_t)((row_texels + 3) / 4) * bc->block_bytes; + size_t source_slice = (size_t)((height_texels + 3) / 4) * source_row; + size_t row_bytes = (size_t)width * bc->texel_bytes; + VkDeviceSize size = (VkDeviceSize)row_bytes * height * slices; + if (!size) + return; + + VkDeviceSize offset; + struct staging_block *block = staging_for(dev, cmd, size, &offset); + if (!block) { + warn_once(1u << 0, "out of staging memory for BC texture uploads; textures left empty"); + return; + } + uint8_t *out = block->data + offset; + const uint8_t *in = source + region->offset; + uint8_t texels[16 * 8]; + for (uint32_t s = 0; s < slices; s++) { + for (uint32_t by = 0; by < (height + 3) / 4; by++) { + for (uint32_t bx = 0; bx < (width + 3) / 4; bx++) { + decode_block(bc, in + s * source_slice + by * source_row + (size_t)bx * bc->block_bytes, texels); + for (uint32_t y = 0; y < 4 && by * 4 + y < height; y++) { + uint32_t columns = width - bx * 4 < 4 ? width - bx * 4 : 4; + memcpy(out + ((size_t)s * height + by * 4 + y) * row_bytes + (size_t)bx * 4 * bc->texel_bytes, + texels + (size_t)y * 4 * bc->texel_bytes, (size_t)columns * bc->texel_bytes); + } + } + } + } + + VkBufferImageCopy copy = { + .bufferOffset = offset, + .imageSubresource = region->subresource, + .imageOffset = region->image_offset, + .imageExtent = region->extent, + }; + copy.imageSubresource.layerCount = layers; + dev->CmdCopyBufferToImage(cmd, block->buffer, dst, layout, 1, ©); +} + +VKAPI_ATTR void VKAPI_CALL CmdCopyBufferToImage(VkCommandBuffer cmd, VkBuffer buffer, VkImage image, + VkImageLayout layout, uint32_t count, + const VkBufferImageCopy *regions) +{ + struct device *dev = device_from_handle(cmd); + const struct bc_image *tracked = tracked_image(dev, image); + if (!tracked) { + dev->CmdCopyBufferToImage(cmd, buffer, image, layout, count, regions); + return; + } + const uint8_t *source = mapped_buffer(dev, buffer); + if (!source) { + warn_once(1u << 1, "BC texture upload from memory that is not mapped; texture left empty"); + return; + } + for (uint32_t i = 0; i < count; i++) { + struct upload region = {regions[i].bufferOffset, regions[i].bufferRowLength, regions[i].bufferImageHeight, + regions[i].imageSubresource, regions[i].imageOffset, regions[i].imageExtent}; + upload_region(dev, cmd, tracked, source, image, layout, ®ion); + } +} + +VKAPI_ATTR void VKAPI_CALL CmdCopyBufferToImage2(VkCommandBuffer cmd, const VkCopyBufferToImageInfo2 *info) +{ + struct device *dev = device_from_handle(cmd); + const struct bc_image *tracked = tracked_image(dev, info->dstImage); + if (!tracked) { + dev->CmdCopyBufferToImage2(cmd, info); + return; + } + const uint8_t *source = mapped_buffer(dev, info->srcBuffer); + if (!source) { + warn_once(1u << 1, "BC texture upload from memory that is not mapped; texture left empty"); + return; + } + for (uint32_t i = 0; i < info->regionCount; i++) { + const VkBufferImageCopy2 *r = &info->pRegions[i]; + struct upload region = {r->bufferOffset, r->bufferRowLength, r->bufferImageHeight, + r->imageSubresource, r->imageOffset, r->imageExtent}; + upload_region(dev, cmd, tracked, source, info->dstImage, info->dstImageLayout, ®ion); + } +} + +/* Reading compressed blocks back would hand out decoded texels the buffer has no room for. */ +VKAPI_ATTR void VKAPI_CALL CmdCopyImageToBuffer(VkCommandBuffer cmd, VkImage image, VkImageLayout layout, + VkBuffer buffer, uint32_t count, const VkBufferImageCopy *regions) +{ + struct device *dev = device_from_handle(cmd); + if (tracked_image(dev, image)) { + warn_once(1u << 2, "BC texture read back into a buffer; skipped"); + return; + } + dev->CmdCopyImageToBuffer(cmd, image, layout, buffer, count, regions); +} + +VKAPI_ATTR void VKAPI_CALL CmdCopyImageToBuffer2(VkCommandBuffer cmd, const VkCopyImageToBufferInfo2 *info) +{ + struct device *dev = device_from_handle(cmd); + if (tracked_image(dev, info->srcImage)) { + warn_once(1u << 2, "BC texture read back into a buffer; skipped"); + return; + } + dev->CmdCopyImageToBuffer2(cmd, info); +} + +/* Between two BC images the texels line up; between BC and anything else they cannot. */ +VKAPI_ATTR void VKAPI_CALL CmdCopyImage(VkCommandBuffer cmd, VkImage src, VkImageLayout src_layout, VkImage dst, + VkImageLayout dst_layout, uint32_t count, const VkImageCopy *regions) +{ + struct device *dev = device_from_handle(cmd); + if (!tracked_image(dev, src) != !tracked_image(dev, dst)) { + warn_once(1u << 3, "copy between a BC texture and an uncompressed one; skipped"); + return; + } + dev->CmdCopyImage(cmd, src, src_layout, dst, dst_layout, count, regions); +} + +VKAPI_ATTR void VKAPI_CALL CmdCopyImage2(VkCommandBuffer cmd, const VkCopyImageInfo2 *info) +{ + struct device *dev = device_from_handle(cmd); + if (!tracked_image(dev, info->srcImage) != !tracked_image(dev, info->dstImage)) { + warn_once(1u << 3, "copy between a BC texture and an uncompressed one; skipped"); + return; + } + dev->CmdCopyImage2(cmd, info); +} + +VKAPI_ATTR VkResult VKAPI_CALL AllocateCommandBuffers(VkDevice device, const VkCommandBufferAllocateInfo *info, + VkCommandBuffer *buffers) +{ + struct device *dev = device_from_handle(device); + VkResult result = dev->AllocateCommandBuffers(device, info, buffers); + if (result != VK_SUCCESS) + return result; + pthread_mutex_lock(&dev->lock); + for (uint32_t i = 0; i < info->commandBufferCount; i++) { + struct command_buffer *state = calloc(1, sizeof(*state)); + state->pool = info->commandPool; + map_put(&dev->command_buffers, HANDLE(buffers[i]), state); + } + pthread_mutex_unlock(&dev->lock); + return result; +} + +VKAPI_ATTR void VKAPI_CALL FreeCommandBuffers(VkDevice device, VkCommandPool pool, uint32_t count, + const VkCommandBuffer *buffers) +{ + struct device *dev = device_from_handle(device); + pthread_mutex_lock(&dev->lock); + for (uint32_t i = 0; i < count; i++) { + struct command_buffer *state = buffers[i] ? map_remove(&dev->command_buffers, HANDLE(buffers[i])) : NULL; + if (state) { + recycle_staging(dev, state); + free(state); + } + } + pthread_mutex_unlock(&dev->lock); + dev->FreeCommandBuffers(device, pool, count, buffers); +} + +static void recycle_command_buffer(struct device *dev, VkCommandBuffer cmd) +{ + pthread_mutex_lock(&dev->lock); + struct command_buffer *state = map_get(&dev->command_buffers, HANDLE(cmd)); + if (state) + recycle_staging(dev, state); + pthread_mutex_unlock(&dev->lock); +} + +VKAPI_ATTR VkResult VKAPI_CALL BeginCommandBuffer(VkCommandBuffer cmd, const VkCommandBufferBeginInfo *info) +{ + struct device *dev = device_from_handle(cmd); + recycle_command_buffer(dev, cmd); + return dev->BeginCommandBuffer(cmd, info); +} + +VKAPI_ATTR VkResult VKAPI_CALL ResetCommandBuffer(VkCommandBuffer cmd, VkCommandBufferResetFlags flags) +{ + struct device *dev = device_from_handle(cmd); + recycle_command_buffer(dev, cmd); + return dev->ResetCommandBuffer(cmd, flags); +} + +/* Every command buffer of the pool is done with its staging; `forget` drops them as well. */ +static void recycle_pool(struct device *dev, VkCommandPool pool, bool forget) +{ + pthread_mutex_lock(&dev->lock); + struct handle_map *map = &dev->command_buffers; + for (uint32_t i = 0; i < map->capacity;) { + struct command_buffer *state = map->values[i]; + if (!map->keys[i] || state->pool != pool) { + i++; + continue; + } + recycle_staging(dev, state); + if (forget) { + /* Removal shifts a later entry into this slot, so look at it again. */ + free(map_remove(map, map->keys[i])); + continue; + } + i++; + } + pthread_mutex_unlock(&dev->lock); +} + +VKAPI_ATTR VkResult VKAPI_CALL ResetCommandPool(VkDevice device, VkCommandPool pool, VkCommandPoolResetFlags flags) +{ + struct device *dev = device_from_handle(device); + recycle_pool(dev, pool, false); + return dev->ResetCommandPool(device, pool, flags); +} + +VKAPI_ATTR void VKAPI_CALL DestroyCommandPool(VkDevice device, VkCommandPool pool, + const VkAllocationCallbacks *allocator) +{ + struct device *dev = device_from_handle(device); + recycle_pool(dev, pool, true); + dev->DestroyCommandPool(device, pool, allocator); +} + +void bc_destroy_device(struct device *dev) +{ + struct handle_map *maps[] = {&dev->images, &dev->buffers, &dev->command_buffers}; + for (size_t m = 0; m < sizeof(maps) / sizeof(maps[0]); m++) { + for (uint32_t i = 0; i < maps[m]->capacity; i++) { + if (!maps[m]->keys[i]) + continue; + if (maps[m] == &dev->command_buffers) + recycle_staging(dev, maps[m]->values[i]); + free(maps[m]->values[i]); + } + map_free(maps[m]); + } + map_free(&dev->memories); + while (dev->free_staging) { + struct staging_block *block = dev->free_staging; + dev->free_staging = block->next; + destroy_block(dev, block); + } +} diff --git a/tools/venus/layer/layer.c b/tools/venus/layer/layer.c new file mode 100644 index 000000000..98e2dc17d --- /dev/null +++ b/tools/venus/layer/layer.c @@ -0,0 +1,617 @@ +#include "mali_compat.h" + +#include +#include +#include +#include + +#define LAYER_NAME "VK_LAYER_DROIDDECK_mali_compat" + +static pthread_mutex_t global_lock = PTHREAD_MUTEX_INITIALIZER; +static struct instance *instances; +static struct device *devices; + +void layer_log(const char *format, ...) +{ + char line[512]; + va_list args; + va_start(args, format); + vsnprintf(line, sizeof(line), format, args); + va_end(args); + fprintf(stderr, "droiddeck-mali-compat: %s\n", line); +} + +/* The loader's dispatch table pointer: shared by an instance and its physical devices, and by a + * device and its queues and command buffers. */ +static void *dispatch_key(const void *dispatchable) +{ + return *(void *const *)dispatchable; +} + +struct instance *instance_from_handle(const void *dispatchable) +{ + void *key = dispatch_key(dispatchable); + pthread_mutex_lock(&global_lock); + struct instance *inst = instances; + while (inst && inst->key != key) + inst = inst->next; + pthread_mutex_unlock(&global_lock); + return inst; +} + +struct device *device_from_handle(const void *dispatchable) +{ + void *key = dispatch_key(dispatchable); + pthread_mutex_lock(&global_lock); + struct device *dev = devices; + while (dev && dev->key != key) + dev = dev->next; + pthread_mutex_unlock(&global_lock); + return dev; +} + +static uint64_t hash(uint64_t key) +{ + key ^= key >> 33; + key *= 0xff51afd7ed558ccdull; + key ^= key >> 33; + return key; +} + +void *map_get(const struct handle_map *map, uint64_t key) +{ + if (!map->capacity) + return NULL; + for (uint32_t i = hash(key) & (map->capacity - 1);; i = (i + 1) & (map->capacity - 1)) { + if (map->keys[i] == key) + return map->values[i]; + if (!map->keys[i]) + return NULL; + } +} + +static void map_grow(struct handle_map *map) +{ + struct handle_map bigger = { + .capacity = map->capacity ? map->capacity * 2 : 64, + }; + bigger.keys = calloc(bigger.capacity, sizeof(*bigger.keys)); + bigger.values = calloc(bigger.capacity, sizeof(*bigger.values)); + for (uint32_t i = 0; i < map->capacity; i++) { + if (map->keys[i]) + map_put(&bigger, map->keys[i], map->values[i]); + } + free(map->keys); + free(map->values); + *map = bigger; +} + +void map_put(struct handle_map *map, uint64_t key, void *value) +{ + if ((map->count + 1) * 4 > map->capacity * 3) + map_grow(map); + uint32_t i = hash(key) & (map->capacity - 1); + while (map->keys[i] && map->keys[i] != key) + i = (i + 1) & (map->capacity - 1); + if (!map->keys[i]) + map->count++; + map->keys[i] = key; + map->values[i] = value; +} + +void *map_remove(struct handle_map *map, uint64_t key) +{ + if (!map->capacity) + return NULL; + uint32_t mask = map->capacity - 1; + uint32_t i = hash(key) & mask; + while (map->keys[i] != key) { + if (!map->keys[i]) + return NULL; + i = (i + 1) & mask; + } + void *value = map->values[i]; + map->count--; + /* Backward-shift deletion keeps every remaining key reachable from its home slot. */ + for (uint32_t j = (i + 1) & mask; map->keys[j]; j = (j + 1) & mask) { + uint32_t home = hash(map->keys[j]) & mask; + if (((j - home) & mask) >= ((j - i) & mask)) { + map->keys[i] = map->keys[j]; + map->values[i] = map->values[j]; + i = j; + } + } + map->keys[i] = 0; + map->values[i] = NULL; + return value; +} + +void map_free(struct handle_map *map) +{ + free(map->keys); + free(map->values); + *map = (struct handle_map){0}; +} + +/* Wine processes only: the Steam client and gamescope run fine without the layer, and their + * Zink would otherwise start using the made-up features too. */ +static bool wanted(void) +{ + const char *setting = getenv("DROIDDECK_MALI_COMPAT"); + if (setting) + return strcmp(setting, "0") != 0; + return getenv("WINEPREFIX") != NULL; +} + +static bool has_extension(struct instance *inst, VkPhysicalDevice physical_device, const char *name) +{ + uint32_t count = 0; + inst->EnumerateDeviceExtensionProperties(physical_device, NULL, &count, NULL); + VkExtensionProperties *extensions = calloc(count, sizeof(*extensions)); + inst->EnumerateDeviceExtensionProperties(physical_device, NULL, &count, extensions); + bool found = false; + for (uint32_t i = 0; i < count && !found; i++) + found = strcmp(extensions[i].extensionName, name) == 0; + free(extensions); + return found; +} + +uint32_t physical_device_gaps(struct instance *inst, VkPhysicalDevice physical_device) +{ + pthread_mutex_lock(&inst->lock); + for (uint32_t i = 0; i < inst->physical_device_count; i++) { + if (inst->physical_devices[i].handle == physical_device) { + uint32_t gaps = inst->physical_devices[i].gaps; + pthread_mutex_unlock(&inst->lock); + return gaps; + } + } + pthread_mutex_unlock(&inst->lock); + + bool robustness2 = has_extension(inst, physical_device, "VK_EXT_robustness2"); + VkPhysicalDeviceRobustness2FeaturesEXT robust = { + .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_ROBUSTNESS_2_FEATURES_EXT, + }; + VkPhysicalDeviceFeatures2 features = { + .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2, + .pNext = robustness2 ? &robust : NULL, + }; + inst->GetPhysicalDeviceFeatures2(physical_device, &features); + uint32_t gaps = 0; + if (!features.features.fillModeNonSolid) + gaps |= GAP_FILL_MODE; + if (!features.features.multiViewport) + gaps |= GAP_MULTI_VIEWPORT; + if (!features.features.shaderClipDistance) + gaps |= GAP_CLIP_DISTANCE; + if (!features.features.shaderCullDistance) + gaps |= GAP_CULL_DISTANCE; + if (!features.features.textureCompressionBC) + gaps |= GAP_BC; + if (robustness2 && !robust.robustBufferAccess2) + gaps |= GAP_ROBUSTNESS2; + + pthread_mutex_lock(&inst->lock); + if (inst->physical_device_count < sizeof(inst->physical_devices) / sizeof(inst->physical_devices[0])) { + inst->physical_devices[inst->physical_device_count++] = + (struct physical_device){.handle = physical_device, .gaps = gaps}; + } + pthread_mutex_unlock(&inst->lock); + return gaps; +} + +static void report_features(VkPhysicalDeviceFeatures *features, uint32_t gaps) +{ + if (gaps & GAP_FILL_MODE) + features->fillModeNonSolid = VK_TRUE; + if (gaps & GAP_MULTI_VIEWPORT) + features->multiViewport = VK_TRUE; + if (gaps & GAP_CLIP_DISTANCE) + features->shaderClipDistance = VK_TRUE; + if (gaps & GAP_CULL_DISTANCE) + features->shaderCullDistance = VK_TRUE; + if (gaps & GAP_BC) + features->textureCompressionBC = VK_TRUE; +} + +/* Which made-up features the application asked for, and off with them before the driver sees them. */ +static uint32_t take_features(VkPhysicalDeviceFeatures *features, uint32_t gaps) +{ + uint32_t requested = 0; + if ((gaps & GAP_FILL_MODE) && features->fillModeNonSolid) { + requested |= GAP_FILL_MODE; + features->fillModeNonSolid = VK_FALSE; + } + if ((gaps & GAP_MULTI_VIEWPORT) && features->multiViewport) { + requested |= GAP_MULTI_VIEWPORT; + features->multiViewport = VK_FALSE; + } + if ((gaps & GAP_CLIP_DISTANCE) && features->shaderClipDistance) { + requested |= GAP_CLIP_DISTANCE; + features->shaderClipDistance = VK_FALSE; + } + if ((gaps & GAP_CULL_DISTANCE) && features->shaderCullDistance) { + requested |= GAP_CULL_DISTANCE; + features->shaderCullDistance = VK_FALSE; + } + if ((gaps & GAP_BC) && features->textureCompressionBC) { + requested |= GAP_BC; + features->textureCompressionBC = VK_FALSE; + } + return requested; +} + +static void report_limits(VkPhysicalDeviceLimits *limits, uint32_t gaps) +{ + if (gaps & GAP_MULTI_VIEWPORT) + limits->maxViewports = 16; + if (gaps & GAPS_SHADER) { + limits->maxClipDistances = 8; + limits->maxCullDistances = 8; + limits->maxCombinedClipAndCullDistances = 8; + } +} + +static VKAPI_ATTR void VKAPI_CALL GetPhysicalDeviceFeatures(VkPhysicalDevice physical_device, + VkPhysicalDeviceFeatures *features) +{ + struct instance *inst = instance_from_handle(physical_device); + inst->GetPhysicalDeviceFeatures(physical_device, features); + report_features(features, physical_device_gaps(inst, physical_device)); +} + +static VKAPI_ATTR void VKAPI_CALL GetPhysicalDeviceFeatures2(VkPhysicalDevice physical_device, + VkPhysicalDeviceFeatures2 *features) +{ + struct instance *inst = instance_from_handle(physical_device); + inst->GetPhysicalDeviceFeatures2(physical_device, features); + uint32_t gaps = physical_device_gaps(inst, physical_device); + report_features(&features->features, gaps); + for (VkBaseOutStructure *s = features->pNext; s; s = s->pNext) { + if (s->sType == VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_ROBUSTNESS_2_FEATURES_EXT && (gaps & GAP_ROBUSTNESS2)) + ((VkPhysicalDeviceRobustness2FeaturesEXT *)s)->robustBufferAccess2 = VK_TRUE; + } +} + +static VKAPI_ATTR void VKAPI_CALL GetPhysicalDeviceProperties(VkPhysicalDevice physical_device, + VkPhysicalDeviceProperties *properties) +{ + struct instance *inst = instance_from_handle(physical_device); + inst->GetPhysicalDeviceProperties(physical_device, properties); + report_limits(&properties->limits, physical_device_gaps(inst, physical_device)); +} + +static VKAPI_ATTR void VKAPI_CALL GetPhysicalDeviceProperties2(VkPhysicalDevice physical_device, + VkPhysicalDeviceProperties2 *properties) +{ + struct instance *inst = instance_from_handle(physical_device); + inst->GetPhysicalDeviceProperties2(physical_device, properties); + report_limits(&properties->properties.limits, physical_device_gaps(inst, physical_device)); +} + +static VKAPI_ATTR VkResult VKAPI_CALL CreateInstance(const VkInstanceCreateInfo *info, + const VkAllocationCallbacks *allocator, VkInstance *instance) +{ + VkLayerInstanceCreateInfo *link = (VkLayerInstanceCreateInfo *)info->pNext; + while (link && !(link->sType == VK_STRUCTURE_TYPE_LOADER_INSTANCE_CREATE_INFO && link->function == VK_LAYER_LINK_INFO)) + link = (VkLayerInstanceCreateInfo *)link->pNext; + if (!link) + return VK_ERROR_INITIALIZATION_FAILED; + PFN_vkGetInstanceProcAddr gipa = link->u.pLayerInfo->pfnNextGetInstanceProcAddr; + link->u.pLayerInfo = link->u.pLayerInfo->pNext; + + PFN_vkCreateInstance create = (PFN_vkCreateInstance)gipa(VK_NULL_HANDLE, "vkCreateInstance"); + VkResult result = create(info, allocator, instance); + if (result != VK_SUCCESS) + return result; + + struct instance *inst = calloc(1, sizeof(*inst)); + inst->key = dispatch_key(*instance); + inst->handle = *instance; + inst->active = wanted(); + inst->gipa = gipa; + pthread_mutex_init(&inst->lock, NULL); +#define LOAD(name) inst->name = (PFN_vk##name)gipa(*instance, "vk" #name); +#define LOAD_KHR(name) \ + if (!inst->name) \ + inst->name = (PFN_vk##name)gipa(*instance, "vk" #name "KHR"); + LOAD(DestroyInstance) + LOAD(EnumerateDeviceExtensionProperties) + LOAD(GetPhysicalDeviceFeatures) + LOAD(GetPhysicalDeviceFeatures2) + LOAD_KHR(GetPhysicalDeviceFeatures2) + LOAD(GetPhysicalDeviceProperties) + LOAD(GetPhysicalDeviceProperties2) + LOAD_KHR(GetPhysicalDeviceProperties2) + LOAD(GetPhysicalDeviceMemoryProperties) + LOAD(GetPhysicalDeviceFormatProperties) + LOAD(GetPhysicalDeviceFormatProperties2) + LOAD_KHR(GetPhysicalDeviceFormatProperties2) + LOAD(GetPhysicalDeviceImageFormatProperties) + LOAD(GetPhysicalDeviceImageFormatProperties2) + LOAD_KHR(GetPhysicalDeviceImageFormatProperties2) +#undef LOAD +#undef LOAD_KHR + /* Without the Vulkan 1.1 queries there is no way to answer the chained ones. */ + if (!inst->GetPhysicalDeviceFeatures2 || !inst->GetPhysicalDeviceProperties2 || + !inst->GetPhysicalDeviceFormatProperties2 || !inst->GetPhysicalDeviceImageFormatProperties2) + inst->active = false; + + pthread_mutex_lock(&global_lock); + inst->next = instances; + instances = inst; + pthread_mutex_unlock(&global_lock); + return VK_SUCCESS; +} + +static VKAPI_ATTR void VKAPI_CALL DestroyInstance(VkInstance instance, const VkAllocationCallbacks *allocator) +{ + void *key = dispatch_key(instance); + pthread_mutex_lock(&global_lock); + struct instance **link = &instances; + while (*link && (*link)->key != key) + link = &(*link)->next; + struct instance *inst = *link; + if (inst) + *link = inst->next; + pthread_mutex_unlock(&global_lock); + if (!inst) + return; + inst->DestroyInstance(instance, allocator); + pthread_mutex_destroy(&inst->lock); + free(inst); +} + +static VKAPI_ATTR VkResult VKAPI_CALL CreateDevice(VkPhysicalDevice physical_device, const VkDeviceCreateInfo *info, + const VkAllocationCallbacks *allocator, VkDevice *device) +{ + VkLayerDeviceCreateInfo *link = (VkLayerDeviceCreateInfo *)info->pNext; + while (link && !(link->sType == VK_STRUCTURE_TYPE_LOADER_DEVICE_CREATE_INFO && link->function == VK_LAYER_LINK_INFO)) + link = (VkLayerDeviceCreateInfo *)link->pNext; + if (!link) + return VK_ERROR_INITIALIZATION_FAILED; + PFN_vkGetInstanceProcAddr gipa = link->u.pLayerInfo->pfnNextGetInstanceProcAddr; + PFN_vkGetDeviceProcAddr gdpa = link->u.pLayerInfo->pfnNextGetDeviceProcAddr; + link->u.pLayerInfo = link->u.pLayerInfo->pNext; + + struct instance *inst = instance_from_handle(physical_device); + PFN_vkCreateDevice create = (PFN_vkCreateDevice)gipa(inst->handle, "vkCreateDevice"); + uint32_t gaps = inst->active ? physical_device_gaps(inst, physical_device) : 0; + + /* The application's own structures are put back after the call: DXVK reads its enabled + * features again later, and must keep seeing what it asked for. */ + VkDeviceCreateInfo copy = *info; + VkPhysicalDeviceFeatures features; + VkPhysicalDeviceFeatures2 *features2 = NULL; + VkPhysicalDeviceRobustness2FeaturesEXT *robust = NULL; + for (VkBaseOutStructure *s = (VkBaseOutStructure *)info->pNext; s; s = s->pNext) { + if (s->sType == VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2) + features2 = (VkPhysicalDeviceFeatures2 *)s; + else if (s->sType == VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_ROBUSTNESS_2_FEATURES_EXT) + robust = (VkPhysicalDeviceRobustness2FeaturesEXT *)s; + } + VkPhysicalDeviceFeatures *enabled = NULL; + VkPhysicalDeviceFeatures saved_features = {0}; + if (info->pEnabledFeatures) { + features = *info->pEnabledFeatures; + copy.pEnabledFeatures = &features; + enabled = &features; + } else if (features2) { + saved_features = features2->features; + enabled = &features2->features; + } + uint32_t requested = enabled ? take_features(enabled, gaps) : 0; + VkBool32 saved_robust = robust ? robust->robustBufferAccess2 : VK_FALSE; + if (robust && (gaps & GAP_ROBUSTNESS2) && robust->robustBufferAccess2) { + requested |= GAP_ROBUSTNESS2; + robust->robustBufferAccess2 = VK_FALSE; + /* The first version of robust buffer access is the nearest the driver has. */ + if (enabled) + enabled->robustBufferAccess = VK_TRUE; + } + + VkResult result = create(physical_device, ©, allocator, device); + + if (features2 && !info->pEnabledFeatures) + features2->features = saved_features; + if (robust) + robust->robustBufferAccess2 = saved_robust; + if (result != VK_SUCCESS) + return result; + + struct device *dev = calloc(1, sizeof(*dev)); + dev->key = dispatch_key(*device); + dev->handle = *device; + dev->gdpa = gdpa; + dev->emulated = requested; + pthread_mutex_init(&dev->lock, NULL); +#define LOAD(name) \ + dev->name = (PFN_vk##name)gdpa(*device, "vk" #name); \ + if (!dev->name) \ + dev->name = (PFN_vk##name)gdpa(*device, "vk" #name "KHR"); \ + if (!dev->name) \ + dev->name = (PFN_vk##name)gdpa(*device, "vk" #name "EXT"); + DEVICE_FUNCTIONS(LOAD) +#undef LOAD + inst->GetPhysicalDeviceMemoryProperties(physical_device, &dev->memory_properties); + VkPhysicalDeviceProperties properties; + inst->GetPhysicalDeviceProperties(physical_device, &properties); + uint32_t components = properties.limits.maxVertexOutputComponents; + if (properties.limits.maxFragmentInputComponents < components) + components = properties.limits.maxFragmentInputComponents; + dev->max_varying_locations = components / 4; + + pthread_mutex_lock(&global_lock); + dev->next = devices; + devices = dev; + pthread_mutex_unlock(&global_lock); + + if (requested) { + layer_log("emulating%s%s%s%s%s%s for %s", (requested & GAP_FILL_MODE) ? " fillModeNonSolid" : "", + (requested & GAP_MULTI_VIEWPORT) ? " multiViewport" : "", + (requested & GAP_CLIP_DISTANCE) ? " shaderClipDistance" : "", + (requested & GAP_CULL_DISTANCE) ? " shaderCullDistance" : "", + (requested & GAP_ROBUSTNESS2) ? " robustBufferAccess2" : "", + (requested & GAP_BC) ? " textureCompressionBC" : "", properties.deviceName); + } + return VK_SUCCESS; +} + +static VKAPI_ATTR void VKAPI_CALL DestroyDevice(VkDevice device, const VkAllocationCallbacks *allocator) +{ + void *key = dispatch_key(device); + pthread_mutex_lock(&global_lock); + struct device **link = &devices; + while (*link && (*link)->key != key) + link = &(*link)->next; + struct device *dev = *link; + if (dev) + *link = dev->next; + pthread_mutex_unlock(&global_lock); + if (!dev) + return; + bc_destroy_device(dev); + dev->DestroyDevice(device, allocator); + for (uint32_t i = 0; i < dev->shader_modules.capacity; i++) { + struct spirv_code *code = dev->shader_modules.values[i]; + if (code) { + free(code->words); + free(code); + } + } + map_free(&dev->shader_modules); + pthread_mutex_destroy(&dev->lock); + free(dev); +} + +struct hook { + const char *name; + PFN_vkVoidFunction function; + uint32_t gaps; +}; + +#define HOOK(name, gaps) {"vk" #name, (PFN_vkVoidFunction)name, gaps} +#define HOOK_ALIAS(name, alias, gaps) {"vk" #alias, (PFN_vkVoidFunction)name, gaps} + +static const struct hook device_hooks[] = { + HOOK(CreateShaderModule, GAPS_SHADER), + HOOK(DestroyShaderModule, GAPS_SHADER), + HOOK(CreateGraphicsPipelines, GAPS_PIPELINE), + HOOK(CmdSetViewport, GAP_MULTI_VIEWPORT), + HOOK(CmdSetScissor, GAP_MULTI_VIEWPORT), + HOOK(CmdSetViewportWithCount, GAP_MULTI_VIEWPORT), + HOOK_ALIAS(CmdSetViewportWithCount, CmdSetViewportWithCountEXT, GAP_MULTI_VIEWPORT), + HOOK(CmdSetScissorWithCount, GAP_MULTI_VIEWPORT), + HOOK_ALIAS(CmdSetScissorWithCount, CmdSetScissorWithCountEXT, GAP_MULTI_VIEWPORT), + HOOK(CreateImage, GAP_BC), + HOOK(DestroyImage, GAP_BC), + HOOK(CreateImageView, GAP_BC), + HOOK(GetDeviceImageMemoryRequirements, GAP_BC), + HOOK_ALIAS(GetDeviceImageMemoryRequirements, GetDeviceImageMemoryRequirementsKHR, GAP_BC), + HOOK(GetDeviceImageSparseMemoryRequirements, GAP_BC), + HOOK_ALIAS(GetDeviceImageSparseMemoryRequirements, GetDeviceImageSparseMemoryRequirementsKHR, GAP_BC), + HOOK(GetDeviceImageSubresourceLayout, GAP_BC), + HOOK_ALIAS(GetDeviceImageSubresourceLayout, GetDeviceImageSubresourceLayoutKHR, GAP_BC), + HOOK(CmdCopyBufferToImage, GAP_BC), + HOOK(CmdCopyBufferToImage2, GAP_BC), + HOOK_ALIAS(CmdCopyBufferToImage2, CmdCopyBufferToImage2KHR, GAP_BC), + HOOK(CmdCopyImageToBuffer, GAP_BC), + HOOK(CmdCopyImageToBuffer2, GAP_BC), + HOOK_ALIAS(CmdCopyImageToBuffer2, CmdCopyImageToBuffer2KHR, GAP_BC), + HOOK(CmdCopyImage, GAP_BC), + HOOK(CmdCopyImage2, GAP_BC), + HOOK_ALIAS(CmdCopyImage2, CmdCopyImage2KHR, GAP_BC), + HOOK(BindBufferMemory, GAP_BC), + HOOK(BindBufferMemory2, GAP_BC), + HOOK_ALIAS(BindBufferMemory2, BindBufferMemory2KHR, GAP_BC), + HOOK(DestroyBuffer, GAP_BC), + HOOK(FreeMemory, GAP_BC), + HOOK(MapMemory, GAP_BC), + HOOK(MapMemory2, GAP_BC), + HOOK_ALIAS(MapMemory2, MapMemory2KHR, GAP_BC), + HOOK(UnmapMemory, GAP_BC), + HOOK(UnmapMemory2, GAP_BC), + HOOK_ALIAS(UnmapMemory2, UnmapMemory2KHR, GAP_BC), + HOOK(AllocateCommandBuffers, GAP_BC), + HOOK(FreeCommandBuffers, GAP_BC), + HOOK(BeginCommandBuffer, GAP_BC), + HOOK(ResetCommandBuffer, GAP_BC), + HOOK(ResetCommandPool, GAP_BC), + HOOK(DestroyCommandPool, GAP_BC), +}; + +static VKAPI_ATTR PFN_vkVoidFunction VKAPI_CALL GetDeviceProcAddr(VkDevice device, const char *name); + +static VKAPI_ATTR PFN_vkVoidFunction VKAPI_CALL GetDeviceProcAddr(VkDevice device, const char *name) +{ + if (!strcmp(name, "vkGetDeviceProcAddr")) + return (PFN_vkVoidFunction)GetDeviceProcAddr; + struct device *dev = device_from_handle(device); + if (!dev) + return NULL; + if (!strcmp(name, "vkDestroyDevice")) + return (PFN_vkVoidFunction)DestroyDevice; + PFN_vkVoidFunction next = dev->gdpa(device, name); + if (!next) + return NULL; + for (size_t i = 0; i < sizeof(device_hooks) / sizeof(device_hooks[0]); i++) { + if ((dev->emulated & device_hooks[i].gaps) && !strcmp(name, device_hooks[i].name)) + return device_hooks[i].function; + } + return next; +} + +static const struct hook physical_device_hooks[] = { + HOOK(GetPhysicalDeviceFeatures, 0), + HOOK(GetPhysicalDeviceFeatures2, 0), + HOOK_ALIAS(GetPhysicalDeviceFeatures2, GetPhysicalDeviceFeatures2KHR, 0), + HOOK(GetPhysicalDeviceProperties, 0), + HOOK(GetPhysicalDeviceProperties2, 0), + HOOK_ALIAS(GetPhysicalDeviceProperties2, GetPhysicalDeviceProperties2KHR, 0), + HOOK(GetPhysicalDeviceFormatProperties, 0), + HOOK(GetPhysicalDeviceFormatProperties2, 0), + HOOK_ALIAS(GetPhysicalDeviceFormatProperties2, GetPhysicalDeviceFormatProperties2KHR, 0), + HOOK(GetPhysicalDeviceImageFormatProperties, 0), + HOOK(GetPhysicalDeviceImageFormatProperties2, 0), + HOOK_ALIAS(GetPhysicalDeviceImageFormatProperties2, GetPhysicalDeviceImageFormatProperties2KHR, 0), +}; + +static VKAPI_ATTR PFN_vkVoidFunction VKAPI_CALL GetInstanceProcAddr(VkInstance instance, const char *name) +{ + if (!strcmp(name, "vkGetInstanceProcAddr")) + return (PFN_vkVoidFunction)GetInstanceProcAddr; + if (!strcmp(name, "vkCreateInstance")) + return (PFN_vkVoidFunction)CreateInstance; + if (!strcmp(name, "vkGetDeviceProcAddr")) + return (PFN_vkVoidFunction)GetDeviceProcAddr; + if (!instance) + return NULL; + struct instance *inst = instance_from_handle(instance); + if (!inst) + return NULL; + if (!strcmp(name, "vkDestroyInstance")) + return (PFN_vkVoidFunction)DestroyInstance; + /* Always ours, even when inactive: each layer moves the device chain one link on. */ + if (!strcmp(name, "vkCreateDevice")) + return (PFN_vkVoidFunction)CreateDevice; + PFN_vkVoidFunction next = inst->gipa(instance, name); + if (!next || !inst->active) + return next; + for (size_t i = 0; i < sizeof(physical_device_hooks) / sizeof(physical_device_hooks[0]); i++) { + if (!strcmp(name, physical_device_hooks[i].name)) + return physical_device_hooks[i].function; + } + return next; +} + +__attribute__((visibility("default"))) VKAPI_ATTR VkResult VKAPI_CALL +vkNegotiateLoaderLayerInterfaceVersion(VkNegotiateLayerInterface *interface) +{ + if (interface->loaderLayerInterfaceVersion < 2) + return VK_ERROR_INITIALIZATION_FAILED; + interface->loaderLayerInterfaceVersion = 2; + interface->pfnGetInstanceProcAddr = GetInstanceProcAddr; + interface->pfnGetDeviceProcAddr = GetDeviceProcAddr; + interface->pfnGetPhysicalDeviceProcAddr = NULL; + return VK_SUCCESS; +} diff --git a/tools/venus/layer/mali_compat.h b/tools/venus/layer/mali_compat.h new file mode 100644 index 000000000..4ab5e12ca --- /dev/null +++ b/tools/venus/layer/mali_compat.h @@ -0,0 +1,246 @@ +/* + * VK_LAYER_DROIDDECK_mali_compat: makes up for what Mali lacks and DXVK requires. + * + * DXVK refuses any GPU without fillModeNonSolid, multiViewport, shaderClipDistance, + * shaderCullDistance, textureCompressionBC and robustBufferAccess2, and Mali has none of them. + * The layer reports each missing feature as supported, keeps it out of the driver's device, and + * handles what uses it: polygon modes draw filled, only the first viewport is kept, clip distances + * become a varying the fragment shader discards on, cull distances are dropped, and BC textures + * are decoded on the CPU into uncompressed images as they are uploaded. + */ +#ifndef MALI_COMPAT_H +#define MALI_COMPAT_H + +#include +#include +#include +#include +#include +#include + +enum gap { + GAP_FILL_MODE = 1u << 0, + GAP_MULTI_VIEWPORT = 1u << 1, + GAP_CLIP_DISTANCE = 1u << 2, + GAP_CULL_DISTANCE = 1u << 3, + GAP_ROBUSTNESS2 = 1u << 4, + GAP_BC = 1u << 5, +}; + +#define GAPS_SHADER (GAP_CLIP_DISTANCE | GAP_CULL_DISTANCE) +#define GAPS_PIPELINE (GAP_FILL_MODE | GAP_MULTI_VIEWPORT | GAPS_SHADER) + +void layer_log(const char *format, ...) __attribute__((format(printf, 1, 2))); + +/* Open addressing from a Vulkan handle to a pointer. Not locked: the owner holds its lock. */ +struct handle_map { + uint64_t *keys; + void **values; + uint32_t capacity; + uint32_t count; +}; + +void *map_get(const struct handle_map *map, uint64_t key); +void map_put(struct handle_map *map, uint64_t key, void *value); +void *map_remove(struct handle_map *map, uint64_t key); +void map_free(struct handle_map *map); + +struct physical_device { + VkPhysicalDevice handle; + uint32_t gaps; +}; + +struct instance { + void *key; + VkInstance handle; + bool active; + PFN_vkGetInstanceProcAddr gipa; + PFN_vkDestroyInstance DestroyInstance; + PFN_vkEnumerateDeviceExtensionProperties EnumerateDeviceExtensionProperties; + PFN_vkGetPhysicalDeviceFeatures GetPhysicalDeviceFeatures; + PFN_vkGetPhysicalDeviceFeatures2 GetPhysicalDeviceFeatures2; + PFN_vkGetPhysicalDeviceProperties GetPhysicalDeviceProperties; + PFN_vkGetPhysicalDeviceProperties2 GetPhysicalDeviceProperties2; + PFN_vkGetPhysicalDeviceMemoryProperties GetPhysicalDeviceMemoryProperties; + PFN_vkGetPhysicalDeviceFormatProperties GetPhysicalDeviceFormatProperties; + PFN_vkGetPhysicalDeviceFormatProperties2 GetPhysicalDeviceFormatProperties2; + PFN_vkGetPhysicalDeviceImageFormatProperties GetPhysicalDeviceImageFormatProperties; + PFN_vkGetPhysicalDeviceImageFormatProperties2 GetPhysicalDeviceImageFormatProperties2; + pthread_mutex_t lock; + struct physical_device physical_devices[8]; + uint32_t physical_device_count; + struct instance *next; +}; + +#define DEVICE_FUNCTIONS(X) \ + X(DestroyDevice) \ + X(CreateShaderModule) \ + X(DestroyShaderModule) \ + X(CreateGraphicsPipelines) \ + X(CmdSetViewport) \ + X(CmdSetScissor) \ + X(CmdSetViewportWithCount) \ + X(CmdSetScissorWithCount) \ + X(CreateImage) \ + X(DestroyImage) \ + X(CreateImageView) \ + X(GetDeviceImageMemoryRequirements) \ + X(GetDeviceImageSparseMemoryRequirements) \ + X(GetDeviceImageSubresourceLayout) \ + X(CmdCopyBufferToImage) \ + X(CmdCopyBufferToImage2) \ + X(CmdCopyImageToBuffer) \ + X(CmdCopyImageToBuffer2) \ + X(CmdCopyImage) \ + X(CmdCopyImage2) \ + X(CreateBuffer) \ + X(DestroyBuffer) \ + X(GetBufferMemoryRequirements) \ + X(BindBufferMemory) \ + X(BindBufferMemory2) \ + X(AllocateMemory) \ + X(FreeMemory) \ + X(MapMemory) \ + X(MapMemory2) \ + X(UnmapMemory) \ + X(UnmapMemory2) \ + X(AllocateCommandBuffers) \ + X(FreeCommandBuffers) \ + X(BeginCommandBuffer) \ + X(ResetCommandBuffer) \ + X(ResetCommandPool) \ + X(DestroyCommandPool) + +struct staging_block; + +struct device { + void *key; + VkDevice handle; + PFN_vkGetDeviceProcAddr gdpa; + /* The gaps the application enabled a made-up feature for. */ + uint32_t emulated; + /* Locations a clip distance varying may use, from the vertex output and fragment input limits. */ + uint32_t max_varying_locations; + VkPhysicalDeviceMemoryProperties memory_properties; +#define DECLARE(name) PFN_vk##name name; + DEVICE_FUNCTIONS(DECLARE) +#undef DECLARE + pthread_mutex_t lock; + struct handle_map shader_modules; /* VkShaderModule -> struct spirv_code, for modules with distances */ + struct handle_map images; /* VkImage -> struct bc_image */ + struct handle_map buffers; /* VkBuffer -> struct bound_buffer */ + struct handle_map memories; /* VkDeviceMemory -> struct mapped_memory */ + struct handle_map command_buffers; /* VkCommandBuffer -> struct command_buffer */ + struct staging_block *free_staging; + struct device *next; +}; + +struct device *device_from_handle(const void *dispatchable); + +struct spirv_code { + uint32_t *words; + size_t count; +}; + +/* spirv.c */ +bool spirv_uses_distances(const uint32_t *words, size_t count); +/* The module with clip and cull distances taken out; false when it cannot be rewritten. */ +bool spirv_strip_distances(const uint32_t *words, size_t count, struct spirv_code *out); + +enum clip_role { CLIP_STRIP, CLIP_PRODUCER, CLIP_CONSUMER }; + +struct clip_info { + bool has_clip; /* a standalone ClipDistance variable this stage could hand on */ + uint32_t clip_count; /* its array length */ + uint32_t locations_used; /* one past the highest varying location, outputs or inputs per role */ +}; + +void spirv_clip_info(const uint32_t *words, size_t count, enum clip_role role, struct clip_info *info); +/* + * Producer: the last stage before rasterization writes its clip distances to `location` instead. + * Consumer: the fragment shader reads them there and discards when any is negative. + * Both drop cull distances. False when the module cannot be rewritten. + */ +bool spirv_emulate_clip(const uint32_t *words, size_t count, enum clip_role role, uint32_t location, + uint32_t clip_count, struct spirv_code *out); + +/* pipeline.c */ +VKAPI_ATTR VkResult VKAPI_CALL CreateShaderModule(VkDevice device, const VkShaderModuleCreateInfo *info, + const VkAllocationCallbacks *allocator, VkShaderModule *module); +VKAPI_ATTR void VKAPI_CALL DestroyShaderModule(VkDevice device, VkShaderModule module, + const VkAllocationCallbacks *allocator); +VKAPI_ATTR VkResult VKAPI_CALL CreateGraphicsPipelines(VkDevice device, VkPipelineCache cache, uint32_t count, + const VkGraphicsPipelineCreateInfo *infos, + const VkAllocationCallbacks *allocator, VkPipeline *pipelines); +VKAPI_ATTR void VKAPI_CALL CmdSetViewport(VkCommandBuffer cmd, uint32_t first, uint32_t count, + const VkViewport *viewports); +VKAPI_ATTR void VKAPI_CALL CmdSetScissor(VkCommandBuffer cmd, uint32_t first, uint32_t count, + const VkRect2D *scissors); +VKAPI_ATTR void VKAPI_CALL CmdSetViewportWithCount(VkCommandBuffer cmd, uint32_t count, + const VkViewport *viewports); +VKAPI_ATTR void VKAPI_CALL CmdSetScissorWithCount(VkCommandBuffer cmd, uint32_t count, const VkRect2D *scissors); + +/* bc.c */ +VkFormat bc_substitute(VkFormat format); +VKAPI_ATTR void VKAPI_CALL GetPhysicalDeviceFormatProperties(VkPhysicalDevice physical_device, VkFormat format, + VkFormatProperties *properties); +VKAPI_ATTR void VKAPI_CALL GetPhysicalDeviceFormatProperties2(VkPhysicalDevice physical_device, VkFormat format, + VkFormatProperties2 *properties); +VKAPI_ATTR VkResult VKAPI_CALL GetPhysicalDeviceImageFormatProperties( + VkPhysicalDevice physical_device, VkFormat format, VkImageType type, VkImageTiling tiling, + VkImageUsageFlags usage, VkImageCreateFlags flags, VkImageFormatProperties *properties); +VKAPI_ATTR VkResult VKAPI_CALL GetPhysicalDeviceImageFormatProperties2( + VkPhysicalDevice physical_device, const VkPhysicalDeviceImageFormatInfo2 *info, + VkImageFormatProperties2 *properties); +VKAPI_ATTR VkResult VKAPI_CALL CreateImage(VkDevice device, const VkImageCreateInfo *info, + const VkAllocationCallbacks *allocator, VkImage *image); +VKAPI_ATTR void VKAPI_CALL DestroyImage(VkDevice device, VkImage image, const VkAllocationCallbacks *allocator); +VKAPI_ATTR VkResult VKAPI_CALL CreateImageView(VkDevice device, const VkImageViewCreateInfo *info, + const VkAllocationCallbacks *allocator, VkImageView *view); +VKAPI_ATTR void VKAPI_CALL GetDeviceImageMemoryRequirements(VkDevice device, + const VkDeviceImageMemoryRequirements *info, + VkMemoryRequirements2 *requirements); +VKAPI_ATTR void VKAPI_CALL GetDeviceImageSparseMemoryRequirements( + VkDevice device, const VkDeviceImageMemoryRequirements *info, uint32_t *count, + VkSparseImageMemoryRequirements2 *requirements); +VKAPI_ATTR void VKAPI_CALL GetDeviceImageSubresourceLayout(VkDevice device, + const VkDeviceImageSubresourceInfo *info, + VkSubresourceLayout2 *layout); +VKAPI_ATTR void VKAPI_CALL CmdCopyBufferToImage(VkCommandBuffer cmd, VkBuffer buffer, VkImage image, + VkImageLayout layout, uint32_t count, + const VkBufferImageCopy *regions); +VKAPI_ATTR void VKAPI_CALL CmdCopyBufferToImage2(VkCommandBuffer cmd, const VkCopyBufferToImageInfo2 *info); +VKAPI_ATTR void VKAPI_CALL CmdCopyImageToBuffer(VkCommandBuffer cmd, VkImage image, VkImageLayout layout, + VkBuffer buffer, uint32_t count, const VkBufferImageCopy *regions); +VKAPI_ATTR void VKAPI_CALL CmdCopyImageToBuffer2(VkCommandBuffer cmd, const VkCopyImageToBufferInfo2 *info); +VKAPI_ATTR void VKAPI_CALL CmdCopyImage(VkCommandBuffer cmd, VkImage src, VkImageLayout src_layout, VkImage dst, + VkImageLayout dst_layout, uint32_t count, const VkImageCopy *regions); +VKAPI_ATTR void VKAPI_CALL CmdCopyImage2(VkCommandBuffer cmd, const VkCopyImageInfo2 *info); +VKAPI_ATTR VkResult VKAPI_CALL BindBufferMemory(VkDevice device, VkBuffer buffer, VkDeviceMemory memory, + VkDeviceSize offset); +VKAPI_ATTR VkResult VKAPI_CALL BindBufferMemory2(VkDevice device, uint32_t count, + const VkBindBufferMemoryInfo *infos); +VKAPI_ATTR void VKAPI_CALL DestroyBuffer(VkDevice device, VkBuffer buffer, const VkAllocationCallbacks *allocator); +VKAPI_ATTR void VKAPI_CALL FreeMemory(VkDevice device, VkDeviceMemory memory, const VkAllocationCallbacks *allocator); +VKAPI_ATTR VkResult VKAPI_CALL MapMemory(VkDevice device, VkDeviceMemory memory, VkDeviceSize offset, + VkDeviceSize size, VkMemoryMapFlags flags, void **data); +VKAPI_ATTR VkResult VKAPI_CALL MapMemory2(VkDevice device, const VkMemoryMapInfo *info, void **data); +VKAPI_ATTR void VKAPI_CALL UnmapMemory(VkDevice device, VkDeviceMemory memory); +VKAPI_ATTR VkResult VKAPI_CALL UnmapMemory2(VkDevice device, const VkMemoryUnmapInfo *info); +VKAPI_ATTR VkResult VKAPI_CALL AllocateCommandBuffers(VkDevice device, const VkCommandBufferAllocateInfo *info, + VkCommandBuffer *buffers); +VKAPI_ATTR void VKAPI_CALL FreeCommandBuffers(VkDevice device, VkCommandPool pool, uint32_t count, + const VkCommandBuffer *buffers); +VKAPI_ATTR VkResult VKAPI_CALL BeginCommandBuffer(VkCommandBuffer cmd, const VkCommandBufferBeginInfo *info); +VKAPI_ATTR VkResult VKAPI_CALL ResetCommandBuffer(VkCommandBuffer cmd, VkCommandBufferResetFlags flags); +VKAPI_ATTR VkResult VKAPI_CALL ResetCommandPool(VkDevice device, VkCommandPool pool, VkCommandPoolResetFlags flags); +VKAPI_ATTR void VKAPI_CALL DestroyCommandPool(VkDevice device, VkCommandPool pool, + const VkAllocationCallbacks *allocator); +void bc_destroy_device(struct device *dev); + +/* layer.c */ +struct instance *instance_from_handle(const void *dispatchable); +/* What the GPU lacks of the features the layer can make up for. */ +uint32_t physical_device_gaps(struct instance *inst, VkPhysicalDevice physical_device); + +#endif diff --git a/tools/venus/layer/pipeline.c b/tools/venus/layer/pipeline.c new file mode 100644 index 000000000..3940209a5 --- /dev/null +++ b/tools/venus/layer/pipeline.c @@ -0,0 +1,281 @@ +#include "mali_compat.h" + +#include +#include + +#define HANDLE(h) ((uint64_t)(uintptr_t)(h)) +#define MAX_STAGES 8 + +/* A module with distances goes to the driver without them; its original stays here, for the + * pipeline that can emulate its clip distances. */ +VKAPI_ATTR VkResult VKAPI_CALL CreateShaderModule(VkDevice device, const VkShaderModuleCreateInfo *info, + const VkAllocationCallbacks *allocator, VkShaderModule *module) +{ + struct device *dev = device_from_handle(device); + size_t count = info->codeSize / sizeof(uint32_t); + if (!spirv_uses_distances(info->pCode, count)) + return dev->CreateShaderModule(device, info, allocator, module); + + struct spirv_code stripped; + if (!spirv_strip_distances(info->pCode, count, &stripped)) { + layer_log("could not take the clip and cull distances out of a shader module"); + return dev->CreateShaderModule(device, info, allocator, module); + } + VkShaderModuleCreateInfo copy = *info; + copy.pCode = stripped.words; + copy.codeSize = stripped.count * sizeof(uint32_t); + VkResult result = dev->CreateShaderModule(device, ©, allocator, module); + free(stripped.words); + if (result != VK_SUCCESS) + return result; + + struct spirv_code *original = malloc(sizeof(*original)); + original->count = count; + original->words = malloc(info->codeSize); + memcpy(original->words, info->pCode, info->codeSize); + pthread_mutex_lock(&dev->lock); + map_put(&dev->shader_modules, HANDLE(*module), original); + pthread_mutex_unlock(&dev->lock); + return VK_SUCCESS; +} + +VKAPI_ATTR void VKAPI_CALL DestroyShaderModule(VkDevice device, VkShaderModule module, + const VkAllocationCallbacks *allocator) +{ + struct device *dev = device_from_handle(device); + pthread_mutex_lock(&dev->lock); + struct spirv_code *original = map_remove(&dev->shader_modules, HANDLE(module)); + pthread_mutex_unlock(&dev->lock); + if (original) { + free(original->words); + free(original); + } + dev->DestroyShaderModule(device, module, allocator); +} + +struct stage_code { + const uint32_t *words; + size_t count; + /* The application's own, when the code comes inline in the stage. */ + VkShaderModuleCreateInfo *inline_info; + /* A module whose distances were already taken out at creation. */ + bool stripped_module; +}; + +static void find_stage_code(struct device *dev, const VkPipelineShaderStageCreateInfo *stage, struct stage_code *code) +{ + memset(code, 0, sizeof(*code)); + for (const VkBaseInStructure *s = stage->pNext; s; s = s->pNext) { + if (s->sType == VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO) { + code->inline_info = (VkShaderModuleCreateInfo *)s; + code->words = code->inline_info->pCode; + code->count = code->inline_info->codeSize / sizeof(uint32_t); + return; + } + } + if (stage->module == VK_NULL_HANDLE) + return; + pthread_mutex_lock(&dev->lock); + struct spirv_code *original = map_get(&dev->shader_modules, HANDLE(stage->module)); + pthread_mutex_unlock(&dev->lock); + if (original) { + code->words = original->words; + code->count = original->count; + code->stripped_module = true; + } +} + +struct pipeline_rewrite { + VkPipelineRasterizationStateCreateInfo rasterization; + VkPipelineViewportStateCreateInfo viewport; + VkPipelineShaderStageCreateInfo stages[MAX_STAGES]; + VkShaderModuleCreateInfo modules[MAX_STAGES]; + struct spirv_code code[MAX_STAGES]; + VkShaderModule temporary[MAX_STAGES]; + /* Rewritten where they are, when they are not first in their stage's chain. */ + VkShaderModuleCreateInfo *in_place[MAX_STAGES]; + const uint32_t *in_place_code[MAX_STAGES]; + size_t in_place_size[MAX_STAGES]; +}; + +static void use_code(struct device *dev, VkGraphicsPipelineCreateInfo *info, struct pipeline_rewrite *rw, + const struct stage_code *code, uint32_t i) +{ + if (info->pStages != rw->stages) { + memcpy(rw->stages, info->pStages, info->stageCount * sizeof(*rw->stages)); + info->pStages = rw->stages; + } + const struct spirv_code *out = &rw->code[i]; + if (code->inline_info && rw->stages[i].pNext == code->inline_info) { + rw->modules[i] = *code->inline_info; + rw->modules[i].pCode = out->words; + rw->modules[i].codeSize = out->count * sizeof(uint32_t); + rw->stages[i].pNext = &rw->modules[i]; + } else if (code->inline_info) { + rw->in_place[i] = code->inline_info; + rw->in_place_code[i] = code->inline_info->pCode; + rw->in_place_size[i] = code->inline_info->codeSize; + code->inline_info->pCode = out->words; + code->inline_info->codeSize = out->count * sizeof(uint32_t); + } else { + VkShaderModuleCreateInfo module_info = { + .sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO, + .codeSize = out->count * sizeof(uint32_t), + .pCode = out->words, + }; + if (dev->CreateShaderModule(dev->handle, &module_info, NULL, &rw->temporary[i]) == VK_SUCCESS) + rw->stages[i].module = rw->temporary[i]; + } +} + +static int stage_index(const VkGraphicsPipelineCreateInfo *info, VkShaderStageFlagBits stage) +{ + for (uint32_t i = 0; i < info->stageCount; i++) { + if (info->pStages[i].stage == stage) + return (int)i; + } + return -1; +} + +/* + * Clip distances are emulated when the last stage before rasterization is the only one that + * hands them on: a vertex shader feeding a geometry shader would need a varying of its own. + * Anywhere else they are dropped, along with every cull distance. + */ +static void rewrite_shaders(struct device *dev, VkGraphicsPipelineCreateInfo *info, struct pipeline_rewrite *rw) +{ + uint32_t n = info->stageCount; + if (!n || n > MAX_STAGES || !info->pStages) + return; + struct stage_code code[MAX_STAGES]; + bool uses[MAX_STAGES]; + bool any = false; + for (uint32_t i = 0; i < n; i++) { + find_stage_code(dev, &info->pStages[i], &code[i]); + uses[i] = code[i].words && spirv_uses_distances(code[i].words, code[i].count); + any |= uses[i]; + } + if (!any) + return; + + int producer = stage_index(info, VK_SHADER_STAGE_GEOMETRY_BIT); + if (producer < 0) + producer = stage_index(info, VK_SHADER_STAGE_TESSELLATION_EVALUATION_BIT); + if (producer < 0) + producer = stage_index(info, VK_SHADER_STAGE_VERTEX_BIT); + int consumer = stage_index(info, VK_SHADER_STAGE_FRAGMENT_BIT); + bool emulate = (dev->emulated & GAP_CLIP_DISTANCE) && producer >= 0 && consumer >= 0 && uses[producer] && + code[consumer].words; + for (uint32_t i = 0; i < n && emulate; i++) { + if ((int)i != producer && (int)i != consumer && uses[i]) + emulate = false; + } + if (emulate) { + struct clip_info produced, consumed; + spirv_clip_info(code[producer].words, code[producer].count, CLIP_PRODUCER, &produced); + spirv_clip_info(code[consumer].words, code[consumer].count, CLIP_CONSUMER, &consumed); + uint32_t location = produced.locations_used > consumed.locations_used ? produced.locations_used + : consumed.locations_used; + emulate = produced.has_clip && location != UINT32_MAX && + location + produced.clip_count <= dev->max_varying_locations && + spirv_emulate_clip(code[producer].words, code[producer].count, CLIP_PRODUCER, location, + produced.clip_count, &rw->code[producer]) && + spirv_emulate_clip(code[consumer].words, code[consumer].count, CLIP_CONSUMER, location, + produced.clip_count, &rw->code[consumer]); + if (!emulate) { + free(rw->code[producer].words); + free(rw->code[consumer].words); + rw->code[producer] = rw->code[consumer] = (struct spirv_code){0}; + } + } + + for (uint32_t i = 0; i < n; i++) { + if (!rw->code[i].words && uses[i] && !code[i].stripped_module && + !spirv_strip_distances(code[i].words, code[i].count, &rw->code[i])) { + layer_log("could not take the clip and cull distances out of a pipeline's shader"); + } + if (rw->code[i].words) + use_code(dev, info, rw, &code[i], i); + } +} + +static void rewrite_pipeline(struct device *dev, VkGraphicsPipelineCreateInfo *info, struct pipeline_rewrite *rw) +{ + if ((dev->emulated & GAP_FILL_MODE) && info->pRasterizationState && + info->pRasterizationState->polygonMode != VK_POLYGON_MODE_FILL) { + rw->rasterization = *info->pRasterizationState; + rw->rasterization.polygonMode = VK_POLYGON_MODE_FILL; + info->pRasterizationState = &rw->rasterization; + } + if ((dev->emulated & GAP_MULTI_VIEWPORT) && info->pViewportState && + (info->pViewportState->viewportCount > 1 || info->pViewportState->scissorCount > 1)) { + rw->viewport = *info->pViewportState; + if (rw->viewport.viewportCount > 1) + rw->viewport.viewportCount = 1; + if (rw->viewport.scissorCount > 1) + rw->viewport.scissorCount = 1; + info->pViewportState = &rw->viewport; + } + if (dev->emulated & GAPS_SHADER) + rewrite_shaders(dev, info, rw); +} + +static void undo_rewrite(struct device *dev, struct pipeline_rewrite *rw) +{ + for (uint32_t i = 0; i < MAX_STAGES; i++) { + if (rw->in_place[i]) { + rw->in_place[i]->pCode = rw->in_place_code[i]; + rw->in_place[i]->codeSize = rw->in_place_size[i]; + } + if (rw->temporary[i] != VK_NULL_HANDLE) + dev->DestroyShaderModule(dev->handle, rw->temporary[i], NULL); + free(rw->code[i].words); + } +} + +VKAPI_ATTR VkResult VKAPI_CALL CreateGraphicsPipelines(VkDevice device, VkPipelineCache cache, uint32_t count, + const VkGraphicsPipelineCreateInfo *infos, + const VkAllocationCallbacks *allocator, VkPipeline *pipelines) +{ + struct device *dev = device_from_handle(device); + VkGraphicsPipelineCreateInfo *copies = malloc(count * sizeof(*copies)); + struct pipeline_rewrite *rewrites = calloc(count, sizeof(*rewrites)); + for (uint32_t i = 0; i < count; i++) { + copies[i] = infos[i]; + rewrite_pipeline(dev, &copies[i], &rewrites[i]); + } + VkResult result = dev->CreateGraphicsPipelines(device, cache, count, copies, allocator, pipelines); + for (uint32_t i = 0; i < count; i++) + undo_rewrite(dev, &rewrites[i]); + free(rewrites); + free(copies); + return result; +} + +/* The driver has one viewport and one scissor; anything past the first is left out. */ +VKAPI_ATTR void VKAPI_CALL CmdSetViewport(VkCommandBuffer cmd, uint32_t first, uint32_t count, + const VkViewport *viewports) +{ + struct device *dev = device_from_handle(cmd); + if (first == 0 && count > 0) + dev->CmdSetViewport(cmd, 0, 1, viewports); +} + +VKAPI_ATTR void VKAPI_CALL CmdSetScissor(VkCommandBuffer cmd, uint32_t first, uint32_t count, const VkRect2D *scissors) +{ + struct device *dev = device_from_handle(cmd); + if (first == 0 && count > 0) + dev->CmdSetScissor(cmd, 0, 1, scissors); +} + +VKAPI_ATTR void VKAPI_CALL CmdSetViewportWithCount(VkCommandBuffer cmd, uint32_t count, const VkViewport *viewports) +{ + struct device *dev = device_from_handle(cmd); + dev->CmdSetViewportWithCount(cmd, count > 1 ? 1 : count, viewports); +} + +VKAPI_ATTR void VKAPI_CALL CmdSetScissorWithCount(VkCommandBuffer cmd, uint32_t count, const VkRect2D *scissors) +{ + struct device *dev = device_from_handle(cmd); + dev->CmdSetScissorWithCount(cmd, count > 1 ? 1 : count, scissors); +} diff --git a/tools/venus/layer/spirv.c b/tools/venus/layer/spirv.c new file mode 100644 index 000000000..734d5ce2d --- /dev/null +++ b/tools/venus/layer/spirv.c @@ -0,0 +1,833 @@ +/* + * Clip and cull distances out of SPIR-V, for a GPU without either. + * + * Stripping: every ClipDistance or CullDistance variable becomes a Private one, so the shader still + * writes and reads it but the rasterizer never sees it. Members of a gl_PerVertex block stay + * declared (as glslang declares them unused anyway) and their accesses move to a Private stand-in. + * Emulating: the last stage before rasterization writes its clip distances to a free varying + * location instead, and the fragment shader reads them there and discards when one is negative. + */ +#include "mali_compat.h" + +#include +#include +#include + +#define OPCODE(word) ((word) & 0xffffu) +#define LENGTH(word) ((word) >> 16) +#define VERSION_1_4 0x00010400u + +struct words { + uint32_t *data; + size_t count; + size_t capacity; +}; + +static void push(struct words *words, uint32_t word) +{ + if (words->count == words->capacity) { + words->capacity = words->capacity ? words->capacity * 2 : 64; + words->data = realloc(words->data, words->capacity * sizeof(uint32_t)); + } + words->data[words->count++] = word; +} + +static void emit(struct words *words, SpvOp op, size_t count, const uint32_t *operands) +{ + push(words, (uint32_t)(count + 1) << 16 | op); + for (size_t i = 0; i < count; i++) + push(words, operands[i]); +} + +#define EMIT(words, op, ...) \ + do { \ + const uint32_t operands_[] = {__VA_ARGS__}; \ + emit(words, op, sizeof(operands_) / sizeof(operands_[0]), operands_); \ + } while (0) + +enum place { BEFORE, INSTEAD, AFTER }; + +struct edit { + size_t offset; + enum place place; + struct words words; +}; + +struct new_type { + SpvOp op; + uint32_t operands[3]; + uint32_t count; + uint32_t id; +}; + +struct module { + uint32_t *w; + size_t count; + uint32_t version; + uint32_t original_bound; + uint32_t bound; + size_t *def; + size_t first_declaration; + size_t first_function; + struct edit *edits; + size_t edit_count; + struct new_type *new_types; + size_t new_type_count; + struct words globals; + uint8_t *private_ids; + uint32_t *interface_add; + size_t interface_add_count; + uint8_t *interface_remove; + bool failed; +}; + +static bool declares_result_first(uint32_t op) +{ + return (op >= SpvOpTypeVoid && op <= SpvOpTypePipe) || op == SpvOpLabel || op == SpvOpExtInstImport || + op == SpvOpString; +} + +static bool declares_result_second(uint32_t op) +{ + switch (op) { + case SpvOpConstantTrue: + case SpvOpConstantFalse: + case SpvOpConstant: + case SpvOpConstantComposite: + case SpvOpConstantNull: + case SpvOpSpecConstantTrue: + case SpvOpSpecConstantFalse: + case SpvOpSpecConstant: + case SpvOpSpecConstantComposite: + case SpvOpSpecConstantOp: + case SpvOpUndef: + case SpvOpVariable: + case SpvOpFunction: + return true; + default: + return false; + } +} + +static bool is_declaration(uint32_t op) +{ + return (op >= SpvOpTypeVoid && op <= SpvOpTypeForwardPointer) || declares_result_second(op); +} + +static void unload(struct module *m) +{ + free(m->w); + free(m->def); + for (size_t i = 0; i < m->edit_count; i++) + free(m->edits[i].words.data); + free(m->edits); + free(m->new_types); + free(m->globals.data); + free(m->private_ids); + free(m->interface_add); + free(m->interface_remove); +} + +static bool load(struct module *m, const uint32_t *words, size_t count) +{ + memset(m, 0, sizeof(*m)); + if (count < 5 || words[0] != SpvMagicNumber || words[3] == 0 || words[3] > (1u << 22)) + return false; + m->w = malloc(count * sizeof(uint32_t)); + memcpy(m->w, words, count * sizeof(uint32_t)); + m->count = count; + m->version = words[1]; + m->original_bound = m->bound = words[3]; + m->def = calloc(m->bound, sizeof(size_t)); + /* Room for every id the rewrite can add. */ + m->private_ids = calloc(m->bound + 4096, 1); + m->interface_remove = calloc(m->bound, 1); + for (size_t i = 5; i < count;) { + uint32_t length = LENGTH(m->w[i]); + uint32_t op = OPCODE(m->w[i]); + if (length == 0 || i + length > count) { + unload(m); + return false; + } + uint32_t result = 0; + if (declares_result_first(op) && length > 1) + result = m->w[i + 1]; + else if (declares_result_second(op) && length > 2) + result = m->w[i + 2]; + if (result && result < m->bound) + m->def[result] = i; + if (!m->first_declaration && is_declaration(op)) + m->first_declaration = i; + if (!m->first_function && op == SpvOpFunction) + m->first_function = i; + i += length; + } + if (!m->first_declaration || !m->first_function) { + unload(m); + return false; + } + return true; +} + +static struct words *edit(struct module *m, size_t offset, enum place place) +{ + m->edits = realloc(m->edits, (m->edit_count + 1) * sizeof(*m->edits)); + struct edit *e = &m->edits[m->edit_count++]; + *e = (struct edit){.offset = offset, .place = place}; + return &e->words; +} + +static uint32_t new_id(struct module *m) +{ + return m->bound++; +} + +static bool is_private(const struct module *m, uint32_t id) +{ + return id < m->original_bound + 4096 && m->private_ids[id]; +} + +static void mark_private(struct module *m, uint32_t id) +{ + if (id < m->original_bound + 4096) + m->private_ids[id] = 1; + else + m->failed = true; +} + +/* The defining instruction of an id from the original module, or 0. */ +static size_t definition(const struct module *m, uint32_t id) +{ + return id < m->original_bound ? m->def[id] : 0; +} + +static uint32_t find_type(struct module *m, SpvOp op, const uint32_t *operands, uint32_t count) +{ + for (size_t i = 5; i < m->first_function; i += LENGTH(m->w[i])) { + if (OPCODE(m->w[i]) == op && LENGTH(m->w[i]) == count + 2 && + !memcmp(&m->w[i + 2], operands, count * sizeof(uint32_t))) + return m->w[i + 1]; + } + for (size_t i = 0; i < m->new_type_count; i++) { + struct new_type *t = &m->new_types[i]; + if (t->op == op && t->count == count && !memcmp(t->operands, operands, count * sizeof(uint32_t))) + return t->id; + } + return 0; +} + +static uint32_t add_type(struct module *m, struct words *where, SpvOp op, const uint32_t *operands, uint32_t count) +{ + uint32_t id = new_id(m); + push(where, (count + 2) << 16 | op); + push(where, id); + for (uint32_t i = 0; i < count; i++) + push(where, operands[i]); + m->new_types = realloc(m->new_types, (m->new_type_count + 1) * sizeof(*m->new_types)); + struct new_type *t = &m->new_types[m->new_type_count++]; + *t = (struct new_type){.op = op, .count = count, .id = id}; + memcpy(t->operands, operands, count * sizeof(uint32_t)); + return id; +} + +/* Right after its pointee, so a global variable declared before the end of the types can use it. */ +static uint32_t pointer_type(struct module *m, uint32_t storage, uint32_t pointee) +{ + uint32_t operands[2] = {storage, pointee}; + uint32_t id = find_type(m, SpvOpTypePointer, operands, 2); + if (id) + return id; + size_t pointee_def = definition(m, pointee); + struct words *where = pointee_def ? edit(m, pointee_def, AFTER) : &m->globals; + return add_type(m, where, SpvOpTypePointer, operands, 2); +} + +static uint32_t scalar_type(struct module *m, SpvOp op, const uint32_t *operands, uint32_t count) +{ + uint32_t id = find_type(m, op, operands, count); + return id ? id : add_type(m, &m->globals, op, operands, count); +} + +static uint32_t float_type(struct module *m) +{ + const uint32_t width = 32; + return scalar_type(m, SpvOpTypeFloat, &width, 1); +} + +static uint32_t uint_type(struct module *m) +{ + const uint32_t operands[2] = {32, 0}; + return scalar_type(m, SpvOpTypeInt, operands, 2); +} + +static uint32_t bool_type(struct module *m) +{ + return scalar_type(m, SpvOpTypeBool, NULL, 0); +} + +static uint32_t constant(struct module *m, uint32_t type, uint32_t value) +{ + for (size_t i = 5; i < m->first_function; i += LENGTH(m->w[i])) { + if (OPCODE(m->w[i]) == SpvOpConstant && LENGTH(m->w[i]) == 4 && m->w[i + 1] == type && m->w[i + 3] == value) + return m->w[i + 2]; + } + uint32_t id = new_id(m); + EMIT(&m->globals, SpvOpConstant, type, id, value); + return id; +} + +static bool constant_value(const struct module *m, uint32_t id, uint32_t *value) +{ + size_t d = definition(m, id); + if (!d || OPCODE(m->w[d]) != SpvOpConstant || LENGTH(m->w[d]) != 4) + return false; + *value = m->w[d + 3]; + return true; +} + +/* What a pointer type points to, and where. */ +static bool pointer_parts(const struct module *m, uint32_t pointer, uint32_t *storage, uint32_t *pointee) +{ + size_t d = definition(m, pointer); + if (!d || OPCODE(m->w[d]) != SpvOpTypePointer) + return false; + *storage = m->w[d + 2]; + *pointee = m->w[d + 3]; + return true; +} + +static bool is_float32(const struct module *m, uint32_t type) +{ + size_t d = definition(m, type); + return d && OPCODE(m->w[d]) == SpvOpTypeFloat && m->w[d + 2] == 32; +} + +/* Varying locations a type takes up. */ +static uint32_t location_span(const struct module *m, uint32_t type, int depth) +{ + size_t d = definition(m, type); + if (!d || depth > 8) + return 1; + uint32_t value; + switch (OPCODE(m->w[d])) { + case SpvOpTypeArray: + return constant_value(m, m->w[d + 3], &value) ? value * location_span(m, m->w[d + 2], depth + 1) : 1; + case SpvOpTypeMatrix: + return m->w[d + 3] * location_span(m, m->w[d + 2], depth + 1); + case SpvOpTypeStruct: { + uint32_t span = 0; + for (uint32_t i = 2; i < LENGTH(m->w[d]); i++) + span += location_span(m, m->w[d + i], depth + 1); + return span; + } + case SpvOpTypeVector: { + size_t component = definition(m, m->w[d + 2]); + bool wide = component && m->w[component + 2] == 64; + return wide && m->w[d + 3] > 2 ? 2 : 1; + } + default: + return 1; + } +} + +struct distance_var { + uint32_t id; + uint32_t built_in; + uint32_t storage; + size_t decoration; +}; + +struct distance_member { + uint32_t structure; + uint32_t member; +}; + +struct analysis { + struct distance_var vars[8]; + uint32_t var_count; + struct distance_member members[8]; + uint32_t member_count; + bool member_locations; +}; + +static bool is_distance(uint32_t built_in) +{ + return built_in == SpvBuiltInClipDistance || built_in == SpvBuiltInCullDistance; +} + +static void analyze(struct module *m, struct analysis *a) +{ + memset(a, 0, sizeof(*a)); + for (size_t i = 5; i < m->first_declaration; i += LENGTH(m->w[i])) { + uint32_t op = OPCODE(m->w[i]); + if (op == SpvOpDecorate && LENGTH(m->w[i]) == 4 && m->w[i + 2] == SpvDecorationBuiltIn && + is_distance(m->w[i + 3])) { + uint32_t storage, pointee; + size_t d = definition(m, m->w[i + 1]); + if (a->var_count < 8 && d && OPCODE(m->w[d]) == SpvOpVariable && + pointer_parts(m, m->w[d + 1], &storage, &pointee)) { + a->vars[a->var_count++] = + (struct distance_var){.id = m->w[i + 1], .built_in = m->w[i + 3], .storage = storage, .decoration = i}; + } else { + m->failed = true; + } + } else if (op == SpvOpMemberDecorate && LENGTH(m->w[i]) == 5 && m->w[i + 3] == SpvDecorationBuiltIn && + is_distance(m->w[i + 4])) { + if (a->member_count < 8) + a->members[a->member_count++] = (struct distance_member){m->w[i + 1], m->w[i + 2]}; + else + m->failed = true; + } else if (op == SpvOpMemberDecorate && LENGTH(m->w[i]) >= 5 && m->w[i + 3] == SpvDecorationLocation) { + a->member_locations = true; + } + } +} + +static bool uses_distances(const uint32_t *words, size_t count) +{ + for (size_t i = 5; i + 1 < count && LENGTH(words[i]); i += LENGTH(words[i])) { + uint32_t op = OPCODE(words[i]); + if (op == SpvOpCapability && (words[i + 1] == SpvCapabilityClipDistance || words[i + 1] == SpvCapabilityCullDistance)) + return true; + if (op != SpvOpCapability && op != SpvOpExtension && op != SpvOpExtInstImport && op != SpvOpMemoryModel) + break; + } + return false; +} + +bool spirv_uses_distances(const uint32_t *words, size_t count) +{ + return count > 5 && words[0] == SpvMagicNumber && uses_distances(words, count); +} + +static void remove_capabilities(struct module *m) +{ + for (size_t i = 5; i < m->first_declaration; i += LENGTH(m->w[i])) { + if (OPCODE(m->w[i]) == SpvOpCapability && + (m->w[i + 1] == SpvCapabilityClipDistance || m->w[i + 1] == SpvCapabilityCullDistance)) + edit(m, i, INSTEAD); + } +} + +static void add_to_interface(struct module *m, uint32_t id) +{ + m->interface_add = realloc(m->interface_add, (m->interface_add_count + 1) * sizeof(uint32_t)); + m->interface_add[m->interface_add_count++] = id; +} + +/* A standalone distance variable becomes Private, and its decorations go. */ +static void privatize(struct module *m, const struct distance_var *var) +{ + size_t d = definition(m, var->id); + uint32_t storage, pointee; + pointer_parts(m, m->w[d + 1], &storage, &pointee); + m->w[d + 1] = pointer_type(m, SpvStorageClassPrivate, pointee); + m->w[d + 3] = SpvStorageClassPrivate; + mark_private(m, var->id); + /* Before SPIR-V 1.4 an entry point lists only its inputs and outputs. */ + if (m->version < VERSION_1_4) + m->interface_remove[var->id] = 1; + for (size_t i = 5; i < m->first_declaration; i += LENGTH(m->w[i])) { + if (OPCODE(m->w[i]) == SpvOpDecorate && m->w[i + 1] == var->id) + edit(m, i, INSTEAD); + } +} + +struct block_var { + uint32_t id; + uint32_t structure; + bool arrayed; + uint32_t array_length; +}; + +struct stand_in { + uint32_t block; + uint32_t member; + uint32_t variable; +}; + +/* Every variable of a block type with a distance member, and a Private stand-in per member. */ +static uint32_t make_stand_ins(struct module *m, const struct analysis *a, struct stand_in *stand_ins, uint32_t max) +{ + uint32_t count = 0; + for (size_t i = m->first_declaration; i < m->first_function; i += LENGTH(m->w[i])) { + if (OPCODE(m->w[i]) != SpvOpVariable) + continue; + uint32_t storage, pointee; + if (!pointer_parts(m, m->w[i + 1], &storage, &pointee)) + continue; + struct block_var var = {.id = m->w[i + 2], .structure = pointee}; + size_t d = definition(m, pointee); + if (d && OPCODE(m->w[d]) == SpvOpTypeArray) { + var.arrayed = true; + var.structure = m->w[d + 2]; + var.array_length = m->w[d + 3]; + } + for (uint32_t k = 0; k < a->member_count; k++) { + if (a->members[k].structure != var.structure) + continue; + if (count == max) { + m->failed = true; + return count; + } + size_t s = definition(m, var.structure); + uint32_t member_type = m->w[s + 2 + a->members[k].member]; + uint32_t type = member_type; + if (var.arrayed) { + const uint32_t operands[2] = {member_type, var.array_length}; + type = add_type(m, &m->globals, SpvOpTypeArray, operands, 2); + } + uint32_t variable = new_id(m); + EMIT(&m->globals, SpvOpVariable, pointer_type(m, SpvStorageClassPrivate, type), variable, + SpvStorageClassPrivate); + if (m->version >= VERSION_1_4) + add_to_interface(m, variable); + stand_ins[count++] = (struct stand_in){var.id, a->members[k].member, variable}; + } + } + return count; +} + +static const struct stand_in *find_stand_in(const struct module *m, const struct stand_in *stand_ins, uint32_t count, + uint32_t base, uint32_t member_index) +{ + uint32_t member; + if (!constant_value(m, member_index, &member)) + return NULL; + for (uint32_t i = 0; i < count; i++) { + if (stand_ins[i].block == base && stand_ins[i].member == member) + return &stand_ins[i]; + } + return NULL; +} + +static bool is_arrayed_block(const struct module *m, uint32_t variable) +{ + size_t d = definition(m, variable); + uint32_t storage, pointee; + if (!d || !pointer_parts(m, m->w[d + 1], &storage, &pointee)) + return false; + size_t p = definition(m, pointee); + return p && OPCODE(m->w[p]) == SpvOpTypeArray; +} + +/* + * One pass over the function bodies: accesses to a distance member go to its stand-in, and every + * pointer derived from a Private one becomes Private too. A Private pointer handed to a function + * or merged through a phi would need its types changed beyond this, so that fails the rewrite. + */ +static void retarget_accesses(struct module *m, const struct stand_in *stand_ins, uint32_t stand_in_count) +{ + for (size_t i = m->first_function; i < m->count; i += LENGTH(m->w[i])) { + uint32_t op = OPCODE(m->w[i]); + uint32_t length = LENGTH(m->w[i]); + switch (op) { + case SpvOpAccessChain: + case SpvOpInBoundsAccessChain: { + uint32_t base = m->w[i + 3]; + bool arrayed = is_arrayed_block(m, base); + uint32_t member_position = arrayed ? 5 : 4; + const struct stand_in *stand_in = + length > member_position + ? find_stand_in(m, stand_ins, stand_in_count, base, m->w[i + member_position]) + : NULL; + uint32_t storage, pointee; + if (stand_in) { + if (!pointer_parts(m, m->w[i + 1], &storage, &pointee)) { + m->failed = true; + break; + } + /* Before edit(): a new pointer type is an edit too, and would move the chain's buffer. */ + uint32_t type = pointer_type(m, SpvStorageClassPrivate, pointee); + struct words *chain = edit(m, i, INSTEAD); + push(chain, (length - 1) << 16 | op); + push(chain, type); + push(chain, m->w[i + 2]); + push(chain, stand_in->variable); + for (uint32_t k = 4; k < length; k++) { + if (k != member_position) + push(chain, m->w[i + k]); + } + mark_private(m, m->w[i + 2]); + } else if (is_private(m, base)) { + if (!pointer_parts(m, m->w[i + 1], &storage, &pointee)) { + m->failed = true; + break; + } + m->w[i + 1] = pointer_type(m, SpvStorageClassPrivate, pointee); + mark_private(m, m->w[i + 2]); + } + break; + } + case SpvOpPtrAccessChain: + case SpvOpCopyObject: + if (is_private(m, m->w[i + 3])) { + uint32_t storage, pointee; + if (!pointer_parts(m, m->w[i + 1], &storage, &pointee)) { + m->failed = true; + break; + } + m->w[i + 1] = pointer_type(m, SpvStorageClassPrivate, pointee); + mark_private(m, m->w[i + 2]); + } + break; + case SpvOpFunctionCall: + for (uint32_t k = 4; k < length; k++) { + if (is_private(m, m->w[i + k])) + m->failed = true; + } + break; + case SpvOpPhi: + case SpvOpSelect: + for (uint32_t k = 3; k < length; k++) { + if (is_private(m, m->w[i + k])) + m->failed = true; + } + break; + default: + break; + } + } +} + +static uint32_t string_words(const uint32_t *words, uint32_t available) +{ + for (uint32_t i = 0; i < available; i++) { + uint32_t word = words[i]; + if (!(word & 0xff) || !(word & 0xff00) || !(word & 0xff0000) || !(word & 0xff000000)) + return i + 1; + } + return available; +} + +static void rewrite_interfaces(struct module *m) +{ + for (size_t i = 5; i < m->first_declaration; i += LENGTH(m->w[i])) { + if (OPCODE(m->w[i]) != SpvOpEntryPoint) + continue; + uint32_t length = LENGTH(m->w[i]); + uint32_t first = 3 + string_words(&m->w[i + 3], length - 3); + struct words *entry = edit(m, i, INSTEAD); + push(entry, 0); + for (uint32_t k = 1; k < first; k++) + push(entry, m->w[i + k]); + for (uint32_t k = first; k < length; k++) { + uint32_t id = m->w[i + k]; + if (id >= m->original_bound || !m->interface_remove[id]) + push(entry, id); + } + for (size_t k = 0; k < m->interface_add_count; k++) + push(entry, m->interface_add[k]); + entry->data[0] = (uint32_t)entry->count << 16 | SpvOpEntryPoint; + } +} + +static void emit_edits(const struct module *m, struct words *out, size_t offset, enum place place, bool *replaced) +{ + for (size_t k = 0; k < m->edit_count; k++) { + const struct edit *e = &m->edits[k]; + if (e->offset != offset || e->place != place) + continue; + for (size_t j = 0; j < e->words.count; j++) + push(out, e->words.data[j]); + if (replaced) + *replaced = true; + } +} + +static bool finish(struct module *m, struct spirv_code *out) +{ + if (m->failed) + return false; + rewrite_interfaces(m); + struct words result = {0}; + for (int i = 0; i < 5; i++) + push(&result, m->w[i]); + result.data[3] = m->bound; + for (size_t i = 5; i < m->count; i += LENGTH(m->w[i])) { + if (i == m->first_function) { + for (size_t j = 0; j < m->globals.count; j++) + push(&result, m->globals.data[j]); + } + emit_edits(m, &result, i, BEFORE, NULL); + bool replaced = false; + emit_edits(m, &result, i, INSTEAD, &replaced); + if (!replaced) { + for (uint32_t j = 0; j < LENGTH(m->w[i]); j++) + push(&result, m->w[i + j]); + } + emit_edits(m, &result, i, AFTER, NULL); + } + out->words = result.data; + out->count = result.count; + return true; +} + +static void strip(struct module *m, const struct analysis *a, uint32_t keep) +{ + for (uint32_t i = 0; i < a->var_count; i++) { + if (a->vars[i].id != keep) + privatize(m, &a->vars[i]); + } + struct stand_in stand_ins[16]; + uint32_t count = make_stand_ins(m, a, stand_ins, 16); + retarget_accesses(m, stand_ins, count); + remove_capabilities(m); +} + +bool spirv_strip_distances(const uint32_t *words, size_t count, struct spirv_code *out) +{ + struct module m; + if (!load(&m, words, count)) + return false; + struct analysis a; + analyze(&m, &a); + strip(&m, &a, 0); + bool done = finish(&m, out); + unload(&m); + return done; +} + +static const struct distance_var *clip_output(const struct module *m, const struct analysis *a, uint32_t *length) +{ + for (uint32_t i = 0; i < a->var_count; i++) { + const struct distance_var *var = &a->vars[i]; + if (var->built_in != SpvBuiltInClipDistance || var->storage != SpvStorageClassOutput) + continue; + uint32_t storage, pointee; + pointer_parts(m, m->w[definition(m, var->id) + 1], &storage, &pointee); + size_t d = definition(m, pointee); + if (d && OPCODE(m->w[d]) == SpvOpTypeArray && is_float32(m, m->w[d + 2]) && + constant_value(m, m->w[d + 3], length)) + return var; + } + return NULL; +} + +void spirv_clip_info(const uint32_t *words, size_t count, enum clip_role role, struct clip_info *info) +{ + memset(info, 0, sizeof(*info)); + struct module m; + if (!load(&m, words, count)) { + info->locations_used = UINT32_MAX; + return; + } + struct analysis a; + analyze(&m, &a); + if (role == CLIP_PRODUCER) + info->has_clip = clip_output(&m, &a, &info->clip_count) != NULL; + uint32_t storage_wanted = role == CLIP_PRODUCER ? SpvStorageClassOutput : SpvStorageClassInput; + for (size_t i = 5; i < m.first_declaration; i += LENGTH(m.w[i])) { + if (OPCODE(m.w[i]) != SpvOpDecorate || LENGTH(m.w[i]) != 4 || m.w[i + 2] != SpvDecorationLocation) + continue; + size_t d = definition(&m, m.w[i + 1]); + uint32_t storage, pointee; + if (!d || OPCODE(m.w[d]) != SpvOpVariable || !pointer_parts(&m, m.w[d + 1], &storage, &pointee) || + storage != storage_wanted) + continue; + uint32_t end = m.w[i + 3] + location_span(&m, pointee, 0); + if (end > info->locations_used) + info->locations_used = end; + } + if (a.member_locations || m.failed) + info->locations_used = UINT32_MAX; + unload(&m); +} + +/* Fragment shader: read the clip distances at `location` and discard when any is negative. */ +static void add_clip_test(struct module *m, uint32_t location, uint32_t clip_count) +{ + uint32_t float_id = float_type(m); + uint32_t uint_id = uint_type(m); + uint32_t bool_id = bool_type(m); + const uint32_t array_operands[2] = {float_id, constant(m, uint_id, clip_count)}; + uint32_t array_id = add_type(m, &m->globals, SpvOpTypeArray, array_operands, 2); + uint32_t input_pointer = pointer_type(m, SpvStorageClassInput, array_id); + uint32_t element_pointer = pointer_type(m, SpvStorageClassInput, float_id); + uint32_t zero = constant(m, float_id, 0); + uint32_t indices[8]; + for (uint32_t i = 0; i < clip_count; i++) + indices[i] = constant(m, uint_id, i); + uint32_t input = new_id(m); + EMIT(&m->globals, SpvOpVariable, input_pointer, input, SpvStorageClassInput); + EMIT(edit(m, m->first_declaration, BEFORE), SpvOpDecorate, input, SpvDecorationLocation, location); + add_to_interface(m, input); + + for (size_t i = 5; i < m->first_declaration; i += LENGTH(m->w[i])) { + if (OPCODE(m->w[i]) != SpvOpEntryPoint || m->w[i + 1] != SpvExecutionModelFragment) + continue; + size_t function = definition(m, m->w[i + 2]); + if (!function) { + m->failed = true; + return; + } + size_t label = function + LENGTH(m->w[function]); + if (label >= m->count || OPCODE(m->w[label]) != SpvOpLabel) { + m->failed = true; + return; + } + /* Function variables must open the entry block, so the test goes after them. */ + size_t last = label; + for (size_t k = label + LENGTH(m->w[label]); k < m->count; k += LENGTH(m->w[k])) { + uint32_t op = OPCODE(m->w[k]); + if (op != SpvOpVariable && op != SpvOpLine && op != SpvOpNoLine) + break; + last = k; + } + uint32_t original_label = m->w[label + 1]; + uint32_t entry = new_id(m), kill = new_id(m), join = new_id(m); + EMIT(edit(m, label, INSTEAD), SpvOpLabel, entry); + struct words *test = edit(m, last, AFTER); + uint32_t any = 0; + for (uint32_t c = 0; c < clip_count; c++) { + uint32_t pointer = new_id(m), value = new_id(m), negative = new_id(m); + EMIT(test, SpvOpAccessChain, element_pointer, pointer, input, indices[c]); + EMIT(test, SpvOpLoad, float_id, value, pointer); + EMIT(test, SpvOpFOrdLessThan, bool_id, negative, value, zero); + if (any) { + uint32_t either = new_id(m); + EMIT(test, SpvOpLogicalOr, bool_id, either, any, negative); + any = either; + } else { + any = negative; + } + } + EMIT(test, SpvOpSelectionMerge, join, SpvSelectionControlMaskNone); + EMIT(test, SpvOpBranchConditional, any, kill, join); + EMIT(test, SpvOpLabel, kill); + push(test, 1u << 16 | SpvOpKill); + EMIT(test, SpvOpLabel, join); + EMIT(test, SpvOpBranch, original_label); + EMIT(test, SpvOpLabel, original_label); + } +} + +bool spirv_emulate_clip(const uint32_t *words, size_t count, enum clip_role role, uint32_t location, + uint32_t clip_count, struct spirv_code *out) +{ + if (clip_count == 0 || clip_count > 8) + return false; + struct module m; + if (!load(&m, words, count)) + return false; + struct analysis a; + analyze(&m, &a); + uint32_t keep = 0; + if (role == CLIP_PRODUCER) { + uint32_t length = 0; + const struct distance_var *clip = clip_output(&m, &a, &length); + if (!clip || length != clip_count) { + unload(&m); + return false; + } + keep = clip->id; + m.w[clip->decoration + 2] = SpvDecorationLocation; + m.w[clip->decoration + 3] = location; + } + strip(&m, &a, keep); + if (role == CLIP_CONSUMER) + add_clip_test(&m, location, clip_count); + bool done = finish(&m, out); + unload(&m); + return done; +} diff --git a/tools/venus/measure-present.patch b/tools/venus/measure-present.patch new file mode 100644 index 000000000..5d8eb87cf --- /dev/null +++ b/tools/venus/measure-present.patch @@ -0,0 +1,153 @@ +diff --git a/src/virtio/vulkan/vn_wsi.c b/src/virtio/vulkan/vn_wsi.c +index f0b41b1..5c08312 100644 +--- a/src/virtio/vulkan/vn_wsi.c ++++ b/src/virtio/vulkan/vn_wsi.c +@@ -23,6 +23,33 @@ + #define DRM_FORMAT_MOD_LINEAR 0 + #endif + ++/* MEASURE: async present costs, logged every 120 presents. */ ++static struct { ++ uint32_t presents, fence_waits, flushes; ++ int64_t present_sum, present_max, fence_sum, fence_max, flush_sum, flush_max; ++} vn_measure; ++ ++#define VN_MEASURE_ADD(name, ns) \ ++ do { \ ++ vn_measure.name##_sum += (ns); \ ++ vn_measure.name##_max = MAX2(vn_measure.name##_max, (ns)); \ ++ } while (0) ++ ++static void ++vn_measure_present(int64_t ns) ++{ ++ VN_MEASURE_ADD(present, ns); ++ if (++vn_measure.presents < 120) ++ return; ++ fprintf(stderr, "MEASURE vn: present thread %.2f/%.2f ms | fence wait %.2f/%.2f ms (%u) | app blocked in flush %.2f/%.2f ms (%u)\n", ++ vn_measure.present_sum / 120 / 1e6, vn_measure.present_max / 1e6, ++ vn_measure.fence_waits ? vn_measure.fence_sum / vn_measure.fence_waits / 1e6 : 0.0, ++ vn_measure.fence_max / 1e6, vn_measure.fence_waits, ++ vn_measure.flushes ? vn_measure.flush_sum / vn_measure.flushes / 1e6 : 0.0, ++ vn_measure.flush_max / 1e6, vn_measure.flushes); ++ memset(&vn_measure, 0, sizeof(vn_measure)); ++} ++ + /* The common WSI support makes some assumptions about the driver. + * + * In wsi_device_init, it assumes VK_EXT_pci_bus_info is available. In +@@ -331,6 +358,7 @@ vn_wsi_memory_type_index(const struct vn_device *dev, + !(type_bits & BITFIELD_BIT(readback_type))) + return alloc_info->memoryTypeIndex; + ++ fprintf(stderr, "MEASURE vn: WSI readback buffer in memory type %d\n", readback_type); + return readback_type; + } + +@@ -479,8 +507,11 @@ vn_wsi_fence_wait(VkQueue queue_handle) + simple_mtx_unlock(&queue->async_present.queue_mutex); + vn_wsi_chains_unlock(dev, queue->async_present.info, /*all=*/false); + ++ int64_t measure_start = os_time_get_nano(); + result = vn_WaitForFences(dev_handle, 1, &queue->async_present.fence, true, + UINT64_MAX); ++ VN_MEASURE_ADD(fence, os_time_get_nano() - measure_start); ++ vn_measure.fence_waits++; + + vn_wsi_chains_lock(dev, queue->async_present.info, /*all=*/false); + if (can_unlock_queue) +@@ -537,8 +568,11 @@ vn_wsi_flush(VkQueue queue_handle) + /* Being able to acquire the lock ensures async present queue access + * has completed. + */ ++ int64_t measure_start = os_time_get_nano(); + simple_mtx_lock(&queue->async_present.queue_mutex); + simple_mtx_unlock(&queue->async_present.queue_mutex); ++ VN_MEASURE_ADD(flush, os_time_get_nano() - measure_start); ++ vn_measure.flushes++; + } + + static VkPresentInfoKHR * +@@ -811,9 +845,11 @@ vn_wsi_present_thread(void *data) + + queue->async_present.pending = false; + ++ int64_t measure_start = os_time_get_nano(); + queue->async_present.result = + wsi_common_queue_present(queue_vk->base.device->physical->wsi_device, + queue_vk, queue->async_present.info); ++ vn_measure_present(os_time_get_nano() - measure_start); + + vn_wsi_chains_unlock(dev, queue->async_present.info, /*all=*/true); + simple_mtx_unlock(&queue->async_present.queue_mutex); +diff --git a/src/vulkan/wsi/wsi_common_wayland.c b/src/vulkan/wsi/wsi_common_wayland.c +index 047c8b3..a153a39 100644 +--- a/src/vulkan/wsi/wsi_common_wayland.c ++++ b/src/vulkan/wsi/wsi_common_wayland.c +@@ -3277,6 +3277,38 @@ set_timestamp(struct wsi_wl_swapchain *chain, + return true; + } + ++/* MEASURE: per-frame cost of the shm copy, logged every 120 presents. */ ++static struct { ++ uint32_t frames; ++ int64_t last_present, interval_sum, interval_max, copy_sum, copy_max, total_sum, total_max; ++} wl_measure; ++ ++static void ++wl_measure_frame(struct wsi_wl_swapchain *chain, int64_t start, int64_t copy_ns, int64_t end) ++{ ++ if (wl_measure.last_present) { ++ int64_t interval = start - wl_measure.last_present; ++ wl_measure.interval_sum += interval; ++ wl_measure.interval_max = MAX2(wl_measure.interval_max, interval); ++ } ++ wl_measure.last_present = start; ++ wl_measure.copy_sum += copy_ns; ++ wl_measure.copy_max = MAX2(wl_measure.copy_max, copy_ns); ++ wl_measure.total_sum += end - start; ++ wl_measure.total_max = MAX2(wl_measure.total_max, end - start); ++ if (++wl_measure.frames < 120) ++ return; ++ fprintf(stderr, "MEASURE wl: %ux%u type %d | interval %.2f/%.2f ms | copy %.2f/%.2f ms (%u KB) | present %.2f/%.2f ms\n", ++ chain->extent.width, chain->extent.height, chain->buffer_type, ++ wl_measure.interval_sum / 119 / 1e6, wl_measure.interval_max / 1e6, ++ wl_measure.copy_sum / 120 / 1e6, wl_measure.copy_max / 1e6, ++ chain->images[0].base.row_pitches[0] * chain->extent.height / 1024, ++ wl_measure.total_sum / 120 / 1e6, wl_measure.total_max / 1e6); ++ int64_t last = wl_measure.last_present; ++ memset(&wl_measure, 0, sizeof(wl_measure)); ++ wl_measure.last_present = last; ++} ++ + static VkResult + wsi_wl_swapchain_queue_present(struct wsi_swapchain *wsi_chain, + uint32_t image_index, +@@ -3286,6 +3318,7 @@ wsi_wl_swapchain_queue_present(struct wsi_swapchain *wsi_chain, + struct wsi_wl_swapchain *chain = (struct wsi_wl_swapchain *)wsi_chain; + bool timestamped = false; + bool queue_dispatched = false; ++ int64_t measure_start = os_time_get_nano(), measure_copy = 0; + + MESA_TRACE_FUNC_FLOW(&chain->images[image_index].wayland_buffer.flow); + +@@ -3314,9 +3347,11 @@ wsi_wl_swapchain_queue_present(struct wsi_swapchain *wsi_chain, + + if (chain->buffer_type == WSI_WL_BUFFER_SHM_MEMCPY) { + struct wsi_wl_image *image = &chain->images[image_index]; ++ int64_t copy_start = os_time_get_nano(); + wsi_image_invalidate_cpu_map(&chain->base, &image->base); + memcpy(image->shm_ptr, image->base.cpu_map, + image->base.row_pitches[0] * chain->extent.height); ++ measure_copy = os_time_get_nano() - copy_start; + } + + VkResult ret = wsi_wl_swapchain_update_colorspace(chain); +@@ -3515,6 +3550,7 @@ wsi_wl_swapchain_queue_present(struct wsi_swapchain *wsi_chain, + + memset(&chain->timing_request, 0, sizeof(chain->timing_request)); + ++ wl_measure_frame(chain, measure_start, measure_copy, os_time_get_nano()); + return VK_SUCCESS; + } + diff --git a/tools/venus/memtypes.c b/tools/venus/memtypes.c new file mode 100644 index 000000000..39520298a --- /dev/null +++ b/tools/venus/memtypes.c @@ -0,0 +1,40 @@ +// Prints the system Vulkan driver's memory types and heaps, as Android apps see them; Venus +// rewrites them for the guest. Build in WSL and run over ADB (from Git Bash, MSYS_NO_PATHCONV=1): +// /home/delo/android-sdk/ndk/27.3.13750724/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android31-clang \ +// -O1 -o memtypes memtypes.c -lvulkan +// adb push memtypes /data/local/tmp/ && adb shell "chmod 755 /data/local/tmp/memtypes && /data/local/tmp/memtypes" +#include +#include + +int main(void) +{ + VkApplicationInfo app = { .sType = VK_STRUCTURE_TYPE_APPLICATION_INFO, .apiVersion = VK_API_VERSION_1_1 }; + VkInstanceCreateInfo ici = { .sType = VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO, .pApplicationInfo = &app }; + VkInstance inst; + if (vkCreateInstance(&ici, NULL, &inst) != VK_SUCCESS) { + printf("vkCreateInstance failed\n"); + return 1; + } + uint32_t count = 1; + VkPhysicalDevice pd; + vkEnumeratePhysicalDevices(inst, &count, &pd); + VkPhysicalDeviceProperties props; + vkGetPhysicalDeviceProperties(pd, &props); + printf("%s\n", props.deviceName); + VkPhysicalDeviceMemoryProperties mp; + vkGetPhysicalDeviceMemoryProperties(pd, &mp); + for (uint32_t i = 0; i < mp.memoryHeapCount; i++) + printf("heap %u: %llu MiB flags 0x%x\n", i, (unsigned long long)(mp.memoryHeaps[i].size >> 20), mp.memoryHeaps[i].flags); + for (uint32_t i = 0; i < mp.memoryTypeCount; i++) { + VkMemoryPropertyFlags f = mp.memoryTypes[i].propertyFlags; + printf("type %u: heap %u flags 0x%x%s%s%s%s%s%s\n", i, mp.memoryTypes[i].heapIndex, f, + f & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT ? " DEVICE_LOCAL" : "", + f & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT ? " HOST_VISIBLE" : "", + f & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT ? " HOST_COHERENT" : "", + f & VK_MEMORY_PROPERTY_HOST_CACHED_BIT ? " HOST_CACHED" : "", + f & VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT ? " LAZY" : "", + f & VK_MEMORY_PROPERTY_PROTECTED_BIT ? " PROTECTED" : ""); + } + vkDestroyInstance(inst, NULL); + return 0; +} diff --git a/tools/venus/mesa-venus-vtest.patch b/tools/venus/mesa-venus-vtest.patch new file mode 100644 index 000000000..e2b569975 --- /dev/null +++ b/tools/venus/mesa-venus-vtest.patch @@ -0,0 +1,541 @@ +diff --git a/src/virtio/vulkan/vn_device.c b/src/virtio/vulkan/vn_device.c +index f42b9f9..6ed3051 100644 +--- a/src/virtio/vulkan/vn_device.c ++++ b/src/virtio/vulkan/vn_device.c +@@ -45,6 +45,7 @@ vn_queue_fini(struct vn_queue *queue) + if (queue->wait_fence != VK_NULL_HANDLE) { + vn_DestroyFence(dev_handle, queue->wait_fence, NULL); + } ++ simple_mtx_destroy(&queue->submit_mutex); + vn_cached_storage_fini(&queue->storage); + vn_queue_base_fini(&queue->base); + } +@@ -65,6 +66,7 @@ vn_queue_init(struct vn_device *dev, + queue->index = array_index; + + vn_cached_storage_init(&queue->storage, &dev->base.vk.alloc); ++ simple_mtx_init(&queue->submit_mutex, mtx_plain); + + if (dev->physical_device->emulate_second_queue == + queue_info->queueFamilyIndex && +diff --git a/src/virtio/vulkan/vn_device_memory.c b/src/virtio/vulkan/vn_device_memory.c +index 8b6eb4c..be146b6 100644 +--- a/src/virtio/vulkan/vn_device_memory.c ++++ b/src/virtio/vulkan/vn_device_memory.c +@@ -93,11 +93,12 @@ vn_device_memory_bo_init(struct vn_device *dev, struct vn_device_memory *mem) + } + + const struct vk_device_memory *mem_vk = &mem->base.vk; +- const VkMemoryType *mem_type = &dev->physical_device->memory_properties +- .memoryTypes[mem_vk->memory_type_index]; ++ /* The renderer's flags: WSI maps a type the app sees as not mappable. */ ++ const VkMemoryPropertyFlags flags = ++ dev->physical_device->renderer_memory_flags[mem_vk->memory_type_index]; + return vn_renderer_bo_create_from_device_memory( +- dev->renderer, batch, mem_vk->size, mem->base.id, +- mem_type->propertyFlags, mem_vk->export_handle_types, &mem->base_bo); ++ dev->renderer, batch, mem_vk->size, mem->base.id, flags, ++ mem_vk->export_handle_types, &mem->base_bo); + } + + static inline void +@@ -374,6 +375,14 @@ vn_AllocateMemory(VkDevice device, + { + struct vn_device *dev = vn_device_from_handle(device); + ++ VkMemoryAllocateInfo local_alloc_info; ++ const uint32_t type_index = vn_wsi_memory_type_index(dev, pAllocateInfo); ++ if (type_index != pAllocateInfo->memoryTypeIndex) { ++ local_alloc_info = *pAllocateInfo; ++ local_alloc_info.memoryTypeIndex = type_index; ++ pAllocateInfo = &local_alloc_info; ++ } ++ + struct vn_device_memory *mem = vk_device_memory_create( + &dev->base.vk, pAllocateInfo, pAllocator, sizeof(*mem)); + if (!mem) +diff --git a/src/virtio/vulkan/vn_physical_device.c b/src/virtio/vulkan/vn_physical_device.c +index f3581aa..c9c50e5 100644 +--- a/src/virtio/vulkan/vn_physical_device.c ++++ b/src/virtio/vulkan/vn_physical_device.c +@@ -982,6 +982,7 @@ vn_physical_device_init_memory_properties( + ring, vn_physical_device_to_handle(physical_dev), &props2); + + physical_dev->memory_properties = props2.memoryProperties; ++ physical_dev->wsi_readback_memory_type = -1; + + /* Kernel makes every mapping coherent. If a memory type is truly + * incoherent, it's better to remove the host-visible flag than silently +@@ -996,12 +997,18 @@ vn_physical_device_init_memory_properties( + VkMemoryPropertyFlags *flags = &props->memoryTypes[i].propertyFlags; + const bool coherent = *flags & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT; + const bool cached = *flags & VK_MEMORY_PROPERTY_HOST_CACHED_BIT; ++ physical_dev->renderer_memory_flags[i] = *flags; + if (coherent) { + if (first_coherent == VK_MAX_MEMORY_TYPES) + first_coherent = i; + if (cached) + has_coherent_cached = true; + } else if (cached) { ++ /* vtest maps the renderer's dma-buf itself and can sync the CPU ++ * cache around a read, which WSI does before copying a frame out. ++ */ ++ if (VN_DEBUG(VTEST) && physical_dev->wsi_readback_memory_type < 0) ++ physical_dev->wsi_readback_memory_type = i; + *flags &= ~(VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | + VK_MEMORY_PROPERTY_HOST_CACHED_BIT); + } +@@ -1060,7 +1067,9 @@ vn_physical_device_init_external_fence_handles( + { + physical_dev->external_fence_handles = 0; + +- if (physical_dev->instance->renderer->info.has_external_sync) { ++ /* Over vtest an export waits on the CPU and returns -1 (vn_sync.c). */ ++ if (physical_dev->instance->renderer->info.has_external_sync || ++ VN_DEBUG(VTEST)) { + #if !DETECT_OS_WINDOWS + physical_dev->external_fence_handles = + VK_EXTERNAL_FENCE_HANDLE_TYPE_SYNC_FD_BIT; +@@ -1095,7 +1104,8 @@ vn_physical_device_init_external_semaphore_handles( + physical_dev->external_binary_semaphore_handles = 0; + physical_dev->external_timeline_semaphore_handles = 0; + +- if (physical_dev->instance->renderer->info.has_external_sync) { ++ if (physical_dev->instance->renderer->info.has_external_sync || ++ VN_DEBUG(VTEST)) { + #if !DETECT_OS_WINDOWS + physical_dev->external_binary_semaphore_handles = + VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_SYNC_FD_BIT; +@@ -1173,7 +1183,8 @@ vn_physical_device_get_native_extensions( + exts->KHR_map_memory2 = true; + #if !DETECT_OS_WINDOWS + exts->EXT_map_memory_placed = true; +- exts->EXT_physical_device_drm = true; ++ /* Over vtest there is no DRM device behind Venus. */ ++ exts->EXT_physical_device_drm = !VN_DEBUG(VTEST); + #endif + /* use common implementation */ + exts->EXT_tooling_info = true; +diff --git a/src/virtio/vulkan/vn_physical_device.h b/src/virtio/vulkan/vn_physical_device.h +index 2c2f4a9..c002a84 100644 +--- a/src/virtio/vulkan/vn_physical_device.h ++++ b/src/virtio/vulkan/vn_physical_device.h +@@ -112,6 +112,12 @@ struct vn_physical_device { + int emulate_second_queue; + + VkPhysicalDeviceMemoryProperties memory_properties; ++ /* The renderer's flags, before memory_properties hides the host-visible ++ * bit of cached types that are not coherent. ++ */ ++ VkMemoryPropertyFlags renderer_memory_flags[VK_MAX_MEMORY_TYPES]; ++ /* A hidden cached type for the CPU copy of WSI images, or -1. */ ++ int wsi_readback_memory_type; + + struct { + VkExternalMemoryHandleTypeFlagBits renderer_handle_type; +diff --git a/src/virtio/vulkan/vn_queue.c b/src/virtio/vulkan/vn_queue.c +index 057842a..dee1244 100644 +--- a/src/virtio/vulkan/vn_queue.c ++++ b/src/virtio/vulkan/vn_queue.c +@@ -1049,11 +1049,16 @@ vn_queue_submission_init_syncs(struct vn_queue_submission *submit) + assert(fence->payload->type == VN_SYNC_TYPE_SYNC); + + submit->temp.syncs[sync_index] = fence->payload->sync; +- submit->temp.sync_vals[sync_index] = 1; ++ submit->temp.sync_vals[sync_index++] = 1; + + /* fence is backed by renderer sync without renderer object */ + submit->fence_handle = VK_NULL_HANDLE; + } ++ ++ /* A signal semaphore counted for an imported sync_file has no renderer ++ * sync to fill in, and the slot would hand vtest stale storage. ++ */ ++ submit->sync_count = sync_index; + } + + static VkResult +@@ -1248,9 +1253,13 @@ vn_queue_submit(struct vn_queue_submission *submit) + struct vn_device *dev = vn_device_from_vk(queue->base.vk.base.device); + VkResult result; + ++ simple_mtx_lock(&queue->submit_mutex); ++ + result = vn_queue_submission_prepare_submit(submit); +- if (result != VK_SUCCESS) ++ if (result != VK_SUCCESS) { ++ simple_mtx_unlock(&queue->submit_mutex); + return vn_error(dev->instance, result); ++ } + + result = vn_queue_submission_do_submit(submit); + if (result != VK_SUCCESS) +@@ -1260,6 +1269,7 @@ vn_queue_submit(struct vn_queue_submission *submit) + + out_cleanup: + vn_queue_submission_cleanup(submit); ++ simple_mtx_unlock(&queue->submit_mutex); + return vn_result(dev->instance, result); + } + +diff --git a/src/virtio/vulkan/vn_queue.h b/src/virtio/vulkan/vn_queue.h +index 3ea6d9d..4160e56 100644 +--- a/src/virtio/vulkan/vn_queue.h ++++ b/src/virtio/vulkan/vn_queue.h +@@ -52,6 +52,11 @@ struct vn_queue { + /* for vn_queue_submission storage */ + struct vn_cached_storage storage; + ++ /* Serializes submissions. The async present thread submits to this queue ++ * while the application does, after vn_wsi_fence_wait drops queue_mutex. ++ */ ++ simple_mtx_t submit_mutex; ++ + /* for async queue present */ + struct { + /* Protects VkQueue host access except async present states. */ +diff --git a/src/virtio/vulkan/vn_renderer_vtest.c b/src/virtio/vulkan/vn_renderer_vtest.c +index fc59d32..c7b20f5 100644 +--- a/src/virtio/vulkan/vn_renderer_vtest.c ++++ b/src/virtio/vulkan/vn_renderer_vtest.c +@@ -13,9 +13,11 @@ + #include + #include + #include ++#include + #include + #include + ++#include "drm-uapi/dma-buf.h" + #include "drm-uapi/virtgpu_drm.h" + #include "util/os_file.h" + #include "util/os_misc.h" +@@ -38,8 +40,10 @@ struct vtest_bo { + struct vn_renderer_bo base; + + uint32_t blob_flags; +- /* might be closed after mmap */ ++ /* might be closed after mmap, but not when the memory is incoherent */ + int res_fd; ++ /* the CPU cache is synced through the dma-buf on flush and invalidate */ ++ bool incoherent; + }; + + struct vtest_sync { +@@ -607,13 +611,35 @@ vtest_sync_create(struct vn_renderer *renderer, + return VK_SUCCESS; + } + ++/* The guest maps the renderer's dma-buf, so a CPU cache sync on it covers the ++ * mapping. Coherent memory needs none. ++ */ ++static void ++vtest_bo_sync_cpu_cache(struct vtest *vtest, ++ struct vtest_bo *bo, ++ uint64_t direction) ++{ ++ if (!bo->incoherent || bo->res_fd < 0) ++ return; ++ ++ struct dma_buf_sync sync = { .flags = DMA_BUF_SYNC_START | direction }; ++ int ret = ioctl(bo->res_fd, DMA_BUF_IOCTL_SYNC, &sync); ++ if (!ret) { ++ sync.flags = DMA_BUF_SYNC_END | direction; ++ ret = ioctl(bo->res_fd, DMA_BUF_IOCTL_SYNC, &sync); ++ } ++ if (ret) ++ vn_log(vtest->instance, "dma-buf cache sync failed: %s", strerror(errno)); ++} ++ + static void + vtest_bo_invalidate(struct vn_renderer *renderer, + struct vn_renderer_bo *bo, + VkDeviceSize offset, + VkDeviceSize size) + { +- /* nop */ ++ vtest_bo_sync_cpu_cache((struct vtest *)renderer, (struct vtest_bo *)bo, ++ DMA_BUF_SYNC_READ); + } + + static void +@@ -622,7 +648,8 @@ vtest_bo_flush(struct vn_renderer *renderer, + VkDeviceSize offset, + VkDeviceSize size) + { +- /* nop */ ++ vtest_bo_sync_cpu_cache((struct vtest *)renderer, (struct vtest_bo *)bo, ++ DMA_BUF_SYNC_WRITE); + } + + static void * +@@ -651,7 +678,7 @@ vtest_bo_map(struct vn_renderer *renderer, + } else { + bo->base.mmap_ptr = ptr; + /* we don't need the fd anymore */ +- if (!shareable) { ++ if (!shareable && !bo->incoherent) { + close(bo->res_fd); + bo->res_fd = -1; + } +@@ -735,6 +762,8 @@ vtest_bo_create_from_device_memory( + }, + .res_fd = res_fd, + .blob_flags = blob_flags, ++ .incoherent = (flags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) && ++ !(flags & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT), + }; + + *out_bo = &bo->base; +diff --git a/src/virtio/vulkan/vn_sync.c b/src/virtio/vulkan/vn_sync.c +index ee0a539..4687520 100644 +--- a/src/virtio/vulkan/vn_sync.c ++++ b/src/virtio/vulkan/vn_sync.c +@@ -26,7 +26,8 @@ vn_sync_payload_release_internal(struct vn_device *dev, + vn_renderer_sync_destroy(dev->renderer, payload->sync); + break; + case VN_SYNC_TYPE_TIMELINE_SYNC: +- for (uint32_t i = 0; i < dev->queue_count; i++) ++ /* One sync per queue plus one for host signals. */ ++ for (uint32_t i = 0; i <= dev->queue_count; i++) + vn_renderer_sync_destroy(dev->renderer, payload->syncs[i]); + vk_free2(&dev->base.vk.alloc, alloc, payload->syncs); + break; +@@ -49,6 +50,24 @@ vn_sync_payload_release(struct vn_device *dev, + vn_sync_payload_release_internal(dev, payload, NULL); + } + ++/* vtest cannot export a sync_file: wait for the payload on the CPU and ++ * return -1, the sync fd that stands for an already signaled payload. ++ */ ++static int ++vn_sync_export_signaled(struct vn_device *dev, struct vn_renderer_sync *sync) ++{ ++ uint64_t val = 1; ++ const struct vn_renderer_wait wait = { ++ .timeout = UINT64_MAX, ++ .syncs = &sync, ++ .sync_values = &val, ++ .sync_count = 1, ++ }; ++ vn_renderer_wait(dev->renderer, &wait); ++ vn_renderer_sync_reset(dev->renderer, sync); ++ return -1; ++} ++ + /* fence commands */ + + static VkResult +@@ -298,7 +317,10 @@ vn_GetFenceFdKHR(VkDevice device, + assert(sync_file); + + int fd = -1; +- if (payload->type == VN_SYNC_TYPE_SYNC) { ++ if (payload->type == VN_SYNC_TYPE_SYNC && ++ !dev->renderer->info.has_external_sync) { ++ fd = vn_sync_export_signaled(dev, payload->sync); ++ } else if (payload->type == VN_SYNC_TYPE_SYNC) { + fd = vn_renderer_sync_export_syncobj(dev->renderer, + fence->payload->sync, sync_file); + vn_renderer_sync_reset(dev->renderer, fence->payload->sync); +@@ -830,7 +852,11 @@ vn_GetSemaphoreFdKHR(VkDevice device, + assert(sync_file); + + int fd = -1; +- if (payload->type == VN_SYNC_TYPE_SYNC) { ++ if (payload->type == VN_SYNC_TYPE_SYNC && ++ !dev->renderer->info.has_external_sync) { ++ assert(sem->sync_fd_export); ++ fd = vn_sync_export_signaled(dev, payload->sync); ++ } else if (payload->type == VN_SYNC_TYPE_SYNC) { + assert(sem->sync_fd_export); + fd = vn_renderer_sync_export_syncobj(dev->renderer, payload->sync, + sync_file); +diff --git a/src/virtio/vulkan/vn_wsi.c b/src/virtio/vulkan/vn_wsi.c +index e73ab48..f0b41b1 100644 +--- a/src/virtio/vulkan/vn_wsi.c ++++ b/src/virtio/vulkan/vn_wsi.c +@@ -287,6 +287,53 @@ vn_wsi_memory_info_init(struct vn_device_memory *mem, + } + } + ++/* The buffer WSI copies each frame into for the CPU (software presentation) ++ * gets the hidden cached type: on vtest the host-visible type the app sees is ++ * uncached, and reading a frame from it took most of a present. WSI ++ * invalidates the buffer before each read. ++ */ ++uint32_t ++vn_wsi_memory_type_index(const struct vn_device *dev, ++ const VkMemoryAllocateInfo *alloc_info) ++{ ++ const struct vn_physical_device *physical_dev = dev->physical_device; ++ const int readback_type = physical_dev->wsi_readback_memory_type; ++ if (readback_type < 0 || !physical_dev->wsi_device.sw) ++ return alloc_info->memoryTypeIndex; ++ ++ const VkMemoryDedicatedAllocateInfo *dedicated_info = NULL; ++ bool wsi = false; ++ vk_foreach_struct_const(pnext, alloc_info->pNext) { ++ switch ((uint32_t)pnext->sType) { ++ case VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO: ++ dedicated_info = (const void *)pnext; ++ break; ++ case VK_STRUCTURE_TYPE_WSI_MEMORY_ALLOCATE_INFO_MESA: ++ wsi = true; ++ break; ++ case VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO: ++ case VK_STRUCTURE_TYPE_IMPORT_MEMORY_HOST_POINTER_INFO_EXT: ++ return alloc_info->memoryTypeIndex; ++ default: ++ break; ++ } ++ } ++ if (!wsi || !dedicated_info || dedicated_info->buffer == VK_NULL_HANDLE) ++ return alloc_info->memoryTypeIndex; ++ ++ const VkMemoryPropertyFlags flags = ++ physical_dev->memory_properties.memoryTypes[alloc_info->memoryTypeIndex] ++ .propertyFlags; ++ const struct vn_buffer *buf = vn_buffer_from_handle(dedicated_info->buffer); ++ const uint32_t type_bits = ++ buf->requirements.memory.memoryRequirements.memoryTypeBits; ++ if (!(flags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) || ++ !(type_bits & BITFIELD_BIT(readback_type))) ++ return alloc_info->memoryTypeIndex; ++ ++ return readback_type; ++} ++ + static uint32_t + vn_modifier_plane_count(struct vn_physical_device *physical_dev, + VkFormat format, +diff --git a/src/virtio/vulkan/vn_wsi.h b/src/virtio/vulkan/vn_wsi.h +index 2231556..1429feb 100644 +--- a/src/virtio/vulkan/vn_wsi.h ++++ b/src/virtio/vulkan/vn_wsi.h +@@ -34,6 +34,10 @@ void + vn_wsi_memory_info_init(struct vn_device_memory *mem, + const VkMemoryAllocateInfo *alloc_info); + ++uint32_t ++vn_wsi_memory_type_index(const struct vn_device *dev, ++ const VkMemoryAllocateInfo *alloc_info); ++ + bool + vn_wsi_validate_image_format_info( + struct vn_physical_device *physical_dev, +@@ -77,6 +81,13 @@ vn_wsi_memory_info_init(struct vn_device_memory *mem, + { + } + ++static inline uint32_t ++vn_wsi_memory_type_index(const struct vn_device *dev, ++ const VkMemoryAllocateInfo *alloc_info) ++{ ++ return alloc_info->memoryTypeIndex; ++} ++ + static inline bool + vn_wsi_validate_image_format_info(struct vn_physical_device *physical_dev, + const VkPhysicalDeviceImageFormatInfo2 *info) +diff --git a/src/vulkan/wsi/wsi_common.c b/src/vulkan/wsi/wsi_common.c +index 411ee3e..ae38115 100644 +--- a/src/vulkan/wsi/wsi_common.c ++++ b/src/vulkan/wsi/wsi_common.c +@@ -235,6 +235,7 @@ wsi_device_init(struct wsi_device *wsi, + WSI_GET_CB(WaitForFences); + WSI_GET_CB(MapMemory); + WSI_GET_CB(UnmapMemory); ++ WSI_GET_CB(InvalidateMappedMemoryRanges); + if (wsi->has_present_wait) + WSI_GET_CB(WaitSemaphores); + #undef WSI_GET_CB +@@ -3577,6 +3578,22 @@ wsi_create_cpu_linear_image_mem(const struct wsi_swapchain *chain, + return VK_SUCCESS; + } + ++void ++wsi_image_invalidate_cpu_map(const struct wsi_swapchain *chain, ++ const struct wsi_image *image) ++{ ++ /* A driver may back the CPU image with cached memory that is not ++ * coherent; for coherent memory this does nothing. ++ */ ++ const VkMappedMemoryRange range = { ++ .sType = VK_STRUCTURE_TYPE_MAPPED_MEMORY_RANGE, ++ .memory = image->blit.memory != VK_NULL_HANDLE ? image->blit.memory ++ : image->memory, ++ .size = VK_WHOLE_SIZE, ++ }; ++ chain->wsi->InvalidateMappedMemoryRanges(chain->device, 1, &range); ++} ++ + static VkResult + wsi_create_cpu_buffer_image_mem(const struct wsi_swapchain *chain, + const struct wsi_image_info *info, +diff --git a/src/vulkan/wsi/wsi_common.h b/src/vulkan/wsi/wsi_common.h +index 803d140..432ba43 100644 +--- a/src/vulkan/wsi/wsi_common.h ++++ b/src/vulkan/wsi/wsi_common.h +@@ -247,6 +247,7 @@ struct wsi_device { + WSI_CB(WaitForFences); + WSI_CB(MapMemory); + WSI_CB(UnmapMemory); ++ WSI_CB(InvalidateMappedMemoryRanges); + WSI_CB(WaitSemaphores); + #undef WSI_CB + +diff --git a/src/vulkan/wsi/wsi_common_drm.c b/src/vulkan/wsi/wsi_common_drm.c +index dbf65b2..2a6b701 100644 +--- a/src/vulkan/wsi/wsi_common_drm.c ++++ b/src/vulkan/wsi/wsi_common_drm.c +@@ -204,6 +204,10 @@ wsi_signal_dma_buf_from_semaphore(const struct wsi_swapchain *chain, + if (result != VK_SUCCESS) + return result; + ++ /* -1 is an already signaled payload: there is nothing to attach. */ ++ if (sync_file_fd < 0) ++ return VK_SUCCESS; ++ + result = wsi_dma_buf_import_sync_file(image->dma_buf_fd, sync_file_fd); + close(sync_file_fd); + return result; +diff --git a/src/vulkan/wsi/wsi_common_private.h b/src/vulkan/wsi/wsi_common_private.h +index 3d2cd7d..e469414 100644 +--- a/src/vulkan/wsi/wsi_common_private.h ++++ b/src/vulkan/wsi/wsi_common_private.h +@@ -410,6 +410,10 @@ wsi_configure_cpu_image(const struct wsi_swapchain *chain, + const struct wsi_cpu_image_params *params, + struct wsi_image_info *info); + ++void ++wsi_image_invalidate_cpu_map(const struct wsi_swapchain *chain, ++ const struct wsi_image *image); ++ + VkResult + wsi_create_buffer_blit_context(const struct wsi_swapchain *chain, + const struct wsi_image_info *info, +diff --git a/src/vulkan/wsi/wsi_common_wayland.c b/src/vulkan/wsi/wsi_common_wayland.c +index 33a4059..047c8b3 100644 +--- a/src/vulkan/wsi/wsi_common_wayland.c ++++ b/src/vulkan/wsi/wsi_common_wayland.c +@@ -3314,6 +3314,7 @@ wsi_wl_swapchain_queue_present(struct wsi_swapchain *wsi_chain, + + if (chain->buffer_type == WSI_WL_BUFFER_SHM_MEMCPY) { + struct wsi_wl_image *image = &chain->images[image_index]; ++ wsi_image_invalidate_cpu_map(&chain->base, &image->base); + memcpy(image->shm_ptr, image->base.cpu_map, + image->base.row_pitches[0] * chain->extent.height); + } +diff --git a/src/vulkan/wsi/wsi_common_x11.c b/src/vulkan/wsi/wsi_common_x11.c +index a077afe..579ba13 100644 +--- a/src/vulkan/wsi/wsi_common_x11.c ++++ b/src/vulkan/wsi/wsi_common_x11.c +@@ -1854,6 +1854,7 @@ x11_present_to_x11_sw(struct x11_swapchain *chain, uint32_t image_index) + xcb_get_geometry_cookie_t geom_cookie = xcb_get_geometry(chain->conn, chain->window); + + xcb_void_cookie_t cookie; ++ wsi_image_invalidate_cpu_map(&chain->base, &image->base); + void *myptr = image->base.cpu_map; + size_t hdr_len = sizeof(xcb_put_image_request_t); + int stride_b = image->base.row_pitches[0]; diff --git a/tools/venus/virglrenderer-android.patch b/tools/venus/virglrenderer-android.patch new file mode 100644 index 000000000..9cf1a0248 --- /dev/null +++ b/tools/venus/virglrenderer-android.patch @@ -0,0 +1,322 @@ +diff --git a/src/venus/vkr_device_memory.c b/src/venus/vkr_device_memory.c +index ffe621b..01cd811 100644 +--- a/src/venus/vkr_device_memory.c ++++ b/src/venus/vkr_device_memory.c +@@ -227,6 +227,136 @@ vkr_gbm_get_fd_info_from_allocation_info(UNUSED struct vkr_physical_device *phys + + #endif /* ENABLE_GBM_ALLOCATION */ + ++#ifdef __ANDROID__ ++#include ++#include ++#include ++#include ++#include ++ ++static int vkr_dma_heap_fd = -1; ++static int vkr_dma_heap_cached_fd = -1; ++ ++/* Uncached for coherent memory types: nobody brackets CPU access with ++ * DMA_BUF_IOCTL_SYNC (the guest maps the memory as host-coherent), so on a ++ * cached heap the CPU and the GPU see each other's writes torn at cache-line ++ * granularity. Cached for cached types, which are not coherent: whoever maps ++ * them syncs the CPU cache through the dma-buf. ++ */ ++static void ++vkr_dma_heap_init_once(void) ++{ ++ vkr_dma_heap_cached_fd = open("/dev/dma_heap/system", O_RDONLY | O_CLOEXEC); ++ vkr_dma_heap_fd = open("/dev/dma_heap/system-uncached", O_RDONLY | O_CLOEXEC); ++ if (vkr_dma_heap_fd < 0) { ++ vkr_log("no uncached dma-buf heap (%s): using the cached one", strerror(errno)); ++ vkr_dma_heap_fd = vkr_dma_heap_cached_fd; ++ } ++ if (vkr_dma_heap_fd < 0) ++ vkr_log("no dma-buf heap (%s): allocating AHardwareBuffers", strerror(errno)); ++} ++ ++/* An AHardwareBuffer's dma-buf: the first fd of its native handle, which the ++ * NDK hands out only through a unix socket. ++ */ ++static int ++vkr_ahb_alloc_dma_buf(size_t size) ++{ ++ const AHardwareBuffer_Desc desc = { ++ .width = (uint32_t)size, ++ .height = 1, ++ .layers = 1, ++ .format = AHARDWAREBUFFER_FORMAT_BLOB, ++ .usage = AHARDWAREBUFFER_USAGE_CPU_READ_OFTEN | AHARDWAREBUFFER_USAGE_CPU_WRITE_OFTEN | ++ AHARDWAREBUFFER_USAGE_GPU_DATA_BUFFER, ++ }; ++ AHardwareBuffer *ahb = NULL; ++ if (AHardwareBuffer_allocate(&desc, &ahb)) { ++ vkr_log("AHardwareBuffer_allocate of %zu failed", size); ++ return -1; ++ } ++ ++ int fd = -1; ++ int sv[2]; ++ if (!socketpair(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0, sv)) { ++ if (!AHardwareBuffer_sendHandleToUnixSocket(ahb, sv[0])) { ++ char data[4096]; ++ char control[CMSG_SPACE(sizeof(int) * 16)]; ++ struct iovec iov = { .iov_base = data, .iov_len = sizeof(data) }; ++ struct msghdr msg = { ++ .msg_iov = &iov, ++ .msg_iovlen = 1, ++ .msg_control = control, ++ .msg_controllen = sizeof(control), ++ }; ++ if (recvmsg(sv[1], &msg, MSG_CMSG_CLOEXEC) > 0) { ++ struct cmsghdr *cmsg = CMSG_FIRSTHDR(&msg); ++ if (cmsg && cmsg->cmsg_level == SOL_SOCKET && cmsg->cmsg_type == SCM_RIGHTS) { ++ const int count = (cmsg->cmsg_len - CMSG_LEN(0)) / sizeof(int); ++ const int *fds = (const int *)CMSG_DATA(cmsg); ++ for (int i = 0; i < count; i++) { ++ if (i == 0) ++ fd = fds[i]; ++ else ++ close(fds[i]); ++ } ++ } ++ } ++ } ++ close(sv[0]); ++ close(sv[1]); ++ } ++ AHardwareBuffer_release(ahb); ++ if (fd < 0) ++ vkr_log("could not get the dma-buf of an AHardwareBuffer"); ++ return fd; ++} ++ ++static VkResult ++vkr_android_get_fd_info_from_allocation_info(const VkMemoryAllocateInfo *alloc_info, ++ bool cached, ++ int *out_dma_buf_fd, ++ VkImportMemoryFdInfoKHR *out_fd_info) ++{ ++ static once_flag dma_heap_once_flag = ONCE_FLAG_INIT; ++ call_once(&dma_heap_once_flag, vkr_dma_heap_init_once); ++ ++ const size_t size = align(alloc_info->allocationSize, getpagesize()); ++ const int heap_fd = ++ cached && vkr_dma_heap_cached_fd >= 0 ? vkr_dma_heap_cached_fd : vkr_dma_heap_fd; ++ int dma_buf_fd = -1; ++ if (heap_fd >= 0) { ++ struct dma_heap_allocation_data data = { ++ .len = size, ++ .fd_flags = O_RDWR | O_CLOEXEC, ++ }; ++ if (!ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &data)) ++ dma_buf_fd = data.fd; ++ else ++ vkr_log("dma-buf heap allocation of %zu failed (%s)", size, strerror(errno)); ++ } ++ if (dma_buf_fd < 0) ++ dma_buf_fd = vkr_ahb_alloc_dma_buf(size); ++ if (dma_buf_fd < 0) ++ return VK_ERROR_OUT_OF_DEVICE_MEMORY; ++ ++ const int fd = os_dupfd_cloexec(dma_buf_fd); ++ if (fd < 0) { ++ close(dma_buf_fd); ++ return VK_ERROR_OUT_OF_HOST_MEMORY; ++ } ++ ++ *out_dma_buf_fd = dma_buf_fd; ++ *out_fd_info = (VkImportMemoryFdInfoKHR){ ++ .sType = VK_STRUCTURE_TYPE_IMPORT_MEMORY_FD_INFO_KHR, ++ .pNext = alloc_info->pNext, ++ .fd = fd, ++ .handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT, ++ }; ++ return VK_SUCCESS; ++} ++#endif /* __ANDROID__ */ ++ + static void + vkr_dispatch_vkAllocateMemory(struct vn_dispatch_context *dispatch, + struct vn_command_vkAllocateMemory *args) +@@ -281,7 +411,14 @@ vkr_dispatch_vkAllocateMemory(struct vn_dispatch_context *dispatch, + int udmabuf_fd = -1; + void *gbm_bo = NULL; + VkExportMemoryAllocateInfo local_export_info; +- if ((property_flags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) && !res_info) { ++ /* Android without driver export: a dma-buf export of any memory type is ++ * served by a dma-buf the server allocates and imports. ++ */ ++ const bool android_export = ++ physical_dev->android_dma_buf_import && export_info && ++ (export_info->handleTypes & VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT); ++ if (((property_flags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) || android_export) && ++ !res_info) { + /* An implementation can support dma_buf import along with opaque fd export/import. + * If the client driver is using external memory and requesting dma_buf, without + * dma_buf fd export support, we must use gbm bo import path instead of forcing +@@ -346,6 +483,13 @@ vkr_dispatch_vkAllocateMemory(struct vn_dispatch_context *dispatch, + */ + args->ret = vkr_udmabuf_get_fd_info_from_allocation_info( + physical_dev, alloc_info, &udmabuf_fd, &local_import_info); ++#ifdef __ANDROID__ ++ } else if (physical_dev->android_dma_buf_import) { ++ /* Kept in udmabuf_fd: exported and closed the same way. */ ++ args->ret = vkr_android_get_fd_info_from_allocation_info( ++ alloc_info, property_flags & VK_MEMORY_PROPERTY_HOST_CACHED_BIT, ++ &udmabuf_fd, &local_import_info); ++#endif + } else { + args->ret = vkr_gbm_get_fd_info_from_allocation_info( + physical_dev, alloc_info, &gbm_bo, &local_import_info); +@@ -367,6 +511,12 @@ vkr_dispatch_vkAllocateMemory(struct vn_dispatch_context *dispatch, + + struct vkr_device_memory *mem = vkr_device_memory_create_and_add(ctx, args); + if (!mem) { ++ vkr_log("vkAllocateMemory failed (%d): type %u flags 0x%x size %" PRIu64 "%s", ++ args->ret, mem_type_index, property_flags, ++ (uint64_t)alloc_info->allocationSize, ++ udmabuf_fd >= 0 ? ", imported dma-buf" : ""); ++ if (udmabuf_fd >= 0) ++ close(udmabuf_fd); + if (local_import_info.fd >= 0) + close(local_import_info.fd); + if (gbm_bo) +diff --git a/src/venus/vkr_physical_device.c b/src/venus/vkr_physical_device.c +index e2c3baa..69ec875 100644 +--- a/src/venus/vkr_physical_device.c ++++ b/src/venus/vkr_physical_device.c +@@ -207,31 +207,54 @@ vkr_physical_device_init_memory_properties(struct vkr_physical_device *physical_ + .sType = VK_STRUCTURE_TYPE_EXTERNAL_BUFFER_PROPERTIES, + }; + ++ /* Exportable and compatible with itself is enough. Requiring the type in ++ * exportFromImportedHandleTypes as well rules out drivers (Mali on Android) ++ * that export fine but do not re-export imported memory. ++ */ + if (physical_dev->EXT_external_memory_dma_buf) { + info.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT, + vk->GetPhysicalDeviceExternalBufferProperties(handle, &info, &props); ++ const VkExternalMemoryProperties *p = &props.externalMemoryProperties; ++ vkr_log("dma_buf buffer: features 0x%x exportFrom 0x%x compatible 0x%x", ++ p->externalMemoryFeatures, p->exportFromImportedHandleTypes, ++ p->compatibleHandleTypes); + physical_dev->is_dma_buf_fd_export_supported = +- (props.externalMemoryProperties.externalMemoryFeatures & +- VK_EXTERNAL_MEMORY_FEATURE_EXPORTABLE_BIT) && +- (props.externalMemoryProperties.exportFromImportedHandleTypes & ++ (p->externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_EXPORTABLE_BIT) && ++ ((p->exportFromImportedHandleTypes | p->compatibleHandleTypes) & + VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT); + } + + if (physical_dev->KHR_external_memory_fd) { + info.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT, + vk->GetPhysicalDeviceExternalBufferProperties(handle, &info, &props); ++ const VkExternalMemoryProperties *p = &props.externalMemoryProperties; ++ vkr_log("opaque_fd buffer: features 0x%x exportFrom 0x%x compatible 0x%x", ++ p->externalMemoryFeatures, p->exportFromImportedHandleTypes, ++ p->compatibleHandleTypes); + physical_dev->is_opaque_fd_export_supported = +- (props.externalMemoryProperties.externalMemoryFeatures & +- VK_EXTERNAL_MEMORY_FEATURE_EXPORTABLE_BIT) && +- (props.externalMemoryProperties.exportFromImportedHandleTypes & ++ (p->externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_EXPORTABLE_BIT) && ++ ((p->exportFromImportedHandleTypes | p->compatibleHandleTypes) & + VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT); + } ++ vkr_log("memory export: dma_buf %d opaque_fd %d", ++ physical_dev->is_dma_buf_fd_export_supported, ++ physical_dev->is_opaque_fd_export_supported); + + /* fallback to gbm allocation with dma-buf import */ ++ physical_dev->android_dma_buf_import = false; ++#ifdef __ANDROID__ ++ if (!physical_dev->is_dma_buf_fd_export_supported && ++ !physical_dev->is_opaque_fd_export_supported && ++ physical_dev->EXT_external_memory_dma_buf) { ++ physical_dev->android_dma_buf_import = true; ++ vkr_log("memory export: none; allocating dma-bufs and importing them"); ++ } ++#else + if (!physical_dev->is_dma_buf_fd_export_supported && + !physical_dev->is_opaque_fd_export_supported && + physical_dev->EXT_external_memory_dma_buf) + physical_dev->gbm_device = vkr_physical_device_get_gbm_device(); ++#endif + + physical_dev->udmabuf_dev_fd = -1; + if (VKR_DEBUG(UDMABUF)) { +@@ -703,6 +726,25 @@ vkr_dispatch_vkGetPhysicalDeviceFormatProperties2( + args->pFormatProperties); + } + ++/* Android without driver export: the server allocates a dma-buf and imports ++ * it (vkr_device_memory.c), so a dma-buf the driver can import, the guest ++ * can export. ++ */ ++static void ++vkr_android_dma_buf_export_fixup(const struct vkr_physical_device *physical_dev, ++ VkExternalMemoryHandleTypeFlagBits handle_type, ++ VkExternalMemoryProperties *props) ++{ ++ if (!physical_dev->android_dma_buf_import || ++ handle_type != VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT || ++ !(props->externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_IMPORTABLE_BIT)) ++ return; ++ ++ props->externalMemoryFeatures |= VK_EXTERNAL_MEMORY_FEATURE_EXPORTABLE_BIT; ++ props->exportFromImportedHandleTypes |= VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT; ++ props->compatibleHandleTypes |= VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT; ++} ++ + static void + vkr_dispatch_vkGetPhysicalDeviceImageFormatProperties2( + UNUSED struct vn_dispatch_context *dispatch, +@@ -715,6 +757,20 @@ vkr_dispatch_vkGetPhysicalDeviceImageFormatProperties2( + vn_replace_vkGetPhysicalDeviceImageFormatProperties2_args_handle(args); + args->ret = vk->GetPhysicalDeviceImageFormatProperties2( + args->physicalDevice, args->pImageFormatInfo, args->pImageFormatProperties); ++ ++ const VkPhysicalDeviceExternalImageFormatInfo *external_info = vkr_find_struct( ++ args->pImageFormatInfo->pNext, ++ VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_EXTERNAL_IMAGE_FORMAT_INFO); ++ VkExternalImageFormatProperties *external_props = vkr_find_struct( ++ args->pImageFormatProperties->pNext, VK_STRUCTURE_TYPE_EXTERNAL_IMAGE_FORMAT_PROPERTIES); ++ if (physical_dev->android_dma_buf_import && external_info && external_props) { ++ if (args->ret == VK_SUCCESS) ++ vkr_android_dma_buf_export_fixup(physical_dev, external_info->handleType, ++ &external_props->externalMemoryProperties); ++ else ++ vkr_log("image format %d tiling %d: no external memory (%d)", ++ args->pImageFormatInfo->format, args->pImageFormatInfo->tiling, args->ret); ++ } + } + + static void +@@ -743,6 +799,10 @@ vkr_dispatch_vkGetPhysicalDeviceExternalBufferProperties( + vn_replace_vkGetPhysicalDeviceExternalBufferProperties_args_handle(args); + vk->GetPhysicalDeviceExternalBufferProperties( + args->physicalDevice, args->pExternalBufferInfo, args->pExternalBufferProperties); ++ ++ vkr_android_dma_buf_export_fixup( ++ physical_dev, args->pExternalBufferInfo->handleType, ++ &args->pExternalBufferProperties->externalMemoryProperties); + } + + static void +diff --git a/src/venus/vkr_physical_device.h b/src/venus/vkr_physical_device.h +index b0b6691..28ba0b7 100644 +--- a/src/venus/vkr_physical_device.h ++++ b/src/venus/vkr_physical_device.h +@@ -33,6 +33,10 @@ struct vkr_physical_device { + bool is_opaque_fd_export_supported; + void *gbm_device; + int udmabuf_dev_fd; ++ /* Android: host-visible memory is a dma-buf the server allocates (dma-buf ++ * heap, else AHardwareBuffer) and imports, since the driver cannot export. ++ */ ++ bool android_dma_buf_import; + + VkQueueFamilyProperties *queue_family_properties; + uint32_t queue_family_property_count; diff --git a/tools/venus/watch-maps.sh b/tools/venus/watch-maps.sh new file mode 100644 index 000000000..2850147b1 --- /dev/null +++ b/tools/venus/watch-maps.sh @@ -0,0 +1,19 @@ +#!/system/bin/sh +# Keeps the memory map of a running game process in the app's cache directory, so a crash address +# in Wine's log (handle_syscall_fault pc=...) can be matched to a library afterwards. +# On the phone, as the app (from Git Bash, with MSYS_NO_PATHCONV=1): +# adb push tools/venus/watch-maps.sh /data/local/tmp/ +# adb shell "run-as com.droiddeck.launcher sh /data/local/tmp/watch-maps.sh 'Hollow Knight Silksong.exe'" +# Runs 15 minutes. Then: adb exec-out "run-as com.droiddeck.launcher cat cache/game-maps.txt" > maps.txt +name=${1:?process name, as ps shows it} +cd /data/user/0/com.droiddeck.launcher || exit 1 +rm -f cache/game-maps.txt +i=0 +while [ $i -lt 900 ]; do + p=$(ps -A -o PID,NAME | grep -F "$name" | grep -v Crash | head -1 | awk '{print $1}') + if [ -n "$p" ]; then + cat /proc/$p/maps > cache/game-maps.tmp 2>/dev/null && mv cache/game-maps.tmp cache/game-maps.txt + fi + sleep 1 + i=$((i + 1)) +done