mirror of
https://github.com/Droid-Deck/DroidDeck.git
synced 2026-11-18 12:28:40 +00:00
204 lines
15 KiB
Markdown
204 lines
15 KiB
Markdown
# Cardboard VR and SteamVR on Turnip: what we learned
|
|
|
|
Written 2026-10-13, after the first stretch of Cardboard mode work on the AYN Thor (Snapdragon 8 Gen 2,
|
|
Adreno 740, Turnip on KGSL). `docs/cardboard-vr.md` describes how the feature works. This page records
|
|
what it took to get there: the traps, why each fix looks the way it does, how to debug a live session,
|
|
and what is still open. Read it before the next round of VR work.
|
|
|
|
## Where it stands
|
|
|
|
| What | Result |
|
|
| --- | --- |
|
|
| Cardboard mode (lens pass, 3DoF head tracking, recentre) | Works on any picture; picture follows SteamVR on and off |
|
|
| SteamVR 2.17.10 ARM64 (the Steam Frame's `deckard-steamvr`) | Starts from the drawer; driver-direct at 60 Hz, dashboard and Steam library in VR |
|
|
| Thumper (356400, D3D11, VR launch option) | Menus and levels at 60 Hz, 7 ms CPU / 13 ms GPU, 23 of 6485 frames dropped |
|
|
| The Thrill of the Fight (494150, Unity) | Gym loads, 60 Hz, 6.5 ms CPU / 8.5 ms GPU |
|
|
| The Lab (450390) | Loads, then Android's low-memory killer takes the whole app |
|
|
| SUPERHOT VR (617830) | Reaches its EULA in the dashboard (not accepted in testing) |
|
|
| SteamVR Home | Blocked: Proton's `vrclient_x64` lacks `IVROverlayView_002` |
|
|
| `helloxr -g Vulkan2` (OpenXR) | Session focused and submitting, picture shows as torn tiles |
|
|
|
|
Screenshots from the Thor: [Thumper gameplay](evidence/cardboard-thumper-vr-gameplay-2026-10-13.png),
|
|
[Thumper title](evidence/cardboard-thumper-vr-cardboard-2026-10-13.png),
|
|
[The Thrill of the Fight](evidence/cardboard-thrill-of-the-fight-cardboard-2026-10-13.png),
|
|
[the SteamVR dashboard with a game's EULA](evidence/cardboard-steamvr-dashboard-2026-10-13.png).
|
|
|
|
Games run as x86-64 Windows builds through Proton ARM64 (FEX, DXVK). Proton's `vrclient_x64` forwards
|
|
OpenVR to SteamVR's ARM64 `vrclient.so`, so no x86 SteamVR is involved.
|
|
|
|
## Turnip on KGSL: the four things SteamVR trips over
|
|
|
|
1. **`VK_KHR_display` makes the GPU disappear.** Turnip on KGSL enumerates no physical device when an
|
|
instance enables it ("I can't KHR_display"). `vrcompositor` always asks for it. The layer takes the
|
|
display extensions off the instance and fakes one display, mode and plane backed by a headless
|
|
surface. Headless surfaces report a `UINT32_MAX` extent, so the swapchain is sized from the mode.
|
|
2. **No exportable semaphores.** SteamVR shares a timeline semaphore with every shared image as an
|
|
opaque fd, and Turnip/KGSL cannot export one. The layer creates the semaphore unshared and hands out
|
|
a memfd (`droiddeck-shared-sem`) holding a 64-bit counter. A poller thread in each process carries
|
|
values both ways every 300 µs. Dropping the semaphores outright, the first attempt, gave torn
|
|
dashboard tiles.
|
|
3. **Wait-before-signal blocks the submitting thread.** Turnip does not defer a submit that waits on
|
|
a timeline value nobody has signalled yet; `vkQueueSubmit` sleeps until it is. SteamVR's client
|
|
waits on a compositor value that the compositor signals only after the client has submitted and
|
|
said so, so every client deadlocked before its first frame:
|
|
- `helloxr` blocked in a wait-idle in `vrclient.so`;
|
|
- DXVK's `dxvk-queue` thread blocked in Proton games.
|
|
|
|
The layer now gives a submit's wait on an imported shared value 3 ms, then drops it, at once when
|
|
the shared counter has not moved for 100 ms. It is kept when the value is already reached, when
|
|
this process has queued the signal itself, or while a Quest session's frame socket exists (checked
|
|
at most once a second). Three details mattered:
|
|
- the waits are judged before the submit's own signals count, because an acquire waits on one
|
|
value and signals the next on the same semaphore;
|
|
- SteamVR's client interop device is Vulkan 1.0, so only the `KHR` entry points resolve there;
|
|
- a Proton game waits on two values a frame that are never signalled (5,500 dropped, none reached
|
|
within 200 ms afterwards). With the grace on each, Angry Birds VR's submit thread lost 6 ms a
|
|
frame and the game ran at 60 fps; without it, 120.
|
|
4. **UBWC layouts must agree.** An imported image has to be created with the same
|
|
`VkImageFormatListCreateInfo` (sRGB + UNORM) as its creator, or Turnip lays out the compression
|
|
differently and the reader sees garbage. The presenter's eye-texture import does this.
|
|
|
|
## Other traps, in the order they cost time
|
|
|
|
- **A still headset goes to standby.** A phone on a table looks unworn, SteamVR enters standby after
|
|
about 15 s, and the compositor stops submitting. The driver's proximity sensor reads "worn" while
|
|
Cardboard is on.
|
|
- **Emulated SysV semaphores never set `sem_otime`.** SteamVR's web helper waits for it to become
|
|
non-zero before using a semaphore set, so the dashboard never appeared. `sysv.c` now stamps
|
|
`sem_otime` and `sem_ctime`.
|
|
- **Steam's web helper needs the layer too.** It shares Steam's in-VR overlays with SteamVR and crashed
|
|
without the layer; after a few crashes Steam quits. Its GPU process outlives SteamVR runs, so
|
|
`steamwebhelper` gets the semaphore bridge whenever SteamVR is installed, not only while SteamVR's
|
|
`active` marker exists. It never gets the display stand-in.
|
|
- **Keep the layer out of ordinary sessions.** As an implicit layer it loads everywhere. Display
|
|
extensions are only taken off while SteamVR runs, a new session clears a leftover `active` marker,
|
|
and without SteamVR installed every process gets the driver's functions unchanged. Review feedback
|
|
on the first version pointed out that gamescope asking for a display extension would have been
|
|
rewritten in an ordinary session.
|
|
- **The `active` marker race.** Clearing a stale marker whenever `vrserver` is not running also
|
|
cleared it during startup, before `vrserver` appeared, because the app polls status every 3 s. Steam
|
|
quit as a result. Only a marker older than 2 minutes is cleared now.
|
|
- **Signal handlers in Proton games.** The layer's crash tracer replaced the `SIGSEGV` handler that FEX
|
|
and Wine rely on, and The Lab crashed in FEX's JIT. It now installs only where nothing handles the
|
|
signal.
|
|
- **mangoapp owns gamescope's single external overlay slot**, so an X11 presenter has to use
|
|
`STEAM_OVERLAY`. The default presenter is now a Wayland window on DroidDeck's own compositor, kept
|
|
on top by app id.
|
|
- **Memory.** Steam, SteamVR and one VR game fit in the Thor's 12 GB. A large game such as The Lab, or
|
|
two games at once, push the device into critical pressure, and the low-memory killer takes
|
|
DroidDeck itself (`ended-without-teardown.txt` in the session logs, `lowmemorykiller` in logcat).
|
|
- **Adreno 840 forced sysmem.** On the Fold with VITURE Luma Ultra, launching SteamVR from the library
|
|
inherited `TU_DEBUG=sysmem` and produced `vkerror=-4`, triggering recovery. Adding `noubwc` still
|
|
failed; replacing the flags with `noubwc` or an empty value worked. The shared launch helper removes
|
|
only `sysmem` on `Adreno840*`, leaving other flags, the parent Steam session and other GPUs unchanged.
|
|
The user confirmed improved landing-area brightness and colors with this configuration and asked
|
|
to preserve them.
|
|
- **Adreno 840 with DD-Turnip v0.2.0.** SteamVR reproduced `vkerror=-4` with `sysmem` removed,
|
|
including on the previous app build and with the presenter disabled. After a phone reboot,
|
|
the unchanged `fcf8c588` baseline rendered for 109 seconds without observed device loss.
|
|
`TU_DEBUG=flushall` initially appeared promising: a global run rendered for over a minute,
|
|
and a SteamVR-only run rendered for about 32 seconds before background suspension. The
|
|
packaged SteamVR-only change then reproduced device loss with `flushall` confirmed in
|
|
`vrcompositor`, so the automatic flag was removed. These bounded runs do not establish the
|
|
underlying cause or a reliable workaround.
|
|
- **Live files are not cache.** Android's `installd` evicted `cache/shm/u10528-*` while SteamVR was
|
|
running, followed by Chromium shared-memory failures and lost IPC. Shared memory, emulated device
|
|
files, sandbox trees and PRoot temporary files now use `no_backup/guest-runtime`, retaining their
|
|
existing cleanup lifetimes. Session state must survive cache eviction.
|
|
- **One host signal owner.** The submit-wait path must not mirror a shared timeline value itself:
|
|
the poller can signal it between the submit thread's counter read and signal, producing an invalid
|
|
duplicate signal. The poller now owns mirroring; a forced-interleaving test covers this race.
|
|
- **Lost hand tracking is not a disconnect.** Publishing a disconnected pose and then a connected,
|
|
untracked pose on every optical update floods SteamVR with device-change notifications. With
|
|
uncalibrated hands this grew its web buffer to about 34 MB and triggered the Web thread watchdog.
|
|
Release inputs and publish only the connected, untracked pose; reserve disconnection for an
|
|
actual controller removal. The controller regression exercises repeated loss and reacquisition.
|
|
- **Failed realtime scheduling can still install a CPU limit.** On the Fold, SteamVR's SDL RTKit
|
|
fallback set `RLIMIT_RTTIME` to 100/200 ms despite realtime priority being refused. The render
|
|
thread later received kernel `SIGXCPU` during `IOCTL_KGSL_GPU_COMMAND`, with policy `SCHED_OTHER`
|
|
and the soft limit advanced to 1100 ms, confirming the realtime watchdog fired. The VR scheduling
|
|
preload rejects lowering this limit only in `vrcompositor` when `RLIMIT_RTPRIO` is zero and
|
|
`CAP_SYS_NICE` is absent. It preserves inherited limits, other processes and other resources;
|
|
it does not intercept signals. SDL's scheduling hint cannot cover SteamVR's direct
|
|
`SDL_LinuxSetThreadPriorityAndPolicy` call. A native preload regression covers rejection and
|
|
forwarding. The exact Android kernel boost/accounting path was not established.
|
|
- **Frames shown out of order.** Video from the glasses jumped back a few frames in bursts while the
|
|
head turned. Two causes, each found by stamping a frame counter into the presenter's output:
|
|
- the presenter copied SteamVR's eye texture a fixed 4 ms after `Present`, with no fence. Under
|
|
GPU load `vrcompositor` had not drawn it yet, so the copy got that texture's previous frame. The
|
|
layer now fences `vrcompositor`'s submits and the presenter waits for the frame's count;
|
|
- DroidDeck's compositor copies a Wayland client's dma-buf without waiting for its implicit fence,
|
|
and KGSL does not order the copy after the client's GPU work. The presenter's blit sat behind the
|
|
game's frame, so the compositor showed the swapchain image's older picture. The presenter now
|
|
waits for its blit before it presents.
|
|
- **Flat games under SteamVR.** With SteamVR running, Steam lists a flat game after its own window in
|
|
gamescope's base layer, so Big Picture keeps the focus and the game never shows, theatre setting or
|
|
not. SteamVR is therefore started only by Play SteamVR, never with the glasses alone. A flat game
|
|
launched while SteamVR runs now opens as a SteamVR panel instead (`droiddeck-steamvr-flat`). Games
|
|
launched from the SteamVR dashboard do not pass through a `SteamClient.Apps.RunGame` hook installed
|
|
later, so the launcher asks Steam at run time rather than marking the click.
|
|
- **A new projection needs an event.** `SetDisplayProjectionRaw` changed what `vrserver` reports, but
|
|
`vrcompositor` kept the frustum it started with, so the glasses' field of view did nothing until
|
|
SteamVR restarted. The driver now follows it with `VREvent_LensDistortionChanged`.
|
|
- **Head-tracking axes.** With the display at `ROTATION_90`, screen x is minus device y and screen y is
|
|
device x. Getting this wrong made a turn at the hips tilt the view. Recentring now keeps the world
|
|
vertical and resets the heading only.
|
|
|
|
## Library launch validation (2026-10-15)
|
|
|
|
Two hands-off launches through the Steam library's Launch button completed 308 seconds and
|
|
188 seconds, with an intentional stop between runs. Each retained its compositor process and
|
|
session, sustained 60 fps, showed dashboard/world transitions, and kept USB control and valid
|
|
6DoF head tracking. Neither run reported device loss, a watchdog abort, recovery, or a session
|
|
restart. Hand controllers remained enabled but uncalibrated. Brightness stayed at 8; no shader
|
|
color changes or temporary graphics overrides were used.
|
|
|
|
These timed runs used APK SHA-256
|
|
`4ce7a623ceb6b1a100c0ed1184530ece2ea8e13a1cb9ee6c0ab24666a8249004`. A newer calibration/recenter
|
|
APK was subsequently installed and verified; see `luma-spatial-evidence.md`. The timed results
|
|
do not establish sustained gameplay or calibrated-hand acceptance for that newer build.
|
|
|
|
## Debugging a live session
|
|
|
|
- **Logs:** `~/.local/share/Steam/logs/{vrserver,vrcompositor,xrclient_*}.txt` and
|
|
`/run/droiddeck/droiddeck-steamvr/vrstartup.log`. Lines from the driver start with `droiddeck:`,
|
|
lines from the layer with `[droiddeck-vr-layer]`. A game's layer messages land in the session log.
|
|
- **Markers beside `active`** in `/run/droiddeck/droiddeck-steamvr/`:
|
|
- `crashtrace` prints backtraces on fatal signals;
|
|
- `semtrace` logs each bridged semaphore's local and shared value every 2 s, named by the memfd
|
|
inode so the same semaphore can be matched across processes, plus the first waits, signals and
|
|
shared image create infos.
|
|
|
|
Both are read when a process starts.
|
|
- **Frame stats:** in `/opt/steamvr/bin/linuxarm64`, `LD_LIBRARY_PATH=. ./vrcmd --stats` reports
|
|
submits, drops and app CPU/GPU times. `./vrcmd --hidedashboard` closes the menu SteamVR opens over a
|
|
new game.
|
|
- **Where a process is stuck:** a stack scan of `/proc/<tid>/mem` from the saved stack pointer (a small
|
|
Python helper) gives module+offset frames. That is how the wait-idle in `vrclient.so` and the
|
|
blocked `dxvk-queue` thread were found.
|
|
- **VR launch options:** `SteamClient.Apps.GetLaunchOptionsForApp(id)` lists them, and
|
|
`SteamClient.Apps.RunGame("<id>", "", <index>, 100)` starts one, through `droiddeckctl cdp eval`.
|
|
- **Quitting:** `droiddeckctl quit <appid>`. Without the id the game keeps running.
|
|
- **Pitfalls when testing:**
|
|
- never `pkill -f` or `pgrep -f` a pattern inside a guest command, because it matches the
|
|
command's own shell; use `pgrep -x`;
|
|
- replace a loaded `.so` by copying to a temporary name and `mv`, never `cp` over it;
|
|
- a session start restages the driver and layer from the APK assets, so a hand-copied build is
|
|
lost on the next start.
|
|
|
|
## Open items
|
|
|
|
- **`helloxr` picture.** Both sides create the swapchain images with identical parameters, so this is
|
|
not the UBWC import issue. Reading an image mid-render (the dropped wait) or binning tiles is the
|
|
next suspicion.
|
|
- **SteamVR Home** needs `IVROverlayView_002` in Proton's `vrclient`, or an ARM64 build of Home.
|
|
- **The Lab and other large games** need memory headroom. Candidates: stop Steam's own web helpers
|
|
while a VR game runs, or lower SteamVR's footprint.
|
|
- **Wait drops are a heuristic.** A 3 ms grace keeps the common case in sync, and an idle counter
|
|
skips it. A submit thread in the layer, deferring submits until their waits are reachable, would be
|
|
exact.
|
|
- **Layers:** only the first layer of each frame is presented. No reprojection.
|
|
- **Controllers** are a gaze pointer driven by the pad. Pad presses did not close the SteamVR menu in
|
|
testing. Untested: whether that came from gamescope focus being on Steam, or from the system button
|
|
mapping.
|