mirror of
https://github.com/Droid-Deck/DroidDeck.git
synced 2026-11-21 13:55:58 +00:00
112 lines
6.6 KiB
Markdown
112 lines
6.6 KiB
Markdown
# Quest OpenXR implementation
|
|
|
|
Branch: `feat/quest-openxr`. The goal is standalone immersive SteamVR on Quest,
|
|
using DroidDeck's Linux guest and an Android OpenXR frontend.
|
|
|
|
## Milestone 1: native headset frontend
|
|
|
|
`questxr` builds a separate `com.droiddeck.questxr` companion so the frontend can
|
|
be validated without replacing the installed DroidDeck runtime. It uses the
|
|
Khronos Android loader, GLES swapchains, predicted stereo views and head poses,
|
|
Touch grip/trigger actions, and trigger-edge haptic feedback. The test scene has
|
|
world-fixed geometry and tracked controller cubes. Controller availability is
|
|
optional for the diagnostic scene; sleeping controllers remain inactive.
|
|
|
|
Build with `:questxr:assembleDebug`. `tools/deploy_quest_xr.ps1` builds, aligns,
|
|
signs with an external key directory, verifies the expected certificate and
|
|
installs the companion. Private keys must remain outside the checkout.
|
|
|
|
Run `python tools/test_quest_xr.py --adb <adb executable>`. The runner restarts
|
|
the companion, simulates wearing the headset using Meta's documented
|
|
`com.oculus.vrpowermanager.prox_close` broadcast, checks fresh advancing frame
|
|
telemetry and valid tracked head pose, and stores local JSON/logcat results in
|
|
the ignored `questxr-validation` directory. Use `--require-controllers` when
|
|
both Touch controllers are awake. That option must pass before claiming tracked
|
|
controller validation. Trigger haptics require an actual trigger press.
|
|
|
|
## Device evidence, 2026-10-14
|
|
|
|
Quest 3, Android 14, DroidDeck 0.4.0. Local Windows build with NDK 27.2.12479018
|
|
passed. The deployed companion certificate matches `keystore/release-signer.sha256`.
|
|
The automated 15-second test passed with 792 rendered stereo frames at the last
|
|
sample, a 13,888,888 ns predicted display period (72 Hz), and head location flags
|
|
15 (position/orientation valid and tracked). A focused state was observed before
|
|
an OS panel took input focus; stereo frame submission continued in VISIBLE.
|
|
Neither controller was active. Controller motion and physical haptics have not
|
|
been validated. This establishes native OpenXR rendering, not SteamVR gameplay.
|
|
|
|
Quest reported `XR_KHR_android_surface_swapchain` support. Next, validate an
|
|
Android-produced surface swapchain as a projection source, then connect the
|
|
existing Android Wayland compositor to it. This may avoid transferring Linux
|
|
dmabufs directly into the runtime's graphics device. Surface support alone does
|
|
not establish that projection, synchronization, or SteamVR rendering works.
|
|
|
|
The first launch left a controller-required OS dialog from an earlier manifest.
|
|
After declaring optional controller/hand capability and restarting VR Shell,
|
|
the app launched. Automated tests should report launch blockers rather than
|
|
treating a successful `am start` as successful rendering.
|
|
|
|
## Milestone 2: Android projection surface and guest integration
|
|
|
|
The Android Surface producer test passed repeatedly, including 1,079 submitted
|
|
stereo frames in a 15-second test at 72 Hz. The new stage-space test reports
|
|
tracked floor-relative position, and the runtime supports monotonic-time
|
|
conversion. A JNI abort during the first surface test was traced to treating
|
|
the runtime's global Surface reference as a local reference and corrected.
|
|
|
|
The same native frontend is now compiled into the main app behind the explicit
|
|
`QuestXrActivity` entrypoint. With `guest_surface=true`, it hands its Android
|
|
projection Surface to the existing compositor. Surface transitions wait for a
|
|
compositor acknowledgement after the old Vulkan swapchain's GPU work drains.
|
|
Successful guest GPU presentations are counted separately from XR submissions.
|
|
The main app's Kotlin and native targets compile locally; an on-device main-app
|
|
guest test is still required before considering this bridge functional.
|
|
|
|
The frontend publishes a versioned 296-byte seqlock tracking record under the
|
|
session runtime directory. It contains floor-relative head/controller poses,
|
|
per-eye view geometry, predicted display time converted to CLOCK_MONOTONIC,
|
|
display period, and Touch action values. The OpenVR driver consumes fresh
|
|
records for real head/controller poses and asymmetric FOV; it marks stale
|
|
tracking invalid. The modified ARM64 Linux driver links successfully in WSL.
|
|
Driver timing, eye-to-head transforms, input completeness, render-pose/image
|
|
association and return haptics still need implementation/validation.
|
|
|
|
The companion's release build now targets Android 14 and passes release lint.
|
|
|
|
## Milestone 3: input, timing and return haptics
|
|
|
|
The driver now supplies eye-to-head transforms and runtime-derived IPD, consumes
|
|
Touch trigger/squeeze/stick/button state, reports predicted pose timestamps and
|
|
paces vsync against converted OpenXR display timing. Vive-profile compatibility
|
|
maps primary buttons to trackpad click and secondary/menu buttons to application
|
|
menu. A versioned 64-byte feedback channel carries bounded driver haptic events
|
|
back to OpenXR. Repeated surface tests still submit 1,079 stereo frames at 72 Hz.
|
|
The native and Kotlin targets compile, release lint passes, and the ARM64 Linux
|
|
driver links with no undefined symbols. IPC tests cover valid packets, odd
|
|
seqlocks, version mismatch, stale/future timestamps, inactive state, excessive
|
|
dimensions and truncated files; they run in CI as well.
|
|
|
|
The first full integration CI APK compiled successfully but failed the ELF
|
|
closure check because its system-library list omitted `libGLESv3.so`. The list
|
|
now recognizes that Android system dependency. A subsequent full CI run is
|
|
required before deployment of the integrated main-app build.
|
|
|
|
On-device prerequisites: the installed app's Debugging tools toggle explicitly
|
|
rejects agent UI activation (`PROTECTED_CONTROL`); it must be switched on-device.
|
|
Steam is signed out and both Touch controllers remain inactive. These block
|
|
guest command diagnostics and actual game/controller/haptic validation, but do
|
|
not block native rendering or independent engineering. No guard was bypassed.
|
|
|
|
## Remaining stages
|
|
|
|
- Android/Linux eye-image bridge and frame ownership/synchronization.
|
|
- OpenVR driver IPC for predicted 6DoF head, per-eye views and tracked hands.
|
|
- Real Touch input mapping and haptics returning through the bridge.
|
|
- Runtime-paced driver timing and render-pose association.
|
|
- Guest SteamVR dashboard and owned-game validation on Quest.
|
|
- Sustained timing, memory and thermal measurements; existing semaphore
|
|
workarounds must be assessed under the real graphics chain.
|
|
|
|
References: [Meta scriptable testing](https://developers.meta.com/vr/documentation/native/android/ts-scriptable-testing/),
|
|
[Khronos Android surface swapchain](https://registry.khronos.org/OpenXR/specs/1.1/man/html/xrCreateSwapchainAndroidSurfaceKHR.html).
|