mirror of
https://github.com/Droid-Deck/DroidDeck.git
synced 2026-11-21 13:55:58 +00:00
Add VITURE XR controls, head tracking, and SteamVR (#732)
* Add VITURE glasses support: hardware control, 3D, head tracking Glasses get their own controller over the MCU (permission, 120 Hz default, brightness, volume, sunblock, 2D/3D) and the game moves onto them when plugged in. The IMU drives Cardboard head tracking and the SteamVR driver. 3D falls back to one eye's view on phones that can't send the 3840x1080 signal. Options open from a glasses button in game and on the Steam page. * Drop the glasses settings like Steam's: Play-style button, Settings lip, tabs * Hang the glasses Settings lip over the top of the button * Space glasses MCU commands so the IMU rate sticks, fit fixed resolutions to the glasses * Keep Steam alive with SteamVR registered, run SteamVR at the glasses' rate, keep the glasses lip on top * Fit VR games on the glasses: full view, seated origin at the head, pad hands reach with the d-pad * Let the pad hands reach a metre out so buttons with deep colliders press * Show every VR frame in order at 120 fps: wait for SteamVR's GPU work, present only finished copies, skip the grace on idle shared counters * Glasses: SteamVR starts and stops with them, closer view with a field of view setting, Recenter on the pad * Glasses: SteamVR field of view defaults to 90° * VR pad hands: sticks move and aim the hands, left stick click switches the sticks to the game, d-pad pulls a held hand * Glasses: Play SteamVR beside the glasses button instead of starting SteamVR with them, pad hands drawn as Index controllers and off while other VR controllers are connected, glasses sheet tabs sized like Steam's * Track glasses mode, refresh-aware glasses window, align Play SteamVR with Play Steam * Fix 3D on the glasses: full-res SteamVR presenter, revert VR 3D at session end, coalesce vsync ticks, no session restart after end * Address review: glasses output after installs and session end, VR revert retry, display fallback, presenter swapchain retry, agent hex checks, drawer bumpers, glasses translations, docs * Harden VITURE stereo negotiation and output refresh tracking * Localize stereo recovery feedback and record Astra reconnect tests * Improve test failure diagnostics and refresh Astra firmware findings * Handle VITURE docks separately and reject truncated USB frames * Glasses: remember a 3D fallback per model, docks as display only, VR timing fixes - A 3D attempt that ends at the 640x480 fallback is remembered for that glasses model, so Play SteamVR no longer blacks out the glasses after every restart; Retry clears it and "3D for VR games" stays reachable - A VITURE Mobile Dock on its own no longer counts as glasses; through a dock Play SteamVR is hidden and the sheet shows the output and why there is no control - HDR is probed on the glasses when the session starts there - Glasses frames with a mismatched checksum are marked in the status frame log - SteamVR layer and driver vblank counters stay monotonic across refresh changes; one-eye render scale applies from activation and on mono/3D switches - Compositor binds announce the refresh clients were last told; the presenter retries a failed first swapchain - Dead strings and helpers removed; docs updated * Glasses: drop USB frames with a bad checksum * Glasses: count frames dropped for a bad checksum * Feat/viture xr (#907) * Add RayNeo Air and GT support to the XR glasses pipeline * Add RayNeo Air and GT support to the XR glasses pipeline * Checkpoint glasses tracking, calibration, and SteamVR launch integration * Checkpoint glasses tracking, calibration, and SteamVR launch integration * Add XREAL 1S support and correct GT and Pro 2 tracking * Add XREAL 1S support and correct GT and Pro 2 tracking * Fix mirrored glasses tracking and recover SteamVR after device loss * Fix mirrored glasses tracking and recover SteamVR after device loss * Document confirmed XREAL 1S brightness control * Document confirmed XREAL 1S brightness control * Fix SteamVR glasses defaults and rendering controls * Fix SteamVR glasses defaults and rendering controls * chore: keep only the 3 newest session log folders (#852) * chore: add agent comment rules (#748) Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com> * Fix SteamVR polling, VITURE checksums, and fence tracking Keep manual SteamVR launches polled, discard corrupt VITURE packets, and retire Vulkan fences out of order without blocking submissions at a fixed capacity. Preserve the contiguous completion watermark used by the presenter. Fix optional Windows launcher configuration and audited helper staging. Add Android, Windows, and native tracker regression coverage. * Fix SteamVR polling, VITURE checksums, and fence tracking Keep manual SteamVR launches polled, discard corrupt VITURE packets, and retire Vulkan fences out of order without blocking submissions at a fixed capacity. Preserve the contiguous completion watermark used by the presenter. Fix optional Windows launcher configuration and audited helper staging. Add Android, Windows, and native tracker regression coverage. * fix: wireless ADB page names the child-process switch the way Android shows it (#848) * l10n: refresh stale and untranslated strings in es, fr, ko, ru (#850) * l10n: refresh stale and untranslated strings in es, fr, ko, ru - mode_added_art_hint: now mentions Steam then SteamGridDB (all four locales). - wincomp_needs_installer / wincomp_installers_note: es and ko still described the old "coming soon" behaviour; align with the current English text. - summary_remove_game: was still English in all four locales. Machine-assisted translation; native-speaker review is welcome. * l10n(es): say registro, not grabación, for the missing installer record --------- Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com> * fix: a failed cloud save recovery no longer ends the app (#846) The cloud-recovery thread started at every app start had nothing to catch an error, so one went to CrashHandler, which halts the process with status 2. In unit tests the thread can outlive the test that started it, fail on the torn-down preferences, and take the whole test run down. The error is now logged and the thread ends. Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com> * fix: an interrupted Linux runtime download resumes instead of starting again (#847) * fix: an interrupted Linux runtime download resumes instead of starting again installOnce() asked the downloader to resume but always deleted the archive in its finally block, so the next attempt began at zero. The archive is now named after the release version and checksum, a failed download keeps what it fetched, and archives of other releases (and the old fixed name) are removed. A checksum or extraction failure still deletes the file. * test: Downloader resumes a cut-off download from the bytes it kept Starts a socket server that promises the whole file, hangs up part way and then answers a Range request, and checks that the second downloadFile call resumes with the right offset and ends with identical bytes. * fix: a resumed download discards a 206 that does not start where the kept bytes end Downloader appended every 206 response to the partial file without looking at Content-Range. A server or proxy that answers a Range request with a different offset would leave a corrupt file. Check that the range starts at the kept length; otherwise delete the partial file and start again without a Range header. Adds a test with a server that answers the Range request from offset 0. --------- Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com> * l10n: German, French, Spanish, Italian, Portuguese and Russian for the zRAM, reset-settings and store free-space strings (#849) * l10n: German, French, Spanish, Italian, Portuguese and Russian for the zRAM, reset-settings and store free-space strings Adds the strings values-de, values-it and values-pt were missing (14 each) and the six that values-fr, values-es and values-ru were missing. Machine-assisted drafts that follow each file's existing register and terms; native review is welcome. * l10n: correct missing active swap translations --------- Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com> Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com> * fix: droiddeckctl reaches variant-package builds (recover #842) (#854) fix: droiddeckctl reaches builds shipped under a variant package The provider authority and the start Activity's component were fixed to com.droiddeck.launcher, so droiddeckctl could not talk to the antutu, ludashi or pubg builds from tools/release/variants.txt. DROIDDECK_PACKAGE now picks the package; the Activity's class name stays the same in every variant. Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com> * feat: droiddeckctl reports on-screen pacing (recover #843) (#855) feat: droiddeckctl pacing reports the frames that reached the screen Reads SurfaceFlinger's timestats for the game's layer (or the session's SurfaceView) over a capture: frames on screen, fps on screen, the present-to-present histogram and how many intervals were off the usual cadence. Records split by a refresh-rate change are summed. A game's own fps counter counts frames handed over, so it can't show dropped or doubled frames. Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com> * fix: Steam can restart more than twice in a session (#754) * fix: Steam can restart more than twice in a session Every exit 42 counted toward the cap that stops an update which will not install, and the count was never cleared, so the third restart of a session ended it however far apart they were. A client that ran for a minute before asking is not that loop: the count starts again. * fix: measure the client's run time as it exits The time spent collecting logs and checking FEX after the client exits counted toward its run, so a slow cleanup could make a quick exit 42 look like a long one and clear the count. --------- Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com> Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com> * fix: the front end keeps its page when the Fold is folded or unfolded (#832) * fix: the front end keeps its page when the Fold is folded or unfolded MainActivity, FileManagerActivity, FilePickerActivity and StoreLoginActivity did not declare the screen-size configuration changes, so folding or unfolding a foldable recreated them and dropped their in-memory state. They now handle orientation, screenSize, screenLayout, smallestScreenSize and density themselves, like SessionActivity already does. uiMode is left out on purpose. * fix: store login keeps rebuilding its toolbar when the display density changes StoreLoginActivity builds its toolbar in code with dp-to-pixel values taken at creation time and has no onConfigurationChanged. It no longer declares density in configChanges, so a density change recreates it as before this PR. Size and layout changes are still handled in place. --------- Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com> * Release pointer buttons without replaying stale motion (#803) * Fix cursor warping on native touch down * Preserve input cleanup without replaying stale pointer motion --------- Co-authored-by: siyuan.qsy <siyuan.qsy@alibaba-inc.com> * l10n: translate the strings added since #284 into zh-CN, zh-HK, zh-TW and ja (#539) * l10n: translate the 184 strings added since #284 (zh-CN/HK/TW, ja) The stores, cloud saves, Windows components, Steam touch, desktop removal and background-downloads strings landed after #284's translation snapshot, so every one of them fell back to English. Translate all 184 into Simplified Chinese, Traditional Chinese (Taiwan and Hong Kong) and Japanese, keeping each locale file's own vocabulary (工作階段/會話, 解除安裝/卸載, 元件/組件, 設定). Also fix four Hong Kong strings that kept the simplified 着色器. * l10n(zh): fix stale and ambiguous strings found in review Severe: summary_remove_game was untranslated English in all three files; mode_added_art_hint, wincomp_installers_note and wincomp_needs_installer still described the pre-#480/#492 English. Also: distinguish the Setup wizard from Android system settings in zh-HK/zh-TW (初始設定 / 系統設定 / Android 設定), unify library as 收藏庫, component as 組件 in zh-HK, session as 會話 in zh-HK, account as 帳戶 in zh-HK, Trending as 趋势/趨勢, Forget as 清除, reword the re-applied-at-launch, action-failed and already-in-Games strings, and play as 執行/開啟遊戲 in zh-TW/zh-HK. * l10n(ja): fix stale and ambiguous strings found in review --------- Co-authored-by: Dantalian <Dantalian-dev@users.noreply.github.com> Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com> * Rebuild and verify Windows VR runtime components * Rebuild and verify Windows VR runtime components * Add top, bottom, and centered picture positions for Steam sessions (#871) * fix: dependency scan ignores folders above the game folder (#859) * DroidDeck-Proton: get releases from the Droid-Deck org repo DroidDeck-Proton is now built and released on Droid-Deck/DroidDeck-Proton (11-9.1 is Latest there), so the Components page, the update check and the guest installer read releases from the org repo instead of The412Banner's. * feat: low-latency controller and touch toggles in the session drawer (#808) * fix: SteamGridDB lookups no longer stick as "no match" after a failure; add a search to the art chooser (#876) * feat: Steam home wall fills from the signed-in library instead of store picks (#878) * test: cover GPU family and support classification (#880) * Fix local builds on Linux failing after the first Comet build (#866) * feat: sort the store library and installed tabs, and switch between card and list views (#798) * feat: three-finger tap toggles Steam's on-screen keyboard in game mode (#782) * fix: keep Quest-patched Turnip as the auto default (#853) * Fix deploy_local.sh and droiddeckctl not finding a phone connected over wireless debugging (#864) * fix: keep proot sessions alive when the kernel loses fork events (#457) * fix(quest): correct desktop and Steam display corruption across sessions (#733) fix(quest): stabilize desktop and Steam layer presentation Keep Quest SDR scenes and the black backdrop off the Android Vulkan swapchain regardless of session order or HDR preferences. Only report the first frame after client content is submitted. Add backdrop lifecycle and first-frame regression coverage. Record the user's successful Quest 3 Test 2 result. Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com> * fix: keep SteamVR syscall hooks and report Steam crashes (#905) * fix: preserve SteamVR syscall hooks * fix: report Steam crashes through gamescope * fix: honor glasses pacing and resolution in SteamVR launches --------- Co-authored-by: connor g <connorsspam1992@gmail.com> Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com> Co-authored-by: Yassine Gherbi <y.gherbi.dev@gmail.com> Co-authored-by: Resul Övür <92828257+ovurrsl@users.noreply.github.com> Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com> Co-authored-by: Kirk <kirtoy.junsay@gmail.com> Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com> Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com> Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com> Co-authored-by: asdf17128 <asdf17128@gmail.com> Co-authored-by: siyuan.qsy <siyuan.qsy@alibaba-inc.com> Co-authored-by: Dantalian <wsu091227@qq.com> Co-authored-by: Dantalian <Dantalian-dev@users.noreply.github.com> Co-authored-by: The412Banner <the412banner@users.noreply.github.com> Co-authored-by: The412Banner <205237651+The412Banner@users.noreply.github.com> Co-authored-by: FX1922 <120828496+FX192@users.noreply.github.com> Co-authored-by: Muaz Kadan <32595066+muazkadan@users.noreply.github.com> Co-authored-by: kingokksa <47888451+kingokksa@users.noreply.github.com> Co-authored-by: Jamie Schenker <jam1esch3nk3r@gmail.com> Co-authored-by: Jagger15 <jagger2333@live.com> * Glasses: keep a pending 2D restore and the SteamVR recovery offer, show Play SteamVR only with a picture * Translate the glasses calibration, XREAL and SteamVR recovery strings * SteamVR recovery: give a finished teardown time to collect logs before restarting * Credit the projects the XR glasses support builds on * Joy-Cons: SteamVR hands from their gyro while SteamVR runs, one combined pad otherwise * Headsets: settings badge on Play SteamVR instead of a top-right button, both shown only once connected * Hand tracking: camera-tracked Joy-Cons and bare hands in SteamVR, drawn as white gloves - Joy-Con movement from the accelerometer (report reordering, per-Joy-Con calibration, zero-velocity updates) - MediaPipe pose and hand landmarks from the device camera, placement check, per-hand Kalman filter with the Joy-Con - No controller connected: bare camera hands with Quest-style gestures, 1€ filtered, rendered as a white glove model built from a distance field - SteamVR safe mode cleared before each start so one crash no longer blocks the driver --------- Co-authored-by: MaxsTechReview <6025299+maxjivi05@users.noreply.github.com> Co-authored-by: informalTechCode <informaltechyt@gmail.com> Co-authored-by: connor g <connorsspam1992@gmail.com> Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com> Co-authored-by: Yassine Gherbi <y.gherbi.dev@gmail.com> Co-authored-by: Resul Övür <92828257+ovurrsl@users.noreply.github.com> Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com> Co-authored-by: Kirk <kirtoy.junsay@gmail.com> Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com> Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com> Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com> Co-authored-by: asdf17128 <asdf17128@gmail.com> Co-authored-by: siyuan.qsy <siyuan.qsy@alibaba-inc.com> Co-authored-by: Dantalian <wsu091227@qq.com> Co-authored-by: Dantalian <Dantalian-dev@users.noreply.github.com> Co-authored-by: The412Banner <the412banner@users.noreply.github.com> Co-authored-by: The412Banner <205237651+The412Banner@users.noreply.github.com> Co-authored-by: FX1922 <120828496+FX192@users.noreply.github.com> Co-authored-by: Muaz Kadan <32595066+muazkadan@users.noreply.github.com> Co-authored-by: kingokksa <47888451+kingokksa@users.noreply.github.com> Co-authored-by: Jamie Schenker <jam1esch3nk3r@gmail.com> Co-authored-by: Jagger15 <jagger2333@live.com>
This commit is contained in:
+13








authored and
GitHub committed 2026-10-17 13:17:05 -04:00
1 parent
8acf3037fd
commit
16f9916825
221 files changed
+24369
-491
No files matched your search
@@ -0,0 +1,130 @@
|
||||
# Astra 2 and VITURE stereo
|
||||
|
||||
## Tested connection
|
||||
|
||||
2026-10-14: REDMAGIC Astra 2, model `NP06J`, Android 16,
|
||||
`RedMagicOS11.5.16_NP06J_GB`, Snapdragon `SM8850P`; VITURE Pro XR
|
||||
(`35ca:101d`), firmware `07.0.03.013_20250422`. The tablet reports a locked
|
||||
bootloader and green verified boot. Tests used the ordinary ADB shell, without root.
|
||||
|
||||
2D negotiates **1920×1080 at 120 Hz**. The glasses accept stereo command
|
||||
`0x0008 [32]` with reply `00 32`, but Android then exposes **only 640×480 at
|
||||
60 Hz**. An accepted USB command is not proof of a working stereo signal.
|
||||
In this state the glasses reject a 2D command (`01 31` or `01 34`) and their
|
||||
mode query returns `01 00`. A physical cable reconnect restored 1920×1080/120
|
||||
with this update; Android then requested USB permission again and control resumed.
|
||||
|
||||
On the fresh attempt, `0x030D [03]` arrived 3.1 seconds after the accepted stereo
|
||||
command. The vendor SDK maps that report to 2D/120 (`0x34`); the subsequent mode
|
||||
query was rejected. The report does not encode a fallback reason or establish a
|
||||
fixed timeout. The earlier command acknowledgement cannot prove stereo retention.
|
||||
|
||||
## What was checked
|
||||
|
||||
| Route | Result on this firmware |
|
||||
| --- | --- |
|
||||
| Pro full SBS command `0x32` | Accepted by MCU; physical output falls to 640×480 |
|
||||
| Android preferred mode 3840×1080/60 | Preference saved; physical supported modes remain 640×480 only; preference cleared after test |
|
||||
| USB source/host → source/device | Request issued; actual port stayed source/host |
|
||||
| USB data disable/enable | Data status changed and was restored; physical display remained 640×480 |
|
||||
| `IUsbManager.resetUsbPort` through ADB | HAL callback reported success (`0`); physical display mode did not change |
|
||||
| External-display power off/reset | Did not recover the signal; glasses still rejected 2D |
|
||||
| Physical cable reconnect | Restored 1920×1080/120 and USB controls after the permission prompt |
|
||||
| OEM mirror/3D controls, source review | Change virtual-display routing, not physical stereo timings |
|
||||
| OEM resolution selection, source review | Chooses cached resolution candidates and prefers stale 16:9 mode |
|
||||
| Pro startup mode / half SBS, SDK review | No supported Pro startup-mode setting or half-SBS input |
|
||||
|
||||
Public driver decompilations from the related REDMAGIC 11 Pro (`NX809J`, SM8850)
|
||||
[retain an active mode override](https://github.com/Coding-BR/android_kernel_nubia_sm8850_qwjujube/blob/efee6f411184886b0fbdaf63fc9db36b3ed96036/decompiled/msm_drm/dp_display_get_modes.c#L37-L44)
|
||||
across a glasses hotplug. Their [USB-PD disconnect path](https://github.com/Coding-BR/android_kernel_nubia_sm8850_qwjujube/blob/efee6f411184886b0fbdaf63fc9db36b3ed96036/decompiled/msm_drm/dp_display_usbpd_disconnect_cb.c#L127-L147)
|
||||
clears that override; their simulated hotplug path does not. This is consistent
|
||||
with the Astra's measured behavior, but does not prove its exact kernel
|
||||
implementation. That driver also [validates saved user modes against EDID](https://github.com/Coding-BR/android_kernel_nubia_sm8850_qwjujube/blob/efee6f411184886b0fbdaf63fc9db36b3ed96036/decompiled/msm_drm/nubia_edid_modes.c#L226-L250),
|
||||
so active overrides and saved preferences must not be treated as the same
|
||||
mechanism. The independent [MagicDesk investigation](https://github.com/mekhontsev/magicdesk/blob/main/docs/xr-resolution-fix.md)
|
||||
documents a related failure on its separately identified REDMAGIC firmware.
|
||||
|
||||
The [VITURE SDK matrix](https://www.viture.com/en-SG/developer/glasses-sdk/glasses)
|
||||
specifies full SBS at 3840×1080/60 for Pro. Persistent startup mode is restricted
|
||||
to Luma/Luma Pro. Among the Luma families supported here, 90 Hz stereo is an
|
||||
Ultra capability. The connected Pro firmware matches both the N6PL entry in the
|
||||
[public updater catalog](https://static.viture.com/dfu-util/js/vers.js) and the newest
|
||||
N6P entry in the [public release-note service](https://cloud.viture.dev/api/v1/system/releasenote)
|
||||
checked on 2026-10-14. This does not rule out unpublished, device-specific updates.
|
||||
|
||||
## Powered dock results
|
||||
|
||||
A VITURE Pro Mobile Dock (`35ca:3011`) was connected between the Astra and Pro XR.
|
||||
Its official read-only P9 version query returned `2.01.03.0.03` with a valid
|
||||
checksum. The Astra negotiated 1920×1080/120, but enumerated only the dock's
|
||||
USB interfaces, not the glasses' `35ca:101d` control/IMU interfaces. A one-second
|
||||
DroidDeck IMU sample returned zero packets.
|
||||
|
||||
DroidDeck treats the glasses as a display behind the dock: no USB permission
|
||||
prompt, video at 1080p/120 on the glasses, no Play SteamVR, refresh, 3D or tracking
|
||||
settings. The dock alone, without glasses, does not count as glasses.
|
||||
|
||||
VITURE's updater calls this dock P9 (app `35ca:3011`, bootloader `35ca:3010`). Its
|
||||
HID messages cover dock versions, battery, inputs and LEDs; none sets a display
|
||||
mode or forwards glasses commands or IMU data. `2.01.03.0.03` is public release
|
||||
`1.03.0.03_20250709`; the latest, `1.03.0.06_20260409`, adds Beast support and
|
||||
updater fixes only. The updater also reads a Lontium LT8668 in the dock, a
|
||||
DisplayPort receiver with its own EDID handling, so the tablet most likely sees the
|
||||
dock as its DisplayPort sink rather than the glasses.
|
||||
|
||||
[VITURE's Pro Dock specifications](https://www.viture.com/product/viture-pro-mobile-dock)
|
||||
confirm 3840×1080/60 video support and exclude phone features requiring
|
||||
communication with the glasses. The vendor SDK also treats the dock as a separate
|
||||
accessory, not an IMU device. This dock may test the video path, but it does not
|
||||
provide DroidDeck's current head-tracking path. Stereo negotiation and mode
|
||||
retention during an upstream-only reconnect still require a physical test.
|
||||
|
||||
## No-root hardware design to test
|
||||
|
||||
A powered pass-through connection may let the tablet see the stereo EDID on a
|
||||
fresh USB-PD connection while the glasses retain their stereo mode:
|
||||
|
||||
1. Power the adapter and glasses independently of the tablet connection.
|
||||
2. Enable stereo on the glasses.
|
||||
3. Disconnect only the tablet's upstream USB-C connection promptly after the
|
||||
stereo acknowledgement, keeping the glasses powered. Compare with a delayed
|
||||
disconnect to test whether the observed mode report affects retention.
|
||||
4. Reconnect the tablet and check its physical supported modes before starting a game.
|
||||
|
||||
This requires an adapter that preserves glasses power and mode during the
|
||||
upstream disconnect, passes 3840×1080/60 video, and forwards USB HID for controls
|
||||
and head tracking. [VITURE's Adapter Ultra documentation](https://www.viture.com/academy/adapters/usb-c-xr-charging-adapter-ultra)
|
||||
explicitly excludes wired glasses data on Android, so it does not satisfy
|
||||
DroidDeck's current USB control/tracking path. Its power and video specifications
|
||||
also do not promise mode retention across upstream disconnect. This design
|
||||
remains **unverified**, not a purchasing recommendation. First verifying stereo
|
||||
retention through the adapter on a compatible host would provide a useful control.
|
||||
An intermediary presenting the stereo EDID before first host
|
||||
connection is another possible design; it must pass the actual stereo timing
|
||||
and preserve HID access, not merely advertise a wider desktop.
|
||||
|
||||
VITURE advertises full Android functionality for its separate
|
||||
[USB-C to Glasses and Charging Adapter](https://www.viture.com/en-MY/product/usb-c-to-glasses-and-charging-adapter).
|
||||
That makes it a candidate for the USB-data experiment, not a confirmed fix.
|
||||
First check USB enumeration, brightness controls and live IMU data through the
|
||||
adapter; then test power/mode retention and the stereo timing.
|
||||
|
||||
Success requires Android to expose and select 3840×1080/60, different left/right
|
||||
eye images to reach the lenses, live head tracking, and repeatable return to 2D.
|
||||
A wide screenshot or `wm size` override alone does not meet those conditions.
|
||||
|
||||
## App safeguards
|
||||
|
||||
DroidDeck waits up to 20 seconds for physical stereo confirmation. When the
|
||||
attempt ends at a low-resolution fallback, DroidDeck remembers it for that glasses
|
||||
model on this device, so Play SteamVR stops requesting 3D after a restart; **Retry** in the
|
||||
glasses settings clears it. A display that stays missing, such as an unanswered
|
||||
cast prompt, fails only that attempt. Automatic retries do not loop.
|
||||
USB mode reports are tracked separately from the physical display. When a session
|
||||
enabled 3D, stopping it requests a return to 2D, including during negotiation;
|
||||
this firmware can still require a cable reconnect. Output timing follows
|
||||
the active glasses display rather than the tablet's requested refresh rate.
|
||||
|
||||
Keep hardware experiments separate from game loads. End the session, force-stop
|
||||
DroidDeck, and reopen its home screen after each game test. Read current HAL
|
||||
temperatures rather than cached thermal-service samples.
|
||||
@@ -44,12 +44,16 @@ OpenVR to SteamVR's ARM64 `vrclient.so`, so no x86 SteamVR is involved.
|
||||
- `helloxr` blocked in a wait-idle in `vrclient.so`;
|
||||
- DXVK's `dxvk-queue` thread blocked in Proton games.
|
||||
|
||||
The layer now gives a submit's wait on an imported shared value 3 ms, then drops it. It is kept
|
||||
when the value is already reached, or when this process has queued the signal itself. Two details
|
||||
mattered:
|
||||
The layer now gives a submit's wait on an imported shared value 3 ms, then drops it, at once when
|
||||
the shared counter has not moved for 100 ms. It is kept when the value is already reached, when
|
||||
this process has queued the signal itself, or while a Quest session's frame socket exists (checked
|
||||
at most once a second). Three details mattered:
|
||||
- the waits are judged before the submit's own signals count, because an acquire waits on one
|
||||
value and signals the next on the same semaphore;
|
||||
- SteamVR's client interop device is Vulkan 1.0, so only the `KHR` entry points resolve there.
|
||||
- SteamVR's client interop device is Vulkan 1.0, so only the `KHR` entry points resolve there;
|
||||
- a Proton game waits on two values a frame that are never signalled (5,500 dropped, none reached
|
||||
within 200 ms afterwards). With the grace on each, Angry Birds VR's submit thread lost 6 ms a
|
||||
frame and the game ran at 60 fps; without it, 120.
|
||||
4. **UBWC layouts must agree.** An imported image has to be created with the same
|
||||
`VkImageFormatListCreateInfo` (sRGB + UNORM) as its creator, or Turnip lays out the compression
|
||||
differently and the reader sees garbage. The presenter's eye-texture import does this.
|
||||
@@ -83,6 +87,21 @@ OpenVR to SteamVR's ARM64 `vrclient.so`, so no x86 SteamVR is involved.
|
||||
- **Memory.** Steam, SteamVR and one VR game fit in the Thor's 12 GB. A large game such as The Lab, or
|
||||
two games at once, push the device into critical pressure, and the low-memory killer takes
|
||||
DroidDeck itself (`ended-without-teardown.txt` in the session logs, `lowmemorykiller` in logcat).
|
||||
- **Frames shown out of order.** Video from the glasses jumped back a few frames in bursts while the
|
||||
head turned. Two causes, each found by stamping a frame counter into the presenter's output:
|
||||
- the presenter copied SteamVR's eye texture a fixed 4 ms after `Present`, with no fence. Under
|
||||
GPU load `vrcompositor` had not drawn it yet, so the copy got that texture's previous frame. The
|
||||
layer now fences `vrcompositor`'s submits and the presenter waits for the frame's count;
|
||||
- DroidDeck's compositor copies a Wayland client's dma-buf without waiting for its implicit fence,
|
||||
and KGSL does not order the copy after the client's GPU work. The presenter's blit sat behind the
|
||||
game's frame, so the compositor showed the swapchain image's older picture. The presenter now
|
||||
waits for its blit before it presents.
|
||||
- **Flat games under SteamVR.** With SteamVR running, Steam lists a flat game after its own window in
|
||||
gamescope's base layer, so Big Picture keeps the focus and the game never shows, theatre setting or
|
||||
not. SteamVR is therefore started only by Play SteamVR, never with the glasses alone.
|
||||
- **A new projection needs an event.** `SetDisplayProjectionRaw` changed what `vrserver` reports, but
|
||||
`vrcompositor` kept the frustum it started with, so the glasses' field of view did nothing until
|
||||
SteamVR restarted. The driver now follows it with `VREvent_LensDistortionChanged`.
|
||||
- **Head-tracking axes.** With the display at `ROTATION_90`, screen x is minus device y and screen y is
|
||||
device x. Getting this wrong made a turn at the hips tilt the view. Recentring now keeps the world
|
||||
vertical and resets the heading only.
|
||||
@@ -123,8 +142,9 @@ OpenVR to SteamVR's ARM64 `vrclient.so`, so no x86 SteamVR is involved.
|
||||
- **SteamVR Home** needs `IVROverlayView_002` in Proton's `vrclient`, or an ARM64 build of Home.
|
||||
- **The Lab and other large games** need memory headroom. Candidates: stop Steam's own web helpers
|
||||
while a VR game runs, or lower SteamVR's footprint.
|
||||
- **Wait drops are a heuristic.** A 3 ms grace keeps the common case in sync. A submit thread in the
|
||||
layer, deferring submits until their waits are reachable, would be exact.
|
||||
- **Wait drops are a heuristic.** A 3 ms grace keeps the common case in sync, and an idle counter
|
||||
skips it. A submit thread in the layer, deferring submits until their waits are reachable, would be
|
||||
exact.
|
||||
- **Layers:** only the first layer of each frame is presented. No reprojection.
|
||||
- **Controllers** are a gaze pointer driven by the pad. Pad presses did not close the SteamVR menu in
|
||||
testing. Untested: whether that came from gamescope focus being on Steam, or from the system button
|
||||
|
||||
@@ -0,0 +1,179 @@
|
||||
# RayNeo Air / GT XR integration
|
||||
|
||||
Built on the current `feat/viture-xr` PR #732 at bdf876d47cc9bb6fa810d014370da5bbb3c28039.
|
||||
The supplied historical hashes a127e66, 7f9dd75, e8e3e6b and f0b3297 were not
|
||||
resolvable in the fetched branch; no native projection or synchronization files are changed.
|
||||
|
||||
## Discovery and transport
|
||||
|
||||
`Glasses.kt` owns USB permission, attachment/detachment and presentation-display
|
||||
selection. There is no manifest USB VID filter: discovery uses `UsbManager.deviceList`.
|
||||
VITURE discovery is retained. `RayNeoModel` accepts the scaffold's confirmed
|
||||
`1bbb:af50` and verncat's confirmed GT identity `3941:af50`; other products under `1bbb` or bridge vendor `22d9` must identify
|
||||
as RayNeo/TCL Air or GT in their USB product name. This covers named Air 2, 2s,
|
||||
3, 4 Pro and GT devices without claiming arbitrary TCL phones or bridge chips.
|
||||
Additional unnamed VID/PID pairs require captures before being allowlisted.
|
||||
Other products under `3941` are rejected. Without a meaningful USB product name,
|
||||
`3941:af50` is labeled "RayNeo GT series"; a supplied GT MAX name is preserved.
|
||||
GT Max was observed on the Samsung SM-F976U with this USB identity on 2026-10-14;
|
||||
USB permission and the command handshake succeeded.
|
||||
EDID/product display names containing RayNeo, Ray Neo, or TCL Air/GT use the existing
|
||||
external-screen path. Android's supported modes, not a synthetic refresh value,
|
||||
determine whether 120 Hz can be selected. Stereo modes may have a lower maximum rate.
|
||||
GT Max's observed display identity is `SmartGlasses`, EDID manufacturer `TCL`,
|
||||
product `980` (1920x1080, 60/120 Hz in 2D). This exact metadata combination is
|
||||
recognized even though neither display name contains RayNeo. Other generic
|
||||
SmartGlasses/TCL products are not accepted by this rule.
|
||||
|
||||
Game output uses the same `GlassesScreen` presentation as VITURE, with the phone
|
||||
retaining the session controls; no screen-capture permission is needed. Discovery
|
||||
checks both Android's display name and its product name. On Android 12 and newer,
|
||||
a missing product-info object is treated like an unnamed panel: a recognized
|
||||
glasses USB device can identify a single presentation display. Unrelated named
|
||||
monitors and multiple unnamed displays are not automatically selected by that
|
||||
fallback. `Game on glasses` remains enabled by default.
|
||||
The presentation holder is selected before `show()`, and the phone SurfaceView
|
||||
is hidden only after the compositor attaches to the glasses holder. Late phone
|
||||
surface callbacks cannot reset the external output.
|
||||
|
||||
`RayNeoLink` claims a paired IN/OUT interface and translates the controller's
|
||||
internal commands to RayNeo packets. For `3941:af50`, selection prefers interface
|
||||
number 5 (not array index 5), as tested in verncat's GT contribution. Only endpoints
|
||||
that can carry 64-byte packets qualify. Firmware with a different interface number
|
||||
can use a unique qualifying IN/OUT pair; ambiguous alternatives are rejected.
|
||||
It requests device info (0x66/0), enables IMU
|
||||
(0x66/1) and waits for acknowledgments, warms up for two seconds, and stops IMU
|
||||
(0x66/2) when no pose consumer remains or the link closes. SBS uses commands 6/7.
|
||||
Unsupported VITURE brightness, volume and electrochromic commands are never sent;
|
||||
the VITURE hardware-control UI is hidden for RayNeo.
|
||||
GT spatial-button notification 0x08 and volume/display ACKs remain non-device-info
|
||||
responses and cannot satisfy unrelated startup commands. The spatial button is
|
||||
left to the glasses' firmware; no undocumented spatial-mode command is sent.
|
||||
|
||||
The parser/filter are adapted from the user's scaffold at
|
||||
581dbf712ed601ece4cf5e4a7c1f52e780299c88. Sensor packets are framed with 0x99/0x65,
|
||||
with little-endian acceleration at byte 4, gyro (degrees/sec) at 16 and a uint32
|
||||
100us device tick at 40. Fragmented and coalesced reports are assembled before decoding.
|
||||
Board 0x3a (Air 4 Pro) receives the scaffold's -20-degree X IMU mounting correction.
|
||||
GT Max (`3941:af50`, board `0x41`) uses the mapping measured on 2026-10-14:
|
||||
gyro and acceleration `(-y, z, -x)`, then +35 degrees around X.
|
||||
At the user's level pose the raw acceleration was approximately
|
||||
`(5.70, 0.00, 7.95)` m/s². A directed left-turn-and-hold capture produced positive
|
||||
X/Z gyro (e.g. `(20.5, 2.0, 28.2)` deg/s), and upward looks produced negative Y
|
||||
gyro. The 35-degree mounting estimate comes from that level capture,
|
||||
not manufacturer calibration; frame fit and other GT variants remain unverified.
|
||||
Gravity initializes orientation before the first published pose. Subsequent
|
||||
body-to-world fusion uses `measuredUp × predictedUp`, correcting tilt toward gravity
|
||||
instead of amplifying the error. These fusion corrections also apply to Air;
|
||||
the GT Max sensor remapping does not apply to Air or VITURE.
|
||||
|
||||
The non-Max GT shares USB `3941:af50` but reports board `0x40`, confirmed by
|
||||
switching between both physical pairs on the same phone. Its sensor axes use
|
||||
the ordinary XYZ basis with a measured -25-degree X mounting correction, rather
|
||||
than the Max permutation. Its confirmed upside-down resting acceleration was
|
||||
approximately `(0.020, -8.819, -4.106)` m/s²; the worn level capture was near
|
||||
`(0.09, 9.26, 3.11)`. Left turns produced positive Y/Z gyro. The mounting estimate
|
||||
uses the resting surface; per-device calibration refines its tilt and gyro bias.
|
||||
Both the pose filter and calibration collector select the same board profile.
|
||||
Non-Max calibration keys have a `:gt-standard-v1` suffix so they cannot replace
|
||||
existing Max profiles when USB identity fields coincide. Calibration loading
|
||||
waits for the device-info response; detached-device callbacks cannot change the
|
||||
replacement device's calibration key. Existing Max keys remain compatible.
|
||||
After installing this correction, the non-Max completed the upside-down
|
||||
calibration on the phone with `saved=true`, phase `SAVED`, progress 100.
|
||||
All 186 glasses/shared-pose/SteamVR-launch/UI regression checks passed and the
|
||||
installed APK's packaged runtime inputs verified. After calibration and the
|
||||
recenter check, the wearer confirmed on 2026-10-14 that the non-Max GT view
|
||||
looks and feels good.
|
||||
|
||||
## Existing pose and SteamVR pipeline
|
||||
|
||||
The scaffold fuses gyro/accel into wxyz. The Android adapter converts its Y-up
|
||||
coordinate basis and component order into VITURE's Z-up xyzw convention before
|
||||
calling the same `Glasses.PoseSink` used by VITURE. `HeadTracker` and `HeadPose`
|
||||
retain their existing recenter/seated-reference behavior. The path continues through
|
||||
`WaylandCompositor.nativeSetCardboardPose`, the compositor's `headpose_shm.h`
|
||||
shared-memory record and `tools/vr/src/driver_droiddeck.cpp`'s pose-file reader.
|
||||
The OpenVR projection, presenter, Vulkan wait logic and vblank counters are untouched.
|
||||
An explicit Play SteamVR request polls the guest runtime independently of display
|
||||
attachment and USB-control readiness. Both RayNeo and VITURE call the same
|
||||
`droiddeck-steamvr start` helper. Stopping cancels the pending launch request;
|
||||
automatic hardware 3D switching still respects the shared Auto 3D setting.
|
||||
|
||||
## Optional stationary calibration
|
||||
|
||||
Glasses → Tracking → Tracking calibration guides the user to rest the connected
|
||||
RayNeo upside down on a level surface. It allows three seconds to settle, then
|
||||
requires five continuous seconds of stationary samples (at least 200 reports).
|
||||
Invalid gravity, the wrong orientation, motion/variance, or gaps longer than 250 ms
|
||||
restart the window. A 30-second deadline fails without replacing a saved profile;
|
||||
USB stream loss also terminates the operation. Cancellation and link replacement
|
||||
invalidate queued callbacks by operation revision and connection generation.
|
||||
|
||||
The profile contains a raw gyro bias and a small mounting tilt correction, applied
|
||||
to both sensor vectors before fusion. It does not estimate heading, full accelerometer
|
||||
scale/bias, or translation. Profiles persist by vendor/product/serial; when a serial
|
||||
is unavailable the product and model name share a fallback profile. Reset deletes
|
||||
only the current profile and restores the model mapping. VITURE's fused SDK poses
|
||||
do not expose the raw inputs this measurement needs, so this control is RayNeo-only.
|
||||
|
||||
While measuring, live pose publication pauses. On completion the user wears the
|
||||
glasses, looks ahead and taps Done to recenter the active session. Dismissing the
|
||||
dialog or leaving the tracking panel cancels an in-progress measurement. Agent
|
||||
diagnostics expose the calibration state through `glasses`; operations are
|
||||
`calibrate`, `cancel-calibration`, and `reset-calibration`.
|
||||
The calibration build passed 157 targeted sensor, lifecycle, pose, launch and
|
||||
Compose UI tests. Installing this feature and validating a real upside-down capture
|
||||
on the phone remain pending a debugging connection.
|
||||
|
||||
## Regression and hardware validation
|
||||
|
||||
### SteamVR library launch routing (2026-10-14)
|
||||
|
||||
DroidDeck's SteamVR library entry (appid 250820), including its game shortcut,
|
||||
now sets the same `EXTRA_STEAMVR` request as Play SteamVR instead of sending a
|
||||
normal Steam game URL. Existing sessions resume with that request as well.
|
||||
Other games retain their normal launch path and arguments.
|
||||
|
||||
Inside Steam's library, the existing SharedJSContext bridge wraps
|
||||
`SteamClient.Apps.RunGame` for exactly 250820 and dispatches it to
|
||||
`droiddeck-steamvr launch`. This calls the same install/start helper used by the
|
||||
button, including OpenVR registration, the ARM64 runtime and Vulkan layer setup.
|
||||
It does not change Steam's saved launch options. The library bridge is installed
|
||||
for both desktop and Big Picture. It relies on the live Steam UI API and needs
|
||||
verification against the actual phone's Steam version after installation.
|
||||
|
||||
The helper serializes installation/startup and avoids restarting an active
|
||||
DroidDeck SteamVR session. Library launch diagnostics are in
|
||||
`${XDG_RUNTIME_DIR:-/tmp}/droiddeck-steamvr/library-launch.log`; normal runtime
|
||||
logs remain in `vrstartup.log` and Steam's logs directory. It does not intercept
|
||||
arbitrary external `steam://` URLs or change VR game launch arguments.
|
||||
|
||||
Validation: seven bridge/dispatch tests passed under Node/Python, five Android
|
||||
SteamVR launch tests passed, shell syntax passed and the debug APK built with
|
||||
runtime input verification. The broader Linux compatibility suite cannot run
|
||||
unmodified on Windows because it requires `fcntl`; no Linux runtime was available
|
||||
for the helper's lock/process integration tests. Installation and live library
|
||||
Play verification are pending while the phone charges.
|
||||
|
||||
Run `./gradlew -PskipRust=true :app:testDebugUnitTest --tests 'com.droiddeck.launcher.glasses.*'`.
|
||||
Protocol tests cover malformed/fragmented packets, unsigned timestamps, model gating,
|
||||
quaternion normalization, yaw basis and tick rollover. USB integration tests cover
|
||||
permissions, GT interface preference/fallback, interleaved GT notifications and
|
||||
ACKs during startup, 2D/3D writes, and stale pose callbacks. Hardware validation is still required per model:
|
||||
USB enumeration, acknowledgment/stream startup, axes and recenter, hot unplug/replug,
|
||||
SBS negotiation, Android's actual 120Hz mode, and SteamVR pose/frame pacing.
|
||||
GT Max was verified on SM-F976U with USB `3941:af50` and the TCL/980 display:
|
||||
the compositor attached to its separate 1920x1080 display, the Play SteamVR
|
||||
request started native `vrserver`/`vrcompositor` through the shared helper, and
|
||||
the presenter logged roughly 60 frames/s while the IMU packet count increased.
|
||||
The test retained the user's disabled Auto 3D setting. After installing the corrected
|
||||
GT sensor mapping, the user confirmed an upright starting view and correct movement
|
||||
in all directions. SteamVR and the glasses IMU remained running. Stereo negotiation,
|
||||
long-term drift, and alternate bridge firmware variants still need their own
|
||||
hardware validation. The targeted glasses, Cardboard pose, and SteamVR launch
|
||||
suite passed 134 tests after the tracking correction. Tracking regression coverage
|
||||
checks initial gravity alignment, tilt convergence, the recorded GT Max level pose,
|
||||
and yaw/pitch/roll through the shared VITURE basis conversion and recenter path.
|
||||
|
||||
Attribution and the scaffold's Apache-2.0 license are in `docs/third-party/rayneo/`.
|
||||
@@ -0,0 +1,92 @@
|
||||
# VITURE Android SDK integration
|
||||
|
||||
The optional adapter is built against the user-supplied VITURE XR Glasses SDK
|
||||
2.4.0. Set `droiddeck.vitureSdk=C:/path/to/VITURE_XR_Glasses_SDK_for_Android`
|
||||
in the ignored `local.properties`, or pass `-PvitureSdkDir=/path/to/sdk`.
|
||||
The directory must contain `include/` and `android/arm64-v8a/`.
|
||||
|
||||
Gradle compiles `droiddeck_viture`, packages `libglasses`, `libcarina_vio`, and
|
||||
`libcloud_protocol`, and includes the SDK's LICENSE and NOTICE under
|
||||
`assets/licenses/viture`. Vendor headers and binaries stay outside the repository.
|
||||
A build without the optional SDK still supports the existing legacy VITURE and
|
||||
RayNeo transports, but does not advertise control for SDK-dependent glasses.
|
||||
|
||||
## Supported app functions
|
||||
|
||||
| Device | Transport and tracking | App controls |
|
||||
| --- | --- | --- |
|
||||
| Pro 2 XR | SDK Gen2 pose callbacks, observed OpenGL axes converted to NWU; negotiated pose frequency | 1080p 2D through 120 Hz, SBS 60 Hz, brightness 0–8, volume 0–8 |
|
||||
| Luma Ultra | SDK Carina USB transport; OpenGL poses converted to NWU | 1200p 2D through 120 Hz, SBS 60/90 Hz, brightness, volume, binary dimming |
|
||||
| Beast | SDK Gen2; native/bypass display commands selected by capability and current mode | 1200p output, SBS, brightness, volume 0–15, dimming toggle |
|
||||
|
||||
Pro 2 identification uses the SDK's PID validation and market name, not a guessed
|
||||
USB ID. Pro 2 has no electrochromic film control. Its display profile uses the
|
||||
[official 50° FOV and 1080p/SBS limits](https://www.viture.com/academy/product/viture-pro-2-xr-glasses).
|
||||
Device firmware can reject individual modes; Android's physical display size
|
||||
must confirm SBS before the app considers the switch successful.
|
||||
|
||||
All three feed the existing `Glasses.PoseSink`, head recentering, compositor and
|
||||
SteamVR pipeline. Ultra explicitly selects 3DoF before SDK initialization because
|
||||
the current pipeline consumes orientation. Positional 6DoF and camera passthrough
|
||||
are outside this integration. The SDK's optional statistics reporter is not enabled.
|
||||
|
||||
The SDK owns USB interface selection and I/O. `UsbDeviceConnection` remains open
|
||||
until stop/shutdown/destroy finish. A generation token rejects callbacks from an
|
||||
old connection. Device brightness, volume and dimming events update the app.
|
||||
Beast native and standard display codes have separate meanings; they are mapped
|
||||
explicitly. Host tracking suspends native DOF to prevent applying rotation twice
|
||||
and restores the previous native DOF on stop. Older firmware rejecting a bypass
|
||||
mode can fall back to the native table, with rollback on failure.
|
||||
|
||||
`Game on glasses` uses Android's existing Presentation output. USB control alone
|
||||
does not create a video display: the phone, cable and adapter must expose a usable
|
||||
external display. Discovery checks both display name and product metadata, with
|
||||
a single unnamed-display fallback when recognized glasses USB is present.
|
||||
|
||||
Screen width and distance apply to **Virtual screen** rendering. In **Automatic**,
|
||||
ordinary 2D games stay flat, ordinary games with a confirmed SBS display use a
|
||||
virtual screen, and a running SteamVR session uses the direct stereo view.
|
||||
The direct view uses SteamVR field of view instead of virtual-screen dimensions
|
||||
while SteamVR is running. A plain side-by-side game fills each eye and does not
|
||||
use the SteamVR field-of-view or recenter controls.
|
||||
The settings UI and session renderer share this mode selection through
|
||||
`GlassesRendering`; home-screen changes configure sessions rather than resizing
|
||||
Android's mirrored launcher. Regular refresh rate applies to 2D output; SBS uses
|
||||
the model's separate 3D refresh rate.
|
||||
Software picture controls remain available while a display is connected even if
|
||||
USB control is lost; hardware controls and head tracking still require USB.
|
||||
Presentation refresh preferences are revalidated on display changes even when
|
||||
the requested Hz is unchanged, and keep both dimensions of the active mode.
|
||||
|
||||
## Validation
|
||||
|
||||
Run `:app:externalNativeBuildDebug` and
|
||||
`:app:testDebugUnitTest --tests 'com.droiddeck.launcher.glasses.*'` with the SDK
|
||||
configured. Compile `app/src/test/cpp/viture_display_modes_test.cpp` with the NDK
|
||||
C++17 compiler and the SDK include directory to check native/standard mappings
|
||||
against the vendor constants. The Kotlin tests cover model capabilities, axis
|
||||
conversion, invalid poses, rejected tracking starts and stale callbacks.
|
||||
|
||||
Physical acceptance checks remain: connect each model, grant USB permission,
|
||||
verify the Play SteamVR control, verify game output moves to the glasses,
|
||||
test head rotation/recenter and SBS, then unplug/reconnect. Building and unit
|
||||
testing do not establish hardware compatibility for every firmware revision.
|
||||
|
||||
### Pro 2 pose-axis correction, 2026-10-14
|
||||
|
||||
The connected Pro 2 delivered approximately 200 normalized poses/sec. The wearer
|
||||
reported yaw moving the view vertically, pitch rolling it, and roll turning it
|
||||
horizontally. Directed captures showed a left turn primarily increasing quaternion
|
||||
Y (approximately 0.004 to 0.253), while repeated upward looks varied quaternion X
|
||||
(approximately -0.187 to +0.082) with Y/Z remaining near zero. This is the OpenGL
|
||||
right/up/back basis, despite the SDK 2.4.0 generic Gen2 callback documentation
|
||||
describing NWU.
|
||||
|
||||
The adapter therefore selects the existing OpenGL-to-shared-glasses conversion
|
||||
for the SDK-identified Pro 2 family as well as Carina. Other Gen2 models retain
|
||||
their previous mapping. Tests exercise model selection and yaw/pitch/roll through
|
||||
the complete shared `HeadPose` conversion and recenter path. This fixes a model
|
||||
coordinate mismatch; it does not claim to run a vendor sensor-calibration command
|
||||
or save a RayNeo-style calibration for VITURE. All 188 glasses/shared-pose/
|
||||
SteamVR-launch/UI regression checks passed, packaged runtime inputs verified,
|
||||
and the corrected build was installed on the phone. Visual verification is pending.
|
||||
@@ -0,0 +1,242 @@
|
||||
# XREAL 1S / One Pro feasibility — 2026-10-14
|
||||
|
||||
The user owns XREAL 1S and also wants One Pro support. Supporting these in DroidDeck
|
||||
is feasible based on existing open drivers, but the supplied Monado Air driver is
|
||||
not their transport. The initial investigation below preceded implementation.
|
||||
An experimental native 1S driver is now enabled; see the hardware validation
|
||||
section at the end for what has actually been verified.
|
||||
|
||||
## Supplied Monado source
|
||||
|
||||
Inspected `C:/Users/conno/Downloads/monado-main/monado-main/src/xrt/drivers/xreal_air/`
|
||||
and `src/xrt/targets/common/target_builder_xreal_air.c`.
|
||||
|
||||
| Model | USB vendor:product | Sensor/control interface | Sensor report |
|
||||
| --- | --- | --- | --- |
|
||||
| Air | 3318:0424 | 3 / 4 | 64 bytes |
|
||||
| Air 2 | 3318:0428 | 3 / 4 | 64 bytes |
|
||||
| Air 2 Pro | 3318:0432 | 3 / 4 | 64 bytes |
|
||||
| Air 2 Ultra | 3318:0426 | 2 / 0 | 512 bytes |
|
||||
|
||||
This driver implements HID IMU input, segmented factory-calibration JSON reads,
|
||||
gravity/gyro fusion, brightness 0–7 and 2D/3D mode commands. It does not register
|
||||
1S or One Pro. Copying those Air HID commands to the new products is unjustified.
|
||||
Its source carries Tobias Frisch's BSL-1.0 notice.
|
||||
|
||||
## Applicable One-family references
|
||||
|
||||
* [XRLinuxDriver](https://github.com/wheaney/XRLinuxDriver), inspected commit
|
||||
`36e2139dd482f5984112d0c3ebafcc07a78f99c0`. `src/devices/xreal.c` lists vendor
|
||||
`3318`, One Pro products `0435/0436`, and 1S products `043d/043e`. These use the
|
||||
non-HID transport. Its README explicitly lists both models and requires current
|
||||
firmware with glasses stabilization/anchor disabled when using external tracking.
|
||||
* [xrealOneDeviceKit](https://github.com/wheaney/xrealOneDeviceKit), inspected commit
|
||||
`f5833f611265f418b1ea549da20dc4764d86a4ec`. Its IMU adapter consumes the One TCP
|
||||
driver, interprets gyro as radians/sec and acceleration as m/s², then feeds fusion.
|
||||
Its factory-calibration JSON hook returns unsupported. The maintained Linux
|
||||
integration disables its SBS-control capability when no MCU connection is open;
|
||||
a static model table alone is not proof of working display switching.
|
||||
* [xreal_one_driver](https://github.com/rohitsangwan01/xreal_one_driver), MIT,
|
||||
inspected commit `3e912315ec16e94e9a47688770d649cfef293624`. Opens TCP
|
||||
`169.254.2.1:52998` with a two-second timeout, assembles 84-byte IMU records,
|
||||
reads a timestamp at byte 14, gyro floats at 34 and acceleration floats at 46.
|
||||
This is a transport/parser reference, not a drop-in Android glasses controller.
|
||||
|
||||
The source check confirms a practical path for both requested models. It does not
|
||||
establish that this Samsung phone exposes the glasses' USB Ethernet interface to
|
||||
ordinary Android apps, or establish the physical tracking axes on this hardware.
|
||||
|
||||
## DroidDeck integration path
|
||||
|
||||
1. Attach the 1S and inspect its USB descriptors, presentation display identity,
|
||||
Ethernet network/link properties and whether the IMU endpoint is reachable.
|
||||
Obtain the same evidence for One Pro before claiming hardware validation.
|
||||
2. Add exact model discovery and external-display selection. Preserve each panel's
|
||||
Android-advertised dimensions and modes; do not assume the Air's resolution.
|
||||
3. Implement a bounded One-family stream parser and cancellable socket transport.
|
||||
On Android, bind the socket to the relevant Ethernet `Network` when available,
|
||||
leaving Steam's Internet route alone. If Android does not expose that link,
|
||||
a USB network transport is additional work; no root requirement is assumed.
|
||||
4. Establish gyro units, timestamp deltas and axis mapping using stationary,
|
||||
directed yaw, pitch and roll captures. Route the validated fused pose through
|
||||
the same glasses/HeadTracker/SteamVR pipeline as VITURE and RayNeo. Avoid layering
|
||||
DroidDeck tracking on top of the glasses' own anchor/stabilization.
|
||||
5. Gate brightness and automatic stereo switching separately on a verified control
|
||||
protocol. IMU streaming alone does not justify showing those controls as working.
|
||||
6. Test malformed/fragmented/coalesced records, missing network, timeout/reconnect,
|
||||
USB detach, screen handoff, SteamVR launch and each physical movement direction.
|
||||
|
||||
At this initial investigation stage, no external SDK was installed, firmware was
|
||||
modified, or XREAL commands were sent.
|
||||
|
||||
## Further source investigation while the phone charges
|
||||
|
||||
Inspected [xr-tools](https://github.com/0xcaff/xr-tools), commit
|
||||
`65c11105434a998f09ac4a5f8e4de8736bb4d36c`, particularly
|
||||
`packages/xreal_one_driver/src/proto/net/`, `src/proto/usb/mod.rs`, and the IMU
|
||||
example. This describes considerably more of the One protocol than the simple
|
||||
TCP IMU reader above:
|
||||
|
||||
* Reports use TCP `169.254.2.1:52998`; controls use `169.254.2.1:52999`.
|
||||
Both have a two-byte message identifier followed by a four-byte **big-endian**
|
||||
body length. Control messages additionally contain a big-endian transaction ID
|
||||
and protobuf payload. A port must bound lengths, use unique transaction IDs,
|
||||
enforce deadlines, and close both sockets on detach.
|
||||
* Sensor messages `28 36` have a **128-byte body**, hence 134 bytes including the
|
||||
header. The older reader's 84-byte extraction is not the complete frame format.
|
||||
Body offsets: device ID 0, device timestamp 8, report type 24, gyro 28,
|
||||
acceleration 40, magnetometer 52, temperature 64 and IMU ID 68. Numeric sensor
|
||||
fields are little-endian; timestamp is documented as nanoseconds, gyro rad/s,
|
||||
acceleration m/s². Type 11 is IMU; type 4 is magnetometer. Do not interpret every
|
||||
report as gyro data or use arrival time as the device's integration interval.
|
||||
* Control `27 1f` reads factory JSON: IMU bias, temperature-dependent gyro bias,
|
||||
and display-to-IMU transforms. Its example subtracts the factory biases, fuses
|
||||
gyro/acceleration and applies the display rotation. This is preferable to
|
||||
guessing a fixed mounting angle, but the example's frame convention still
|
||||
needs conversion and directed-motion verification in DroidDeck.
|
||||
* `27 1c` sets brightness 0–9; `27 27` sets dimming; `28 22` selects regular/SBS
|
||||
input; `27 5e/5f` reads/sets display configuration; `28 a6/a7` reads/sets EIS.
|
||||
Documented display configurations are 1920×1080 at 60/90/120 Hz and 3840×1080
|
||||
at 60 Hz. These are evidence for One/One Pro, not confirmed 1S modes. Preserve
|
||||
the Android-advertised modes and verify controls separately on 1S firmware.
|
||||
* `28 29` is labelled buttons-enabled/buttons-disabled scene mode. Do not assume
|
||||
that command disables stabilization; the separate EIS control exists.
|
||||
* This project's USB detection labels `3318:0436` One Pro and `3318:0438` One as
|
||||
normal devices, and `0435/0437` as **recovery** devices. XRLinuxDriver accepting
|
||||
both IDs does not justify opening recovery products as ordinary glasses or
|
||||
sending firmware commands. xr-tools does not register 1S; its applicability
|
||||
there is a protocol-family inference, not hardware validation.
|
||||
|
||||
No license file or Cargo license declaration was found in this inspected xr-tools
|
||||
revision. No implementation was copied into DroidDeck. Use protocol observations
|
||||
for an independently written adapter, or resolve source licensing before a port.
|
||||
The MIT TCP driver and GPL XRLinuxDriver remain explicitly licensed references.
|
||||
|
||||
Also inspected XRLinuxDriver commit
|
||||
[`05d0cfb`](https://github.com/wheaney/XRLinuxDriver/commit/05d0cfb86abaacbf073e1a285ad4eabf681ccb05).
|
||||
It moves per-device pitch adjustment into the core pose path, after coordinate
|
||||
conversion; it does not add an Android transport or new One control protocol.
|
||||
Its pitch constants should not replace the GT mapping already verified by the user.
|
||||
|
||||
Next hardware check remains read-only: enumerate the 1S USB interfaces, display
|
||||
and Ethernet link; attempt bounded reads from both TCP services, save the factory
|
||||
configuration, then capture stationary/yaw/pitch/roll data. No XREAL support is
|
||||
advertised in this build. Android Ethernet availability and 1S control compatibility
|
||||
are still unverified.
|
||||
|
||||
## Official XREAL documentation and SDK inspection
|
||||
|
||||
Checked the user's [official documentation](https://docs.xreal.com/) on
|
||||
2026-10-14. This establishes an additional integration candidate alongside the
|
||||
direct TCP driver; it does not make that candidate a drop-in Android controller.
|
||||
|
||||
* The [device page](https://docs.xreal.com/XREALDevices/XREAL%20Glasses) explicitly
|
||||
includes One, One Pro and 1S in the X1-based One family with rotational tracking.
|
||||
* [SDK 3.1.0 release notes](https://docs.xreal.com/Release%20Note/XREAL%20SDK%203.1.0)
|
||||
add positional tracking for the One series with XREAL Eye and updated firmware.
|
||||
Beam Pro and Samsung S25 are its listed tested hosts. This does not establish
|
||||
6DoF on a camera-less 1S or validation on the user's SM-F976U.
|
||||
* [ControlGlasses](https://docs.xreal.com/Tools/ControlGlasses) is a required server
|
||||
for official SDK apps on compatible Android phones. It supplies device services,
|
||||
3D switching and dynamically loaded algorithm libraries. Beam Pro uses MyGlasses
|
||||
instead. An SDK integration would have a companion-app/version dependency.
|
||||
* The [quickstart](https://docs.xreal.com/Getting%20Started%20with%20XREAL%20SDK)
|
||||
describes Unity XR Plugin integration, ARM64 and OpenGL ES3. The
|
||||
[dual-display guide](https://docs.xreal.com/Tools/Dual%20Screen%20Display)
|
||||
confirms separate phone/glasses views and continued glasses output while the
|
||||
phone runs another app. This demonstrates the desired behavior in the SDK;
|
||||
it does not itself supply a native Kotlin implementation.
|
||||
* The [compatibility page](https://docs.xreal.com/XREALDevices/Compatibility)
|
||||
currently flags unresolved Android 16 compatibility. The
|
||||
[FAQ](https://docs.xreal.com/Frequently%20Asked%20Questions) also says DeX can
|
||||
interfere with SDK display operation, and Android 16 USB permission may require
|
||||
camera permission first when the device exposes an enabled UVC interface. Apply
|
||||
that permission requirement only when relevant; ordinary tracking must not
|
||||
start camera capture or enable UVC automatically.
|
||||
|
||||
Downloaded, but did not install or bundle, the official release archive linked
|
||||
from the 3.1.0 release notes:
|
||||
`https://public-resource.xreal.com/download/XREALSDK_Release_3.1.0.20251125/com.xreal.xr.tar.gz`.
|
||||
SHA-256: `fd7d0fce103712e7acc85bfcbb8b127007e1d84f64596efb6c11580bea058d90`.
|
||||
Size: 248,169,074 bytes.
|
||||
|
||||
Package inspection found Android AARs including `nr_api` and `nr_loader`, ARM64
|
||||
`libXREALXRPlugin.so` / `libXREALNativeSessionManager.so`, and Java
|
||||
`com.xreal.sdk.display.GlassesDisplay` plus display/surface listeners. The C# layer
|
||||
calls native exports for tracking type, resolution, refresh rate, component poses
|
||||
and session lifecycle; the loader also integrates Unity subsystems. Presence of
|
||||
these libraries does not establish a supported standalone JNI API or permission
|
||||
to redistribute a stripped subset. No C headers were present in the archive file
|
||||
listing inspected. No SDK source or binaries were added to the app.
|
||||
|
||||
Decision: retain both candidates. Before choosing the official native-library
|
||||
route, establish non-Unity initialization, head-pose access, ControlGlasses
|
||||
compatibility, display-surface ownership and distribution terms. Before choosing
|
||||
direct TCP, establish the Android network link and 1S protocol behavior. Compare
|
||||
both against DroidDeck's existing compositor instead of replacing its rendering
|
||||
architecture merely to fit the SDK's Unity sample.
|
||||
|
||||
## Native 1S implementation and hardware validation
|
||||
|
||||
The pre-implementation checkpoint is `4b045519`. The new driver uses independently
|
||||
written Kotlin TCP framing, control transactions, sensor parsing and gyro/gravity
|
||||
fusion. It does not bundle XREAL SDK binaries or copy the unlicensed xr-tools
|
||||
source. Protocol observations came from the references above and
|
||||
[xr-tools](https://github.com/0xcaff/xr-tools), inspected commit
|
||||
`65c11105434a998f09ac4a5f8e4de8736bb4d36c`.
|
||||
|
||||
Verified on the connected SM-F976U (Android reports release 17):
|
||||
|
||||
* Normal 1S USB identity is `3318:043e`, product name `XREAL 1S`. Only that exact
|
||||
product enables this driver; Air, One Pro and alternate/recovery IDs are excluded.
|
||||
* Android exposes NCM/ECM Ethernet networks at `169.254.2.10/24` and
|
||||
`169.254.1.10/24`. Individual sockets bound to the first network reach the
|
||||
glasses at `169.254.2.1`, control port 52999 and report port 52998. The process
|
||||
default network remains unchanged, preserving Steam's Internet connectivity.
|
||||
* Requests require the high bit of the transaction ID; responses clear that bit.
|
||||
Replies must match both command and transaction before values are published.
|
||||
* GET_DISPLAY returned payload `22 02 10 1d`: mode **29**, with Android reporting
|
||||
**1920 x 1200 at 60 Hz**. This differs from the older One's mode 2 (1080 lines).
|
||||
The 1S default 60 Hz mapping preserves mode 29.
|
||||
* A two-second capture delivered **2,000 valid IMU poses (1,000 Hz)**. Reports
|
||||
use the 128-byte body, type 11, little-endian device nanoseconds and float
|
||||
radians/sec / m/s² format. The framing handles partial and coalesced TCP reads.
|
||||
|
||||
The driver plugs into the shared glasses display and head-pose paths, including
|
||||
SteamVR. The FOV calculation now uses the actual per-eye aspect ratio, including
|
||||
the 1S's 16:10 display. Controls use acknowledged commands, bounded queues and
|
||||
timeouts; connection failures clear tracking state and retry. Volume is hidden
|
||||
because no volume protocol has been established. Missing Ethernet is reported
|
||||
without requesting HID permission or claiming USB interfaces from Android.
|
||||
|
||||
Directed wearer captures on the same device established the 1S sensor basis:
|
||||
level acceleration is approximately `(0, -9.65, -1.07)` m/s²; turning left gives
|
||||
negative gyro Y; looking up gives positive gyro X; tilting toward the right
|
||||
shoulder gives positive gyro Z with negative acceleration X. Both vectors are
|
||||
mapped as `(x, -y, -z)` into the shared right/up/back basis before initial gravity
|
||||
alignment and fusion. No RayNeo GT mounting angle is reused. Regression tests
|
||||
exercise those measured signs through `HeadPose`, check an upright first pose
|
||||
before recenter, and preserve the device's radians/sec and nanosecond units.
|
||||
|
||||
Stereo switching was verified at 3840 x 1080 / 60 Hz, returning to native
|
||||
1920 x 1200. On 1S, SET_INPUT performs the EDID switch itself; immediately sending
|
||||
SET_DISPLAY returns busy status 5001 (`22 03 08 89 27`). The driver now polls
|
||||
GET_DISPLAY across the temporary empty reply (`22 00`) and only sends a second
|
||||
mode command if the settled mode differs. Hardware retesting kept the connection
|
||||
and sensor stream running through the switch. The busy status is never accepted
|
||||
as a successful acknowledgement.
|
||||
|
||||
After installing the corrected build on 2026-10-14, the wearer confirmed that
|
||||
the view and motion feel good in response to the upright-view, yaw, pitch and
|
||||
head-tilt check. The SteamVR-requested session was running with a rendered first
|
||||
frame, stereo output and active tracking. All 181 glasses, shared-pose, SteamVR
|
||||
launch and glasses-UI regression checks passed; packaged runtime inputs verified.
|
||||
|
||||
The wearer also confirmed on 2026-10-14 that XREAL 1S brightness control works.
|
||||
Higher refresh rates were not tested.
|
||||
|
||||
Still experimental: dimmer, 90/120 Hz compatibility modes and
|
||||
extended gameplay stability need hardware validation. One successful view/motion
|
||||
check does not establish long-session stability. Users must select Follow mode and disable the glasses' own
|
||||
stabilization. Factory calibration and XREAL Eye/6DoF are not implemented.
|
||||
One Pro is not enabled or claimed as tested.
|
||||
Reference in new issue
Block a user