Add app-wide 3DoF for VITURE, RayNeo and XREAL glasses and 6DoF for Luma Ultra (#1035)

* Add VITURE glasses support: hardware control, 3D, head tracking

Glasses get their own controller over the MCU (permission, 120 Hz default, brightness, volume, sunblock, 2D/3D) and the game moves onto them when plugged in. The IMU drives Cardboard head tracking and the SteamVR driver. 3D falls back to one eye's view on phones that can't send the 3840x1080 signal. Options open from a glasses button in game and on the Steam page.

* Drop the glasses settings like Steam's: Play-style button, Settings lip, tabs

* Hang the glasses Settings lip over the top of the button

* Space glasses MCU commands so the IMU rate sticks, fit fixed resolutions to the glasses

* Keep Steam alive with SteamVR registered, run SteamVR at the glasses' rate, keep the glasses lip on top

* Fit VR games on the glasses: full view, seated origin at the head, pad hands reach with the d-pad

* Let the pad hands reach a metre out so buttons with deep colliders press

* Show every VR frame in order at 120 fps: wait for SteamVR's GPU work, present only finished copies, skip the grace on idle shared counters

* Glasses: SteamVR starts and stops with them, closer view with a field of view setting, Recenter on the pad

* Glasses: SteamVR field of view defaults to 90°

* VR pad hands: sticks move and aim the hands, left stick click switches the sticks to the game, d-pad pulls a held hand

* Glasses: Play SteamVR beside the glasses button instead of starting SteamVR with them, pad hands drawn as Index controllers and off while other VR controllers are connected, glasses sheet tabs sized like Steam's

* Track glasses mode, refresh-aware glasses window, align Play SteamVR with Play Steam

* Fix 3D on the glasses: full-res SteamVR presenter, revert VR 3D at session end, coalesce vsync ticks, no session restart after end

* Address review: glasses output after installs and session end, VR revert retry, display fallback, presenter swapchain retry, agent hex checks, drawer bumpers, glasses translations, docs

* Harden VITURE stereo negotiation and output refresh tracking

* Localize stereo recovery feedback and record Astra reconnect tests

* Improve test failure diagnostics and refresh Astra firmware findings

* Handle VITURE docks separately and reject truncated USB frames

* Glasses: remember a 3D fallback per model, docks as display only, VR timing fixes

- A 3D attempt that ends at the 640x480 fallback is remembered for that glasses
  model, so Play SteamVR no longer blacks out the glasses after every restart;
  Retry clears it and "3D for VR games" stays reachable
- A VITURE Mobile Dock on its own no longer counts as glasses; through a dock
  Play SteamVR is hidden and the sheet shows the output and why there is no control
- HDR is probed on the glasses when the session starts there
- Glasses frames with a mismatched checksum are marked in the status frame log
- SteamVR layer and driver vblank counters stay monotonic across refresh changes;
  one-eye render scale applies from activation and on mono/3D switches
- Compositor binds announce the refresh clients were last told; the presenter
  retries a failed first swapchain
- Dead strings and helpers removed; docs updated

* Glasses: drop USB frames with a bad checksum

* Glasses: count frames dropped for a bad checksum

* Feat/viture xr (#907)

* Add RayNeo Air and GT support to the XR glasses pipeline

* Add RayNeo Air and GT support to the XR glasses pipeline

* Checkpoint glasses tracking, calibration, and SteamVR launch integration

* Checkpoint glasses tracking, calibration, and SteamVR launch integration

* Add XREAL 1S support and correct GT and Pro 2 tracking

* Add XREAL 1S support and correct GT and Pro 2 tracking

* Fix mirrored glasses tracking and recover SteamVR after device loss

* Fix mirrored glasses tracking and recover SteamVR after device loss

* Document confirmed XREAL 1S brightness control

* Document confirmed XREAL 1S brightness control

* Fix SteamVR glasses defaults and rendering controls

* Fix SteamVR glasses defaults and rendering controls

* chore: keep only the 3 newest session log folders (#852)

* chore: add agent comment rules (#748)

Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com>

* Fix SteamVR polling, VITURE checksums, and fence tracking

Keep manual SteamVR launches polled, discard corrupt VITURE packets, and retire Vulkan fences out of order without blocking submissions at a fixed capacity. Preserve the contiguous completion watermark used by the presenter.

Fix optional Windows launcher configuration and audited helper staging. Add Android, Windows, and native tracker regression coverage.

* Fix SteamVR polling, VITURE checksums, and fence tracking

Keep manual SteamVR launches polled, discard corrupt VITURE packets, and retire Vulkan fences out of order without blocking submissions at a fixed capacity. Preserve the contiguous completion watermark used by the presenter.

Fix optional Windows launcher configuration and audited helper staging. Add Android, Windows, and native tracker regression coverage.

* fix: wireless ADB page names the child-process switch the way Android shows it (#848)

* l10n: refresh stale and untranslated strings in es, fr, ko, ru (#850)

* l10n: refresh stale and untranslated strings in es, fr, ko, ru

- mode_added_art_hint: now mentions Steam then SteamGridDB (all four locales).
- wincomp_needs_installer / wincomp_installers_note: es and ko still described
  the old "coming soon" behaviour; align with the current English text.
- summary_remove_game: was still English in all four locales.

Machine-assisted translation; native-speaker review is welcome.

* l10n(es): say registro, not grabación, for the missing installer record

---------

Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com>

* fix: a failed cloud save recovery no longer ends the app (#846)

The cloud-recovery thread started at every app start had nothing to
catch an error, so one went to CrashHandler, which halts the process
with status 2. In unit tests the thread can outlive the test that
started it, fail on the torn-down preferences, and take the whole test
run down. The error is now logged and the thread ends.

Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com>

* fix: an interrupted Linux runtime download resumes instead of starting again (#847)

* fix: an interrupted Linux runtime download resumes instead of starting again

installOnce() asked the downloader to resume but always deleted the archive in its finally block, so the next attempt began at zero. The archive is now named after the release version and checksum, a failed download keeps what it fetched, and archives of other releases (and the old fixed name) are removed. A checksum or extraction failure still deletes the file.

* test: Downloader resumes a cut-off download from the bytes it kept

Starts a socket server that promises the whole file, hangs up part way and then answers a Range request, and checks that the second downloadFile call resumes with the right offset and ends with identical bytes.

* fix: a resumed download discards a 206 that does not start where the kept bytes end

Downloader appended every 206 response to the partial file without
looking at Content-Range. A server or proxy that answers a Range request
with a different offset would leave a corrupt file. Check that the range
starts at the kept length; otherwise delete the partial file and start
again without a Range header.

Adds a test with a server that answers the Range request from offset 0.

---------

Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com>

* l10n: German, French, Spanish, Italian, Portuguese and Russian for the zRAM, reset-settings and store free-space strings (#849)

* l10n: German, French, Spanish, Italian, Portuguese and Russian for the zRAM, reset-settings and store free-space strings

Adds the strings values-de, values-it and values-pt were missing (14 each) and the six that values-fr, values-es and values-ru were missing. Machine-assisted drafts that follow each file's existing register and terms; native review is welcome.

* l10n: correct missing active swap translations

---------

Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com>
Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com>

* fix: droiddeckctl reaches variant-package builds (recover #842) (#854)

fix: droiddeckctl reaches builds shipped under a variant package

The provider authority and the start Activity's component were fixed to
com.droiddeck.launcher, so droiddeckctl could not talk to the antutu,
ludashi or pubg builds from tools/release/variants.txt. DROIDDECK_PACKAGE
now picks the package; the Activity's class name stays the same in every
variant.

Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com>

* feat: droiddeckctl reports on-screen pacing (recover #843) (#855)

feat: droiddeckctl pacing reports the frames that reached the screen

Reads SurfaceFlinger's timestats for the game's layer (or the session's
SurfaceView) over a capture: frames on screen, fps on screen, the
present-to-present histogram and how many intervals were off the usual
cadence. Records split by a refresh-rate change are summed. A game's own fps
counter counts frames handed over, so it can't show dropped or doubled frames.

Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com>

* fix: Steam can restart more than twice in a session (#754)

* fix: Steam can restart more than twice in a session

Every exit 42 counted toward the cap that stops an update which will
not install, and the count was never cleared, so the third restart of
a session ended it however far apart they were. A client that ran for
a minute before asking is not that loop: the count starts again.

* fix: measure the client's run time as it exits

The time spent collecting logs and checking FEX after the client exits
counted toward its run, so a slow cleanup could make a quick exit 42
look like a long one and clear the count.

---------

Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com>
Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com>

* fix: the front end keeps its page when the Fold is folded or unfolded (#832)

* fix: the front end keeps its page when the Fold is folded or unfolded

MainActivity, FileManagerActivity, FilePickerActivity and StoreLoginActivity did not declare the screen-size configuration changes, so folding or unfolding a foldable recreated them and dropped their in-memory state. They now handle orientation, screenSize, screenLayout, smallestScreenSize and density themselves, like SessionActivity already does. uiMode is left out on purpose.

* fix: store login keeps rebuilding its toolbar when the display density changes

StoreLoginActivity builds its toolbar in code with dp-to-pixel values taken at creation time and has no onConfigurationChanged. It no longer declares density in configChanges, so a density change recreates it as before this PR. Size and layout changes are still handled in place.

---------

Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com>

* Release pointer buttons without replaying stale motion (#803)

* Fix cursor warping on native touch down

* Preserve input cleanup without replaying stale pointer motion

---------

Co-authored-by: siyuan.qsy <siyuan.qsy@alibaba-inc.com>

* l10n: translate the strings added since #284 into zh-CN, zh-HK, zh-TW and ja (#539)

* l10n: translate the 184 strings added since #284 (zh-CN/HK/TW, ja)

The stores, cloud saves, Windows components, Steam touch, desktop
removal and background-downloads strings landed after #284's
translation snapshot, so every one of them fell back to English.
Translate all 184 into Simplified Chinese, Traditional Chinese
(Taiwan and Hong Kong) and Japanese, keeping each locale file's own
vocabulary (工作階段/會話, 解除安裝/卸載, 元件/組件, 設定).

Also fix four Hong Kong strings that kept the simplified 着色器.

* l10n(zh): fix stale and ambiguous strings found in review

Severe: summary_remove_game was untranslated English in all three
files; mode_added_art_hint, wincomp_installers_note and
wincomp_needs_installer still described the pre-#480/#492 English.

Also: distinguish the Setup wizard from Android system settings in
zh-HK/zh-TW (初始設定 / 系統設定 / Android 設定), unify library as
收藏庫, component as 組件 in zh-HK, session as 會話 in zh-HK, account
as 帳戶 in zh-HK, Trending as 趋势/趨勢, Forget as 清除, reword the
re-applied-at-launch, action-failed and already-in-Games strings, and
play as 執行/開啟遊戲 in zh-TW/zh-HK.

* l10n(ja): fix stale and ambiguous strings found in review

---------

Co-authored-by: Dantalian <Dantalian-dev@users.noreply.github.com>
Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com>

* Rebuild and verify Windows VR runtime components

* Rebuild and verify Windows VR runtime components

* Add top, bottom, and centered picture positions for Steam sessions (#871)

* fix: dependency scan ignores folders above the game folder (#859)

* DroidDeck-Proton: get releases from the Droid-Deck org repo

DroidDeck-Proton is now built and released on Droid-Deck/DroidDeck-Proton
(11-9.1 is Latest there), so the Components page, the update check and the
guest installer read releases from the org repo instead of The412Banner's.

* feat: low-latency controller and touch toggles in the session drawer (#808)

* fix: SteamGridDB lookups no longer stick as "no match" after a failure; add a search to the art chooser (#876)

* feat: Steam home wall fills from the signed-in library instead of store picks (#878)

* test: cover GPU family and support classification (#880)

* Fix local builds on Linux failing after the first Comet build (#866)

* feat: sort the store library and installed tabs, and switch between card and list views (#798)

* feat: three-finger tap toggles Steam's on-screen keyboard in game mode (#782)

* fix: keep Quest-patched Turnip as the auto default (#853)

* Fix deploy_local.sh and droiddeckctl not finding a phone connected over wireless debugging (#864)

* fix: keep proot sessions alive when the kernel loses fork events (#457)

* fix(quest): correct desktop and Steam display corruption across sessions (#733)

fix(quest): stabilize desktop and Steam layer presentation

Keep Quest SDR scenes and the black backdrop off the Android Vulkan swapchain regardless of session order or HDR preferences. Only report the first frame after client content is submitted.

Add backdrop lifecycle and first-frame regression coverage. Record the user's successful Quest 3 Test 2 result.

Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com>

* fix: keep SteamVR syscall hooks and report Steam crashes (#905)

* fix: preserve SteamVR syscall hooks

* fix: report Steam crashes through gamescope

* fix: honor glasses pacing and resolution in SteamVR launches

---------

Co-authored-by: connor g <connorsspam1992@gmail.com>
Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com>
Co-authored-by: Yassine Gherbi <y.gherbi.dev@gmail.com>
Co-authored-by: Resul Övür <92828257+ovurrsl@users.noreply.github.com>
Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com>
Co-authored-by: Kirk <kirtoy.junsay@gmail.com>
Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com>
Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com>
Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com>
Co-authored-by: asdf17128 <asdf17128@gmail.com>
Co-authored-by: siyuan.qsy <siyuan.qsy@alibaba-inc.com>
Co-authored-by: Dantalian <wsu091227@qq.com>
Co-authored-by: Dantalian <Dantalian-dev@users.noreply.github.com>
Co-authored-by: The412Banner <the412banner@users.noreply.github.com>
Co-authored-by: The412Banner <205237651+The412Banner@users.noreply.github.com>
Co-authored-by: FX1922 <120828496+FX192@users.noreply.github.com>
Co-authored-by: Muaz Kadan <32595066+muazkadan@users.noreply.github.com>
Co-authored-by: kingokksa <47888451+kingokksa@users.noreply.github.com>
Co-authored-by: Jamie Schenker <jam1esch3nk3r@gmail.com>
Co-authored-by: Jagger15 <jagger2333@live.com>

* Glasses: keep a pending 2D restore and the SteamVR recovery offer, show Play SteamVR only with a picture

* Translate the glasses calibration, XREAL and SteamVR recovery strings

* SteamVR recovery: give a finished teardown time to collect logs before restarting

* Credit the projects the XR glasses support builds on

* Joy-Cons: SteamVR hands from their gyro while SteamVR runs, one combined pad otherwise

* Add experimental Luma tracking and stabilize SteamVR launches

Add native Luma Ultra spatial tracking, optical controllers, and stereo
calibration with DroidDeck styling, imperial/metric measurements, and
pending recenter recovery. Include the pinned hand model and calibration board.

Keep live guest and SDK files outside Android's evictable cache. Filter the
Adreno 840 sysmem override in the shared SteamVR helper, serialize timeline
mirroring, retain invalid optical controllers without event floods, and
prevent an unprivileged scheduling fallback from lowering compositor RTTIME.

Validation: latest APK build and runtime freshness checks pass; 247 affected
glasses/calibration tests pass. Native VR, helper, and packaging regressions
pass. Two earlier phone library launches sustained 60 fps for 308 and 188
seconds without recovery or restart. The newer calibration APK is installed
and checksum-verified; calibrated-hand and sustained gameplay acceptance
remain pending. One existing Android test requires Windows symlink privileges.

* Headsets: settings badge on Play SteamVR instead of a top-right button, both shown only once connected

* Hand tracking: camera-tracked Joy-Cons and bare hands in SteamVR, drawn as white gloves

- Joy-Con movement from the accelerometer (report reordering, per-Joy-Con calibration, zero-velocity updates)
- MediaPipe pose and hand landmarks from the device camera, placement check, per-hand Kalman filter with the Joy-Con
- No controller connected: bare camera hands with Quest-style gestures, 1€ filtered, rendered as a white glove model built from a distance field
- SteamVR safe mode cleared before each start so one crash no longer blocks the driver

* Improve Luma Ultra tracking and isolate device-specific controls

Refine stereo hand and Joy-Con tracking, gesture release, calibration,
recovery UI, diagnostics, display modes, and compositor handling.
Keep Ultra controls and input routing behind a live SDK-confirmed Carina
connection; clear retained spatial mode when other glasses connect.
Add optional UVC transport with pinned dependencies and packaged notices.

Validation: 456 Android tests, ARM64 VR build and native regressions,
47 tooling tests, SDK/UVC-enabled and disabled builds, and 150 packaged
runtime checksums. Two shader compilation tests skipped because the
compiler is unavailable. Hardware interaction gates remain experimental.

* Delay Luma Ultra 6DoF until SteamVR settles

Suspend spatial SDK startup while SteamVR launches, then restore it two seconds
after RUNNING is confirmed. Preserve saved tracking preferences and automatically
recenter same-device restoration, including cancellation and early stop.

Discard superseded launch requests and status replies after session teardown.

Validation: 128 focused JVM tests passed on the isolated commit snapshot; 129
passed with the current local hand-tracking work. Debug build and runtime checks
passed, and the reviewed APK was installed and hash-verified on the phone.

* Clarify glasses and phone camera tracking options

Separate Luma Ultra camera controls from phone-camera tracking so the two
hand-tracking switches have clear sources. Label bare-hand controllers and
Joy-Con position tracking explicitly, explain physical Joy-Con inputs and
camera priority, and update the camera prompt and calibration guidance.

Use fresh phone section and toggle labels so existing translations do not
hide the distinction. Preserve tracking behavior and saved preferences.

Validation: 39 existing UI tests, debug build, and packaged runtime verification
passed. Reviewed changes and installed the APK on the connected phone.

* Fix tracking handoff and shared runtime regressions

Release spatial ownership and controller overrides when output returns to the phone. Preserve USB-loss recovery, keep Vulkan submissions asynchronous, clean persistent guest scratch on runtime removal, and clarify the 6DoF head-tracking control.

* Apply smart-glasses virtual screens across DroidDeck menus

* Finalize Luma hand baseline and document smart-glasses support

Keep raw grayscale-to-RGB Full hand inference, wrist-only Joy-Con reconstruction, bounded visual holds, and the dead-zone rollback. Retain opt-in steady aim, recenter controls, reproducible Lite models, and the JPEG-padding fix.

Remove unsuccessful benchmark-only and USB transport experiments. Document the cross-brand 3DoF matrix, Luma Ultra 6DoF, SDK requirements, and app-wide screen behavior.

* Fix executable permission for SteamVR launcher

* Address hand tracking and VR review findings

* Bound stereo recovery test image comparison memory

* Localize smart-glasses setup and tracking diagnostics

Move calibration controls, status messages, validation errors, distance labels,
and printed-board instructions into Android string resources. Add and update
machine-assisted translations in all 32 supported non-English locale catalogs,
with corrections for recentering, hand tracking, product names, and measurements.

Keep print instructions within the page using text layout that supports wrapping
and right-to-left text. Correct the Lite model toggle description to reflect its
Lite palm detector and Lite landmark model.

Add locale coverage, format-argument, line-break, and measurement checks, plus
French and Arabic calibration UI tests. Preserve tracking calculations and the
shared runtime storage behavior.

Validation: Android resource and Kotlin compilation; 3 localization checks;
68 focused tests; 1,597 JVM tests at a 512 MB heap; French and Arabic UI renders.

* Correct smart-glasses translations from PR review

Fix the 42 localization findings covering hand and head terminology, disabled states, calibration dimensions, camera visibility, capture rejection, and menu references. Correct the repeated capture-error ambiguity in related locales and clarify its English source. Keep Romanian setup instructions consistent with the existing informal voice.

Extend localization measurement validation to distance hints and retry guidance. Verify the recovery screen's palette-dependent text color with one row-sized pixel buffer and guaranteed bitmap cleanup.

Validation: Android resource and Kotlin compilation; 3 localization checks across all supported locales; 68 focused JVM tests at a 512 MB heap.

* Work around SteamVR device loss with Adreno 840 Turnip v0.2.0

* Withdraw ineffective SteamVR flushall workaround

---------

Co-authored-by: maxjivi05 <maxjividen@gmail.com>
Co-authored-by: MaxsTechReview <6025299+maxjivi05@users.noreply.github.com>
Co-authored-by: connor g <connorsspam1992@gmail.com>
Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com>
Co-authored-by: Yassine Gherbi <y.gherbi.dev@gmail.com>
Co-authored-by: Resul Övür <92828257+ovurrsl@users.noreply.github.com>
Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com>
Co-authored-by: Kirk <kirtoy.junsay@gmail.com>
Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com>
Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com>
Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com>
Co-authored-by: asdf17128 <asdf17128@gmail.com>
Co-authored-by: siyuan.qsy <siyuan.qsy@alibaba-inc.com>
Co-authored-by: Dantalian <wsu091227@qq.com>
Co-authored-by: Dantalian <Dantalian-dev@users.noreply.github.com>
Co-authored-by: The412Banner <the412banner@users.noreply.github.com>
Co-authored-by: The412Banner <205237651+The412Banner@users.noreply.github.com>
Co-authored-by: FX1922 <120828496+FX192@users.noreply.github.com>
Co-authored-by: Muaz Kadan <32595066+muazkadan@users.noreply.github.com>
Co-authored-by: kingokksa <47888451+kingokksa@users.noreply.github.com>
Co-authored-by: Jamie Schenker <jam1esch3nk3r@gmail.com>
Co-authored-by: Jagger15 <jagger2333@live.com>
This commit is contained in:
213 files changed
+22572 -919

No files matched your search

+69
View File
@@ -0,0 +1,69 @@
"""Build the pinned lite-palm/lite-landmark MediaPipe bundle (mediapipe==0.10.21)."""
import argparse
import hashlib
import io
from pathlib import Path
import zipfile
FULL_SHA = 'fbc2a30080c3c557093b5ddfc334698132eb341044ccee322ccf8bcf3607cde1'
LITE_SHA = 'd7fde8ac11f8ce03f8663775bfc323f4fc9f2a38062b4f4efa142874ef5b2a48'
PALM_SHA = 'e9a4aaddf90dda56a87235303cf00e4c2d3fb28725f68fd88772997dac905c18'
def write_entry(archive, name, data):
entry = zipfile.ZipInfo(name, (1980, 1, 1, 0, 0, 0))
entry.create_system = 3
entry.external_attr = 0o100644 << 16
archive.writestr(entry, data)
def canonicalize_associated_files(data):
with zipfile.ZipFile(io.BytesIO(data)) as archive:
entries = archive.infolist()
if not entries:
return data
prefix = data[:min(entry.header_offset for entry in entries)]
files = {entry.filename: archive.read(entry) for entry in entries}
output = io.BytesIO(prefix)
with zipfile.ZipFile(output, 'a', compression=zipfile.ZIP_STORED) as archive:
for name in sorted(files):
write_entry(archive, name, files[name])
return output.getvalue()
def build(full, lite, palm, output):
import mediapipe
from mediapipe.tasks.python.metadata import metadata
from mediapipe.tasks.metadata import schema_py_generated as schema
if mediapipe.__version__ != '0.10.21':
raise ValueError('Use mediapipe==0.10.21 for reproducible metadata')
for path, expected in [(full, FULL_SHA), (lite, LITE_SHA), (palm, PALM_SHA)]:
if hashlib.sha256(path.read_bytes()).hexdigest() != expected:
raise ValueError(f'Model source checksum mismatch: {path}')
with zipfile.ZipFile(full) as task:
references = {name: task.read(name) for name in ['hand_detector.tflite', 'hand_landmarks_detector.tflite']}
def contract(data):
graph = schema.Model.GetRootAsModel(data, 0).Subgraphs(0)
return [[(graph.Tensors(int(i)).Name(), graph.Tensors(int(i)).ShapeAsNumpy().tolist(), graph.Tensors(int(i)).Type())
for i in indices] for indices in (graph.InputsAsNumpy(), graph.OutputsAsNumpy())]
models = {}
for name, path in [('hand_detector.tflite', palm), ('hand_landmarks_detector.tflite', lite)]:
data = path.read_bytes()
reference = references[name]
if contract(data) != contract(reference):
raise ValueError(f'Lite tensor contract differs: {name}')
populator = metadata.MetadataPopulator.with_model_buffer(data)
populator.load_metadata_and_associated_files(reference)
populator.populate()
models[name] = canonicalize_associated_files(populator.get_model_buffer())
output.parent.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(output, 'w', compression=zipfile.ZIP_STORED) as task:
for name, data in models.items():
write_entry(task, name, data)
print(hashlib.sha256(output.read_bytes()).hexdigest())
if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--full', type=Path, default=Path('app/src/main/assets/models/hand_landmarker.task'))
parser.add_argument('--lite', type=Path, required=True, help='Pinned hand_landmark_lite.tflite from storage.googleapis.com/mediapipe-assets/')
parser.add_argument('--palm', type=Path, required=True, help='Pinned palm_detection_lite.tflite from storage.googleapis.com/mediapipe-assets/')
parser.add_argument('--output', type=Path, default=Path('app/src/main/assets/models/hand_landmarker_lite.task'))
args = parser.parse_args()
build(args.full, args.lite, args.palm, args.output)
+179
View File
@@ -0,0 +1,179 @@
"""Build checksum-pinned Android RGB UVC libraries and their fault-injection test."""
import argparse
import hashlib
import json
import os
from pathlib import Path
import shutil
import subprocess
import tarfile
import urllib.request
import zipfile
ROOT = Path(__file__).resolve().parents[1]
RECIPES = ROOT / 'tools/uvc'
PACKAGES = (
{'name': 'libusb', 'version': '1.0.30', 'license': 'LGPL-2.1-or-later',
'url': 'https://github.com/libusb/libusb/releases/download/v1.0.30/libusb-1.0.30.tar.bz2',
'archive': 'libusb-1.0.30.tar.bz2',
'sha256': 'fea36f34f9156400209595e300840767ab1a385ede1dc7ee893015aea9c6dbaf'},
{'name': 'libuvc', 'version': '0.0.8', 'license': 'BSD-3-Clause',
'url': 'https://github.com/libuvc/libuvc/archive/refs/tags/v0.0.8.zip',
'archive': 'libuvc-0.0.8.zip',
'sha256': '67922938ad81eb82b8ce14576f15a68ac75e30af738728a1f3070b6bbc938bce'},
)
def sha256(path):
return hashlib.sha256(path.read_bytes()).hexdigest()
def download(package, cache):
archive = cache / package['archive']
if not archive.exists():
request = urllib.request.Request(package['url'], headers={'User-Agent': 'DroidDeckXR-UVC-Build'})
with urllib.request.urlopen(request, timeout=60) as response:
payload = response.read()
if hashlib.sha256(payload).hexdigest() != package['sha256']:
raise ValueError(f"Downloaded checksum mismatch: {package['archive']}")
archive.write_bytes(payload)
if sha256(archive) != package['sha256']:
raise ValueError(f'Cached checksum mismatch: {archive}')
return archive
def extract(archive, destination):
if archive.suffix == '.zip':
with zipfile.ZipFile(archive) as bundle:
for item in bundle.infolist():
(destination / item.filename).resolve().relative_to(destination.resolve())
if (item.external_attr >> 16) & 0o170000 == 0o120000:
raise ValueError('Symlink in ZIP archive')
bundle.extractall(destination)
else:
with tarfile.open(archive) as bundle:
bundle.extractall(destination, filter='data')
def patch_source(source, patch):
old, new = [], []
content = source.read_text(encoding='utf-8')
hunks = []
for line in patch.read_text(encoding='utf-8').splitlines(keepends=True):
if line.startswith(('--- ', '+++ ')):
continue
if line.startswith('@@ '):
if old or new:
hunks.append((''.join(old), ''.join(new)))
old, new = [], []
elif line.startswith(' '):
old.append(line[1:])
new.append(line[1:])
elif line.startswith('-'):
old.append(line[1:])
elif line.startswith('+'):
new.append(line[1:])
else:
raise ValueError(f'Unsupported patch line: {line!r}')
hunks.append((''.join(old), ''.join(new)))
for before, after in hunks:
if not before or content.count(before) != 1:
raise ValueError(f'Patch context mismatch: {source}')
content = content.replace(before, after, 1)
source.write_text(content, encoding='utf-8', newline='\n')
def run(command, log):
result = subprocess.run([str(part) for part in command], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True, encoding='utf-8')
log.write_text(result.stdout, encoding='utf-8')
print(result.stdout, end='')
result.check_returncode()
def build(args):
output, cache = args.output.resolve(), args.cache.resolve()
source, binary, dist = output / 'src', output / 'android-arm64', output / 'dist'
for folder in (output, cache, source):
folder.mkdir(parents=True, exist_ok=True)
for package in PACKAGES:
extract(download(package, cache), source)
patch = RECIPES / 'libuvc-submit-failure.patch'
stream = source / 'libuvc-0.0.8/src/stream.c'
patch_source(stream, patch)
ndk, cmake = args.ndk.resolve(), args.cmake.resolve()
ninja = args.ninja.resolve() if args.ninja else cmake.parent / ('ninja.exe' if os.name == 'nt' else 'ninja')
configure = [cmake, '-S', RECIPES, '-B', binary, '-G', 'Ninja',
f'-DCMAKE_MAKE_PROGRAM={ninja.as_posix()}',
f'-DCMAKE_TOOLCHAIN_FILE={(ndk / "build/cmake/android.toolchain.cmake").as_posix()}',
f'-DRGB_UVC_SOURCE_ROOT={source.as_posix()}', '-DANDROID_ABI=arm64-v8a',
'-DANDROID_PLATFORM=android-26', '-DANDROID_STL=c++_static',
'-DCMAKE_BUILD_TYPE=RelWithDebInfo']
compile_command = [cmake, '--build', binary, '--parallel', str(args.jobs)]
run(configure, output / 'configure.log')
run(compile_command, output / 'build.log')
copies = {
binary / 'libusb.so': dist / 'arm64-v8a/libusb.so',
binary / 'libuvc.a': dist / 'arm64-v8a/libuvc.a',
binary / 'uvc_submit_failure_test': dist / 'arm64-v8a/uvc_submit_failure_test',
source / 'libusb-1.0.30/libusb/libusb.h': dist / 'include/libusb.h',
source / 'libuvc-0.0.8/include/libuvc/libuvc.h': dist / 'include/libuvc/libuvc.h',
binary / 'generated/libuvc/libuvc_config.h': dist / 'include/libuvc/libuvc_config.h',
source / 'libusb-1.0.30/COPYING': dist / 'licenses/libusb-COPYING',
source / 'libuvc-0.0.8/LICENSE.txt': dist / 'licenses/libuvc-LICENSE.txt',
}
for original, target in copies.items():
target.parent.mkdir(parents=True, exist_ok=True)
shutil.copyfile(original, target)
prebuilt = list((ndk / 'toolchains/llvm/prebuilt').iterdir())
if len(prebuilt) != 1:
raise ValueError(f'Expected one NDK host toolchain: {prebuilt}')
readelf = prebuilt[0] / 'bin' / ('llvm-readelf.exe' if os.name == 'nt' else 'llvm-readelf')
reports = []
for name in ('libusb.so', 'uvc_submit_failure_test'):
result = subprocess.check_output([str(readelf), '-h', '-l', '-d', str(binary / name)],
text=True, encoding='utf-8')
if 'AArch64' not in result or 'TEXTREL' in result:
raise ValueError(f'Unexpected ELF ABI or text relocations: {name}')
loads = [line for line in result.splitlines() if line.lstrip().startswith('LOAD')]
if not loads or any(not line.rstrip().endswith('0x4000') for line in loads):
raise ValueError(f'ELF must use 16 KiB LOAD alignment: {name}')
if name == 'uvc_submit_failure_test' and ('[libusb.so]' not in result or 'libglasses' in result):
raise ValueError('Fault test must depend on standalone libusb')
reports.append(result)
(output / 'elf-validation.txt').write_text('\n'.join(reports), encoding='utf-8')
manifest = {
'packages': PACKAGES,
'patch': {'path': str(patch.relative_to(ROOT)), 'sha256': sha256(patch),
'sourceSha256': sha256(stream),
'reason': 'Preserve USB submission errors; remove unsubmitted transfers under cb_mutex, then cancel/drain submitted transfers and join the callback thread.'},
'build': {'ndk': str(ndk), 'ndkSourceProperties': (ndk / 'source.properties').read_text(encoding='utf-8'),
'configure': [str(part) for part in configure], 'compile': [str(part) for part in compile_command],
'api': 26, 'abi': 'arm64-v8a', 'pic': True, 'queueBuffers': 8,
'isoPacketsPerTransferMaximum': 32, 'jpegConversion': False,
'libusbShared': True, 'libuvcStatic': True,
'recipeHashes': {str(path.relative_to(ROOT)): sha256(path) for path in
(Path(__file__), RECIPES / 'CMakeLists.txt', RECIPES / 'submit_failure_test.cpp')}},
'validation': {'nativeBuild': 'passed', 'linkNoUndefined': 'passed', 'machine': 'AArch64',
'loadAlignmentBytes': 16384, 'textRelocations': False,
'deviceRuntimeTest': 'not run by build script'},
'artifacts': [{'path': str(path.relative_to(output)), 'sha256': sha256(path), 'size': path.stat().st_size}
for path in copies.values()],
}
(output / 'provenance.json').write_text(json.dumps(manifest, indent=2) + '\n', encoding='utf-8')
print(f'Libraries and headers: {dist}')
print(f'Provenance: {output / "provenance.json"}')
if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--ndk', type=Path, required=True)
parser.add_argument('--cmake', type=Path, required=True)
parser.add_argument('--ninja', type=Path)
parser.add_argument('--output', type=Path, default=ROOT / 'build/rgb-uvc')
parser.add_argument('--cache', type=Path, default=ROOT / 'build/uvc-source-cache')
parser.add_argument('--jobs', type=int, default=4)
build(parser.parse_args())
+50
View File
@@ -1,6 +1,7 @@
#include "cardboard.h"
#include <math.h>
#include <stdio.h>
#include <string.h>
static int failures;
#define CHECK(c) do { if (!(c)) { printf("FAIL %s:%d: %s\n", __FILE__, __LINE__, #c); failures++; } } while (0)
@@ -54,10 +55,59 @@ int main(void) {
cardboard_push_constants(&q, pose, 3840, 1080, 3840, 1080, pc);
CHECK(near(pc[14], tanf(glasses_fovs[i] * 0.5f * (float)M_PI / 180.0f)));
}
const struct cardboard_params screen = {
.enabled = 1, .source = CARDBOARD_SOURCE_SCREEN, .ipd_m = 0.064f,
.screen_dist_m = 3.0f, .screen_width_m = 2.0f, .fov_deg = 25.8f,
};
const int source_sizes[][2] = {{1280, 720}, {1600, 900}, {1920, 1080}};
const int panel_heights[] = {1080, 1200};
for (unsigned panel = 0; panel < sizeof(panel_heights) / sizeof(panel_heights[0]); panel++) {
float native[24];
const int height = panel_heights[panel];
cardboard_push_constants(&screen, pose, 1920, 1080, 3840, height, native);
for (unsigned size = 0; size < sizeof(source_sizes) / sizeof(source_sizes[0]); size++) {
cardboard_push_constants(&screen, pose, source_sizes[size][0], source_sizes[size][1], 3840, height, pc);
for (unsigned field = 0; field < 24; field++) CHECK(near(pc[field], native[field]));
CHECK(near(pc[17], 16.0f / 9.0f));
CHECK(near(pc[18], CARDBOARD_SOURCE_SCREEN));
CHECK(near(pc[19], 1920.0f / height));
CHECK(near(pc[20], 0.0f));
}
}
q.source = CARDBOARD_SOURCE_FLAT;
cardboard_clamp(&q);
CHECK(q.source == CARDBOARD_SOURCE_FLAT);
for (unsigned panel = 0; panel < sizeof(panel_heights) / sizeof(panel_heights[0]); panel++) {
for (unsigned size = 0; size < sizeof(source_sizes) / sizeof(source_sizes[0]); size++) {
cardboard_push_constants(&q, pose, source_sizes[size][0], source_sizes[size][1], 3840, panel_heights[panel], pc);
CHECK(near(pc[17], 16.0f / 9.0f));
CHECK(near(pc[18], CARDBOARD_SOURCE_FLAT));
CHECK(near(pc[19], 1920.0f / panel_heights[panel]));
CHECK(near(pc[20], 0.0f));
}
}
q.fov_deg = 0.0f;
cardboard_clamp(&q);
CHECK(q.fov_deg > 0.0f && q.fov_deg < glasses_fovs[0]);
const float translation[3] = {0.25f, -0.3f, 0.5f};
q.source = CARDBOARD_SOURCE_SCREEN;
cardboard_push_constants(&q, pose, 1920, 1080, 3840, 1200, pc);
float original[24]; memcpy(original, pc, sizeof(pc));
cardboard_push_translation(&q, translation, pc);
for (int i = 0; i < 21; ++i) CHECK(near(pc[i], original[i]));
for (int i = 0; i < 3; ++i) CHECK(near(pc[21+i], translation[i]));
q.source = CARDBOARD_SOURCE_SBS;
cardboard_push_translation(&q, translation, pc);
CHECK(near(pc[21], 0) && near(pc[22], 0) && near(pc[23], 0));
q.source = CARDBOARD_SOURCE_FLAT;
cardboard_push_translation(&q, translation, pc);
CHECK(near(pc[21], 0) && near(pc[22], 0) && near(pc[23], 0));
q.source = CARDBOARD_SOURCE_SCREEN;
const float zero[3] = {0};
cardboard_push_translation(&q, zero, pc);
for (int i = 0; i < 24; ++i) CHECK(near(pc[i], original[i]));
if (failures) return 1;
printf("PASS: cardboard params\n");
return 0;
+205
View File
@@ -0,0 +1,205 @@
#!/usr/bin/env python3
"""Compile and render the tracked-menu GLES shaders with headless Mesa (opt-in)."""
import argparse
import ctypes as c
import hashlib
import json
import math
import os
import pathlib
import re
import struct
import zlib
def main():
os.environ.setdefault('EGL_PLATFORM', 'surfaceless')
os.environ.setdefault('LIBGL_ALWAYS_SOFTWARE', '1')
os.environ.setdefault('MESA_GLES_VERSION_OVERRIDE', '2.0')
root = pathlib.Path(__file__).resolve().parents[1]
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--output-dir', type=pathlib.Path, default=root / 'build/glasses-menu-shader')
args = parser.parse_args()
out = args.output_dir.resolve()
out.mkdir(parents=True, exist_ok=True)
source_path = root / 'app/src/main/java/com/droiddeck/launcher/glasses/GlassesMenuSurface.kt'
source = source_path.read_text()
shaders = {name: re.search(r'private const val '+name+r' = """(.*?)"""', source, re.S).group(1).encode()
for name in ('VERTEX', 'FRAGMENT')}
try:
egl = c.CDLL('libEGL.so.1')
gl = c.CDLL('libGLESv2.so.2')
except OSError as error:
raise SystemExit('This opt-in check needs Linux EGL/GLES and Mesa software rendering. '
'On Debian/Ubuntu install libegl1 libegl-mesa0 libgles2 libgl1-mesa-dri. '
f'Library load failed: {error}') from error
def bind(lib, name, result, *args):
fn = getattr(lib, name)
fn.restype, fn.argtypes = result, args
return fn
P, I, U, F = c.c_void_p, c.c_int, c.c_uint, c.c_float
eglGetDisplay = bind(egl, 'eglGetDisplay', P, P)
eglInitialize = bind(egl, 'eglInitialize', U, P, c.POINTER(I), c.POINTER(I))
eglChooseConfig = bind(egl, 'eglChooseConfig', U, P, c.POINTER(I), c.POINTER(P), I, c.POINTER(I))
eglCreatePbufferSurface = bind(egl, 'eglCreatePbufferSurface', P, P, P, c.POINTER(I))
eglCreateContext = bind(egl, 'eglCreateContext', P, P, P, P, c.POINTER(I))
eglMakeCurrent = bind(egl, 'eglMakeCurrent', U, P, P, P, P)
eglQueryString = bind(egl, 'eglQueryString', c.c_char_p, P, I)
eglGetProcAddress = bind(egl, 'eglGetProcAddress', P, c.c_char_p)
display = eglGetDisplay(None)
major, minor = I(), I()
assert eglInitialize(display, c.byref(major), c.byref(minor))
attrs = (I * 15)(0x3033, 1, 0x3040, 4, 0x3024, 8, 0x3023, 8, 0x3022, 8, 0x3021, 8, 0x3025, 0, 0x3038)
config, count = P(), I()
assert eglChooseConfig(display, attrs, c.byref(config), 1, c.byref(count)) and count.value
surface = eglCreatePbufferSurface(display, config, (I * 5)(0x3057, 512, 0x3056, 256, 0x3038))
context = eglCreateContext(display, config, None, (I * 3)(0x3098, 2, 0x3038))
assert surface and context and eglMakeCurrent(display, surface, surface, context)
getString = bind(gl, 'glGetString', c.c_char_p, U)
getError = bind(gl, 'glGetError', U)
metadata = {'source_sha256': hashlib.sha256(source.encode()).hexdigest(),
'shader_sha256': {key: hashlib.sha256(value).hexdigest() for key, value in shaders.items()},
'egl': eglQueryString(display, 0x3054).decode(),
'gles': getString(0x1F02).decode(), 'renderer': getString(0x1F01).decode(),
'assumption': 'External texture contains top-row-first Canvas buffer pixels; consumer transform flips Y, as Android GLConsumer does. Device SurfaceTexture/Canvas integration still needs validation.'}
assert b'GL_OES_EGL_image_external' in getString(0x1F03)
createShader = bind(gl, 'glCreateShader', U, U)
shaderSource = bind(gl, 'glShaderSource', None, U, I, c.POINTER(c.c_char_p), c.POINTER(I))
compileShader = bind(gl, 'glCompileShader', None, U)
getShaderiv = bind(gl, 'glGetShaderiv', None, U, U, c.POINTER(I))
getShaderLog = bind(gl, 'glGetShaderInfoLog', None, U, I, c.POINTER(I), c.c_char_p)
compiled = []
for name, kind in [('VERTEX', 0x8B31), ('FRAGMENT', 0x8B30)]:
shader = createShader(kind)
text = c.c_char_p(shaders[name])
shaderSource(shader, 1, c.byref(text), None)
compileShader(shader)
status, log = I(), c.create_string_buffer(8192)
getShaderiv(shader, 0x8B81, c.byref(status))
getShaderLog(shader, len(log), None, log)
assert status.value, name + ': ' + log.value.decode()
compiled.append(shader)
createProgram = bind(gl, 'glCreateProgram', U)
attachShader = bind(gl, 'glAttachShader', None, U, U)
linkProgram = bind(gl, 'glLinkProgram', None, U)
getProgramiv = bind(gl, 'glGetProgramiv', None, U, U, c.POINTER(I))
getProgramLog = bind(gl, 'glGetProgramInfoLog', None, U, I, c.POINTER(I), c.c_char_p)
program = createProgram()
for shader in compiled:
attachShader(program, shader)
linkProgram(program)
status, log = I(), c.create_string_buffer(8192)
getProgramiv(program, 0x8B82, c.byref(status))
getProgramLog(program, len(log), None, log)
assert status.value, log.value.decode()
useProgram = bind(gl, 'glUseProgram', None, U)
useProgram(program)
getUniform = bind(gl, 'glGetUniformLocation', I, U, c.c_char_p)
uniform1i = bind(gl, 'glUniform1i', None, I, I)
uniform3f = bind(gl, 'glUniform3f', None, I, F, F, F)
uniform4f = bind(gl, 'glUniform4f', None, I, F, F, F, F)
uniformMatrix4 = bind(gl, 'glUniformMatrix4fv', None, I, I, U, c.POINTER(F))
loc = lambda name: getUniform(program, name.encode())
uniform1i(loc('uTexture'), 0)
flip = (F * 16)(1,0,0,0, 0,-1,0,0, 0,0,1,0, 0,1,0,1)
uniformMatrix4(loc('uTextureMatrix'), 1, 0, flip)
genTextures = bind(gl, 'glGenTextures', None, I, c.POINTER(U))
bindTexture = bind(gl, 'glBindTexture', None, U, U)
texParameteri = bind(gl, 'glTexParameteri', None, U, U, I)
texImage2D = bind(gl, 'glTexImage2D', None, U, I, I, I, I, I, U, U, P)
names = (U * 2)()
genTextures(2, names)
bindTexture(0x0DE1, names[0])
pixels = bytes(value for y in range(256) for x in range(256) for value in (x, y, 37, 255))
pixel_data = (c.c_ubyte * len(pixels)).from_buffer_copy(pixels)
texImage2D(0x0DE1, 0, 0x1908, 256, 256, 0, 0x1908, 0x1401, pixel_data)
for param, value in [(0x2801,0x2601),(0x2800,0x2601),(0x2802,0x812F),(0x2803,0x812F)]:
texParameteri(0x0DE1, param, value)
createImage = c.CFUNCTYPE(P, P, P, U, P, c.POINTER(I))(eglGetProcAddress(b'eglCreateImageKHR'))
imageTarget = c.CFUNCTYPE(None, U, P)(eglGetProcAddress(b'glEGLImageTargetTexture2DOES'))
image = createImage(display, context, 0x30B1, P(names[0]), (I * 3)(0x30BC, 0, 0x3038))
assert image
bindTexture(0x8D65, names[1])
imageTarget(0x8D65, image)
for param, value in [(0x2801,0x2601),(0x2800,0x2601),(0x2802,0x812F),(0x2803,0x812F)]:
texParameteri(0x8D65, param, value)
assert getError() == 0
getAttrib = bind(gl, 'glGetAttribLocation', I, U, c.c_char_p)
enableAttrib = bind(gl, 'glEnableVertexAttribArray', None, U)
attribPointer = bind(gl, 'glVertexAttribPointer', None, U, I, U, U, I, P)
vertices = (F * 8)(-1,-1, 1,-1, -1,1, 1,1)
position = getAttrib(program, b'aPosition')
enableAttrib(position)
attribPointer(position, 2, 0x1406, 0, 0, vertices)
viewport = bind(gl, 'glViewport', None, I, I, I, I)
draw = bind(gl, 'glDrawArrays', None, U, I, I)
read = bind(gl, 'glReadPixels', None, I, I, I, I, U, U, P)
finish = bind(gl, 'glFinish', None)
cases = []
def render(name, mono=True, position=(0,0,0), yaw=0, width=512, height=256, coords=((64,32),(448,32),(64,224),(448,224)), backdrop=False):
viewport(0, 0, width, height)
cs, sn = math.cos(yaw), math.sin(yaw)
for row, values in enumerate(((cs,0,sn),(0,1,0),(-sn,0,cs))):
uniform3f(loc('uRotation'+str(row)), *values)
uniform3f(loc('uHead'), *position)
uniform3f(loc('uScreen'), 1, 1, 2)
uniform4f(loc('uOptics'), .5, width/height/(1 if mono else 2), 0 if mono else .032, 1 if mono else 0)
draw(0x0005, 0, 4)
finish()
assert getError() == 0
actual = (c.c_ubyte * (width*height*4))()
read(0,0,width,height,0x1908,0x1401,actual)
samples = []
for x, y in coords:
# Readback is GL bottom-up; coordinates below are image top-left.
offset = ((height-1-y)*width+x)*4
rgb = list(actual[offset:offset+3])
eye = 0 if mono or x < width/2 else 1
ux = (x+.5)/width if mono else (x+.5)/(width/2)-eye
vy = (y+.5)/height
rayx = (ux-.5)*(width/height/(1 if mono else 2))
rayy = .5-vy
dx, dy, dz = cs*rayx-sn, rayy, -sn*rayx-cs
eo = 0 if mono else (-.032 if eye == 0 else .032)
ox, oy, oz = position[0]+cs*eo, position[1], position[2]-sn*eo
t = (-1-oz)/dz if dz else -1
uv = ((ox+dx*t)*.5+.5, -(oy+dy*t)+.5)
valid = dz < -.0001 and t > 0 and all(0 <= v <= 1 for v in uv) and not backdrop
expected = [round(max(0,min(255,v*256-.5))) for v in uv]+[37] if valid else [4,4,5]
assert all(abs(a-b) <= 2 for a,b in zip(rgb,expected)), (name,x,y,rgb,expected,uv)
samples.append({'at':[x,y],'rgb':rgb,'expected':expected})
cases.append({'name':name,'samples':samples})
if name == 'mono_upright':
rows = bytearray()
for y in range(height-1,-1,-1):
for x in range(width):
start=(y*width+x)*4
rows.extend(actual[start:start+3])
(out/'menu-shader-mono.ppm').write_bytes(f'P6\n{width} {height}\n255\n'.encode()+rows)
def chunk(name, data):
return struct.pack('>I',len(data))+name+data+struct.pack('>I',zlib.crc32(name+data)&0xffffffff)
scanlines=b''.join(b'\x00'+rows[y*width*3:(y+1)*width*3] for y in range(height))
(out/'menu-shader-mono.png').write_bytes(b'\x89PNG\r\n\x1a\n'+
chunk(b'IHDR',struct.pack('>IIBBBBB',width,height,8,2,0,0,0))+
chunk(b'IDAT',zlib.compress(scanlines))+chunk(b'IEND',b''))
render('mono_upright')
render('sbs_both_eyes', mono=False, width=512, height=128,
coords=((32,16),(224,16),(32,112),(224,112),(288,16),(480,16),(288,112),(480,112),(128,64),(384,64)))
render('six_dof_translation', position=(.2,.1,.1), coords=((256,128),(200,96),(300,160)))
render('row_major_yaw', yaw=math.radians(10), coords=((256,128),(200,96),(300,160)))
render('behind_plane', position=(0,0,-2), coords=((256,128),), backdrop=True)
render('back_facing', yaw=math.pi, coords=((256,128),), backdrop=True)
metadata['cases'] = cases
metadata['result'] = 'PASS'
(out/'menu-shader-validation.json').write_text(json.dumps(metadata, indent=2)+'\n')
print(json.dumps(metadata, indent=2), flush=True)
if __name__ == '__main__':
main()
+11 -1
View File
@@ -676,6 +676,10 @@ def execute(args: argparse.Namespace, adb: str, serial: str) -> dict:
request["mode"] = args.mode
elif args.glasses_op == "imu":
request["ms"] = args.ms
elif args.glasses_op == "spatial-diagnostics":
request["enabled"] = not args.off
elif args.glasses_op == "spatial-recenter":
request["screen"] = args.screen
elif args.glasses_op == "raw":
request.update(msgId=args.msg_id, payload=args.payload or "")
return add_serial(checked(provider_call(adb, serial, "glasses", request=request), "GLASSES_FAILED"), serial)
@@ -853,7 +857,7 @@ def build_parser() -> argparse.ArgumentParser:
glasses_ops = glasses.add_subparsers(dest="glasses_op")
glasses_ops.add_parser("status")
glasses_ops.add_parser("allow", help="ask for USB control of the glasses again")
glasses_set = glasses_ops.add_parser("set", help="refreshHz, sbsHz, brightness, volume, sunblock, 3d, gameOnGlasses, headTracking, auto3d, picture3d, screenWidthDm, screenDistanceDm, vrFovDeg")
glasses_set = glasses_ops.add_parser("set", help="refreshHz, sbsHz, resolution2d, resolution3d, resolutionVr (auto or WIDTHxHEIGHT per eye), brightness, volume, sunblock, 3d, 3dof, gameOnGlasses, headTracking, auto3d, picture3d, screenWidthDm, screenDistanceDm, vrFovDeg")
glasses_set.add_argument("key")
glasses_set.add_argument("value")
glasses_sim = glasses_ops.add_parser("simulate", help="feed a moving head pose (yaw, pitch and roll sweeps), or hold one")
@@ -862,6 +866,12 @@ def build_parser() -> argparse.ArgumentParser:
glasses_sim.add_argument("--pitch", type=float, help="hold this pitch (degrees, down positive) instead of sweeping")
glasses_ops.add_parser("stop-simulate")
glasses_ops.add_parser("query", help="ask the glasses for their display mode, brightness, volume and IMU state")
spatial_diagnostics = glasses_ops.add_parser("spatial-diagnostics", help="start stereo diagnostics without saving images")
spatial_diagnostics.add_argument("--off", action="store_true")
spatial_recenter = glasses_ops.add_parser("spatial-recenter", help="recenter the current spatial origin")
spatial_recenter.add_argument("--screen", action="store_true")
glasses_ops.add_parser("spatial-floor", help="set floor using configured eye height")
glasses_ops.add_parser("spatial-restart", help="restart the provider for explicit tracking recovery")
glasses_mode = glasses_ops.add_parser("mode", help="send a display mode byte, e.g. 0x31 (1080p60), 0x34 (1080p120), 0x32 (3D)")
glasses_mode.add_argument("mode")
glasses_imu = glasses_ops.add_parser("imu", help="stream the glasses' IMU for a moment and show what it reports")
+20
View File
@@ -23,6 +23,24 @@ quest_settings() {
"$HOME/.local/share/Steam/config/steamvr.vrsettings" "$backup"
}
configure_graphics() {
case "$(cat /sys/class/kgsl/kgsl-3d0/gpu_model 2>/dev/null)" in
Adreno840*)
# The session's forced sysmem mode loses the Vulkan device in SteamVR.
export TU_DEBUG="$(
IFS=', '
set -f
separator=
for flag in ${TU_DEBUG:-}; do
case "$flag" in ''|sysmem) continue ;; esac
printf '%s%s' "$separator" "$flag"
separator=,
done
)"
;;
esac
}
stop() {
rm -f "$state/active"
for n in vrcompositor vrserver vrmonitor vrdashboard vrstartup vrwebhelper; do
@@ -176,6 +194,8 @@ JSON
export VK_LAYER_PATH="$layer${VK_LAYER_PATH:+:$VK_LAYER_PATH}"
export VK_INSTANCE_LAYERS="VK_LAYER_DROIDDECK_vr${VK_INSTANCE_LAYERS:+:$VK_INSTANCE_LAYERS}"
export VK_LOADER_LAYERS_DISABLE="${VK_LOADER_LAYERS_DISABLE:-~implicit~}"
configure_graphics
export LD_PRELOAD="$layer/libdroiddeck_vr_scheduling.so${LD_PRELOAD:+:$LD_PRELOAD}"
if [ "${DROIDDECK_VR_PRESENT:-}" = questwsi ]; then
quest_settings apply || exit 1
startup_handed_off=0
+9 -1
View File
@@ -42,7 +42,15 @@ LOCAL_BUILD_PLUMBING = {'app/build.gradle', 'tools/release/runtime_inputs.py'}
LOCAL_ANDROID_SOURCES = {'app/src/main/cpp/CMakeLists.txt',
'app/src/main/cpp/viture/sdk_bridge.cpp',
'app/src/main/cpp/viture/display_modes.h',
'app/src/main/cpp/waylandcomp/src/cardboard.c'}
'app/src/main/cpp/viture/stereo_frame_gate.h',
'app/src/main/cpp/viture/uvc_bridge.cpp',
'app/src/main/cpp/waylandcomp/src/cardboard.c',
'app/src/main/cpp/waylandcomp/src/cardboard.h',
'app/src/main/cpp/waylandcomp/src/cardboard.frag',
'app/src/main/cpp/waylandcomp/src/cardboard_frag.h',
'app/src/main/cpp/waylandcomp/src/waylandcomp_jni.c',
'app/src/main/cpp/waylandcomp/src/spatialpose.c',
'app/src/main/cpp/waylandcomp/CMakeLists.txt'}
# These audited helper updates are copied from this checkout by runtime.record().
# Permit them only while source_assets() still guarantees that staging path.
LOCAL_SESSION_HELPERS = {'tools/linuxfs/overlay/usr/local/bin/droiddeck-steam-compat',
+5 -2
View File
@@ -19,6 +19,7 @@ SOURCE_PATHS = ['app/build.gradle', 'app/src/main/cpp/fakeinput_steam.cpp', 'app
'tools/release/runtime_inputs.py', 'app/src/main/assets/desktop-removal',
'tools/quest', 'tools/vr/src', 'tools/vr/driver_droiddeck', 'tools/vr/layer', 'tools/vr/build.sh',
'app/src/main/cpp/waylandcomp/src/headpose_shm.h',
'app/src/main/cpp/waylandcomp/src/spatialpose_shm.h',
'app/src/main/cpp/adrenotools/src/hook/kgsl.h',
'app/src/main/assets/graphics_driver/adrenotools-turnip25.1.0.tzst']
@@ -75,9 +76,11 @@ def required():
paths += ['assets/linuxfs/usr/local/lib/droiddeck-msitools/' + name for name in
['msiinfo', 'cabextract', '7z', '7z.so', 'libmsi-1.0.so.0', 'libgsf-1.so.114', 'libgcab-1.0.so.0']]
paths += ['assets/linuxfs/usr/local/lib/droiddeck-vr/' + name for name in
['droiddeck/bin/linuxarm64/driver_droiddeck.so', 'droiddeck/driver.vrdrivermanifest',
['droiddeck/bin/linuxarm64/driver_droiddeck.so', 'droiddeck/bin/linuxarm64/xr-display-mode',
'droiddeck/driver.vrdrivermanifest',
'droiddeck/resources/settings/default.vrsettings', 'layer/libVkLayer_droiddeck_vr.so',
'layer/VK_LAYER_DROIDDECK_vr.json', 'layer/VK_LAYER_DROIDDECK_vr_implicit.json']]
'layer/VK_LAYER_DROIDDECK_vr.json', 'layer/VK_LAYER_DROIDDECK_vr_implicit.json',
'layer/libdroiddeck_vr_scheduling.so']]
for side in ['left', 'right']:
paths += [f'assets/linuxfs/usr/local/lib/droiddeck-vr/droiddeck/resources/rendermodels/droiddeck_hand_{side}/{name}'
for name in [f'droiddeck_hand_{side}.json', 'hand.mtl', 'hand.png', 'hand_open.obj', 'hand_fist.obj',
+35
View File
@@ -0,0 +1,35 @@
import importlib.util
import io
from pathlib import Path
import unittest
import zipfile
spec = importlib.util.spec_from_file_location('build_hand_lite', Path(__file__).parents[1] / 'build_hand_lite.py')
builder = importlib.util.module_from_spec(spec)
spec.loader.exec_module(builder)
class HandLiteBundleTest(unittest.TestCase):
def test_embedded_labels_are_reproducible_without_changing_model_bytes(self):
model = b'\x08\x00\x00\x00TFL3model tensors and weights'
outputs = []
for year, order in [(2020, ['labels.txt', 'handedness.txt']),
(2026, ['handedness.txt', 'labels.txt'])]:
source = io.BytesIO(model)
with zipfile.ZipFile(source, 'a') as archive:
for name in order:
entry = zipfile.ZipInfo(name, (year, 10, 16, 12, 30, 0))
archive.writestr(entry, name.encode())
output = builder.canonicalize_associated_files(source.getvalue())
self.assertEqual(model, output[:len(model)])
self.assertEqual(output, builder.canonicalize_associated_files(output))
with zipfile.ZipFile(io.BytesIO(output)) as archive:
for name in order:
self.assertEqual(name.encode(), archive.read(name))
outputs.append(output)
self.assertEqual(*outputs)
if __name__ == '__main__':
unittest.main()
+71
View File
@@ -0,0 +1,71 @@
import collections
from pathlib import Path
import re
import unicodedata
import unittest
import xml.etree.ElementTree as ET
ROOT = Path(__file__).resolve().parents[2]
RES = ROOT / "app/src/main/res"
FORMAT = re.compile(r"%(?:\d+\$)?[-#+ 0,(]*\d*(?:\.\d+)?[a-zA-Z%]")
def strings(path):
return {
element.attrib["name"]: "".join(element.itertext())
for element in ET.parse(path).getroot()
if element.tag == "string" and element.get("translatable") != "false"
}
class GlassesLocalizationTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.english = strings(RES / "values/strings.xml")
cls.required = {
key: value for key, value in cls.english.items()
if key.startswith(("glasses_", "spatial_", "tracking_recovery_", "setup_glasses_", "phone_camera_tracking_"))
or key in ("setup_tab_glasses", "hand_camera_prompt_title", "hand_camera_prompt_body", "hand_camera_setting_hint")
}
language_source = (ROOT / "app/src/main/java/com/droiddeck/launcher/core/AppLanguage.kt").read_text(encoding="utf-8")
tags = re.findall(r'"([a-zA-Z0-9-]+)"', re.search(r"val supported:.*?listOf\((.*?)\)", language_source, re.S).group(1))
special = {"id": "in", "es-419": "b+es+419", "pt-BR": "pt"}
cls.locales = {}
for tag in tags:
if tag == "en":
continue
qualifier = special.get(tag, tag.replace("-", "-r"))
cls.locales[tag] = strings(RES / f"values-{qualifier}/strings.xml")
def test_every_supported_language_has_nonempty_glasses_and_calibration_text(self):
for locale, translated in self.locales.items():
for key in self.required:
with self.subTest(locale=locale, key=key):
self.assertTrue(translated.get(key, "").strip(), "Missing translation would fall back to English")
def test_translations_preserve_format_arguments_and_explicit_line_breaks(self):
for locale, translated in self.locales.items():
for key, source in self.required.items():
with self.subTest(locale=locale, key=key):
target = translated.get(key, "")
self.assertEqual(collections.Counter(FORMAT.findall(source)), collections.Counter(FORMAT.findall(target)))
self.assertEqual(source.count(r"\n"), target.count(r"\n"))
def test_printed_board_and_distance_guidance_preserve_measurements(self):
keys = ("spatial_print_hint", "spatial_print_scale", "spatial_print_check", "spatial_distance_12in",
"spatial_distance_30cm", "spatial_distance_20in", "spatial_distance_50cm",
"spatial_distance_hint", "spatial_distance_corners")
def numbers(value):
value = "".join(str(unicodedata.decimal(char)) if char.isdecimal() else char for char in value)
return collections.Counter(number.replace(",", ".") for number in re.findall(r"\d+(?:[.,]\d+)?", value))
for locale, translated in self.locales.items():
for key in keys:
with self.subTest(locale=locale, key=key):
self.assertEqual(numbers(self.english[key]), numbers(translated.get(key, "")))
if __name__ == "__main__":
unittest.main()
+23
View File
@@ -0,0 +1,23 @@
#include "../../app/src/main/cpp/viture/mjpeg_payload.h"
#include <cassert>
#include <cstdio>
#include <cstring>
int main() {
const uint8_t jpeg[] = {0xff,0xd8,0xff,0xc0,0,4,1,2,0xff,0xd9};
uint8_t frame[32]{};
std::memcpy(frame,jpeg,sizeof(jpeg));
for (std::size_t padding=0; padding<=7; ++padding)
assert(mjpeg_payload_size(frame,sizeof(jpeg)+padding) == sizeof(jpeg));
assert(mjpeg_payload_size(frame,sizeof(jpeg)+8) == 0);
for (std::size_t size=0; size<sizeof(jpeg); ++size)
assert(mjpeg_payload_size(frame,size) == 0);
frame[sizeof(jpeg)] = 0x42;
assert(mjpeg_payload_size(frame,sizeof(jpeg)+1) == 0);
frame[sizeof(jpeg)] = 0;
frame[0] = 0;
assert(mjpeg_payload_size(frame,sizeof(jpeg)) == 0);
assert(mjpeg_payload_size(nullptr,10) == 0);
assert(mjpeg_payload_size(jpeg,4*1024*1024+1) == 0);
std::puts("PASS: JPEG boundaries, 0-7 padding bytes, truncation, garbage, and bounds");
}
+3 -2
View File
@@ -162,10 +162,11 @@ class WindowsRuntimePreparationTest(unittest.TestCase):
with mock.patch.object(prepare, 'source_assets', return_value={}), \
mock.patch.object(prepare, 'changed', side_effect=[set(), {'app/src/main/cpp/CMakeLists.txt'}, set()]), \
mock.patch.object(prepare, 'git', side_effect=[prepare.DONOR_TREE.encode(),
b'app/src/main/cpp/viture/sdk_bridge.cpp\0']):
b'app/src/main/cpp/viture/sdk_bridge.cpp\0app/src/main/cpp/viture/stereo_frame_gate.h\0']):
classifications = prepare.check_sources()
self.assertEqual(set(classifications), {'app/src/main/cpp/CMakeLists.txt',
'app/src/main/cpp/viture/sdk_bridge.cpp'})
'app/src/main/cpp/viture/sdk_bridge.cpp',
'app/src/main/cpp/viture/stereo_frame_gate.h'})
self.assertTrue(all('compiled by Gradle' in action for action in classifications.values()))
with mock.patch.object(prepare, 'changed', side_effect=[set(), set()]), \
mock.patch.object(prepare, 'git', side_effect=[prepare.DONOR_TREE.encode(),
+79
View File
@@ -0,0 +1,79 @@
import os
from pathlib import Path
import shutil
import subprocess
import tempfile
import time
import unittest
HELPER = Path(__file__).resolve().parents[1] / 'linuxfs/overlay/usr/local/bin/droiddeck-steamvr'
@unittest.skipUnless(shutil.which('sh'), 'requires a POSIX shell')
class SteamVrGraphicsTest(unittest.TestCase):
def launch(self, model, debug=None, action='start', preload=None):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
bindir = root / 'bin'
bindir.mkdir()
runtime = root / 'steamvr/bin/linuxarm64'
runtime.mkdir(parents=True)
(runtime / 'vrstartup').touch(mode=0o755)
driver = root / 'driver/droiddeck/bin/linuxarm64'
driver.mkdir(parents=True)
(driver / 'driver_droiddeck.so').touch()
result = root / 'launch-env'
preloads = root / 'launch-preload'
scripts = {
'cat': 'if [ "$1" = /sys/class/kgsl/kgsl-3d0/gpu_model ]; then printf "%s" "$TEST_GPU_MODEL"; else exec /bin/cat "$@"; fi',
'pgrep': 'exit 1',
'setsid': 'printf "%s" "$LD_PRELOAD" > "$TEST_LAUNCH_PRELOAD"; printf "%s" "${TU_DEBUG-unset}" > "$TEST_LAUNCH_ENV"',
}
for name, script in scripts.items():
path = bindir / name
path.write_text('#!/bin/sh\n' + script + '\n')
path.chmod(0o755)
env = dict(os.environ, HOME=str(root), XDG_RUNTIME_DIR=str(root / 'run'),
STEAMVR=str(root / 'steamvr'), DROIDDECK_VR_DIR=str(root / 'driver'),
PATH=str(bindir) + os.pathsep + os.environ['PATH'],
TEST_GPU_MODEL=model, TEST_LAUNCH_ENV=str(result), TEST_LAUNCH_PRELOAD=str(preloads))
env.pop('TU_DEBUG', None)
env.pop('LD_PRELOAD', None)
if debug is not None:
env['TU_DEBUG'] = debug
if preload is not None:
env['LD_PRELOAD'] = preload
run = subprocess.run(['sh', str(HELPER), action], env=env,
capture_output=True, text=True, timeout=10)
self.assertEqual(run.returncode, 0, run.stderr)
until = time.monotonic() + 2
while not result.exists() and time.monotonic() < until:
time.sleep(0.01)
self.assertEqual(preloads.read_text(), str(root / 'driver/layer/libdroiddeck_vr_scheduling.so') +
(':' + preload if preload else ''))
return result.read_text()
def test_adreno_840_start_removes_forced_sysmem(self):
self.assertEqual(self.launch('Adreno840v2', 'sysmem'), '')
self.assertEqual(self.launch('Adreno840v2'), '')
def test_preserves_other_debug_flags_and_removes_repeated_sysmem(self):
self.assertEqual(self.launch('Adreno840v2', 'sysmem,forcecb,sysmem,noubwc'), 'forcecb,noubwc')
self.assertEqual(self.launch('Adreno840v2', 'forcecb sysmem flushall'), 'forcecb,flushall')
def test_library_launch_uses_the_same_graphics_setup(self):
self.assertEqual(self.launch('Adreno840v2', 'sysmem,forcecb', action='launch'), 'forcecb')
def test_scheduling_preload_preserves_existing_preloads(self):
self.assertEqual(self.launch('Adreno840v2', 'sysmem', action='launch',
preload='/existing/preload.so'), '')
def test_other_and_unknown_gpus_keep_their_environment(self):
self.assertEqual(self.launch('Adreno740'), 'unset')
self.assertEqual(self.launch('Adreno740', 'sysmem,forcecb'), 'sysmem,forcecb')
self.assertEqual(self.launch('', 'sysmem'), 'sysmem')
if __name__ == '__main__':
unittest.main()
+11
View File
@@ -14,6 +14,14 @@ SOURCE = Path(__file__).resolve().parents[1] / 'linuxfs/overlay/usr/local/bin/dr
class SteamVrHelperTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.compiler_output = tempfile.TemporaryDirectory()
cls.addClassCleanup(cls.compiler_output.cleanup)
cls.scheduling = Path(cls.compiler_output.name) / 'libdroiddeck_vr_scheduling.so'
source = Path(__file__).resolve().parents[1] / 'vr/src/scheduling.c'
subprocess.run(['cc', '-shared', '-fPIC', '-o', str(cls.scheduling), str(source), '-ldl'], check=True)
def setUp(self):
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
@@ -37,6 +45,9 @@ class SteamVrHelperTest(unittest.TestCase):
self.executable(self.steamvr / 'bin/linuxarm64/vrstartup',
'echo "display=${DROIDDECK_VR_DISPLAY_MODE:-unset}"')
(self.driver / 'driver_droiddeck.so').touch()
layer = self.root / 'driver/layer'
layer.mkdir()
shutil.copyfile(self.scheduling, layer / self.scheduling.name)
self.environment = dict(os.environ)
self.environment.pop('DROIDDECK_VR_DISPLAY_MODE', None)
self.environment.pop('DROIDDECK_VR_PRESENT', None)
+66
View File
@@ -0,0 +1,66 @@
#include "../../app/src/main/cpp/viture/stereo_frame_gate.h"
#include <cassert>
#include <cstdio>
#include <initializer_list>
#include <limits>
int main() {
StereoFrameGate gate;
gate.SetHalfRate(true);
int copies = 0;
for (int i = 0; i < 250; ++i) {
const bool accepted = gate.Accept(1 + i*.04);
assert(accepted == (i%2 == 0));
copies += accepted;
}
assert(copies == 125 && gate.received == 250 && gate.skipped == 125);
gate.SetHalfRate(false);
for (int i = 0; i < 25; ++i) assert(gate.Accept(20 + i*.04));
assert(gate.received == 275 && gate.skipped == 125);
gate.SetHalfRate(true);
assert(gate.Accept(30));
gate.SetHalfRate(true);
assert(!gate.Accept(30.04));
assert(gate.Accept(30.08));
assert(gate.Accept(1));
assert(!gate.Accept(1.04));
const auto count = gate.received;
assert(!gate.Accept(std::numeric_limits<double>::quiet_NaN()));
assert(!gate.Accept(0));
assert(gate.received == count);
gate = {};
gate.SetHalfRate(true);
assert(gate.Accept(1) && gate.received == 1 && gate.skipped == 0);
for (int rate : {500, 600, 800, 1000}) {
gate = {};
gate.SetRatePermille(rate);
copies = 0;
int previousCopy = 0;
for (int i = 0; i < 250; ++i) {
gate.SetRatePermille(rate);
const bool accepted = gate.Accept(1 + i*.04);
if (i == 0) assert(accepted);
if (accepted) {
assert(i-previousCopy <= 2);
previousCopy = i;
++copies;
}
}
assert(copies == 250*rate/1000);
assert(gate.received == 250 && gate.skipped == uint64_t(250-copies));
assert(gate.Accept(.5));
}
gate = {};
for (int rate : {500, 600, 800, 1000, 800, 600, 500}) {
gate.SetRatePermille(rate);
assert(gate.Accept(1 + gate.received*.04));
}
for (int invalid : {-1, 0, 499, 700, 1001}) {
gate = {};
gate.SetRatePermille(invalid);
assert(gate.Accept(1));
assert(!gate.Accept(1.04));
assert(gate.Accept(1.08));
}
puts("PASS: fractional stereo copies, stable phase, full-rate setup, source resets, invalid settings and reconnect reset");
}
+3 -1
View File
@@ -26,11 +26,13 @@ class WindowsVrBuildTest(unittest.TestCase):
for name, source in vr.CONFIGS.items():
self.write(self.root / source, b'{"name": "current"}\n')
self.write(self.out / name, b'{"name": "current"}\n')
for name in vr.LIBRARIES:
for name in vr.LIBRARIES | vr.EXECUTABLES:
data = bytearray(64)
data[:6] = b'\x7fELF\x02\x01'
struct.pack_into('<HH', data, 16, 3, 183)
self.write(self.out / name, data)
for name in vr.HAND_MODELS:
self.write(self.out / name, b'generated hand model\n')
for name in ('vulkan/vulkan_core.h', 'vulkan/vk_layer.h'):
self.write(self.include / name, b'vulkan header\n')
self.commit = mock.patch.object(vr, 'commit', return_value='current-revision')
+60
View File
@@ -0,0 +1,60 @@
cmake_minimum_required(VERSION 3.22)
project(droiddeck_rgb_uvc VERSION 0.0.8 LANGUAGES C CXX)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(RGB_UVC_SOURCE_ROOT "" CACHE PATH "Extracted checksum-pinned libusb and libuvc sources")
if(NOT RGB_UVC_SOURCE_ROOT)
message(FATAL_ERROR "Set RGB_UVC_SOURCE_ROOT or use tools/build_rgb_uvc.py")
endif()
set(USB_SOURCE "${RGB_UVC_SOURCE_ROOT}/libusb-1.0.30")
set(UVC_SOURCE "${RGB_UVC_SOURCE_ROOT}/libuvc-0.0.8")
add_library(rgb_usb SHARED
${USB_SOURCE}/libusb/core.c
${USB_SOURCE}/libusb/descriptor.c
${USB_SOURCE}/libusb/hotplug.c
${USB_SOURCE}/libusb/io.c
${USB_SOURCE}/libusb/sync.c
${USB_SOURCE}/libusb/strerror.c
${USB_SOURCE}/libusb/os/linux_usbfs.c
${USB_SOURCE}/libusb/os/events_posix.c
${USB_SOURCE}/libusb/os/threads_posix.c
${USB_SOURCE}/libusb/os/linux_netlink.c
)
set_target_properties(rgb_usb PROPERTIES OUTPUT_NAME usb)
target_include_directories(rgb_usb PUBLIC ${USB_SOURCE}/libusb PRIVATE ${USB_SOURCE}/android ${USB_SOURCE}/libusb/os)
target_compile_options(rgb_usb PRIVATE -fvisibility=hidden -pthread)
target_link_options(rgb_usb PRIVATE -Wl,--no-undefined -Wl,-z,max-page-size=16384)
target_link_libraries(rgb_usb PUBLIC log)
set(libuvc_VERSION_MAJOR 0)
set(libuvc_VERSION_MINOR 0)
set(libuvc_VERSION_PATCH 8)
set(libuvc_VERSION 0.0.8)
set(LIBUVC_HAS_JPEG OFF)
configure_file(${UVC_SOURCE}/include/libuvc/libuvc_config.h.in generated/libuvc/libuvc_config.h @ONLY)
add_library(rgb_uvc STATIC
${UVC_SOURCE}/src/ctrl.c
${UVC_SOURCE}/src/ctrl-gen.c
${UVC_SOURCE}/src/device.c
${UVC_SOURCE}/src/diag.c
${UVC_SOURCE}/src/frame.c
${UVC_SOURCE}/src/init.c
${UVC_SOURCE}/src/stream.c
${UVC_SOURCE}/src/misc.c
)
set_target_properties(rgb_uvc PROPERTIES OUTPUT_NAME uvc)
target_include_directories(rgb_uvc PUBLIC ${UVC_SOURCE}/include ${CMAKE_CURRENT_BINARY_DIR}/generated)
target_compile_definitions(rgb_uvc PRIVATE UVC_DEBUGGING LIBUVC_NUM_TRANSFER_BUFS=8)
target_compile_options(rgb_uvc PRIVATE -fvisibility=hidden -pthread)
target_link_libraries(rgb_uvc PUBLIC rgb_usb log)
add_executable(uvc_submit_failure_test submit_failure_test.cpp)
target_compile_features(uvc_submit_failure_test PRIVATE cxx_std_17)
target_compile_definitions(uvc_submit_failure_test PRIVATE LIBUVC_NUM_TRANSFER_BUFS=8)
target_link_libraries(uvc_submit_failure_test PRIVATE rgb_uvc)
target_link_options(uvc_submit_failure_test PRIVATE
-Wl,--no-undefined -Wl,-z,max-page-size=16384
-Wl,--wrap=libusb_submit_transfer -Wl,--wrap=libusb_cancel_transfer
-Wl,--wrap=libusb_set_interface_alt_setting
-Wl,--wrap=libusb_get_ss_endpoint_companion_descriptor
)
+19
View File
@@ -0,0 +1,19 @@
--- a/src/stream.c
+++ b/src/stream.c
@@ -1281,13 +1281,15 @@
}
if ( ret != UVC_SUCCESS && transfer_id >= 0 ) {
+ pthread_mutex_lock(&strmh->cb_mutex);
for ( ; transfer_id < LIBUVC_NUM_TRANSFER_BUFS; transfer_id++) {
free ( strmh->transfers[transfer_id]->buffer );
libusb_free_transfer ( strmh->transfers[transfer_id]);
strmh->transfers[transfer_id] = 0;
}
- ret = UVC_SUCCESS;
+ pthread_mutex_unlock(&strmh->cb_mutex);
+ uvc_stream_stop(strmh);
}
UVC_EXIT(ret);
return ret;
+112
View File
@@ -0,0 +1,112 @@
#include <libusb.h>
#include <libuvc/libuvc.h>
#include <libuvc/libuvc_internal.h>
#include <cstdio>
#include <thread>
#include <vector>
static int failAt;
static int submissions;
static int cancellations;
static bool invalidCancellation;
static std::vector<std::thread> completions;
extern "C" int __wrap_libusb_submit_transfer(libusb_transfer* transfer) {
if (!transfer) return LIBUSB_ERROR_INVALID_PARAM;
return submissions++ == failAt ? LIBUSB_ERROR_NO_MEM : LIBUSB_SUCCESS;
}
extern "C" int __wrap_libusb_cancel_transfer(libusb_transfer* transfer) {
if (!transfer) {
invalidCancellation = true;
return LIBUSB_ERROR_INVALID_PARAM;
}
++cancellations;
completions.emplace_back([transfer] {
transfer->status = LIBUSB_TRANSFER_CANCELLED;
transfer->callback(transfer);
});
return LIBUSB_SUCCESS;
}
extern "C" int __wrap_libusb_set_interface_alt_setting(libusb_device_handle*, int, int) {
return LIBUSB_SUCCESS;
}
extern "C" int __wrap_libusb_get_ss_endpoint_companion_descriptor(
libusb_context*, const libusb_endpoint_descriptor*, libusb_ss_endpoint_companion_descriptor**) {
return LIBUSB_ERROR_NOT_FOUND;
}
static void onFrame(uvc_frame_t*, void*) {}
static bool checkFailure(int failureIndex) {
failAt = failureIndex;
submissions = cancellations = 0;
invalidCancellation = false;
uvc_frame_desc_t frame{};
frame.bFrameIndex = 1;
frame.wWidth = 640;
frame.wHeight = 480;
uvc_format_desc_t format{};
format.bFormatIndex = 1;
format.frame_descs = &frame;
memcpy(format.guidFormat, "MJPG", 4);
frame.parent = &format;
uvc_streaming_interface_t streamInterface{};
streamInterface.bInterfaceNumber = 1;
streamInterface.bEndpointAddress = 0x81;
streamInterface.format_descs = &format;
format.parent = &streamInterface;
libusb_endpoint_descriptor endpoint{};
endpoint.bEndpointAddress = 0x81;
endpoint.bmAttributes = LIBUSB_TRANSFER_TYPE_ISOCHRONOUS;
endpoint.wMaxPacketSize = 512;
libusb_interface_descriptor alternatives[2]{};
alternatives[0].bInterfaceNumber = alternatives[1].bInterfaceNumber = 1;
alternatives[1].bAlternateSetting = 1;
alternatives[1].bNumEndpoints = 1;
alternatives[1].endpoint = &endpoint;
libusb_interface interfaces[2]{};
interfaces[1].num_altsetting = 2;
interfaces[1].altsetting = alternatives;
libusb_config_descriptor config{};
config.bNumInterfaces = 2;
config.interface = interfaces;
uvc_device_info_t info{};
info.config = &config;
uvc_device_handle_t camera{};
camera.info = &info;
uvc_stream_handle_t stream{};
stream.devh = &camera;
stream.stream_if = &streamInterface;
stream.cur_ctrl.bInterfaceNumber = 1;
stream.cur_ctrl.bFormatIndex = 1;
stream.cur_ctrl.bFrameIndex = 1;
stream.cur_ctrl.dwMaxPayloadTransferSize = 512;
stream.cur_ctrl.dwMaxVideoFrameSize = 8192;
pthread_mutex_init(&stream.cb_mutex, nullptr);
pthread_cond_init(&stream.cb_cond, nullptr);
const int result = uvc_stream_start(&stream, onFrame, nullptr, 0);
if (result == UVC_SUCCESS) uvc_stream_stop(&stream);
for (auto& completion : completions) completion.join();
completions.clear();
bool valid = result == (failureIndex < 8 ? LIBUSB_ERROR_NO_MEM : UVC_SUCCESS)
&& !stream.running && cancellations == failureIndex && !invalidCancellation;
for (auto* transfer : stream.transfers) valid = valid && transfer == nullptr;
pthread_cond_destroy(&stream.cb_cond);
pthread_mutex_destroy(&stream.cb_mutex);
if (!valid) std::fprintf(stderr, "failureIndex=%d result=%d cancellations=%d\n", failureIndex, result, cancellations);
return valid;
}
int main() {
for (int iteration = 0; iteration < 20; ++iteration) {
for (int failureIndex : {0, 3, 8}) {
if (!checkFailure(failureIndex)) return 1;
}
}
std::puts("PASS: 60 starts covering zero/partial/full submissions; no USB hardware calls");
return 0;
}
+25 -1
View File
@@ -31,9 +31,31 @@ compile() {
cc -std=c11 -O2 -Wall -Wextra -Werror -Ibuild/vr/include -Itools/vr/src \
-o build/vr/test_fence_tracker tools/vr/tests/test_fence_tracker.c
build/vr/test_fence_tracker
cc -std=c11 -O2 -Wall -Wextra -Wno-unused-parameter -pthread -Ibuild/vr/include \
-o build/vr/test_timeline_bridge tools/vr/tests/test_timeline_bridge.c -ldl
build/vr/test_timeline_bridge
cc -std=c11 -O2 -Wall -Wextra -Wno-unused-parameter -pthread -Ibuild/vr/include \
-o build/vr/test_compositor_submit tools/vr/tests/test_compositor_submit.c -ldl
build/vr/test_compositor_submit
c++ -std=c++17 -O2 -Wall -Wextra -Werror -Itools/vr/src \
-o build/vr/test_render_target tools/vr/tests/test_render_target.cpp
build/vr/test_render_target
c++ -std=c++17 -O2 -Wall -Wextra -Werror -Ibuild/vr/include -Itools/vr/src \
-o build/vr/test_presenter_image tools/vr/tests/test_presenter_image.cpp
build/vr/test_presenter_image
c++ -std=c++17 -O2 -Wall -Wextra -Werror -Itools/vr/src \
-o build/vr/test_spatial_tracking tools/vr/tests/test_spatial_tracking.cpp
build/vr/test_spatial_tracking
c++ -std=c++17 -O2 -Wall -Wextra -Werror -Itools/vr/src \
-o build/vr/test_adaptive_pose tools/vr/tests/test_adaptive_pose.cpp
build/vr/test_adaptive_pose
c++ -std=c++17 -O2 -Wall -Wextra -Wno-unused-parameter -pthread -Itools/vr/src \
-o build/vr/test_optical_controller tools/vr/tests/test_optical_controller.cpp
build/vr/test_optical_controller
cc -shared -fPIC -O2 -Wall -Wextra -Werror \
-o build/vr/libdroiddeck_vr_scheduling.so tools/vr/src/scheduling.c -ldl
cc -O2 -Wall -Wextra -Werror -o build/vr/test_scheduling tools/vr/tests/test_scheduling.c
LD_PRELOAD="$PWD/build/vr/libdroiddeck_vr_scheduling.so" build/vr/test_scheduling
c++ -std=c++17 -O2 -Wall -Wextra -Werror -Wno-unused-parameter -Itools/vr/src \
-o build/vr/test_joycon tools/vr/tests/test_joycon.cpp
build/vr/test_joycon
@@ -58,8 +80,10 @@ compile() {
aarch64-linux-gnu-g++ -std=c++17 -shared -fPIC -O2 -Wall -Wextra -Wno-missing-field-initializers -fvisibility=hidden -pthread \
-static-libstdc++ -static-libgcc -Wl,-z,defs -Wl,-z,nodelete -Ibuild/vr/include \
-o build/vr/out/layer/libVkLayer_droiddeck_vr.so build/vr/vk_layer.o tools/vr/src/wsi_capture.cpp -ldl
aarch64-linux-gnu-gcc -shared -fPIC -O2 -Wall -Wextra -Werror -Wl,-z,defs \
-o build/vr/out/layer/libdroiddeck_vr_scheduling.so tools/vr/src/scheduling.c -ldl
aarch64-linux-gnu-strip --strip-unneeded build/vr/out/droiddeck/bin/linuxarm64/driver_droiddeck.so \
build/vr/out/layer/libVkLayer_droiddeck_vr.so')
build/vr/out/layer/libVkLayer_droiddeck_vr.so build/vr/out/layer/libdroiddeck_vr_scheduling.so')
if [ "$native" = 1 ]; then
(cd "$repo" && "${cmd[@]}")
else
+19 -7
View File
@@ -14,18 +14,27 @@ IMAGE = 'droiddeck-local-cross:24.04-v3'
MANIFEST = 'vr-build.json'
SOURCE_TREES = ('tools/vr/src', 'tools/vr/tests', 'tools/vr/third_party',
'tools/vr/driver_droiddeck', 'tools/vr/layer')
SOURCE_FILES = ('tools/vr/build.sh', 'tools/vr/build_windows.py', 'tools/local-cross.Dockerfile',
SOURCE_FILES = ('tools/vr/build.sh', 'tools/vr/build_windows.py', 'tools/vr/make_hand_models.cpp', 'tools/local-cross.Dockerfile',
'app/src/main/cpp/waylandcomp/src/headpose_shm.h',
'app/src/main/cpp/waylandcomp/src/spatialpose_shm.h',
'app/src/main/cpp/waylandcomp/generated/xdg-shell-protocol.c',
'app/src/main/cpp/waylandcomp/generated/xdg-shell-client-protocol.h')
LIBRARIES = {'droiddeck/bin/linuxarm64/driver_droiddeck.so', 'layer/libVkLayer_droiddeck_vr.so'}
LIBRARIES = {'droiddeck/bin/linuxarm64/driver_droiddeck.so', 'layer/libVkLayer_droiddeck_vr.so',
'layer/libdroiddeck_vr_scheduling.so'}
CONFIGS = {
'droiddeck/driver.vrdrivermanifest': 'tools/vr/driver_droiddeck/driver.vrdrivermanifest',
'droiddeck/resources/settings/default.vrsettings': 'tools/vr/driver_droiddeck/resources/settings/default.vrsettings',
'layer/VK_LAYER_DROIDDECK_vr.json': 'tools/vr/layer/VK_LAYER_DROIDDECK_vr.json',
'layer/VK_LAYER_DROIDDECK_vr_implicit.json': 'tools/vr/layer/VK_LAYER_DROIDDECK_vr_implicit.json',
}
OUTPUTS = LIBRARIES | CONFIGS.keys()
EXECUTABLES = {'droiddeck/bin/linuxarm64/xr-display-mode'}
HAND_MODELS = {
f'droiddeck/resources/rendermodels/droiddeck_hand_{side}/{name}'
for side in ('left', 'right')
for name in (f'droiddeck_hand_{side}.json', 'hand.mtl', 'hand.png', 'hand_open.obj',
'hand_fist.obj', 'thumb_index_open.obj', 'thumb_index_pinch.obj')
}
OUTPUTS = LIBRARIES | CONFIGS.keys() | EXECUTABLES | HAND_MODELS
def is_source(path):
@@ -75,12 +84,15 @@ def read_outputs(folder, root):
if path.is_symlink() or not path.resolve().is_relative_to(folder.resolve()):
raise ValueError('VR output escapes its build directory: ' + name)
data = path.read_bytes()
if name in LIBRARIES:
if name in LIBRARIES or name in EXECUTABLES:
if (len(data) < 64 or data[:6] != b'\x7fELF\x02\x01' or
struct.unpack_from('<HH', data, 16) != (3, 183)):
raise ValueError('VR library is not an ARM64 ELF shared library: ' + name)
elif json.loads(data) != json.loads((root / CONFIGS[name]).read_bytes()):
struct.unpack_from('<H', data, 16)[0] not in ((3,) if name in LIBRARIES else (2, 3)) or
struct.unpack_from('<H', data, 18)[0] != 183):
raise ValueError('VR output is not an ARM64 ELF shared library or executable: ' + name)
elif name in CONFIGS and json.loads(data) != json.loads((root / CONFIGS[name]).read_bytes()):
raise ValueError('VR configuration differs from current source: ' + name)
elif not data:
raise ValueError('Empty VR output: ' + name)
files[name] = data
return files
+46
View File
@@ -0,0 +1,46 @@
#pragma once
#include "arm_model.h"
#include <cstdint>
namespace droiddeck {
class AdaptivePositionMotion {
public:
explicit AdaptivePositionMotion(double minimumCutoff = 1.5, double positionGain = 25.0)
: minimumCutoff_(minimumCutoff), positionGain_(positionGain) {}
void Reset() { time_ = 0; velocity_ = {}; }
void Observe(const Vec3 &position, int64_t time) {
if (time == time_) return;
const double dt = double(time - time_) * 1e-9;
if (time_ > 0 && dt > 0 && dt <= .2)
velocity_ = velocity_ + ((position - position_) * (1.0 / dt) - velocity_) * Alpha(dt, 1.0);
else velocity_ = {};
position_ = position; time_ = time;
}
double PositionAlpha(double dt) const {
return Alpha(dt, std::clamp(minimumCutoff_ + positionGain_ * Length(velocity_), minimumCutoff_, 25.0));
}
private:
const double minimumCutoff_, positionGain_;
static double Alpha(double dt, double cutoff) { return -std::expm1(-2 * M_PI * cutoff * std::clamp(dt, 0.0, .2)); }
int64_t time_ = 0;
Vec3 position_, velocity_;
};
inline Quat ShoulderAim(const Vec3 &head, const Quat &headRotation, const Vec3 &palm, bool right, const Quat &fallback) {
const Vec3 headRight = Rotate(headRotation, {1, 0, 0});
double yaw = 0;
if (!Heading(headRotation, &yaw)) yaw = std::atan2(-headRight.z, headRight.x);
const Quat torso = AboutY(yaw);
const Vec3 shoulder = head + Rotate(torso, {right ? .18 : -.18, -.28, 0});
const Vec3 reach = palm - shoulder;
const double distance = Length(reach);
if (!std::isfinite(distance) || distance < .05) return fallback;
const Vec3 forward = Rotate(torso, {0, 0, -1});
return Normalize(FromTo(forward, reach * (1.0 / distance)) * torso);
}
}
+251 -50
View File
@@ -32,6 +32,8 @@
#include "index_pose.h"
#include "joycon_shm.h"
#include "presenter.h"
#include "spatial_tracking.h"
#include "luma_joycon.h"
#include "render_target.h"
#include "xr_tracking_file.h"
#include "xr_frame_math.h"
@@ -278,7 +280,8 @@ public:
const uint32_t create = 0x8 ;
for (int i = 0; i < 3; i++) {
SharedTextureHandle_t h = 0;
if (!VRIPCResourceManager()->NewSharedVulkanImage(d->nFormat, d->nWidth, d->nHeight, true, false, true, 1, 1,
// Compute access makes SteamVR substitute UNORM for sRGB, darkening direct compositor output.
if (!VRIPCResourceManager()->NewSharedVulkanImage(d->nFormat, d->nWidth, d->nHeight, true, false, false, 1, 1,
create, usage, &h)) {
Log("NewSharedVulkanImage failed");
return;
@@ -529,7 +532,7 @@ bool JoyConLive(const droiddeck_joycon_hand &hand, int64_t nowNs) {
class HandController : public ITrackedDeviceServerDriver {
public:
explicit HandController(bool right) : right_(right), arm_(right), camera_(right) {}
explicit HandController(bool right) : right_(right), arm_(right), camera_(right), luma_(right) {}
EVRInitError Activate(uint32_t id) override {
auto props = VRProperties();
@@ -617,12 +620,41 @@ public:
}
}
bool Update(const droiddeck_joycon_hand *hand, bool bare, const HmdQuaternion_t &headPose, int64_t nowNs) {
bool Update(const droiddeck_joycon_hand *hand, bool bare, const HmdQuaternion_t &headPose, int64_t nowNs,
const droiddeck_spatialpose *spatial = nullptr) {
const uint32_t id = id_.load(std::memory_order_acquire);
if (id == k_unTrackedDeviceIndexInvalid) return false;
const auto now = std::chrono::steady_clock::now();
const double dt = std::min(std::chrono::duration<double>(now - lastUpdate_).count(), 0.1);
lastUpdate_ = now;
if (lumaMode_ != (spatial != nullptr) || (spatial &&
(lumaConnection_ != spatial->connection || lumaOrigin_ != spatial->origin))) {
Disconnect(id);
luma_.Reset();
lumaMode_ = spatial != nullptr;
if (spatial) { lumaConnection_ = spatial->connection; lumaOrigin_ = spatial->origin; }
}
if (spatial && !SpatialFresh(spatial->head, nowNs)) {
if (connected_ && hand) {
luma_.Reset();
lumaSeen_ = false;
DriverPose_t p = GetPose();
p.poseIsValid = false;
p.result = TrackingResult_Running_OutOfRange;
p.poseTimeOffset = 0;
std::fill_n(p.vecVelocity, 3, 0.0);
std::fill_n(p.vecAngularVelocity, 3, 0.0);
droiddeck_joycon_hand idle{};
SendInputs(idle, 1.0);
{
std::lock_guard<std::mutex> lock(poseMutex_);
pose_ = p;
}
VRServerDriverHost()->TrackedDevicePoseUpdated(id, p, sizeof(p));
} else Disconnect(id);
LeaveBare(id);
return true;
}
if (!hand) {
Disconnect(id);
if (bare) {
@@ -634,6 +666,16 @@ public:
}
LeaveBare(id);
const Quat head{headPose.w, headPose.x, headPose.y, headPose.z};
if (spatial) {
double norm = 0;
for (float v : hand->rotation) norm += v * v;
if (!std::isfinite(norm) || norm < 0.5 || norm > 1.5 ||
!std::all_of(std::begin(hand->angular_velocity), std::end(hand->angular_velocity),
[](float v) { return std::isfinite(v); })) {
Disconnect(id);
return true;
}
}
const Quat q = Normalize({hand->rotation[0], hand->rotation[1], hand->rotation[2], hand->rotation[3]});
if (!connected_ || hand->recenter != recenter_) {
recenter_ = hand->recenter;
@@ -647,41 +689,57 @@ public:
arm_.Reset();
follower_.Reset();
kalman_.Reset();
luma_.Reset();
shown_ = false;
Log("%s Joy-Con facing forward", right_ ? "right" : "left");
}
const Quat turn = AboutY(yaw_);
const Quat aligned = turn * q;
const Vec3 rate{hand->angular_velocity[0], hand->angular_velocity[1], hand->angular_velocity[2]};
const Vec3 palm = arm_.Update(kHeadPosition, head, aligned, Length(rate), dt);
const Vec3 moved = Rotate(turn, {hand->displacement[0], hand->displacement[1], hand->displacement[2]});
const Vec3 inertial = Rotate(turn, {hand->inertial_velocity[0], hand->inertial_velocity[1], hand->inertial_velocity[2]});
const bool holding = (hand->buttons & (DROIDDECK_JOYCON_TRIGGER | DROIDDECK_JOYCON_GRIP)) != 0;
kalman_.Input(hand->time_ns - kJoyConDelayNs, inertial, Length(rate));
const bool camera = kalman_.Ready() && kalman_.TimeNs() - kalman_.SeenNs() <= kCameraHoldNs;
bool switched = false;
if (camera != cameraPlaced_) {
cameraPlaced_ = camera;
switched = true;
Log("%s Joy-Con hand placed by %s", right_ ? "right" : "left", camera ? "the camera" : "the arm model");
if (!camera && shown_) follower_.Place(shownAt_, moved);
}
const double settle = holding ? kHoldingSettleSeconds : kSettleSeconds;
const Vec3 source = camera ? kalman_.Position() : follower_.Update(palm, moved, dt, settle, kMaxOffset);
const Vec3 velocity = camera ? kalman_.Velocity() : Rotate(turn, {hand->velocity[0], hand->velocity[1], hand->velocity[2]});
if (shown_ && (switched || Length(source - source_) > kJump)) blend_ = shownAt_ - source;
else if (!shown_) blend_ = Vec3{};
blend_ = blend_ * std::exp(-dt / kBlendSeconds);
const Vec3 grip = source + blend_;
source_ = source;
shownAt_ = grip;
shown_ = true;
if (FILE *trace = gHandTrace.File()) {
const Vec3 k = kalman_.Ready() ? kalman_.Position() : Vec3{};
fprintf(trace, "J,%lld,%d,%lld,%.4f,%.4f,%.4f,%.4f,%.4f,%.4f,%d,%.4f,%.4f,%.4f,%.4f,%.4f,%.4f,%.3f,%.4f,%.4f,%.4f,%u,%.4f,%.4f,%.4f,%.4f,%.4f,%.4f,%.4f,%d\n",
(long long)nowNs, right_ ? 1 : 0, (long long)hand->time_ns, moved.x, moved.y, moved.z, palm.x, palm.y, palm.z,
camera ? 1 : 0, k.x, k.y, k.z, grip.x, grip.y, grip.z, Length(rate), velocity.x, velocity.y, velocity.z,
hand->buttons, aligned.w, aligned.x, aligned.y, aligned.z, inertial.x, inertial.y, inertial.z, kalman_.Ready() ? 1 : 0);
Vec3 grip, velocity;
if (spatial) {
const auto tracked = luma_.Update(*spatial, aligned, Length(rate), nowNs, dt);
if (!tracked.valid || aligning_) {
Disconnect(id);
return true;
}
grip = tracked.position;
if (!lumaSeen_ || lumaCamera_ != tracked.camera) {
Log("%s Luma Joy-Con position: %s", right_ ? "right" : "left",
tracked.camera ? "stereo hand" : "6DoF arm estimate");
lumaSeen_ = true; lumaCamera_ = tracked.camera;
}
} else {
const Vec3 palm = arm_.Update(kHeadPosition, head, aligned, Length(rate), dt);
const Vec3 moved = Rotate(turn, {hand->displacement[0], hand->displacement[1], hand->displacement[2]});
const Vec3 inertial = Rotate(turn, {hand->inertial_velocity[0], hand->inertial_velocity[1], hand->inertial_velocity[2]});
const bool holding = (hand->buttons & (DROIDDECK_JOYCON_TRIGGER | DROIDDECK_JOYCON_GRIP)) != 0;
kalman_.Input(hand->time_ns - kJoyConDelayNs, inertial, Length(rate));
const bool camera = kalman_.Ready() && kalman_.TimeNs() - kalman_.SeenNs() <= kCameraHoldNs;
bool switched = false;
if (camera != cameraPlaced_) {
cameraPlaced_ = camera;
switched = true;
Log("%s Joy-Con hand placed by %s", right_ ? "right" : "left", camera ? "the camera" : "the arm model");
if (!camera && shown_) follower_.Place(shownAt_, moved);
}
const double settle = holding ? kHoldingSettleSeconds : kSettleSeconds;
const Vec3 source = camera ? kalman_.Position() : follower_.Update(palm, moved, dt, settle, kMaxOffset);
velocity = camera ? kalman_.Velocity() : Rotate(turn, {hand->velocity[0], hand->velocity[1], hand->velocity[2]});
if (shown_ && (switched || Length(source - source_) > kJump)) blend_ = shownAt_ - source;
else if (!shown_) blend_ = Vec3{};
blend_ = blend_ * std::exp(-dt / kBlendSeconds);
grip = source + blend_;
source_ = source;
shownAt_ = grip;
shown_ = true;
if (FILE *trace = gHandTrace.File()) {
const Vec3 k = kalman_.Ready() ? kalman_.Position() : Vec3{};
fprintf(trace, "J,%lld,%d,%lld,%.4f,%.4f,%.4f,%.4f,%.4f,%.4f,%d,%.4f,%.4f,%.4f,%.4f,%.4f,%.4f,%.3f,%.4f,%.4f,%.4f,%u,%.4f,%.4f,%.4f,%.4f,%.4f,%.4f,%.4f,%d\n",
(long long)nowNs, right_ ? 1 : 0, (long long)hand->time_ns, moved.x, moved.y, moved.z, palm.x, palm.y, palm.z,
camera ? 1 : 0, k.x, k.y, k.z, grip.x, grip.y, grip.z, Length(rate), velocity.x, velocity.y, velocity.z,
hand->buttons, aligned.w, aligned.x, aligned.y, aligned.z, inertial.x, inertial.y, inertial.z, kalman_.Ready() ? 1 : 0);
}
}
Quat raw;
Vec3 position;
@@ -700,7 +758,7 @@ public:
p.vecAngularVelocity[0] = spin.x;
p.vecAngularVelocity[1] = spin.y;
p.vecAngularVelocity[2] = spin.z;
p.poseTimeOffset = -std::max(0.0, (nowNs - hand->time_ns) * 1e-9);
p.poseTimeOffset = spatial ? 0.0 : -std::max(0.0, (nowNs - hand->time_ns) * 1e-9);
p.willDriftInYaw = true;
p.poseIsValid = true;
p.result = TrackingResult_Running_OK;
@@ -833,8 +891,10 @@ private:
void Disconnect(uint32_t id) {
if (!connected_) return;
connected_ = false;
lumaSeen_ = false;
follower_.Reset();
kalman_.Reset();
luma_.Reset();
shown_ = false;
DriverPose_t p = {};
p.qWorldFromDriverRotation.w = 1;
@@ -845,9 +905,9 @@ private:
std::lock_guard<std::mutex> lock(poseMutex_);
pose_ = p;
}
VRServerDriverHost()->TrackedDevicePoseUpdated(id, p, sizeof(p));
droiddeck_joycon_hand idle = {};
SendInputs(idle, 1.0);
VRServerDriverHost()->TrackedDevicePoseUpdated(id, p, sizeof(p));
Log("%s Joy-Con hand disconnected", right_ ? "right" : "left");
}
@@ -935,6 +995,9 @@ private:
HandFollower follower_;
HandKalman kalman_;
CameraHand camera_;
LumaJoyConPosition luma_;
bool lumaMode_ = false, lumaSeen_ = false, lumaCamera_ = false;
uint64_t lumaConnection_ = 0, lumaOrigin_ = 0;
std::atomic<bool> bare_{false}, expectBare_{false};
bool bareSeen_ = false;
Vec3 source_, shownAt_, blend_;
@@ -960,7 +1023,9 @@ private:
class CardboardController : public ITrackedDeviceServerDriver {
public:
explicit CardboardController(bool right) : right_(right) {}
explicit CardboardController(bool right) : right_(right) {
pose_.qWorldFromDriverRotation.w = pose_.qDriverFromHeadRotation.w = pose_.qRotation.w = 1;
}
EVRInitError Activate(uint32_t id) override {
id_ = id;
@@ -1003,13 +1068,35 @@ public:
Log("%s hand activated (%s)", right_ ? "right" : "left", xrMode_ ? "Quest Touch" : "game pad");
return VRInitError_None;
}
void Deactivate() override { id_ = k_unTrackedDeviceIndexInvalid; }
void Deactivate() override { ClearInputs(); id_ = k_unTrackedDeviceIndexInvalid; }
void EnterStandby() override {}
void *GetComponent(const char *) override { return nullptr; }
void DebugRequest(const char *, char *resp, uint32_t size) override { if (size) resp[0] = 0; }
DriverPose_t GetPose() override { return pose_; }
void ClearInputs() {
const bool optical = opticalActive_;
opticalInputs_.Reset();
opticalPose_.Reset();
steadyHandAim_ = false;
opticalActive_ = false;
lastOpticalState_ = ~0u;
if (id_ == k_unTrackedDeviceIndexInvalid) return;
auto in = VRDriverInput();
EVRInputError error = VRInputError_None;
for (VRInputComponentHandle_t h : {system_, menu_, grip_, triggerClick_, padClick_, padTouch_}) {
const auto result = in->UpdateBooleanComponent(h, false, 0);
if (result != VRInputError_None) error = result;
}
for (VRInputComponentHandle_t h : {trigger_, padX_, padY_}) {
const auto result = in->UpdateScalarComponent(h, 0.0f, 0);
if (result != VRInputError_None) error = result;
}
if (optical || error != VRInputError_None)
Log("%s controller inputs cleared: error=%d", right_ ? "right" : "left", int(error));
}
void UpdateXr(const DroidDeckXrHand &hand, int64_t predicted) {
if (id_ == k_unTrackedDeviceIndexInvalid) return;
DriverPose_t pose{};
@@ -1051,6 +1138,7 @@ public:
bool XrMode() const { return xrMode_; }
void Disconnect() {
ClearInputs();
if (id_ == k_unTrackedDeviceIndexInvalid) return;
DriverPose_t p = {};
p.qWorldFromDriverRotation.w = 1;
@@ -1059,9 +1147,55 @@ public:
p.result = TrackingResult_Uninitialized;
pose_ = p;
VRServerDriverHost()->TrackedDevicePoseUpdated(id_, p, sizeof(p));
}
void UpdateOptical(const dd_spatial_hand &hand, bool headValid, const PadState *pad, int64_t now,
uint64_t connection, uint64_t origin, uint16_t inputEpochs = 0,
const dd_spatial_pose *aimHead = nullptr) {
if (id_ == k_unTrackedDeviceIndexInvalid) return;
opticalActive_ = true;
const bool inputValid = opticalInputs_.Update(hand, headValid, now, connection, origin, inputEpochs);
if (steadyHandAim_ != (aimHead != nullptr)) {
opticalPose_.Reset();
steadyHandAim_ = aimHead != nullptr;
}
auto visualHand = hand;
if (aimHead && headValid && SpatialFresh(*aimHead, now) && SpatialHandFresh(hand, now)) {
const auto &h = *aimHead;
const auto &p = hand.pose;
const auto q = droiddeck::ShoulderAim({h.position[0], h.position[1], h.position[2]},
{h.rotation[3], h.rotation[0], h.rotation[1], h.rotation[2]},
{p.position[0], p.position[1], p.position[2]}, right_,
{p.rotation[3], p.rotation[0], p.rotation[1], p.rotation[2]});
visualHand.pose.rotation[0] = q.x; visualHand.pose.rotation[1] = q.y;
visualHand.pose.rotation[2] = q.z; visualHand.pose.rotation[3] = q.w;
}
const bool valid = opticalPose_.Update(visualHand, headValid, now, connection, origin);
DriverPose_t p{};
p.qWorldFromDriverRotation.w = p.qDriverFromHeadRotation.w = p.qRotation.w = 1;
if (valid) {
const auto &observed = opticalPose_.rendered;
p.qRotation = {observed.rotation[3], observed.rotation[0], observed.rotation[1], observed.rotation[2]};
for (int i=0; i<3; ++i) p.vecPosition[i] = observed.position[i];
p.poseTimeOffset = double(observed.time_ns-now)/1e9;
}
p.deviceIsConnected = true; p.poseIsValid = valid;
p.result = valid ? TrackingResult_Running_OK : TrackingResult_Running_OutOfRange;
auto in = VRDriverInput();
for (VRInputComponentHandle_t h : {system_, menu_, grip_, triggerClick_, padClick_, padTouch_}) in->UpdateBooleanComponent(h, false, 0);
for (VRInputComponentHandle_t h : {trigger_, padX_, padY_}) in->UpdateScalarComponent(h, 0.0f, 0);
const uint32_t b = pad ? pad->buttons : 0;
in->UpdateBooleanComponent(system_, right_ && (b & PAD_GUIDE), 0);
in->UpdateBooleanComponent(menu_, (b & (right_ ? PAD_START : PAD_SELECT)) != 0, 0);
const auto gripError = in->UpdateBooleanComponent(grip_, opticalInputs_.grip, 0);
const auto clickError = in->UpdateBooleanComponent(triggerClick_, opticalInputs_.click, 0);
const auto triggerError = in->UpdateScalarComponent(trigger_, opticalInputs_.trigger, 0);
const float x = pad ? pad->axes[right_ ? 2 : 0]/32767.0f : 0;
const float y = pad ? -pad->axes[right_ ? 3 : 1]/32767.0f : 0;
in->UpdateScalarComponent(padX_, x, 0); in->UpdateScalarComponent(padY_, y, 0);
in->UpdateBooleanComponent(padTouch_, x*x+y*y > 0.01f || (b & (right_ ? PAD_A : PAD_X)), 0);
in->UpdateBooleanComponent(padClick_, (b & (right_ ? (PAD_A|PAD_R3) : PAD_X)) != 0, 0);
LogOpticalInputs(hand, inputValid, valid, now, connection, origin, inputEpochs, gripError, clickError, triggerError);
pose_ = p;
VRServerDriverHost()->TrackedDevicePoseUpdated(id_, p, sizeof(p));
}
static constexpr float kReachSpeed = 0.6f, kReachMin = -0.2f, kReachMax = 1.0f;
@@ -1070,8 +1204,10 @@ public:
return pad.axes[right ? 4 : 5] > kGrabTrigger || (pad.buttons & (right ? PAD_RB : PAD_LB));
}
void Update(const HmdQuaternion_t &head, const PadState *pad, bool sticksMoveHands, float reach) {
void Update(const DriverPose_t &headPose, const PadState *pad, bool sticksMoveHands, float reach) {
const auto &head = headPose.qRotation;
if (id_ == k_unTrackedDeviceIndexInvalid) return;
if (opticalActive_) ClearInputs();
const auto now = std::chrono::steady_clock::now();
const float dt = std::min(std::chrono::duration<float>(now - lastUpdate_).count(), 0.1f);
lastUpdate_ = now;
@@ -1105,7 +1241,7 @@ public:
m[0][0] = 1 - 2 * (y * y + z * z); m[0][1] = 2 * (x * y - z * w); m[0][2] = 2 * (x * z + y * w);
m[1][0] = 2 * (x * y + z * w); m[1][1] = 1 - 2 * (x * x + z * z); m[1][2] = 2 * (y * z - x * w);
m[2][0] = 2 * (x * z - y * w); m[2][1] = 2 * (y * z + x * w); m[2][2] = 1 - 2 * (x * x + y * y);
for (int i = 0; i < 3; i++) p.vecPosition[i] = (i == 1 ? 1.6 : 0.0) + m[i][0] * off[0] + m[i][1] * off[1] + m[i][2] * off[2];
for (int i = 0; i < 3; i++) p.vecPosition[i] = headPose.vecPosition[i] + m[i][0] * off[0] + m[i][1] * off[1] + m[i][2] * off[2];
p.poseIsValid = true;
p.result = TrackingResult_Running_OK;
p.deviceIsConnected = true;
@@ -1128,6 +1264,36 @@ public:
}
private:
void LogOpticalInputs(const dd_spatial_hand &hand, bool inputValid, bool poseValid, int64_t now,
uint64_t connection, uint64_t origin, uint16_t inputEpochs, EVRInputError gripError,
EVRInputError clickError, EVRInputError triggerError) {
const uint32_t triggerBand = !std::isfinite(hand.trigger) ? 3 : hand.trigger < .25f ? 0 : hand.trigger > .65f ? 2 : 1;
const uint32_t state = uint32_t(poseValid) | uint32_t(inputValid)<<1 | uint32_t(opticalInputs_.click)<<2 |
uint32_t(opticalInputs_.grip)<<3 | uint32_t(opticalInputs_.armed)<<4 | uint32_t(opticalInputs_.gripArmed)<<5 |
(hand.pose.flags & 7u)<<6 | triggerBand<<9 | uint32_t(hand.grip != 0)<<11;
const int errors[] = {int(gripError),int(clickError),int(triggerError)};
if (state == lastOpticalState_ && connection == lastOpticalConnection_ && origin == lastOpticalOrigin_ &&
inputEpochs == lastOpticalEpochs_ &&
std::equal(errors,errors+3,lastOpticalErrors_)) return;
lastOpticalState_ = state; lastOpticalConnection_ = connection; lastOpticalOrigin_ = origin;
lastOpticalEpochs_ = inputEpochs;
std::copy_n(errors,3,lastOpticalErrors_);
Log("%s optical input pose=%d input=%d flags=%u raw=%.3f/%u out=%.3f/%d/%d armed=%d/%d error=%d/%d/%d ageMs=%.1f connection=%llu origin=%llu epochs=%u/%u",
right_ ? "right" : "left", int(poseValid), int(inputValid), hand.pose.flags, double(hand.trigger), hand.grip,
double(opticalInputs_.trigger), int(opticalInputs_.click), int(opticalInputs_.grip),
int(opticalInputs_.armed), int(opticalInputs_.gripArmed), errors[0], errors[1], errors[2],
hand.pose.time_ns > 0 ? double(now-hand.pose.time_ns)/1e6 : -1.,
static_cast<unsigned long long>(connection), static_cast<unsigned long long>(origin),
unsigned(inputEpochs & 255), unsigned(inputEpochs >> 8));
}
OpticalInputs opticalInputs_;
OpticalPose opticalPose_;
bool steadyHandAim_ = false;
bool opticalActive_ = false;
uint32_t lastOpticalState_ = ~0u;
uint64_t lastOpticalConnection_ = 0, lastOpticalOrigin_ = 0;
uint16_t lastOpticalEpochs_ = 0;
int lastOpticalErrors_[3]{};
static constexpr int16_t kGrabTrigger = 153;
static constexpr float kStickDeadzone = 0.12f, kHandRangeX = 0.40f, kHandRangeUp = 0.50f, kHandRangeDown = 0.30f, kHandSmoothing = 0.05f, kHandPivot = 0.45f;
bool right_;
@@ -1244,7 +1410,10 @@ public:
void DebugRequest(const char *, char *resp, uint32_t size) override { if (size) resp[0] = 0; }
DriverPose_t GetPose() override {
DriverPose_t GetPose() override { return GetPose(nullptr); }
DriverPose_t GetPose(droiddeck_spatialpose *snapshot) {
std::lock_guard<std::mutex> lock(spatialMutex_);
DriverPose_t p = {};
p.qWorldFromDriverRotation.w = 1;
p.qDriverFromHeadRotation.w = 1;
@@ -1274,6 +1443,26 @@ public:
p.poseIsValid = true;
p.result = TrackingResult_Running_OK;
p.deviceIsConnected = true;
droiddeck_spatialpose spatial{};
const bool spatialRead = spatial_.Read(&spatial);
if (spatialRead) lastSpatialSnapshot_ = spatial;
else {
spatial = lastSpatialSnapshot_;
spatial.head.flags = 0;
for (auto &hand : spatial.hands) hand.pose.flags = 0;
}
if (snapshot) *snapshot = spatial;
if (spatial.flags & DD_SPATIAL_ACTIVE) {
if (SpatialFresh(spatial.head, SpatialNow())) {
for (int i=0; i<3; ++i) p.vecPosition[i] = spatial.head.position[i];
p.qRotation = {spatial.head.rotation[3], spatial.head.rotation[0], spatial.head.rotation[1], spatial.head.rotation[2]};
lastSpatial_ = p;
} else {
if (lastSpatial_.deviceIsConnected) p = lastSpatial_;
p.poseIsValid = false;
p.result = TrackingResult_Running_OutOfRange;
}
}
return p;
}
@@ -1383,7 +1572,7 @@ private:
haveJoyCons = true;
}
const bool joyConsChanged = haveJoyCons && joyCons.seq != lastJoyConSeq_;
if ((fresh || padChanged || joyConsChanged || now - lastSent > std::chrono::milliseconds(100)) && id_ != k_unTrackedDeviceIndexInvalid) {
if ((fresh || padChanged || joyConsChanged || now - lastSent > std::chrono::milliseconds(10)) && id_ != k_unTrackedDeviceIndexInvalid) {
if (joyConsChanged) lastJoyConSeq_ = joyCons.seq;
if (fresh) {
lastSeq = hp.seq;
@@ -1400,9 +1589,12 @@ private:
Log("headset %s (Cardboard mode %s)", worn ? "worn" : "taken off", worn ? "on" : "off");
}
}
DriverPose_t head = GetPose();
droiddeck_spatialpose spatial{};
DriverPose_t head = GetPose(&spatial);
VRServerDriverHost()->TrackedDevicePoseUpdated(id_, head, sizeof(DriverPose_t));
if (havePad && (pad.buttons & PAD_L3) && !(lastPad_.buttons & PAD_L3)) {
const bool lumaJoyCons = (spatial.flags & DD_SPATIAL_ACTIVE) && (spatial.flags & DD_SPATIAL_JOYCONS);
const bool optical = !lumaJoyCons && (spatial.flags & DD_SPATIAL_ACTIVE) && (spatial.flags & DD_SPATIAL_HANDS);
if (!optical && havePad && (pad.buttons & PAD_L3) && !(lastPad_.buttons & PAD_L3)) {
sticksMoveHands_ = !sticksMoveHands_;
Log("sticks now move %s", sticksMoveHands_ ? "the hands" : "in the game");
}
@@ -1429,31 +1621,36 @@ private:
const bool running = (cam.flags & DROIDDECK_HANDCAM_RUNNING) != 0;
if (cam.frame != lastCamFrame_) {
lastCamFrame_ = cam.frame;
if (running && std::llabs(nowNs - cam.time_ns) < kHandCamStaleNs) SeeCamera(cam, nowNs);
if (!lumaJoyCons && !optical && running && std::llabs(nowNs - cam.time_ns) < kHandCamStaleNs) SeeCamera(cam, nowNs);
}
bareFlag_ = running && (cam.flags & DROIDDECK_HANDCAM_BARE_HANDS);
bareTimeNs_ = cam.time_ns;
}
const bool bare = bareFlag_ && std::llabs(nowNs - bareTimeNs_) < kBareHandsStaleNs;
const bool bare = !otherControllers_ && !optical && !lumaJoyCons && bareFlag_ && std::llabs(nowNs - bareTimeNs_) < kBareHandsStaleNs;
if (bare != bareHands_) {
bareHands_ = bare;
Log("%s", bare ? "no controllers: the camera tracks bare hands" : "camera bare hands off");
}
for (int side = 0; side < 2; side++) {
const droiddeck_joycon_hand *joyCon =
haveJoyCons && JoyConLive(joyCons.hands[side], nowNs) ? &joyCons.hands[side] : nullptr;
!otherControllers_ && !optical && haveJoyCons && JoyConLive(joyCons.hands[side], nowNs) ? &joyCons.hands[side] : nullptr;
if (handControllers_[side]) handControllers_[side]->ExpectBare(!joyCon && bare);
if (joyCon || bare) wantHand_[side] = true;
const bool covered = handControllers_[side] && handControllers_[side]->Update(joyCon, bare, head.qRotation, nowNs);
const bool covered = handControllers_[side] && handControllers_[side]->Update(joyCon, bare, head.qRotation, nowNs,
lumaJoyCons ? &spatial : nullptr);
CardboardController *hand = hands_[side];
if (!hand) continue;
if (otherControllers_ || covered || bare) {
if (otherControllers_ || covered || bare || lumaJoyCons) {
if (!padHandOff_[side]) hand->Disconnect();
padHandOff_[side] = true;
continue;
}
padHandOff_[side] = false;
hand->Update(head.qRotation, havePad ? &pad : nullptr, sticksMoveHands_, reach_);
if (optical) hand->UpdateOptical(spatial.hands[side], SpatialFresh(spatial.head, SpatialNow()),
havePad ? &pad : nullptr, SpatialNow(), spatial.connection, spatial.origin,
uint16_t(spatial.input_epochs >> (side*16)),
spatial.flags & DD_SPATIAL_STEADY_AIM ? &spatial.head : nullptr);
else hand->Update(head, havePad ? &pad : nullptr, sticksMoveHands_, reach_);
}
if (havePad) lastPad_ = pad;
lastSent = now;
@@ -1521,6 +1718,10 @@ public:
private:
PoseFile pose_;
SpatialFile spatial_;
std::mutex spatialMutex_;
DriverPose_t lastSpatial_{};
droiddeck_spatialpose lastSpatialSnapshot_{};
XrTrackingFile xr_;
XrChaperoneOrigin chaperone_;
bool xrMode_ = false;
+67
View File
@@ -0,0 +1,67 @@
#pragma once
#include "arm_model.h"
#include "spatial_tracking.h"
namespace droiddeck {
struct LumaJoyConPose {
Vec3 position;
bool valid = false, camera = false;
};
class LumaJoyConPosition {
public:
explicit LumaJoyConPosition(bool right) : side_(right ? 1 : 0), arm_(right) {}
void Reset() { arm_.Reset(); motion_.Reset(); shown_ = false; blend_ = {}; lastCamera_ = {}; }
LumaJoyConPose Update(const droiddeck_spatialpose &spatial, const Quat &rotation,
double angularSpeed, int64_t now, double dt) {
if (!(spatial.flags & DD_SPATIAL_ACTIVE) || !(spatial.flags & DD_SPATIAL_JOYCONS) ||
!SpatialFresh(spatial.head, now)) {
Reset();
return {};
}
if (connection_ != spatial.connection || origin_ != spatial.origin) {
Reset();
connection_ = spatial.connection; origin_ = spatial.origin;
}
dt = std::clamp(dt, 0.0, 0.1);
const auto &h = spatial.head;
const Vec3 head{h.position[0], h.position[1], h.position[2]};
const Quat headRotation{h.rotation[3], h.rotation[0], h.rotation[1], h.rotation[2]};
const Vec3 estimate = arm_.Update(head, headRotation, rotation, angularSpeed, dt);
const auto &observation = spatial.hands[side_];
const auto &p = observation.pose.position;
const Vec3 measured{p[0], p[1], p[2]};
if (SpatialHandFresh(observation, now) && Length(measured - head) <= 1.5 &&
observation.pose.time_ns > lastCamera_.time_ns) {
motion_.Observe(measured, observation.pose.time_ns);
lastCamera_ = observation.pose;
}
const bool camera = SpatialFresh(lastCamera_, now, 200000000LL);
if (camera) {
const Vec3 source{lastCamera_.position[0], lastCamera_.position[1], lastCamera_.position[2]};
position_ = shown_ ? position_ + (source - position_) * motion_.PositionAlpha(dt) : source;
blend_ = {};
} else {
motion_.Reset();
if (shown_ && camera_) blend_ = position_ - estimate;
blend_ = blend_ * std::exp(-dt / .18);
position_ = estimate + blend_;
}
shown_ = true; camera_ = camera;
return {position_, true, camera};
}
private:
const int side_;
ArmModel arm_;
AdaptivePositionMotion motion_;
uint64_t connection_ = 0, origin_ = 0;
bool shown_ = false, camera_ = false;
Vec3 position_, blend_;
dd_spatial_pose lastCamera_{};
};
}
+27 -30
View File
@@ -10,6 +10,7 @@ typedef unsigned long VisualID;
#include "../../../app/src/main/cpp/waylandcomp/src/headpose_shm.h"
#include "presenter.h"
#include "presenter_image.h"
#include "present_sync.h"
#include "xr_tracking_file.h"
#include "xr_frame_channel.h"
@@ -107,6 +108,7 @@ struct Imported {
VkDeviceMemory memory = VK_NULL_HANDLE;
uint32_t width = 0, height = 0;
uint32_t producerPid = 0;
VkFormat format = VK_FORMAT_UNDEFINED;
};
struct PendingImport {
@@ -133,14 +135,6 @@ private:
bool gpuComplete_ = true;
};
VkFormat UnormOf(uint32_t f) {
switch (f) {
case VK_FORMAT_R8G8B8A8_SRGB: return VK_FORMAT_R8G8B8A8_UNORM;
case VK_FORMAT_B8G8R8A8_SRGB: return VK_FORMAT_B8G8R8A8_UNORM;
default: return (VkFormat)f;
}
}
class Impl {
public:
static Impl &Get() {
@@ -276,7 +270,9 @@ private:
} else if (f.texture[0] && !frameChannel_) {
std::this_thread::sleep_until(at + guard_);
}
if (f.texture[0] && wayland_) FollowPanel();
if (!frameChannel_) {
if (const Imported *eye = Find(f.texture[0])) FollowPanel(eye->format);
}
if (f.texture[0] && (frameChannel_ ? Capture(f) : Draw(f))) shown++;
{
std::lock_guard<std::mutex> lock(mutex_);
@@ -335,12 +331,13 @@ private:
return false;
}
void FollowPanel() {
uint32_t w = 0, h = 0;
if (!PanelSize(&w, &h)) return;
void FollowPanel(VkFormat eyeFormat) {
uint32_t w = width_, h = height_;
if (wayland_ && !PanelSize(&w, &h)) return;
const bool resized = w != width_ || h != height_;
const bool encodingChanged = SrgbEyeFormat(eyeFormat) != SrgbEyeFormat(eyeFormat_);
const auto now = std::chrono::steady_clock::now();
if (!resized && (swapchain_ || now < retryAt_)) return;
if (!resized && !encodingChanged && (swapchain_ || now < retryAt_)) return;
vk_.DeviceWaitIdle(dev_);
if (swapchain_) vk_.DestroySwapchainKHR(dev_, swapchain_, nullptr);
swapchain_ = VK_NULL_HANDLE;
@@ -348,6 +345,7 @@ private:
if (resized) Log("presenter: panel now %ux%u (was %ux%u)", w, h, width_, height_);
width_ = w;
height_ = h;
eyeFormat_ = eyeFormat;
if (!CreateSwapchain()) retryAt_ = now + std::chrono::seconds(1);
}
@@ -539,7 +537,6 @@ private:
VkSemaphoreCreateInfo semci = {VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO, nullptr, 0};
vk_.CreateSemaphore(dev_, &semci, nullptr, &acquired_);
vk_.CreateSemaphore(dev_, &semci, nullptr, &rendered_);
if (!frameChannel_ && !CreateSwapchain() && !wayland_) return false;
Log("presenter: Vulkan on %s, queue family %u", props.deviceName, qfam_);
return true;
}
@@ -551,9 +548,11 @@ private:
vk_.GetPhysicalDeviceSurfaceFormatsKHR(pd_, surface_, &nf, nullptr);
std::vector<VkSurfaceFormatKHR> fmts(nf);
vk_.GetPhysicalDeviceSurfaceFormatsKHR(pd_, surface_, &nf, fmts.data());
VkSurfaceFormatKHR fmt = fmts.empty() ? VkSurfaceFormatKHR{VK_FORMAT_B8G8R8A8_UNORM, VK_COLOR_SPACE_SRGB_NONLINEAR_KHR} : fmts[0];
for (auto &f : fmts)
if (f.format == VK_FORMAT_B8G8R8A8_UNORM || f.format == VK_FORMAT_R8G8B8A8_UNORM) { fmt = f; break; }
VkSurfaceFormatKHR fmt{};
if (!EyeSurfaceFormat(eyeFormat_, fmts.data(), nf, &fmt)) {
Log("presenter: no surface format preserves eye format %d", eyeFormat_);
return false;
}
uint32_t npm = 0;
vk_.GetPhysicalDeviceSurfacePresentModesKHR(pd_, surface_, &npm, nullptr);
std::vector<VkPresentModeKHR> pms(npm);
@@ -606,17 +605,10 @@ private:
im.width = p.width;
im.height = p.height;
im.producerPid = p.producerPid;
const char *fl = getenv("DROIDDECK_VR_IMPORT_FORMAT_LIST");
const bool formatList = !fl || strcmp(fl, "0") != 0;
VkFormat formats[2] = {(VkFormat)p.format, UnormOf(p.format)};
VkImageFormatListCreateInfo list = {VK_STRUCTURE_TYPE_IMAGE_FORMAT_LIST_CREATE_INFO, nullptr, 2, formats};
VkExternalMemoryImageCreateInfo ext = {VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO, formatList ? &list : nullptr,
VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT};
VkImageCreateInfo ici = {VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO, &ext, VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT, VK_IMAGE_TYPE_2D,
UnormOf(p.format), {p.width, p.height, 1}, 1, 1, VK_SAMPLE_COUNT_1_BIT, VK_IMAGE_TILING_OPTIMAL,
VK_IMAGE_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_INPUT_ATTACHMENT_BIT,
VK_SHARING_MODE_EXCLUSIVE, 0, nullptr, VK_IMAGE_LAYOUT_UNDEFINED};
VkResult r = vk_.CreateImage(dev_, &ici, nullptr, &im.image);
im.format = (VkFormat)p.format;
// Dedicated OPAQUE_FD imports must match SteamVR's exported image description.
SharedEyeImage description(im.format, p.width, p.height);
VkResult r = vk_.CreateImage(dev_, &description.info, nullptr, &im.image);
if (r != VK_SUCCESS) { Log("presenter: import image %d", r); close(p.fd); return; }
VkMemoryRequirements req;
vk_.GetImageMemoryRequirements(dev_, im.image, &req);
@@ -632,8 +624,11 @@ private:
vk_.DestroyImage(dev_, im.image, nullptr);
return;
}
if (vk_.BindImageMemory(dev_, im.image, im.memory, 0) != VK_SUCCESS) {
vk_.DestroyImage(dev_, im.image, nullptr); vk_.FreeMemory(dev_, im.memory, nullptr);
r = vk_.BindImageMemory(dev_, im.image, im.memory, 0);
if (r != VK_SUCCESS) {
Log("presenter: bind imported image %d", r);
vk_.DestroyImage(dev_, im.image, nullptr);
vk_.FreeMemory(dev_, im.memory, nullptr);
return;
}
textures_.push_back(im);
@@ -798,6 +793,7 @@ private:
bool Draw(const EyeFrame &f) {
const Imported *eye[2] = {Find(f.texture[0]), Find(f.texture[1])};
if (!eye[0] || !eye[1] || !swapchain_) return false;
if (SrgbEyeFormat(eye[0]->format) != SrgbEyeFormat(eye[1]->format)) return false;
vk_.WaitForFences(dev_, 1, &fence_, VK_TRUE, UINT64_MAX);
uint32_t idx = 0;
VkResult r = vk_.AcquireNextImageKHR(dev_, swapchain_, 100000000ull, acquired_, VK_NULL_HANDLE, &idx);
@@ -902,6 +898,7 @@ private:
VkFence fence_ = VK_NULL_HANDLE;
VkSemaphore acquired_ = VK_NULL_HANDLE, rendered_ = VK_NULL_HANDLE;
VkSwapchainKHR swapchain_ = VK_NULL_HANDLE;
VkFormat eyeFormat_ = VK_FORMAT_UNDEFINED;
VkExtent2D extent_{};
std::vector<VkImage> images_;
std::vector<Imported> textures_;
+57
View File
@@ -0,0 +1,57 @@
#pragma once
#include <vulkan/vulkan_core.h>
namespace droiddeck {
inline VkFormat AlternateEyeFormat(VkFormat format) {
switch (format) {
case VK_FORMAT_R8G8B8A8_SRGB: return VK_FORMAT_R8G8B8A8_UNORM;
case VK_FORMAT_R8G8B8A8_UNORM: return VK_FORMAT_R8G8B8A8_SRGB;
case VK_FORMAT_B8G8R8A8_SRGB: return VK_FORMAT_B8G8R8A8_UNORM;
case VK_FORMAT_B8G8R8A8_UNORM: return VK_FORMAT_B8G8R8A8_SRGB;
default: return format;
}
}
inline bool SrgbEyeFormat(VkFormat format) {
return format == VK_FORMAT_R8G8B8A8_SRGB || format == VK_FORMAT_B8G8R8A8_SRGB;
}
struct SharedEyeImage {
VkFormat formats[2];
VkImageFormatListCreateInfo list;
VkExternalMemoryImageCreateInfo external;
VkImageCreateInfo info;
SharedEyeImage(VkFormat format, uint32_t width, uint32_t height)
: formats{format, AlternateEyeFormat(format)},
list{VK_STRUCTURE_TYPE_IMAGE_FORMAT_LIST_CREATE_INFO, nullptr, formats[0] == formats[1] ? 1u : 2u, formats},
external{VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO, &list, VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT},
info{VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO, &external, VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT, VK_IMAGE_TYPE_2D,
format, {width, height, 1}, 1, 1, VK_SAMPLE_COUNT_1_BIT, VK_IMAGE_TILING_OPTIMAL,
VK_IMAGE_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_SAMPLED_BIT |
VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT | VK_IMAGE_USAGE_INPUT_ATTACHMENT_BIT,
VK_SHARING_MODE_EXCLUSIVE, 0, nullptr, VK_IMAGE_LAYOUT_UNDEFINED} {}
SharedEyeImage(const SharedEyeImage &) = delete;
SharedEyeImage &operator=(const SharedEyeImage &) = delete;
};
inline bool EyeSurfaceFormat(VkFormat eye, const VkSurfaceFormatKHR *formats, uint32_t count, VkSurfaceFormatKHR *out) {
const bool srgb = SrgbEyeFormat(eye);
if (count == 1 && formats[0].format == VK_FORMAT_UNDEFINED) {
*out = {srgb ? VK_FORMAT_B8G8R8A8_SRGB : VK_FORMAT_B8G8R8A8_UNORM, formats[0].colorSpace};
return out->colorSpace == VK_COLOR_SPACE_SRGB_NONLINEAR_KHR;
}
for (uint32_t i = 0; i < count; i++) {
const auto &f = formats[i];
if (f.colorSpace != VK_COLOR_SPACE_SRGB_NONLINEAR_KHR) continue;
if (srgb ? SrgbEyeFormat(f.format) :
f.format == VK_FORMAT_R8G8B8A8_UNORM || f.format == VK_FORMAT_B8G8R8A8_UNORM) {
*out = f;
return true;
}
}
return false;
}
}
+38
View File
@@ -0,0 +1,38 @@
#define _GNU_SOURCE
#include <dlfcn.h>
#include <errno.h>
#include <linux/capability.h>
#include <stdint.h>
#include <string.h>
#include <sys/resource.h>
#include <sys/syscall.h>
#include <unistd.h>
extern char *program_invocation_short_name;
static int cannot_schedule_realtime(void) {
struct rlimit priority;
struct __user_cap_header_struct header = {_LINUX_CAPABILITY_VERSION_3, 0};
struct __user_cap_data_struct caps[2] = {{0}};
if (getrlimit(RLIMIT_RTPRIO, &priority) || priority.rlim_cur != 0 ||
syscall(SYS_capget, &header, caps)) return 0;
return !(caps[CAP_SYS_NICE / 32].effective & (UINT32_C(1) << (CAP_SYS_NICE % 32)));
}
int setrlimit(__rlimit_resource_t resource, const struct rlimit *limit) {
int (*next)(__rlimit_resource_t, const struct rlimit *) = dlsym(RTLD_NEXT, "setrlimit");
if (!next) { errno = ENOSYS; return -1; }
const int saved = errno;
if (resource == RLIMIT_RTTIME &&
!strcmp(program_invocation_short_name, "vrcompositor") && cannot_schedule_realtime()) {
struct rlimit current;
if (!getrlimit(resource, &current) &&
(limit->rlim_cur < current.rlim_cur || limit->rlim_max < current.rlim_max)) {
// SDL's failed RTKit fallback leaves a fatal RT budget on Android GPU submissions.
errno = EPERM;
return -1;
}
}
errno = saved;
return next(resource, limit);
}
+175
View File
@@ -0,0 +1,175 @@
#pragma once
#include "../../../app/src/main/cpp/waylandcomp/src/spatialpose_shm.h"
#include "adaptive_pose.h"
#include <algorithm>
#include <cmath>
#include <cstring>
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <time.h>
#include <unistd.h>
inline int64_t SpatialNow() {
timespec t{}; clock_gettime(CLOCK_MONOTONIC, &t);
return int64_t(t.tv_sec)*1000000000LL+t.tv_nsec;
}
inline bool SpatialFresh(const dd_spatial_pose &p, int64_t now, int64_t maxAge = 100000000LL) {
if (!(p.flags & DD_SPATIAL_VALID) || p.time_ns <= 0 || now < p.time_ns || now-p.time_ns > maxAge) return false;
for (float v : p.position) if (!std::isfinite(v)) return false;
float n = 0;
for (float v : p.rotation) { if (!std::isfinite(v)) return false; n += v*v; }
return n > 0.99f && n < 1.01f;
}
inline bool SpatialHandFresh(const dd_spatial_hand &hand, int64_t now) {
return SpatialFresh(hand.pose, now, 200000000LL);
}
struct OpticalPose {
static constexpr int64_t kVisualHoldNs = 350000000LL;
dd_spatial_hand last{};
dd_spatial_pose filtered{}, rendered{};
int64_t receivedAt = 0, renderedAt = 0;
float velocity[3]{};
unsigned observations = 0;
bool predict = false;
uint64_t connection = 0, origin = 0;
void Reset() {
last = {}; filtered = rendered = {};
receivedAt = renderedAt = 0;
observations = 0; predict = false; std::fill_n(velocity,3,0);
}
bool HoldLastObservation(int64_t now) {
observations = 0; predict = false; receivedAt = 0; renderedAt = now;
if (SpatialFresh(last.pose,now,kVisualHoldNs)) return true;
Reset();
return false;
}
bool Update(const dd_spatial_hand &hand, bool headValid, int64_t now, uint64_t nextConnection, uint64_t nextOrigin) {
if (!headValid || connection != nextConnection || origin != nextOrigin) Reset();
connection = nextConnection; origin = nextOrigin;
if (!headValid) return false;
if (!SpatialHandFresh(hand, now) || hand.pose.time_ns < last.pose.time_ns)
return HoldLastObservation(now);
if (!observations && hand.pose.time_ns == last.pose.time_ns) return HoldLastObservation(now);
const bool interpolate = observations > 0 && renderedAt > 0 && now >= renderedAt && SpatialHandFresh(last,now);
if (hand.pose.time_ns > last.pose.time_ns) {
const float dt = float(hand.pose.time_ns-last.pose.time_ns)/1e9f;
if (observations && dt > 0 && dt <= .2f) {
float speed2 = 0, previousSpeed2 = 0, direction = 0, acceleration2 = 0;
for (int i=0; i<3; ++i) {
const float v = (hand.pose.position[i]-last.pose.position[i])/dt;
speed2 += v*v;
previousSpeed2 += velocity[i]*velocity[i];
direction += v*velocity[i];
const float acceleration = (v-velocity[i])/dt;
acceleration2 += acceleration*acceleration;
velocity[i] = v;
}
predict = observations >= 2 && speed2 <= 2.25f && acceleration2 <= 36.f &&
direction > 0 && direction*direction >= .64f*speed2*previousSpeed2;
} else {
observations = 0; predict = false; std::fill_n(velocity,3,0);
}
filtered = hand.pose;
float norm = 0;
for (float v : filtered.rotation) norm += v*v;
for (float &v : filtered.rotation) v /= std::sqrt(norm);
++observations; last = hand; receivedAt = now;
}
auto target = filtered;
if (predict) {
// Inference already consumes most capture age; receipt-relative prediction advances between results.
float seconds = std::clamp(float(now-receivedAt)/1e9f,0.f,.08f);
float speed2 = 0;
for (float v : velocity) speed2 += v*v;
if (speed2 > 0) seconds = std::min(seconds,.02f/std::sqrt(speed2));
for (int i=0; i<3; ++i) target.position[i] += velocity[i]*seconds;
}
if (interpolate) {
const float alpha = 1.f-std::exp(-float(now-renderedAt)/15000000.f);
for (int i=0; i<3; ++i) rendered.position[i] += alpha*(target.position[i]-rendered.position[i]);
float dot = 0, norm = 0;
for (int i=0; i<4; ++i) dot += rendered.rotation[i]*target.rotation[i];
for (int i=0; i<4; ++i) {
rendered.rotation[i] += alpha*((dot < 0 ? -1 : 1)*target.rotation[i]-rendered.rotation[i]);
norm += rendered.rotation[i]*rendered.rotation[i];
}
for (float &v : rendered.rotation) v /= std::sqrt(norm);
rendered.time_ns = target.time_ns; rendered.flags = target.flags;
} else rendered = target;
renderedAt = now;
return SpatialHandFresh(last, now);
}
};
struct OpticalInputs {
bool armed = false, gripArmed = false, click = false, grip = false;
float trigger = 0;
uint64_t connection = 0, origin = 0;
int64_t lastObservation = 0;
uint16_t inputEpochs = 0;
void Release() { armed = gripArmed = click = grip = false; trigger = 0; }
void Reset() { Release(); lastObservation = 0; inputEpochs = 0; }
bool Update(const dd_spatial_hand &hand, bool headValid, int64_t now, uint64_t nextConnection, uint64_t nextOrigin,
uint16_t nextEpochs = 0) {
if (connection != nextConnection || origin != nextOrigin) {
Reset(); connection = nextConnection; origin = nextOrigin;
}
if ((inputEpochs & 0xff) != (nextEpochs & 0xff)) { armed = click = false; trigger = 0; }
if ((inputEpochs >> 8) != (nextEpochs >> 8)) gripArmed = grip = false;
inputEpochs = nextEpochs;
if (!headValid || !SpatialHandFresh(hand, now)) {
Release(); return false;
}
if (hand.pose.time_ns < lastObservation) { Release(); return false; }
const bool newObservation = hand.pose.time_ns > lastObservation;
if (lastObservation && now-lastObservation > 200000000LL) Release();
const bool triggerValid = (hand.pose.flags & DD_SPATIAL_TRIGGER_VALID) && std::isfinite(hand.trigger) && hand.trigger >= 0 && hand.trigger <= 1;
const bool gripValid = (hand.pose.flags & DD_SPATIAL_GRIP_VALID) && hand.grip <= 1;
if (!triggerValid) { armed = click = false; trigger = 0; }
if (!gripValid) gripArmed = grip = false;
if (!newObservation) return true;
lastObservation = hand.pose.time_ns;
if (triggerValid) {
if (hand.trigger < 0.25f) armed = true;
trigger = armed ? hand.trigger : 0;
click = click ? trigger > 0.45f : trigger > 0.65f;
}
if (gripValid) {
if (!hand.grip) gripArmed = true;
grip = gripArmed && hand.grip;
}
return true;
}
};
class SpatialFile {
public:
explicit SpatialFile(const char *path = "/run/droiddeck/" DROIDDECK_SPATIALPOSE_FILE) : path_(path) {}
SpatialFile(const SpatialFile &) = delete;
SpatialFile &operator=(const SpatialFile &) = delete;
bool Read(droiddeck_spatialpose *out) {
if (!map_) {
int fd = open(path_, O_RDONLY | O_CLOEXEC);
if (fd < 0) return false;
struct stat st{};
void *p = MAP_FAILED;
if (fstat(fd, &st) == 0 && st.st_size == sizeof(*out))
p = mmap(nullptr, sizeof(*out), PROT_READ, MAP_SHARED, fd, 0);
close(fd);
if (p == MAP_FAILED) return false;
map_ = static_cast<droiddeck_spatialpose *>(p);
}
for (int i = 0; i < 8; ++i) {
const auto seq = __atomic_load_n(&map_->seq, __ATOMIC_ACQUIRE);
if (seq & 1u) continue;
memcpy(out, map_, sizeof(*out));
__atomic_thread_fence(__ATOMIC_ACQUIRE);
if (seq == __atomic_load_n(&map_->seq, __ATOMIC_RELAXED))
return out->magic == DROIDDECK_SPATIALPOSE_MAGIC && out->version == DROIDDECK_SPATIALPOSE_VERSION && out->size == sizeof(*out);
}
return false;
}
~SpatialFile() { if (map_) munmap(map_, sizeof(*map_)); }
private:
const char *path_;
droiddeck_spatialpose *map_ = nullptr;
};
+19 -19
View File
@@ -969,26 +969,20 @@ static int submit_drop_wait(VkSemaphore s, uint64_t value) {
const uint64_t queued = b->submitted;
const uint64_t seen_us = b->seen_us;
VkDevice dev = b->dev;
_Atomic uint64_t *counter = b->counter;
pthread_mutex_unlock(&g_br_lock);
if (!value || value <= queued) return 0;
struct dev *d = dev_for(dev);
if (!d || quest_frames()) return 0;
PFN_vkGetSemaphoreCounterValue get = (PFN_vkGetSemaphoreCounterValue)d->gdpa(dev, "vkGetSemaphoreCounterValue");
if (!get) get = (PFN_vkGetSemaphoreCounterValue)d->gdpa(dev, "vkGetSemaphoreCounterValueKHR");
PFN_vkSignalSemaphore sig = (PFN_vkSignalSemaphore)d->gdpa(dev, "vkSignalSemaphore");
if (!sig) sig = (PFN_vkSignalSemaphore)d->gdpa(dev, "vkSignalSemaphoreKHR");
if (!get) return 0;
const uint64_t start = now_us();
const uint64_t until = start + (start - seen_us > SHARED_IDLE_US ? 0 : SUBMIT_GRACE_US);
for (;;) {
uint64_t local = 0;
if (get(dev, s, &local) == VK_SUCCESS && local >= value) return 0;
uint64_t shared = atomic_load(counter);
if (shared >= value && sig) {
VkSemaphoreSignalInfo si = {VK_STRUCTURE_TYPE_SEMAPHORE_SIGNAL_INFO, NULL, s, shared};
if (sig(dev, &si) == VK_SUCCESS) return 0;
}
/* Only the poller mirrors shared values. A second host signal here can
* race its signal and submit the same timeline value twice. */
if (now_us() >= until) break;
usleep(100);
}
@@ -1110,11 +1104,7 @@ static void sync_init(void) {
say("%s", "frames go to the presenter once their GPU work is done");
}
static struct tracked *track_take(VkQueue q) {
pthread_once(&g_sync_once, sync_init);
if (!g_sync) return NULL;
struct dev *d = dev_for(q);
if (!d) return NULL;
static struct tracked *track_take(struct dev *d) {
pthread_mutex_lock(&g_trk_lock);
struct tracked *t = track_pool_take(&g_trk, d->handle, d->gdpa);
pthread_mutex_unlock(&g_trk_lock);
@@ -1138,20 +1128,30 @@ static void track_forget_device(VkDevice dev) {
}
static VkResult submit_tracked(PFN_vkQueueSubmit next, VkQueue q, uint32_t count, const VkSubmitInfo *submits, VkFence fence) {
struct tracked *slot = track_take(q);
if (!slot) return next(q, count, submits, fence);
if (strcmp(program_invocation_short_name, "vrcompositor")) return next(q, count, submits, fence);
pthread_once(&g_sync_once, sync_init);
struct dev *d = dev_for(q);
if (!g_sync || !d) return next(q, count, submits, fence);
struct tracked *slot = track_take(d);
if (!slot) return VK_ERROR_OUT_OF_HOST_MEMORY;
const VkFence mine = slot->fence;
VkResult r = next(q, count, submits, fence ? fence : mine);
track_submitted(slot, r == VK_SUCCESS && fence ? next(q, 0, NULL, mine) : r);
if (r == VK_SUCCESS && fence) r = next(q, 0, NULL, mine);
track_submitted(slot, r);
return r;
}
static VkResult submit2_tracked(PFN_vkQueueSubmit2 next, VkQueue q, uint32_t count, const VkSubmitInfo2 *submits, VkFence fence) {
struct tracked *slot = track_take(q);
if (!slot) return next(q, count, submits, fence);
if (strcmp(program_invocation_short_name, "vrcompositor")) return next(q, count, submits, fence);
pthread_once(&g_sync_once, sync_init);
struct dev *d = dev_for(q);
if (!g_sync || !d) return next(q, count, submits, fence);
struct tracked *slot = track_take(d);
if (!slot) return VK_ERROR_OUT_OF_HOST_MEMORY;
const VkFence mine = slot->fence;
VkResult r = next(q, count, submits, fence ? fence : mine);
track_submitted(slot, r == VK_SUCCESS && fence ? next(q, 0, NULL, mine) : r);
if (r == VK_SUCCESS && fence) r = next(q, 0, NULL, mine);
track_submitted(slot, r);
return r;
}
+58
View File
@@ -0,0 +1,58 @@
#include "adaptive_pose.h"
#include <cassert>
#include <cstdio>
using namespace droiddeck;
int main() {
for (const int period : {10, 20}) {
AdaptivePositionMotion motion;
double position = 0, previous = 0, energy = 0, oldEnergy = 0, target = 0;
for (int frame = 0; frame < 2000; ++frame) {
if (frame % period == 0) {
target = (frame / period % 2 ? 1 : -1) * .004;
motion.Observe({target, 0, 0}, 1000000000LL + frame * 4000000LL);
}
position += (target - position) * motion.PositionAlpha(.004);
previous += (target - previous) * -std::expm1(-.004 / .06);
if (frame > 1000) { energy += position * position; oldEnergy += previous * previous; }
}
assert(energy < oldEnergy * .81);
motion.Reset(); position = previous = 0;
for (int frame = 0; frame < 500; ++frame) {
target = .5 * frame * .004;
motion.Observe({target, 0, 0}, 1000000000LL + frame * 4000000LL);
position += (target - position) * motion.PositionAlpha(.004);
previous += (target - previous) * -std::expm1(-.004 / .06);
}
assert(target - position < target - previous);
assert(target - position < .006);
}
AdaptivePositionMotion motion;
const double idle = motion.PositionAlpha(.01);
motion.Observe({}, 1000000000);
motion.Observe({.1, 0, 0}, 1080000000);
assert(motion.PositionAlpha(.01) > idle);
const double alpha = motion.PositionAlpha(.01);
motion.Observe({9, 9, 9}, 1080000000);
assert(motion.PositionAlpha(.01) == alpha);
motion.Observe({}, 2000000000);
assert(motion.PositionAlpha(.01) == idle);
motion.Reset();
assert(motion.PositionAlpha(.01) == idle);
assert(motion.PositionAlpha(0) == 0);
for (bool right : {false, true}) for (double yaw : {0., .8, -2.}) {
const Vec3 head{1, 1.6, -2};
const auto turn = AboutY(yaw);
const Vec3 shoulder = head + Rotate(turn, {right ? .18 : -.18, -.28, 0});
const Vec3 ray = Rotate(turn, {.2, .1, -.5});
const auto aim = ShoulderAim(head, turn, shoulder + ray, right, AboutX(1));
assert(Length(Rotate(aim, {0, 0, -1}) - ray * (1 / Length(ray))) < 1e-8);
const auto flippedPalm = ShoulderAim(head, turn, shoulder + ray, right, AboutX(-2));
assert(Length(Rotate(flippedPalm, {0, 0, -1}) - Rotate(aim, {0, 0, -1})) < 1e-8);
assert(ShoulderAim(head, turn, shoulder, right, AboutX(1)).x == AboutX(1).x);
}
const auto vertical = ShoulderAim({}, AboutX(M_PI / 2), {.3, -.1, -.5}, true, {});
assert(std::isfinite(vertical.w) && std::isfinite(vertical.y));
puts("PASS: adaptive jitter, motion response, capture time, reset, and shoulder aim");
}
+221
View File
@@ -0,0 +1,221 @@
#define _GNU_SOURCE
#include <ucontext.h>
#include <assert.h>
#include <stdint.h>
#include "../src/vk_layer_droiddeck_vr.c"
struct fake_fence { VkResult result; unsigned resets; };
static VkDevice device;
static VkResult submit_result = VK_SUCCESS, completion = VK_SUCCESS, marker_result = VK_SUCCESS;
static unsigned submits_seen, waits_seen, markers_seen, freed;
static int create_fails;
static VkFence caller_fence = (VkFence)(uintptr_t)99;
static struct fake_fence *fake(VkFence f) { return (struct fake_fence *)(uintptr_t)f; }
static VkResult VKAPI_CALL create(VkDevice d, const VkFenceCreateInfo *ci, const VkAllocationCallbacks *a, VkFence *out) {
assert(d == device && ci->flags == 0);
if (create_fails) return VK_ERROR_OUT_OF_HOST_MEMORY;
struct fake_fence *f = calloc(1, sizeof(*f));
f->result = VK_TIMEOUT;
*out = (VkFence)(uintptr_t)f;
return VK_SUCCESS;
}
static VkResult VKAPI_CALL wait_fence(VkDevice d, uint32_t n, const VkFence *f, VkBool32 all, uint64_t timeout) {
assert(d == device && n == 1 && all);
assert(timeout == 0);
waits_seen++;
return fake(*f)->result;
}
static VkResult VKAPI_CALL reset(VkDevice d, uint32_t n, const VkFence *f) {
assert(d == device && n == 1);
fake(*f)->resets++;
fake(*f)->result = VK_TIMEOUT;
return VK_SUCCESS;
}
static void VKAPI_CALL destroy(VkDevice d, VkFence f, const VkAllocationCallbacks *a) {
assert(d == device); free(fake(f)); freed++;
}
static PFN_vkVoidFunction VKAPI_CALL gdpa(VkDevice d, const char *name) {
assert(d == device);
if (!strcmp(name, "vkCreateFence")) return (PFN_vkVoidFunction)create;
if (!strcmp(name, "vkWaitForFences")) return (PFN_vkVoidFunction)wait_fence;
if (!strcmp(name, "vkResetFences")) return (PFN_vkVoidFunction)reset;
if (!strcmp(name, "vkDestroyFence")) return (PFN_vkVoidFunction)destroy;
assert(0); return NULL;
}
static VkResult submitted(uint32_t n, VkFence f) {
submits_seen++;
if (!strcmp(program_invocation_short_name, "vrserver")) return submit_result;
assert(g_trk && g_trk->state == TRACK_RESERVED);
if (!n) markers_seen++;
VkResult r = n ? submit_result : marker_result;
if (r == VK_SUCCESS && f != caller_fence) fake(f)->result = completion;
return r;
}
static VkResult VKAPI_CALL submit1(VkQueue q, uint32_t n, const VkSubmitInfo *info, VkFence f) {
assert((n == 0) == (info == NULL)); return submitted(n, f);
}
static VkResult VKAPI_CALL submit2(VkQueue q, uint32_t n, const VkSubmitInfo2 *info, VkFence f) {
assert((n == 0) == (info == NULL)); return submitted(n, f);
}
static void no_init(void) {}
static VkQueue forwarded_queue;
static const void *forwarded_info;
static VkFence forwarded_fence;
static VkResult VKAPI_CALL forward1(VkQueue q, uint32_t n, const VkSubmitInfo *info, VkFence fence) {
assert(q == forwarded_queue && n == 1 && info == forwarded_info && fence == forwarded_fence);
submits_seen++;
return submit_result;
}
static VkResult VKAPI_CALL forward2(VkQueue q, uint32_t n, const VkSubmitInfo2 *info, VkFence fence) {
assert(q == forwarded_queue && n == 1 && info == forwarded_info && fence == forwarded_fence);
submits_seen++;
return submit_result;
}
static void unavailable_tracking(int modern, int missing_sync, VkFence fence, VkResult result) {
struct droiddeck_present_sync *saved_sync = g_sync;
const uint32_t issued = saved_sync->issued, done = saved_sync->done;
const unsigned freed_before = freed;
uintptr_t unknown_dispatch = 2;
forwarded_queue = missing_sync ? (VkQueue)device : (VkQueue)&unknown_dispatch;
if (missing_sync) g_sync = NULL;
VkSubmitInfo info = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO };
VkSubmitInfo2 info2 = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO_2 };
forwarded_info = modern ? (const void *)&info2 : (const void *)&info;
forwarded_fence = fence;
submits_seen = waits_seen = markers_seen = 0;
submit_result = result;
VkResult r = modern ? submit2_tracked(forward2, forwarded_queue, 1, &info2, fence)
: submit_tracked(forward1, forwarded_queue, 1, &info, fence);
assert(r == result && submits_seen == 1 && !waits_seen && !markers_seen && !g_trk);
assert(freed == freed_before && saved_sync->issued == issued && saved_sync->done == done);
g_sync = saved_sync;
}
static void scenario(int modern, VkFence fence, VkResult submitStatus, VkResult markerStatus, VkResult completeStatus) {
submit_result = submitStatus; marker_result = markerStatus; completion = completeStatus;
submits_seen = waits_seen = markers_seen = 0;
VkSubmitInfo info = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO };
VkSubmitInfo2 info2 = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO_2 };
VkResult r = modern ? submit2_tracked(submit2, (VkQueue)device, 1, &info2, fence)
: submit_tracked(submit1, (VkQueue)device, 1, &info, fence);
VkResult tracking = submitStatus == VK_SUCCESS && fence ? markerStatus : submitStatus;
assert(r == tracking);
assert(waits_seen == 0);
assert(markers_seen == (submitStatus == VK_SUCCESS && fence ? 1u : 0u));
assert(submits_seen == 1 + markers_seen);
if (tracking == VK_SUCCESS) {
assert(g_trk->state == TRACK_SUBMITTED && fake(g_trk->fence)->resets == 0);
if (completeStatus == VK_TIMEOUT) {
uint32_t done;
assert(track_pool_reap(g_trk, g_sync->issued, &done));
assert(g_trk->state == TRACK_SUBMITTED && fake(g_trk->fence)->resets == 0);
}
fake(g_trk->fence)->result = VK_SUCCESS;
uint32_t done;
assert(!track_pool_reap(g_trk, g_sync->issued, &done));
}
assert(!track_pool_device_busy(g_trk, device));
track_pool_forget(&g_trk, device);
}
static VkFence dependency_fence;
static VkQueue dependency_queue;
static VkResult dependency_submit(VkQueue q, VkFence fence, uint64_t wait, uint64_t signal) {
assert(g_trk && g_trk->state == TRACK_RESERVED);
if (q == dependency_queue) {
assert(wait == 1 && signal == 0 && dependency_fence == VK_NULL_HANDLE);
dependency_fence = fence;
fake(fence)->result = VK_TIMEOUT;
} else {
assert(wait == 0 && signal == 1 && dependency_fence != VK_NULL_HANDLE);
fake(dependency_fence)->result = fake(fence)->result = VK_SUCCESS;
}
return VK_SUCCESS;
}
static VkResult VKAPI_CALL dependent_submit1(VkQueue q, uint32_t n, const VkSubmitInfo *info, VkFence fence) {
assert(n == 1);
const VkTimelineSemaphoreSubmitInfo *timeline = info->pNext;
assert(timeline->sType == VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO);
return dependency_submit(q, fence, info->waitSemaphoreCount ? timeline->pWaitSemaphoreValues[0] : 0,
info->signalSemaphoreCount ? timeline->pSignalSemaphoreValues[0] : 0);
}
static VkResult VKAPI_CALL dependent_submit2(VkQueue q, uint32_t n, const VkSubmitInfo2 *info, VkFence fence) {
assert(n == 1);
return dependency_submit(q, fence, info->waitSemaphoreInfoCount ? info->pWaitSemaphoreInfos[0].value : 0,
info->signalSemaphoreInfoCount ? info->pSignalSemaphoreInfos[0].value : 0);
}
static void wait_before_signal(int modern) {
uintptr_t dispatch = *(uintptr_t *)device;
VkQueue signalling = (VkQueue)&dispatch;
dependency_queue = (VkQueue)device;
dependency_fence = VK_NULL_HANDLE;
VkSemaphore semaphore = (VkSemaphore)(uintptr_t)123;
uint64_t value = 1;
VkPipelineStageFlags stage = VK_PIPELINE_STAGE_ALL_COMMANDS_BIT;
VkTimelineSemaphoreSubmitInfo timeline_wait = { .sType = VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO,
.waitSemaphoreValueCount = 1, .pWaitSemaphoreValues = &value };
VkTimelineSemaphoreSubmitInfo timeline_signal = { .sType = VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO,
.signalSemaphoreValueCount = 1, .pSignalSemaphoreValues = &value };
VkSubmitInfo wait = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO, .pNext = &timeline_wait,
.waitSemaphoreCount = 1, .pWaitSemaphores = &semaphore, .pWaitDstStageMask = &stage };
VkSubmitInfo signal = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO, .pNext = &timeline_signal,
.signalSemaphoreCount = 1, .pSignalSemaphores = &semaphore };
VkSemaphoreSubmitInfo dependency = { .sType = VK_STRUCTURE_TYPE_SEMAPHORE_SUBMIT_INFO,
.semaphore = semaphore, .value = 1, .stageMask = VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT };
VkSubmitInfo2 wait2 = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO_2,
.waitSemaphoreInfoCount = 1, .pWaitSemaphoreInfos = &dependency };
VkSubmitInfo2 signal2 = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO_2,
.signalSemaphoreInfoCount = 1, .pSignalSemaphoreInfos = &dependency };
assert((modern ? submit2_tracked(dependent_submit2, dependency_queue, 1, &wait2, VK_NULL_HANDLE)
: submit_tracked(dependent_submit1, dependency_queue, 1, &wait, VK_NULL_HANDLE)) == VK_SUCCESS);
uint32_t done;
assert(track_pool_reap(g_trk, g_sync->issued, &done));
assert(done == g_sync->issued - 1);
assert((modern ? submit2_tracked(dependent_submit2, signalling, 1, &signal2, VK_NULL_HANDLE)
: submit_tracked(dependent_submit1, signalling, 1, &signal, VK_NULL_HANDLE)) == VK_SUCCESS);
assert(!track_pool_reap(g_trk, g_sync->issued, &done));
assert(done == g_sync->issued);
track_pool_forget(&g_trk, device);
}
int main(void) {
uintptr_t dispatch = 1;
device = (VkDevice)&dispatch;
g_dev[0] = (struct dev){ .handle = device, .gdpa = gdpa };
struct droiddeck_present_sync sync = {0};
g_sync = &sync;
pthread_once(&g_sync_once, no_init);
program_invocation_short_name = "vrcompositor";
for (int modern = 0; modern < 2; modern++) {
scenario(modern, VK_NULL_HANDLE, VK_SUCCESS, VK_SUCCESS, VK_SUCCESS);
scenario(modern, caller_fence, VK_SUCCESS, VK_SUCCESS, VK_SUCCESS);
scenario(modern, VK_NULL_HANDLE, VK_SUCCESS, VK_SUCCESS, VK_TIMEOUT);
scenario(modern, caller_fence, VK_SUCCESS, VK_SUCCESS, VK_ERROR_DEVICE_LOST);
scenario(modern, caller_fence, VK_ERROR_DEVICE_LOST, VK_SUCCESS, VK_SUCCESS);
scenario(modern, caller_fence, VK_SUCCESS, VK_ERROR_DEVICE_LOST, VK_SUCCESS);
}
assert(freed == 12);
wait_before_signal(0);
wait_before_signal(1);
for (int modern = 0; modern < 2; modern++) {
for (int missing_sync = 0; missing_sync < 2; missing_sync++) {
unavailable_tracking(modern, missing_sync, VK_NULL_HANDLE, VK_SUCCESS);
unavailable_tracking(modern, missing_sync, caller_fence, VK_SUCCESS);
unavailable_tracking(modern, missing_sync, caller_fence, VK_ERROR_DEVICE_LOST);
}
}
create_fails = 1;
submits_seen = 0;
VkSubmitInfo info = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO };
VkSubmitInfo2 info2 = { .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO_2 };
assert(submit_tracked(submit1, (VkQueue)device, 1, &info, VK_NULL_HANDLE) == VK_ERROR_OUT_OF_HOST_MEMORY);
assert(submit2_tracked(submit2, (VkQueue)device, 1, &info2, caller_fence) == VK_ERROR_OUT_OF_HOST_MEMORY);
assert(!submits_seen && !g_trk);
program_invocation_short_name = "vrserver";
submit_result = VK_SUCCESS; waits_seen = 0;
assert(submit_tracked(submit1, (VkQueue)device, 1, &info, caller_fence) == VK_SUCCESS);
assert(submits_seen == 1 && waits_seen == 0 && !g_trk);
puts("PASS: compositor submissions stay asynchronous, preserve caller fences, and allow later cross-queue signals");
}
+79
View File
@@ -6,6 +6,7 @@
#include "index_pose.h"
#include "handcam_shm.h"
#include "joycon_shm.h"
#include "luma_joycon.h"
#include <cstdio>
#include <cstdlib>
@@ -516,7 +517,85 @@ static void handCameraLayoutMatchesTheAppWriter() {
"camera flags match HandCameraShm.kt");
}
static void lumaStereoFallsBackInTheHeadTrackingOrigin() {
int64_t now = 1000000000;
droiddeck_spatialpose s{};
s.flags = DD_SPATIAL_ACTIVE | DD_SPATIAL_JOYCONS;
s.connection = s.origin = 1;
s.head = {now, {2, 1.7f, 3}, {0, 0, 0, 1}, DD_SPATIAL_VALID};
s.hands[0].pose = {now, {1.7f, 1.3f, 2.6f}, {0, 0, 0, 1}, DD_SPATIAL_VALID};
s.hands[1].pose = {now, {2.3f, 1.3f, 2.6f}, {0, 0, 0, 1}, DD_SPATIAL_VALID};
LumaJoyConPosition left(false), right(true);
auto l = left.Update(s, {}, 0, now, .01);
auto r = right.Update(s, {}, 0, now, .01);
check(l.valid && l.camera && r.camera && near(l.position.x, 1.7) && near(r.position.x, 2.3),
"stereo positions retain their world coordinates and handedness");
s.head.position[0] += .1f;
l = left.Update(s, {}, 0, now, .01);
check(near(l.position.x, 1.7), "moving the head does not move a world-space stereo observation");
now += 200000001;
s.head.time_ns = now;
l = left.Update(s, {}, 0, now, .01);
check(l.valid && !l.camera && Length(l.position - Vec3{1.7, 1.3, 2.6}) < .04,
"stale stereo falls back with a continuous transition");
for (int i=0; i<100; ++i) l = left.Update(s, {}, 0, now, .01);
ArmModel expected(false);
const auto anchor = expected.Update({2.1, 1.7, 3}, {}, {}, 0, .01);
check(Length(l.position - anchor) < .003, "fallback settles to the translated 6DoF head anchor");
s.hands[0].pose.time_ns = now;
auto reacquired = left.Update(s, {}, 0, now, .01);
check(reacquired.camera && Length(reacquired.position - l.position) < .04, "reacquisition blends without a snap");
s.origin++;
s.head.position[0] = 0;
s.hands[0].pose = {};
l = left.Update(s, {}, 0, now, .01);
check(l.valid && !l.camera && near(l.position.x, -.195), "origin changes discard the old world position and blend");
s.head.time_ns = now - 100000001;
check(!left.Update(s, {}, 0, now, .01).valid, "head loss invalidates instead of guessing a world origin");
s.head.time_ns = now;
s.hands[0].pose = {now + 1, {0, 1.3f, 3}, {0,0,0,1}, DD_SPATIAL_VALID};
check(!left.Update(s, {}, 0, now, .01).camera, "future camera observations are rejected");
s.hands[0].pose.time_ns = now;
s.hands[0].pose.position[0] = 99;
check(!left.Update(s, {}, 0, now, .01).camera, "implausible hand positions use the arm model");
s.flags = DD_SPATIAL_ACTIVE;
check(!left.Update(s, {}, 0, now, .01).valid, "the experimental flag is required");
}
static void lumaReachSurvivesBriefDetectionMissesWithoutRefreshingOldDepth() {
int64_t now = 1000000000;
droiddeck_spatialpose s{};
s.flags = DD_SPATIAL_ACTIVE | DD_SPATIAL_JOYCONS;
s.connection = s.origin = 1;
s.head = {now, {0, 1.7f, 0}, {0,0,0,1}, DD_SPATIAL_VALID};
s.hands[0].pose = {now, {-.2f, 1.3f, -.3f}, {0,0,0,1}, DD_SPATIAL_VALID};
LumaJoyConPosition tracker(false);
auto pose = tracker.Update(s, {}, 0, now, .01);
s.hands[0] = {};
for (int i=0; i<19; ++i) {
now += 10000000; s.head.time_ns = now;
pose = tracker.Update(s, {}, 0, now, .01);
check(pose.camera && near(pose.position.z, -.3), "brief misses retain the last measured reach");
}
now += 20000000; s.head.time_ns = now;
check(!tracker.Update(s, {}, 0, now, .02).camera, "held depth expires at its original capture timestamp");
for (int i=0; i<100; ++i) {
now += 10000000; s.head.time_ns = now;
if (i % 16 == 0)
s.hands[0].pose = {now - 60000000, {-.2f, 1.3f, -.7f}, {0,0,0,1}, DD_SPATIAL_VALID};
if (i % 16 == 8) s.hands[0] = {};
pose = tracker.Update(s, {}, 0, now, .01);
}
check(pose.position.z < -.65, "intermittent stereo observations extend reach instead of repeatedly resetting to the arm model");
s.connection++;
s.hands[0] = {};
pose = tracker.Update(s, {}, 0, now, .01);
check(!pose.camera && near(pose.position.z, -.305), "reconnection cannot reuse cached camera depth");
}
int main() {
lumaReachSurvivesBriefDetectionMissesWithoutRefreshingOldDepth();
lumaStereoFallsBackInTheHeadTrackingOrigin();
restingHandsHangInFrontAtElbowHeight();
raisingTheControllerExtendsTheArm();
theTorsoOnlyFollowsTheHeadWhileTheControllerMoves();
+263
View File
@@ -0,0 +1,263 @@
#include "../src/driver_droiddeck.cpp"
#include <cassert>
#include <map>
void droiddeck::Presenter::AddTextures(const uint64_t[3], const int[3], uint32_t, uint32_t, uint32_t, uint32_t) {}
void droiddeck::Presenter::RemoveTexture(uint64_t) {}
void droiddeck::Presenter::Submit(const EyeFrame &) {}
void droiddeck::Presenter::Shutdown() {}
struct Host : IVRServerDriverHost {
std::vector<DriverPose_t> poses;
const std::map<VRInputComponentHandle_t, float> *inputs = nullptr;
std::vector<std::map<VRInputComponentHandle_t, float>> inputsAtPose;
bool TrackedDeviceAdded(const char *, ETrackedDeviceClass, ITrackedDeviceServerDriver *) override { return true; }
void TrackedDevicePoseUpdated(uint32_t, const DriverPose_t &p, uint32_t) override {
poses.push_back(p); inputsAtPose.push_back(*inputs);
}
void VsyncEvent(double) override {}
void VendorSpecificEvent(uint32_t, EVREventType, const VREvent_Data_t &, double) override {}
bool IsExiting() override { return false; }
bool PollNextEvent(VREvent_t *, uint32_t) override { return false; }
void GetRawTrackedDevicePoses(float, TrackedDevicePose_t *, uint32_t) override {}
void RequestRestart(const char *, const char *, const char *, const char *) override {}
uint32_t GetFrameTimings(Compositor_FrameTiming *, uint32_t) override { return 0; }
void SetDisplayEyeToHead(uint32_t, const HmdMatrix34_t &, const HmdMatrix34_t &) override {}
void SetDisplayProjectionRaw(uint32_t, const HmdRect2_t &, const HmdRect2_t &) override {}
void SetRecommendedRenderTargetSize(uint32_t, uint32_t, uint32_t) override {}
};
struct Properties : IVRProperties {
ETrackedPropertyError ReadPropertyBatch(PropertyContainerHandle_t, PropertyRead_t *, uint32_t) override { return TrackedProp_Success; }
ETrackedPropertyError WritePropertyBatch(PropertyContainerHandle_t, PropertyWrite_t *, uint32_t) override { return TrackedProp_Success; }
const char *GetPropErrorNameFromEnum(ETrackedPropertyError) override { return "success"; }
PropertyContainerHandle_t TrackedDeviceToPropertyContainer(TrackedDeviceIndex_t id) override { return id+1; }
};
struct Input : IVRDriverInput {
std::map<VRInputComponentHandle_t, float> values;
std::map<std::string, VRInputComponentHandle_t> handles;
std::map<VRInputComponentHandle_t, EVRInputError> failures;
EVRInputError CreateBooleanComponent(PropertyContainerHandle_t, const char *name, VRInputComponentHandle_t *h) override {
*h = values.size()+1; values[*h]=0; handles[name]=*h; return VRInputError_None;
}
EVRInputError UpdateBooleanComponent(VRInputComponentHandle_t h, bool v, double) override {
if (failures[h] != VRInputError_None) return failures[h];
values[h]=v; return VRInputError_None;
}
EVRInputError CreateScalarComponent(PropertyContainerHandle_t c, const char *n, VRInputComponentHandle_t *h, EVRScalarType, EVRScalarUnits) override { return CreateBooleanComponent(c,n,h); }
EVRInputError UpdateScalarComponent(VRInputComponentHandle_t h, float v, double) override {
if (failures[h] != VRInputError_None) return failures[h];
values[h]=v; return VRInputError_None;
}
EVRInputError CreateHapticComponent(PropertyContainerHandle_t c, const char *n, VRInputComponentHandle_t *h) override { return CreateBooleanComponent(c,n,h); }
EVRInputError CreateSkeletonComponent(PropertyContainerHandle_t, const char *, const char *, const char *, EVRSkeletalTrackingLevel, const VRBoneTransform_t *, uint32_t, VRInputComponentHandle_t *) override { return VRInputError_None; }
EVRInputError UpdateSkeletonComponent(VRInputComponentHandle_t, EVRSkeletalMotionRange, const VRBoneTransform_t *, uint32_t) override { return VRInputError_None; }
EVRInputError CreatePoseComponent(PropertyContainerHandle_t, const char *, VRInputComponentHandle_t *) override { return VRInputError_None; }
EVRInputError UpdatePoseComponent(VRInputComponentHandle_t, const HmdMatrix34_t *, double) override { return VRInputError_None; }
EVRInputError CreateEyeTrackingComponent(PropertyContainerHandle_t, const char *, VRInputComponentHandle_t *) override { return VRInputError_None; }
EVRInputError UpdateEyeTrackingComponent(VRInputComponentHandle_t, const VREyeTrackingData_t *, double) override { return VRInputError_None; }
};
struct DriverLog : IVRDriverLog {
std::vector<std::string> lines;
void Log(const char *message) override { lines.emplace_back(message); }
};
struct Context : IVRDriverContext {
Host host;
Input input;
Properties properties;
DriverLog log;
Context() { host.inputs = &input.values; }
void *GetGenericInterface(const char *name, EVRInitError *) override {
if (!strcmp(name, IVRServerDriverHost_Version)) return &host;
if (!strcmp(name, IVRDriverInput_Version)) return &input;
if (!strcmp(name, IVRProperties_Version)) return &properties;
if (!strcmp(name, IVRDriverLog_Version)) return &log;
return nullptr;
}
DriverHandle_t GetDriverHandle() override { return 1; }
};
int main() {
Context context;
VRDriverContext() = &context;
CardboardController controller(false);
assert(controller.Activate(1) == VRInitError_None);
const auto gripHandle=context.input.handles.at("/input/grip/click");
const auto clickHandle=context.input.handles.at("/input/trigger/click");
const auto triggerHandle=context.input.handles.at("/input/trigger/value");
const auto assertReleasedAtPose = [&] {
const auto &inputs=context.host.inputsAtPose.back();
assert(inputs.at(gripHandle)==0 && inputs.at(clickHandle)==0 && inputs.at(triggerHandle)==0);
};
dd_spatial_hand hand{};
for (int i=0; i<500; ++i) controller.UpdateOptical(hand, false, nullptr, 1000000000+i, 1, 1);
assert(context.host.poses.size() == 500);
assert(context.log.lines.size()==2);
for (const auto &pose : context.host.poses) {
assert(pose.deviceIsConnected && !pose.poseIsValid);
assert(pose.result == TrackingResult_Running_OutOfRange);
assert(pose.qRotation.w == 1);
}
hand.pose = {1000000000, {1,2,3}, {0,0,0,1}, DD_SPATIAL_VALID | DD_SPATIAL_TRIGGER_VALID | DD_SPATIAL_GRIP_VALID};
int64_t now=1000000000;
uint64_t connection=1,origin=1;
uint16_t epochs=0;
const auto observe = [&](float trigger,uint32_t grip) {
now+=80000000;hand.pose.time_ns=now;hand.trigger=trigger;hand.grip=grip;
controller.UpdateOptical(hand,true,nullptr,now,connection,origin,epochs);
};
const auto missing = [&](bool headValid,int64_t elapsed) {
now+=elapsed;controller.UpdateOptical({},headValid,nullptr,now,connection,origin,epochs);
};
const auto assertPressed = [&] {
assert(context.host.poses.back().poseIsValid);
assert(context.input.values.at(gripHandle)==1 && context.input.values.at(clickHandle)==1 &&
context.input.values.at(triggerHandle)==1);
};
observe(0,0);observe(1,1);assertPressed();
now+=200000000;controller.UpdateOptical(hand,true,nullptr,now,connection,origin,epochs);
assertPressed();
++now;controller.UpdateOptical(hand,true,nullptr,now,connection,origin,epochs);
assert(context.host.poses.back().deviceIsConnected && context.host.poses.back().poseIsValid);
assertReleasedAtPose();
now=hand.pose.time_ns+350000000;controller.UpdateOptical(hand,true,nullptr,now,connection,origin,epochs);
assert(context.host.poses.back().poseIsValid);
assert(std::abs(context.host.poses.back().poseTimeOffset+.35)<1e-9);
assertReleasedAtPose();
++now;controller.UpdateOptical(hand,true,nullptr,now,connection,origin,epochs);
assert(context.host.poses.back().deviceIsConnected && !context.host.poses.back().poseIsValid);
assertReleasedAtPose();
observe(0,0);observe(1,1);assertPressed();
missing(true,50000000);
assert(context.host.poses.back().poseIsValid && context.host.poses.back().vecPosition[0]==1);
assert(std::abs(context.host.poses.back().poseTimeOffset+.05)<1e-9);
assertReleasedAtPose();
missing(true,150000000);
assert(context.host.poses.back().poseIsValid);
assert(std::abs(context.host.poses.back().poseTimeOffset+.2)<1e-9);
missing(true,150000000);
assert(context.host.poses.back().poseIsValid);
assert(std::abs(context.host.poses.back().poseTimeOffset+.35)<1e-9);
assertReleasedAtPose();
missing(true,1);assert(!context.host.poses.back().poseIsValid);
observe(1,1);assertReleasedAtPose();
observe(0,0);observe(1,1);assertPressed();
missing(false,10000000);
assert(!context.host.poses.back().poseIsValid);assertReleasedAtPose();
missing(true,10000000);assert(!context.host.poses.back().poseIsValid);
observe(0,0);observe(1,1);assertPressed();
epochs=1;observe(1,1);
assert(context.host.poses.back().poseIsValid);
assert(context.input.values.at(gripHandle)==1 && context.input.values.at(clickHandle)==0 && context.input.values.at(triggerHandle)==0);
epochs=0x101;observe(1,1);assertReleasedAtPose();
hand.trigger=0;hand.grip=0;
controller.UpdateOptical(hand,true,nullptr,now+1,connection,origin,epochs);
observe(1,1);assertReleasedAtPose();
observe(0,0);observe(1,1);assertPressed();
++origin;missing(true,10000000);
assert(!context.host.poses.back().poseIsValid);assertReleasedAtPose();
observe(1,1);assertReleasedAtPose();
observe(0,0);observe(1,1);assertPressed();
++connection;missing(true,10000000);
assert(!context.host.poses.back().poseIsValid);assertReleasedAtPose();
observe(0,0);observe(1,1);assertPressed();
controller.Disconnect();
assert(!context.host.poses.back().deviceIsConnected);assertReleasedAtPose();
missing(true,10000000);assert(!context.host.poses.back().poseIsValid);
observe(1,1);assertReleasedAtPose();
controller.Update(controller.GetPose(),nullptr,false,0);
missing(true,10000000);assert(!context.host.poses.back().poseIsValid);
PadState pad{};pad.axes[0]=16384;pad.buttons=PAD_SELECT;
controller.UpdateOptical({},true,&pad,++now,connection,origin,epochs);
assert(!context.host.poses.back().poseIsValid);
assert(std::any_of(context.input.values.begin(),context.input.values.end(),[](const auto &v){ return v.second!=0; }));
observe(0,0);observe(1,1);assertPressed();
context.input.failures[gripHandle]=VRInputError_IPCError;
missing(false,10000000);
assert(context.input.values.at(gripHandle)==1 && context.input.values.at(triggerHandle)==0);
assert(context.log.lines.back().find("error=7/0/0")!=std::string::npos);
const auto failedLogCount=context.log.lines.size();
for(int i=0;i<10;++i) missing(false,1);
assert(context.log.lines.size()==failedLogCount);
context.input.failures[gripHandle]=VRInputError_None;
missing(false,10000000);
assertReleasedAtPose();
assert(context.log.lines.back().find("error=0/0/0")!=std::string::npos);
assert(context.log.lines.size()==failedLogCount+1);
observe(1,1);assertReleasedAtPose();
observe(0,0);observe(1,1);assertPressed();
const dd_spatial_pose aimHead{now, {0,1.6f,0}, {0,0,0,1}, DD_SPATIAL_VALID};
controller.UpdateOptical(hand,true,nullptr,now,connection,origin,epochs,&aimHead);
assertPressed();
const auto aimed = controller.GetPose();
const auto expectedAim = ShoulderAim({0,1.6f,0}, {}, {1,2,3}, false, {});
assert(std::abs(aimed.qRotation.w - expectedAim.w) < 1e-6);
assert(std::abs(aimed.qRotation.y - expectedAim.y) < 1e-6);
assert(aimed.poseTimeOffset == 0);
controller.UpdateOptical(hand,true,nullptr,now,connection,origin,epochs);
assertPressed();
assert(controller.GetPose().qRotation.w == 1);
const auto palmRotation=Normalize(AboutX(.4)*AboutY(-.6));
hand.pose.rotation[0]=palmRotation.x;hand.pose.rotation[1]=palmRotation.y;
hand.pose.rotation[2]=palmRotation.z;hand.pose.rotation[3]=palmRotation.w;
controller.UpdateOptical(hand,true,nullptr,now,connection,++origin,epochs);
const auto normal=controller.GetPose();
assert(std::abs(normal.qRotation.w-palmRotation.w)<1e-6);
assert(std::abs(normal.qRotation.x-palmRotation.x)<1e-6);
assert(std::abs(normal.qRotation.y-palmRotation.y)<1e-6);
assert(std::abs(normal.qRotation.z-palmRotation.z)<1e-6);
assert(normal.vecPosition[0]==hand.pose.position[0] && normal.poseTimeOffset==0);
controller.UpdateOptical({},true,nullptr,now+1,connection,origin,epochs,&aimHead);
assertReleasedAtPose();
assert(!controller.GetPose().poseIsValid);
controller.Deactivate();
for(const auto &value:context.input.values) assert(value.second==0);
const auto poseCount=context.host.poses.size();
controller.UpdateOptical(hand,true,nullptr,now+1,connection,origin,epochs);
assert(context.host.poses.size()==poseCount);
HandController joyCon(false);
assert(joyCon.Activate(2) == VRInitError_None);
const auto joyTrigger = context.input.handles.at("/input/trigger/click");
droiddeck_joycon_hand joy{};
joy.rotation[0] = 1;
joy.time_ns = now;
joy.buttons = DROIDDECK_JOYCON_TRIGGER;
droiddeck_spatialpose spatial{};
spatial.flags = DD_SPATIAL_ACTIVE | DD_SPATIAL_JOYCONS;
spatial.connection = spatial.origin = 1;
spatial.head = {now, {2, 1.6f, 3}, {0,0,0,1}, DD_SPATIAL_VALID};
spatial.hands[0].pose = {now, {1.8f, 1.2f, 2.6f}, {0,0,0,1}, DD_SPATIAL_VALID};
assert(joyCon.Update(&joy, false, {1,0,0,0}, now, &spatial));
auto pose = joyCon.GetPose();
assert(pose.poseIsValid && context.input.values.at(joyTrigger) == 1);
Quat expectedRotation; Vec3 expectedPosition;
IndexRawPose(false, {}, {1.8f, 1.2f, 2.6f}, &expectedRotation, &expectedPosition);
assert(std::abs(pose.vecPosition[0] - expectedPosition.x) < 1e-5);
assert(std::abs(pose.qRotation.w - expectedRotation.w) < 1e-5);
spatial.hands[0].pose.flags = 0;
joyCon.Update(&joy, false, {1,0,0,0}, now, &spatial);
assert(joyCon.GetPose().poseIsValid && context.input.values.at(joyTrigger) == 1);
const Quat turned = AboutY(.7);
joy.rotation[0] = turned.w; joy.rotation[2] = turned.y;
joyCon.Update(&joy, false, {1,0,0,0}, now, &spatial);
const auto beforeLoss = joyCon.GetPose().qRotation;
spatial.head.flags = 0;
joyCon.Update(&joy, false, {1,0,0,0}, now, &spatial);
assert(!joyCon.GetPose().poseIsValid && context.input.values.at(joyTrigger) == 0);
assert(context.host.inputsAtPose.back().at(joyTrigger) == 0);
spatial.head.flags = DD_SPATIAL_VALID;
joyCon.Update(&joy, false, {1,0,0,0}, now, &spatial);
assert(std::abs(joyCon.GetPose().qRotation.w - beforeLoss.w) < 1e-5);
assert(std::abs(joyCon.GetPose().qRotation.y - beforeLoss.y) < 1e-5);
spatial.origin++;
spatial.head.position[0] = -2;
joyCon.Update(&joy, false, {1,0,0,0}, now, &spatial);
assert(joyCon.GetPose().poseIsValid && joyCon.GetPose().vecPosition[0] < -2);
joyCon.Update(nullptr, false, {1,0,0,0}, now, &spatial);
assert(!joyCon.GetPose().deviceIsConnected && context.input.values.at(joyTrigger) == 0);
puts("PASS: optical controller recovery and Luma Joy-Con camera, fallback, head loss and origin changes");
}
+41
View File
@@ -0,0 +1,41 @@
#include "presenter_image.h"
#include <cassert>
#include <cstdio>
#include <initializer_list>
int main() {
droiddeck::SharedEyeImage image(VK_FORMAT_R8G8B8A8_SRGB, 1728, 1080);
const auto &info = image.info;
// SteamVR's shared-eye creation trace on the Fold7: format 43, flags 8, usage 0x97, list 43/37.
assert(info.format == 43 && info.flags == 8 && info.usage == 0x97);
assert(info.extent.width == 1728 && info.extent.height == 1080 && info.extent.depth == 1);
assert(info.samples == VK_SAMPLE_COUNT_1_BIT && info.mipLevels == 1 && info.arrayLayers == 1);
assert(info.tiling == VK_IMAGE_TILING_OPTIMAL && info.sharingMode == VK_SHARING_MODE_EXCLUSIVE);
const auto *external = static_cast<const VkExternalMemoryImageCreateInfo *>(info.pNext);
assert(external->handleTypes == VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT);
const auto *list = static_cast<const VkImageFormatListCreateInfo *>(external->pNext);
assert(list->viewFormatCount == 2 && list->pViewFormats[0] == 43 && list->pViewFormats[1] == 37);
const VkSurfaceFormatKHR choices[] = {
{VK_FORMAT_B8G8R8A8_UNORM, VK_COLOR_SPACE_SRGB_NONLINEAR_KHR},
{VK_FORMAT_R8G8B8A8_UNORM, VK_COLOR_SPACE_SRGB_NONLINEAR_KHR},
{VK_FORMAT_B8G8R8A8_SRGB, VK_COLOR_SPACE_SRGB_NONLINEAR_KHR},
{VK_FORMAT_R8G8B8A8_SRGB, VK_COLOR_SPACE_SRGB_NONLINEAR_KHR},
};
VkSurfaceFormatKHR selected{};
for (VkFormat eye : {VK_FORMAT_R8G8B8A8_SRGB, VK_FORMAT_B8G8R8A8_SRGB}) {
assert(droiddeck::EyeSurfaceFormat(eye, choices, 4, &selected));
assert(droiddeck::SrgbEyeFormat(selected.format));
assert(!droiddeck::EyeSurfaceFormat(eye, choices, 2, &selected));
}
for (VkFormat eye : {VK_FORMAT_R8G8B8A8_UNORM, VK_FORMAT_B8G8R8A8_UNORM}) {
assert(droiddeck::EyeSurfaceFormat(eye, choices, 4, &selected));
assert(!droiddeck::SrgbEyeFormat(selected.format));
assert(!droiddeck::EyeSurfaceFormat(eye, choices + 2, 2, &selected));
}
assert(!droiddeck::EyeSurfaceFormat(VK_FORMAT_R8G8B8A8_SRGB, nullptr, 0, &selected));
const VkSurfaceFormatKHR any{VK_FORMAT_UNDEFINED, VK_COLOR_SPACE_SRGB_NONLINEAR_KHR};
assert(droiddeck::EyeSurfaceFormat(VK_FORMAT_R8G8B8A8_SRGB, &any, 1, &selected));
assert(selected.format == VK_FORMAT_B8G8R8A8_SRGB);
std::puts("PASS: shared eye imports match SteamVR and preserve color encoding through presentation");
}
+58
View File
@@ -0,0 +1,58 @@
#define _GNU_SOURCE
#include <assert.h>
#include <errno.h>
#include <linux/capability.h>
#include <stdint.h>
#include <stdio.h>
#include <sys/resource.h>
#include <sys/syscall.h>
#include <sys/wait.h>
#include <unistd.h>
extern char *program_invocation_short_name;
static int realtime_unavailable(void) {
struct rlimit priority;
struct __user_cap_header_struct header = {_LINUX_CAPABILITY_VERSION_3, 0};
struct __user_cap_data_struct caps[2] = {{0}};
assert(getrlimit(RLIMIT_RTPRIO, &priority) == 0);
assert(syscall(SYS_capget, &header, caps) == 0);
return priority.rlim_cur == 0 &&
!(caps[CAP_SYS_NICE / 32].effective & (UINT32_C(1) << (CAP_SYS_NICE % 32)));
}
static void check(int compositor) {
program_invocation_short_name = compositor ? "vrcompositor" : "vrserver";
struct rlimit before, after, low = {100000, 200000};
assert(getrlimit(RLIMIT_RTTIME, &before) == 0);
assert(before.rlim_cur > low.rlim_cur && before.rlim_max > low.rlim_max);
const int blocked = compositor && realtime_unavailable();
int result = setrlimit(RLIMIT_RTTIME, &low);
if (blocked) {
assert(result == -1 && errno == EPERM);
assert(getrlimit(RLIMIT_RTTIME, &after) == 0);
assert(after.rlim_cur == before.rlim_cur && after.rlim_max == before.rlim_max);
assert(setrlimit(RLIMIT_RTTIME, &before) == 0);
} else {
assert(result == 0);
assert(getrlimit(RLIMIT_RTTIME, &after) == 0);
assert(after.rlim_cur == low.rlim_cur && after.rlim_max == low.rlim_max);
}
assert(getrlimit(RLIMIT_NOFILE, &before) == 0);
before.rlim_cur = 64;
assert(setrlimit(RLIMIT_NOFILE, &before) == 0);
assert(getrlimit(RLIMIT_NOFILE, &after) == 0 && after.rlim_cur == 64);
assert(setrlimit(-1, &before) == -1 && errno == EINVAL);
}
int main(void) {
for (int compositor = 0; compositor <= 1; ++compositor) {
pid_t child = fork();
assert(child >= 0);
if (!child) { check(compositor); _exit(0); }
int status;
assert(waitpid(child, &status, 0) == child);
assert(WIFEXITED(status) && WEXITSTATUS(status) == 0);
}
puts("SteamVR scheduling regression passed");
}
+387
View File
@@ -0,0 +1,387 @@
#include "spatial_tracking.h"
#include <cassert>
#include <cstddef>
#include <cstdio>
#include <limits>
static float PoseAngle(const float *a, const float *b) {
double dot=0,normA=0,normB=0;
for (int i=0; i<4; ++i) {
dot+=double(a[i])*b[i];normA+=double(a[i])*a[i];normB+=double(b[i])*b[i];
}
return float(2*std::acos(std::clamp(std::abs(dot)/std::sqrt(normA*normB),0.,1.)));
}
static void CheckPredictionDirection() {
for (const int framesPerSample : {10,20}) {
OpticalPose pose;
dd_spatial_hand hand{};
for (int frame=0; frame<2000; ++frame) {
if (frame%framesPerSample==0) {
const int sample=frame/framesPerSample;
const float sign=sample%2 ? -1.f : 1.f;
const float position=sample ? sign*.004f : 0;
const float angle=sample ? sign*.05f : 0;
hand.pose={30000000000LL+frame*4000000LL,{position,0,0},
{0,sign*std::sin(angle*.5f),0,sign*std::cos(angle*.5f)},DD_SPATIAL_VALID};
}
assert(pose.Update(hand,true,30060000000LL+frame*4000000LL,1,1));
assert(!pose.predict);
assert(std::abs(pose.rendered.position[0])<=.004f);
}
}
}
static void CheckSmallMotion() {
for (const int framesPerSample : {10,20}) {
OpticalPose pose;
dd_spatial_hand hand{};
const float identity[]={0,0,0,1};
float previousPosition=0,previousAngle=0;
for (int sample=0; sample<56; ++sample) {
const int step=sample<16 ? 0 : sample<28 ? sample-15 : sample<44 ? 12 : 55-sample;
const float position=step*.0002f,angle=step*.000872665f;
hand.pose={70000000000LL+sample*framesPerSample*4000000LL,{position,0,0},
{0,std::sin(angle*.5f),0,std::cos(angle*.5f)},DD_SPATIAL_VALID};
for (int frame=0; frame<framesPerSample; ++frame) {
assert(pose.Update(hand,true,hand.pose.time_ns+60000000LL+frame*4000000LL,1,1));
if (frame==0 && (sample==16 || sample==44)) {
const float direction=sample==16 ? 1.f : -1.f;
assert(direction*(pose.rendered.position[0]-previousPosition)>.00002f);
assert(direction*(PoseAngle(identity,pose.rendered.rotation)-previousAngle)>.00008f);
}
assert(pose.rendered.time_ns==hand.pose.time_ns);
}
const float actualAngle=PoseAngle(identity,pose.rendered.rotation);
assert(std::abs(pose.rendered.position[0]-position)<.0002f);
assert(std::abs(actualAngle-angle)<.00012f);
if ((sample>=16 && sample<28) || sample>=44) {
const float direction=sample<28 ? 1.f : -1.f;
assert(direction*(pose.rendered.position[0]-previousPosition)>.0001f);
assert(direction*(actualAngle-previousAngle)>.0005f);
}
previousPosition=pose.rendered.position[0];previousAngle=actualAngle;
}
}
}
static void CheckPalmMotion() {
for (const int framesPerSample : {10,20}) {
OpticalPose pose;
dd_spatial_hand hand{};
const float identity[]={0,0,0,1};
float maximumPositionError=0,maximumAngleError=0;
for (int frame=0; frame<1000; ++frame) {
if (frame%framesPerSample==0) {
const float elapsed=frame*.004f;
const float angle=elapsed*.1f;
hand.pose={40000000000LL+frame*4000000LL,{elapsed*.01f,0,0},
{0,std::sin(angle*.5f),0,std::cos(angle*.5f)},DD_SPATIAL_VALID};
}
assert(pose.Update(hand,true,40060000000LL+frame*4000000LL,1,1));
if (frame>250) {
const float observedAngle=float(hand.pose.time_ns-40000000000LL)*1e-9f*.1f;
maximumPositionError=std::max(maximumPositionError,std::abs(pose.rendered.position[0]-hand.pose.position[0]));
maximumAngleError=std::max(maximumAngleError,std::abs(PoseAngle(identity,pose.rendered.rotation)-observedAngle));
}
}
std::printf("Palm %.1fHz slow motion maximum error position=%.3fmm rotation=%.3fdeg\n",
250./framesPerSample,maximumPositionError*1000,maximumAngleError*180/M_PI);
assert(maximumPositionError<.001f && maximumAngleError<.013f);
pose.Reset();
hand.pose={50000000000LL,{0,0,0},{0,0,0,1},DD_SPATIAL_VALID};
for (int frame=0; frame<framesPerSample; ++frame)
assert(pose.Update(hand,true,50060000000LL+frame*4000000LL,1,1));
hand.pose.time_ns+=framesPerSample*4000000LL;
hand.pose.position[0]=.1f;
hand.pose.rotation[1]=std::sin(float(M_PI/4));
hand.pose.rotation[3]=std::cos(float(M_PI/4));
int positionResponseMs=-1,rotationResponseMs=-1;
for (int frame=0; frame<20; ++frame) {
assert(pose.Update(hand,true,hand.pose.time_ns+60000000LL+frame*4000000LL,1,1));
const auto &rendered=pose.rendered;
assert(rendered.position[0]>=0 && rendered.position[0]<=.100001f && !pose.predict);
if (positionResponseMs<0 && rendered.position[0]>=.095f) positionResponseMs=frame*4;
if (rotationResponseMs<0 && PoseAngle(identity,rendered.rotation)>=float(M_PI*.475))
rotationResponseMs=frame*4;
assert(rendered.time_ns==hand.pose.time_ns);
}
assert(positionResponseMs>=36 && positionResponseMs<=52);
assert(rotationResponseMs>=36 && rotationResponseMs<=52);
std::printf("Palm %.1fHz 95%% step response position=%dms rotation=%dms after receipt\n",
250./framesPerSample,positionResponseMs,rotationResponseMs);
const auto observed=pose.last;
hand.pose.position[0]=2;
hand.pose.rotation[1]=0;hand.pose.rotation[3]=1;
assert(pose.Update(hand,true,hand.pose.time_ns+140000000LL,1,1));
assert(std::memcmp(&pose.last,&observed,sizeof(observed))==0);
}
}
static void CheckVisualHold() {
OpticalPose pose;
OpticalInputs input;
dd_spatial_hand hand{};
for (int sample=0; sample<3; ++sample) {
hand.pose={60000000000LL+sample*80000000LL,{sample*.02f,0,0},{0,0,0,1},
DD_SPATIAL_VALID|DD_SPATIAL_TRIGGER_VALID|DD_SPATIAL_GRIP_VALID};
hand.trigger=sample==2 ? 1.f : 0.f;hand.grip=sample==2 ? 1u : 0u;
assert(input.Update(hand,true,hand.pose.time_ns+60000000LL,1,1));
for (int frame=0; frame<20; ++frame)
assert(pose.Update(hand,true,hand.pose.time_ns+60000000LL+frame*4000000LL,1,1));
}
assert(pose.predict && input.click && input.grip);
const auto held=pose.rendered;
const int64_t capture=hand.pose.time_ns;
assert(pose.Update({},true,capture+140000000LL,1,1));
assert(!input.Update({},true,capture+140000000LL,1,1));
assert(!input.click && !input.grip && input.trigger==0);
assert(!pose.predict && !pose.receivedAt && !pose.observations);
auto repeated=hand;repeated.pose.position[0]=9;
for (int ageMs : {150,200,201,300,350}) {
const int64_t now=capture+ageMs*1000000LL;
assert(pose.Update(repeated,true,now,1,1));
input.Update(repeated,true,now,1,1);
assert(!input.click && !input.grip && input.trigger==0);
assert(pose.last.pose.time_ns==capture);
assert(std::memcmp(&pose.rendered,&held,sizeof(held))==0);
assert(!pose.predict && !pose.receivedAt && !pose.observations);
}
auto stale=hand;stale.pose.time_ns+=20000000LL;stale.pose.position[0]=5;
assert(pose.Update(stale,true,capture+350000000LL,1,1));
assert(pose.last.pose.time_ns==capture && std::memcmp(&pose.rendered,&held,sizeof(held))==0);
assert(!pose.Update(stale,true,capture+350000001LL,1,1));
assert(!pose.last.pose.time_ns && !pose.rendered.flags);
assert(!pose.Update(repeated,true,capture+351000000LL,1,1));
OpticalPose unseen;
assert(!unseen.Update(stale,true,stale.pose.time_ns+200000001LL,1,1));
auto fresh=hand;fresh.pose.time_ns=capture+350000000LL;fresh.pose.position[0]=.5f;
fresh.pose.rotation[1]=std::sin(.3f);fresh.pose.rotation[3]=std::cos(.3f);
assert(pose.Update(fresh,true,capture+400000000LL,1,1));
assert(pose.rendered.position[0]==.5f && !pose.predict);
assert(PoseAngle(pose.rendered.rotation,fresh.pose.rotation)<1e-6f);
assert(pose.rendered.time_ns==fresh.pose.time_ns);
assert(input.Update(fresh,true,capture+400000000LL,1,1));
assert(!input.click && !input.grip && input.trigger==0);
for (int reset=0; reset<3; ++reset) {
OpticalPose resetPose;
assert(resetPose.Update(hand,true,capture,1,1));
assert(resetPose.Update({},true,capture+150000000LL,1,1));
const uint64_t connection=reset==1 ? 2 : 1,origin=reset==2 ? 2 : 1;
assert(!resetPose.Update({},reset!=0,capture+160000000LL,connection,origin));
assert(!resetPose.last.pose.time_ns && !resetPose.rendered.flags);
assert(!resetPose.Update({},true,capture+170000000LL,connection,origin));
}
}
int main() {
CheckPredictionDirection();
CheckSmallMotion();
CheckPalmMotion();
CheckVisualHold();
static_assert(sizeof(droiddeck_spatialpose) == 176);
static_assert(offsetof(droiddeck_spatialpose, head) == 40);
static_assert(offsetof(droiddeck_spatialpose, hands) == 80);
char path[] = "/tmp/droiddeck-spatial-XXXXXX";
int fd = mkstemp(path); assert(fd >= 0);
droiddeck_spatialpose out{}, s{};
s.magic = DROIDDECK_SPATIALPOSE_MAGIC; s.version = DROIDDECK_SPATIALPOSE_VERSION; s.size = sizeof(s);
s.head = {1000000000LL, {1,2,3}, {0,0,0,1}, DD_SPATIAL_VALID};
s.hands[0].pose = s.head; s.hands[0].pose.time_ns -= 50000000;
{
SpatialFile empty(path); assert(!empty.Read(&out));
assert(write(fd, &s, sizeof(s)-1) == sizeof(s)-1); assert(!empty.Read(&out));
}
assert(pwrite(fd, &s, sizeof(s), 0) == sizeof(s));
{
SpatialFile reader(path); assert(reader.Read(&out));
assert(out.hands[0].pose.time_ns == 950000000);
s.seq = 1; assert(pwrite(fd, &s, sizeof(s), 0) == sizeof(s)); assert(!reader.Read(&out));
s.seq = 2; s.version = 99; assert(pwrite(fd, &s, sizeof(s), 0) == sizeof(s)); assert(!reader.Read(&out));
s.version = 1; assert(pwrite(fd, &s, sizeof(s), 0) == sizeof(s)); assert(!reader.Read(&out));
s.version = 2; assert(pwrite(fd, &s, sizeof(s), 0) == sizeof(s)); assert(!reader.Read(&out));
s.input_epochs = 0x04030201;
s.version = DROIDDECK_SPATIALPOSE_VERSION; s.head.time_ns += 90000000; assert(pwrite(fd, &s, sizeof(s), 0) == sizeof(s)); assert(reader.Read(&out));
assert(SpatialFresh(out.head, 1190000000)); assert(!SpatialFresh(out.head,1190000001));
assert(out.hands[0].pose.time_ns == 950000000);
assert(out.input_epochs == 0x04030201);
assert(SpatialHandFresh(out.hands[0],1150000000)); assert(!SpatialHandFresh(out.hands[0],1150000001));
}
close(fd); unlink(path);
assert(!SpatialFresh(s.head,1));
s.head.position[0] = std::numeric_limits<float>::quiet_NaN(); assert(!SpatialFresh(s.head,1100000000));
auto hand=s.hands[0]; hand.pose.time_ns=1000000000; hand.trigger=1; hand.grip=1;
hand.pose.flags |= DD_SPATIAL_TRIGGER_VALID | DD_SPATIAL_GRIP_VALID;
OpticalInputs input;
assert(input.Update(hand,true,1000000000,7,1)); assert(!input.click && !input.grip);
auto updateInput = [&](float trigger, uint32_t grip, uint16_t epochs = 0) {
hand.pose.time_ns += 10000000;
hand.trigger=trigger; hand.grip=grip;
return input.Update(hand,true,hand.pose.time_ns,7,1,epochs);
};
updateInput(0,0); assert(input.armed && input.gripArmed);
updateInput(.8f,1); assert(input.click && input.grip);
updateInput(.5f,1); assert(input.click);
updateInput(.4f,1); assert(!input.click && input.grip);
updateInput(1,1); assert(input.click && input.grip);
assert(input.Update(hand,true,hand.pose.time_ns+200000000,7,1)); assert(input.click && input.grip);
assert(!input.Update(hand,true,hand.pose.time_ns+200000001,7,1)); assert(!input.armed && !input.grip && input.trigger==0);
hand.pose.time_ns += 210000000;
updateInput(1,1); assert(!input.click && !input.grip);
updateInput(0,0); updateInput(1,1); assert(input.click && input.grip);
hand.pose.flags &= ~DD_SPATIAL_TRIGGER_VALID;
updateInput(1,1); assert(!input.armed && !input.click && input.trigger==0 && input.grip);
hand.pose.flags |= DD_SPATIAL_TRIGGER_VALID;
updateInput(1,1); assert(!input.click && input.grip);
updateInput(0,1); updateInput(1,1); assert(input.click);
hand.pose.flags &= ~DD_SPATIAL_GRIP_VALID;
updateInput(1,1); assert(input.click && !input.grip && !input.gripArmed);
hand.pose.flags |= DD_SPATIAL_GRIP_VALID;
updateInput(1,1); assert(input.click && !input.grip);
updateInput(1,0); updateInput(1,1); assert(input.grip);
updateInput(1,1,0x0100); assert(input.click && !input.grip && !input.gripArmed);
updateInput(1,1,0x0100); assert(input.click && !input.grip);
updateInput(1,0,0x0100); updateInput(1,1,0x0100); assert(input.grip);
updateInput(1,1,0x0101); assert(!input.click && !input.armed && input.grip);
updateInput(0,1,0x0101); updateInput(1,1,0x0101); assert(input.click && input.grip);
updateInput(1,1,0x0202); assert(!input.click && !input.grip);
updateInput(0,0,0x0202); updateInput(1,1,0x0202); assert(input.click && input.grip);
input.Update({},false,hand.pose.time_ns+1,7,1,0x0202);
hand.trigger=0; hand.grip=0;
input.Update(hand,true,hand.pose.time_ns+2,7,1,0x0202); assert(!input.armed && !input.gripArmed);
updateInput(1,1,0x0202); assert(!input.click && !input.grip);
updateInput(0,0,0x0202); updateInput(1,1,0x0202); assert(input.click && input.grip);
hand.pose.time_ns+=210000000;
updateInput(1,1,0x0202); assert(!input.click && !input.grip);
updateInput(0,0,0x0202); updateInput(1,1,0x0202);
auto backwards=hand; backwards.pose.time_ns-=1000000; backwards.trigger=0; backwards.grip=0;
assert(!input.Update(backwards,true,hand.pose.time_ns,7,1,0x0202)); assert(!input.armed && !input.gripArmed);
updateInput(0,0,0x0202); updateInput(1,1,0x0202);
input.Update(hand,true,hand.pose.time_ns,7,2,0x0202); assert(!input.click && !input.grip);
hand.pose.time_ns=1210000000;
OpticalPose pose;
assert(pose.Update(hand,true,1210000000,7,2));
auto corrupt = hand; corrupt.pose.position[0] = std::numeric_limits<float>::quiet_NaN();
assert(pose.Update(corrupt,true,1220000000,7,2));
assert(pose.last.pose.position[0] == hand.pose.position[0]);
auto future = hand; future.pose.time_ns = 1500000000;
assert(pose.Update(future,true,1230000000,7,2));
assert(pose.last.pose.time_ns == 1210000000);
auto old = hand; old.pose.time_ns -= 10000000; old.pose.position[0] = 99;
assert(pose.Update(old,true,1240000000,7,2));
assert(pose.last.pose.time_ns == 1210000000);
assert(pose.Update({},true,1410000000,7,2));
assert(pose.Update({},true,1560000000,7,2));
assert(!pose.Update({},true,1560000001,7,2));
for (const int64_t delay : {60000000LL, 80000000LL, 120000000LL}) {
pose.Reset();
hand.pose = {2000000000LL, {0,0,0}, {0,0,0,1.002f}, DD_SPATIAL_VALID};
float previous = 0;
for (int sample=0; sample<8; ++sample) {
hand.pose.time_ns = 2000000000LL+sample*80000000LL;
hand.pose.position[0] = sample*.02f;
const float sign = sample%2 ? -1.f : 1.f;
hand.pose.rotation[1] = sign*1.002f*std::sin(sample*.05f);
hand.pose.rotation[3] = sign*1.002f*std::cos(sample*.05f);
float midway = 0;
for (int frame=0; frame<20; ++frame) {
const int64_t now = hand.pose.time_ns+delay+frame*4000000LL;
assert(pose.Update(hand,true,now,7,3));
assert(pose.receivedAt==hand.pose.time_ns+delay);
assert(pose.rendered.time_ns==hand.pose.time_ns);
assert(pose.rendered.position[0]<=hand.pose.position[0]+.020001f);
float norm=0; for(float v:pose.rendered.rotation) norm+=v*v;
assert(std::abs(norm-1)<1e-5);
if (sample>=4) {
assert(pose.predict);
assert(pose.rendered.position[0]>previous+.0005f);
assert(pose.rendered.position[0]<previous+.006f);
const float movingTarget = hand.pose.position[0]+frame*.001f;
assert(std::abs(pose.rendered.position[0]-movingTarget)<.006f);
}
if (frame==10) midway = pose.rendered.position[0];
if (sample>=4 && frame==19) assert(pose.rendered.position[0]-midway>.005f);
previous = pose.rendered.position[0];
}
}
const int64_t stoppedAt = hand.pose.time_ns+80000000;
const float beforeStop = pose.rendered.position[0];
hand.pose.time_ns = stoppedAt;
for (int frame=0; frame<20; ++frame) {
assert(pose.Update(hand,true,stoppedAt+delay+frame*4000000LL,7,3));
assert(pose.rendered.position[0]<=beforeStop);
assert(pose.rendered.position[0]>=hand.pose.position[0]);
}
assert(std::abs(pose.rendered.position[0]-hand.pose.position[0])<.0001f);
hand.pose.time_ns+=80000000; hand.pose.position[0]-=.04f;
previous = pose.rendered.position[0];
for (int frame=0; frame<20; ++frame) {
assert(pose.Update(hand,true,hand.pose.time_ns+delay+frame*4000000LL,7,3));
assert(!pose.predict);
assert(pose.rendered.position[0]<=previous);
assert(pose.rendered.position[0]>=hand.pose.position[0]);
previous=pose.rendered.position[0];
}
hand.pose.time_ns+=80000000; hand.pose.position[0]-=.04f;
pose.Update(hand,true,hand.pose.time_ns+delay,7,3);
assert(pose.predict);
for (int frame=1; frame<20; ++frame) {
pose.Update(hand,true,hand.pose.time_ns+delay+frame*4000000LL,7,3);
assert(pose.rendered.position[0]>=hand.pose.position[0]-.020001f);
}
hand.pose.time_ns+=80000000; hand.pose.position[0]+=1.f;
pose.Update(hand,true,hand.pose.time_ns+delay,7,3);
assert(!pose.predict);
for (int frame=1; frame<20; ++frame) {
pose.Update(hand,true,hand.pose.time_ns+delay+frame*4000000LL,7,3);
assert(pose.rendered.position[0]<=hand.pose.position[0]);
}
}
const auto held = pose.rendered;
pose.Update({},true,hand.pose.time_ns+196000000,7,3);
assert(!pose.predict && !pose.receivedAt);
assert(std::memcmp(&pose.rendered,&held,sizeof(held))==0);
pose.Update(hand,true,hand.pose.time_ns+198000000,7,3);
assert(std::memcmp(&pose.rendered,&held,sizeof(held))==0);
assert(pose.Update({},true,hand.pose.time_ns+200000000,7,3));
assert(pose.Update({},true,hand.pose.time_ns+350000000,7,3));
assert(!pose.Update({},true,hand.pose.time_ns+350000001,7,3));
hand.pose.time_ns+=400000000; hand.pose.position[0]=.5f;
pose.Update(hand,true,hand.pose.time_ns+60000000,7,3);
assert(pose.rendered.position[0]==.5f && !pose.predict);
hand.pose.time_ns+=80000000; hand.pose.position[0]=.6f;
pose.Update(hand,true,hand.pose.time_ns+60000000,7,4);
assert(pose.rendered.position[0]==.6f && !pose.predict);
hand.pose.time_ns+=80000000; hand.pose.position[0]=.7f;
pose.Update(hand,true,hand.pose.time_ns+60000000,8,4);
assert(pose.rendered.position[0]==.7f && !pose.predict);
pose.Update({},false,hand.pose.time_ns+64000000,8,4);
assert(!pose.last.pose.time_ns && !pose.receivedAt && !pose.predict);
pose.Reset();
const int64_t delays[] = {60000000,120000000,80000000,60000000,120000000,80000000};
int sample=0;
hand.pose = {6000000000LL, {0,0,0}, {0,0,0,1}, DD_SPATIAL_VALID};
float previous=0;
for (int64_t now=6060000000LL; now<6556000000LL; now+=4000000) {
if (sample<5 && now>=6000000000LL+(sample+1)*80000000LL+delays[sample+1]) {
++sample;
hand.pose.time_ns=6000000000LL+sample*80000000LL;
hand.pose.position[0]=sample*.02f;
}
assert(pose.Update(hand,true,now,8,4));
assert(pose.receivedAt==hand.pose.time_ns+delays[sample]);
assert(pose.rendered.position[0]>=previous);
assert(pose.rendered.position[0]<=previous+.01f);
assert(pose.rendered.position[0]<=hand.pose.position[0]+.020001f);
assert(pose.rendered.time_ns==hand.pose.time_ns);
previous=pose.rendered.position[0];
}
puts("PASS: spatial layout, files, freshness, independent timestamps and optical input recovery");
}
+63
View File
@@ -0,0 +1,63 @@
#define _GNU_SOURCE
#include <ucontext.h>
#include <assert.h>
#include <stdint.h>
#include "../src/vk_layer_droiddeck_vr.c"
static _Atomic uint64_t local_value;
static _Atomic unsigned duplicate_signals;
static pthread_t submit_thread;
static int first_read = 1;
static VkResult VKAPI_CALL get_value(VkDevice device, VkSemaphore semaphore, uint64_t *value) {
(void)device;
(void)semaphore;
*value = atomic_load(&local_value);
if (pthread_equal(pthread_self(), submit_thread) && first_read) {
first_read = 0;
assert(pthread_create(&g_poller, NULL, poller, NULL) == 0);
const uint64_t until = now_us() + 2000000;
while (!atomic_load(&local_value) && now_us() < until) usleep(100);
assert(atomic_load(&local_value) == 1);
}
return VK_SUCCESS;
}
static VkResult VKAPI_CALL signal_value(VkDevice device, const VkSemaphoreSignalInfo *info) {
(void)device;
if (info->value <= atomic_load(&local_value)) {
atomic_fetch_add(&duplicate_signals, 1);
return VK_ERROR_DEVICE_LOST;
}
atomic_store(&local_value, info->value);
return VK_SUCCESS;
}
static PFN_vkVoidFunction VKAPI_CALL get_proc(VkDevice device, const char *name) {
(void)device;
if (!strcmp(name, "vkGetSemaphoreCounterValue")) return (PFN_vkVoidFunction)get_value;
if (!strcmp(name, "vkSignalSemaphore")) return (PFN_vkVoidFunction)signal_value;
assert(0);
return NULL;
}
int main(void) {
uintptr_t dispatch = 1;
VkDevice device = (VkDevice)&dispatch;
VkSemaphore semaphore = (VkSemaphore)(uintptr_t)2;
_Atomic uint64_t shared = 1;
g_dev[0] = (struct dev){.handle = device, .gdpa = get_proc};
g_br[0] = (struct bridged){.sem = semaphore, .dev = device, .counter = &shared,
.fd = -1, .timeline = 1, .seen = 1, .seen_us = now_us(), .imported = 1};
submit_thread = pthread_self();
submit_drop_wait(semaphore, 1);
assert(atomic_load(&local_value) == 1);
assert(atomic_load(&duplicate_signals) == 0);
assert(submit_drop_wait(semaphore, 1) == 0);
pthread_cancel(g_poller);
pthread_join(g_poller, NULL);
puts("PASS: a concurrent bridge signal cannot be repeated by the submit thread");
return 0;
}