Add VITURE XR controls, head tracking, and SteamVR (#732)

* Add VITURE glasses support: hardware control, 3D, head tracking

Glasses get their own controller over the MCU (permission, 120 Hz default, brightness, volume, sunblock, 2D/3D) and the game moves onto them when plugged in. The IMU drives Cardboard head tracking and the SteamVR driver. 3D falls back to one eye's view on phones that can't send the 3840x1080 signal. Options open from a glasses button in game and on the Steam page.

* Drop the glasses settings like Steam's: Play-style button, Settings lip, tabs

* Hang the glasses Settings lip over the top of the button

* Space glasses MCU commands so the IMU rate sticks, fit fixed resolutions to the glasses

* Keep Steam alive with SteamVR registered, run SteamVR at the glasses' rate, keep the glasses lip on top

* Fit VR games on the glasses: full view, seated origin at the head, pad hands reach with the d-pad

* Let the pad hands reach a metre out so buttons with deep colliders press

* Show every VR frame in order at 120 fps: wait for SteamVR's GPU work, present only finished copies, skip the grace on idle shared counters

* Glasses: SteamVR starts and stops with them, closer view with a field of view setting, Recenter on the pad

* Glasses: SteamVR field of view defaults to 90°

* VR pad hands: sticks move and aim the hands, left stick click switches the sticks to the game, d-pad pulls a held hand

* Glasses: Play SteamVR beside the glasses button instead of starting SteamVR with them, pad hands drawn as Index controllers and off while other VR controllers are connected, glasses sheet tabs sized like Steam's

* Track glasses mode, refresh-aware glasses window, align Play SteamVR with Play Steam

* Fix 3D on the glasses: full-res SteamVR presenter, revert VR 3D at session end, coalesce vsync ticks, no session restart after end

* Address review: glasses output after installs and session end, VR revert retry, display fallback, presenter swapchain retry, agent hex checks, drawer bumpers, glasses translations, docs

* Harden VITURE stereo negotiation and output refresh tracking

* Localize stereo recovery feedback and record Astra reconnect tests

* Improve test failure diagnostics and refresh Astra firmware findings

* Handle VITURE docks separately and reject truncated USB frames

* Glasses: remember a 3D fallback per model, docks as display only, VR timing fixes

- A 3D attempt that ends at the 640x480 fallback is remembered for that glasses
  model, so Play SteamVR no longer blacks out the glasses after every restart;
  Retry clears it and "3D for VR games" stays reachable
- A VITURE Mobile Dock on its own no longer counts as glasses; through a dock
  Play SteamVR is hidden and the sheet shows the output and why there is no control
- HDR is probed on the glasses when the session starts there
- Glasses frames with a mismatched checksum are marked in the status frame log
- SteamVR layer and driver vblank counters stay monotonic across refresh changes;
  one-eye render scale applies from activation and on mono/3D switches
- Compositor binds announce the refresh clients were last told; the presenter
  retries a failed first swapchain
- Dead strings and helpers removed; docs updated

* Glasses: drop USB frames with a bad checksum

* Glasses: count frames dropped for a bad checksum

* Feat/viture xr (#907)

* Add RayNeo Air and GT support to the XR glasses pipeline

* Add RayNeo Air and GT support to the XR glasses pipeline

* Checkpoint glasses tracking, calibration, and SteamVR launch integration

* Checkpoint glasses tracking, calibration, and SteamVR launch integration

* Add XREAL 1S support and correct GT and Pro 2 tracking

* Add XREAL 1S support and correct GT and Pro 2 tracking

* Fix mirrored glasses tracking and recover SteamVR after device loss

* Fix mirrored glasses tracking and recover SteamVR after device loss

* Document confirmed XREAL 1S brightness control

* Document confirmed XREAL 1S brightness control

* Fix SteamVR glasses defaults and rendering controls

* Fix SteamVR glasses defaults and rendering controls

* chore: keep only the 3 newest session log folders (#852)

* chore: add agent comment rules (#748)

Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com>

* Fix SteamVR polling, VITURE checksums, and fence tracking

Keep manual SteamVR launches polled, discard corrupt VITURE packets, and retire Vulkan fences out of order without blocking submissions at a fixed capacity. Preserve the contiguous completion watermark used by the presenter.

Fix optional Windows launcher configuration and audited helper staging. Add Android, Windows, and native tracker regression coverage.

* Fix SteamVR polling, VITURE checksums, and fence tracking

Keep manual SteamVR launches polled, discard corrupt VITURE packets, and retire Vulkan fences out of order without blocking submissions at a fixed capacity. Preserve the contiguous completion watermark used by the presenter.

Fix optional Windows launcher configuration and audited helper staging. Add Android, Windows, and native tracker regression coverage.

* fix: wireless ADB page names the child-process switch the way Android shows it (#848)

* l10n: refresh stale and untranslated strings in es, fr, ko, ru (#850)

* l10n: refresh stale and untranslated strings in es, fr, ko, ru

- mode_added_art_hint: now mentions Steam then SteamGridDB (all four locales).
- wincomp_needs_installer / wincomp_installers_note: es and ko still described
  the old "coming soon" behaviour; align with the current English text.
- summary_remove_game: was still English in all four locales.

Machine-assisted translation; native-speaker review is welcome.

* l10n(es): say registro, not grabación, for the missing installer record

---------

Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com>

* fix: a failed cloud save recovery no longer ends the app (#846)

The cloud-recovery thread started at every app start had nothing to
catch an error, so one went to CrashHandler, which halts the process
with status 2. In unit tests the thread can outlive the test that
started it, fail on the torn-down preferences, and take the whole test
run down. The error is now logged and the thread ends.

Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com>

* fix: an interrupted Linux runtime download resumes instead of starting again (#847)

* fix: an interrupted Linux runtime download resumes instead of starting again

installOnce() asked the downloader to resume but always deleted the archive in its finally block, so the next attempt began at zero. The archive is now named after the release version and checksum, a failed download keeps what it fetched, and archives of other releases (and the old fixed name) are removed. A checksum or extraction failure still deletes the file.

* test: Downloader resumes a cut-off download from the bytes it kept

Starts a socket server that promises the whole file, hangs up part way and then answers a Range request, and checks that the second downloadFile call resumes with the right offset and ends with identical bytes.

* fix: a resumed download discards a 206 that does not start where the kept bytes end

Downloader appended every 206 response to the partial file without
looking at Content-Range. A server or proxy that answers a Range request
with a different offset would leave a corrupt file. Check that the range
starts at the kept length; otherwise delete the partial file and start
again without a Range header.

Adds a test with a server that answers the Range request from offset 0.

---------

Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com>

* l10n: German, French, Spanish, Italian, Portuguese and Russian for the zRAM, reset-settings and store free-space strings (#849)

* l10n: German, French, Spanish, Italian, Portuguese and Russian for the zRAM, reset-settings and store free-space strings

Adds the strings values-de, values-it and values-pt were missing (14 each) and the six that values-fr, values-es and values-ru were missing. Machine-assisted drafts that follow each file's existing register and terms; native review is welcome.

* l10n: correct missing active swap translations

---------

Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com>
Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com>

* fix: droiddeckctl reaches variant-package builds (recover #842) (#854)

fix: droiddeckctl reaches builds shipped under a variant package

The provider authority and the start Activity's component were fixed to
com.droiddeck.launcher, so droiddeckctl could not talk to the antutu,
ludashi or pubg builds from tools/release/variants.txt. DROIDDECK_PACKAGE
now picks the package; the Activity's class name stays the same in every
variant.

Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com>

* feat: droiddeckctl reports on-screen pacing (recover #843) (#855)

feat: droiddeckctl pacing reports the frames that reached the screen

Reads SurfaceFlinger's timestats for the game's layer (or the session's
SurfaceView) over a capture: frames on screen, fps on screen, the
present-to-present histogram and how many intervals were off the usual
cadence. Records split by a refresh-rate change are summed. A game's own fps
counter counts frames handed over, so it can't show dropped or doubled frames.

Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com>

* fix: Steam can restart more than twice in a session (#754)

* fix: Steam can restart more than twice in a session

Every exit 42 counted toward the cap that stops an update which will
not install, and the count was never cleared, so the third restart of
a session ended it however far apart they were. A client that ran for
a minute before asking is not that loop: the count starts again.

* fix: measure the client's run time as it exits

The time spent collecting logs and checking FEX after the client exits
counted toward its run, so a slow cleanup could make a quick exit 42
look like a long one and clear the count.

---------

Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com>
Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com>

* fix: the front end keeps its page when the Fold is folded or unfolded (#832)

* fix: the front end keeps its page when the Fold is folded or unfolded

MainActivity, FileManagerActivity, FilePickerActivity and StoreLoginActivity did not declare the screen-size configuration changes, so folding or unfolding a foldable recreated them and dropped their in-memory state. They now handle orientation, screenSize, screenLayout, smallestScreenSize and density themselves, like SessionActivity already does. uiMode is left out on purpose.

* fix: store login keeps rebuilding its toolbar when the display density changes

StoreLoginActivity builds its toolbar in code with dp-to-pixel values taken at creation time and has no onConfigurationChanged. It no longer declares density in configChanges, so a density change recreates it as before this PR. Size and layout changes are still handled in place.

---------

Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com>

* Release pointer buttons without replaying stale motion (#803)

* Fix cursor warping on native touch down

* Preserve input cleanup without replaying stale pointer motion

---------

Co-authored-by: siyuan.qsy <siyuan.qsy@alibaba-inc.com>

* l10n: translate the strings added since #284 into zh-CN, zh-HK, zh-TW and ja (#539)

* l10n: translate the 184 strings added since #284 (zh-CN/HK/TW, ja)

The stores, cloud saves, Windows components, Steam touch, desktop
removal and background-downloads strings landed after #284's
translation snapshot, so every one of them fell back to English.
Translate all 184 into Simplified Chinese, Traditional Chinese
(Taiwan and Hong Kong) and Japanese, keeping each locale file's own
vocabulary (工作階段/會話, 解除安裝/卸載, 元件/組件, 設定).

Also fix four Hong Kong strings that kept the simplified 着色器.

* l10n(zh): fix stale and ambiguous strings found in review

Severe: summary_remove_game was untranslated English in all three
files; mode_added_art_hint, wincomp_installers_note and
wincomp_needs_installer still described the pre-#480/#492 English.

Also: distinguish the Setup wizard from Android system settings in
zh-HK/zh-TW (初始設定 / 系統設定 / Android 設定), unify library as
收藏庫, component as 組件 in zh-HK, session as 會話 in zh-HK, account
as 帳戶 in zh-HK, Trending as 趋势/趨勢, Forget as 清除, reword the
re-applied-at-launch, action-failed and already-in-Games strings, and
play as 執行/開啟遊戲 in zh-TW/zh-HK.

* l10n(ja): fix stale and ambiguous strings found in review

---------

Co-authored-by: Dantalian <Dantalian-dev@users.noreply.github.com>
Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com>

* Rebuild and verify Windows VR runtime components

* Rebuild and verify Windows VR runtime components

* Add top, bottom, and centered picture positions for Steam sessions (#871)

* fix: dependency scan ignores folders above the game folder (#859)

* DroidDeck-Proton: get releases from the Droid-Deck org repo

DroidDeck-Proton is now built and released on Droid-Deck/DroidDeck-Proton
(11-9.1 is Latest there), so the Components page, the update check and the
guest installer read releases from the org repo instead of The412Banner's.

* feat: low-latency controller and touch toggles in the session drawer (#808)

* fix: SteamGridDB lookups no longer stick as "no match" after a failure; add a search to the art chooser (#876)

* feat: Steam home wall fills from the signed-in library instead of store picks (#878)

* test: cover GPU family and support classification (#880)

* Fix local builds on Linux failing after the first Comet build (#866)

* feat: sort the store library and installed tabs, and switch between card and list views (#798)

* feat: three-finger tap toggles Steam's on-screen keyboard in game mode (#782)

* fix: keep Quest-patched Turnip as the auto default (#853)

* Fix deploy_local.sh and droiddeckctl not finding a phone connected over wireless debugging (#864)

* fix: keep proot sessions alive when the kernel loses fork events (#457)

* fix(quest): correct desktop and Steam display corruption across sessions (#733)

fix(quest): stabilize desktop and Steam layer presentation

Keep Quest SDR scenes and the black backdrop off the Android Vulkan swapchain regardless of session order or HDR preferences. Only report the first frame after client content is submitted.

Add backdrop lifecycle and first-frame regression coverage. Record the user's successful Quest 3 Test 2 result.

Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com>

* fix: keep SteamVR syscall hooks and report Steam crashes (#905)

* fix: preserve SteamVR syscall hooks

* fix: report Steam crashes through gamescope

* fix: honor glasses pacing and resolution in SteamVR launches

---------

Co-authored-by: connor g <connorsspam1992@gmail.com>
Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com>
Co-authored-by: Yassine Gherbi <y.gherbi.dev@gmail.com>
Co-authored-by: Resul Övür <92828257+ovurrsl@users.noreply.github.com>
Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com>
Co-authored-by: Kirk <kirtoy.junsay@gmail.com>
Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com>
Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com>
Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com>
Co-authored-by: asdf17128 <asdf17128@gmail.com>
Co-authored-by: siyuan.qsy <siyuan.qsy@alibaba-inc.com>
Co-authored-by: Dantalian <wsu091227@qq.com>
Co-authored-by: Dantalian <Dantalian-dev@users.noreply.github.com>
Co-authored-by: The412Banner <the412banner@users.noreply.github.com>
Co-authored-by: The412Banner <205237651+The412Banner@users.noreply.github.com>
Co-authored-by: FX1922 <120828496+FX192@users.noreply.github.com>
Co-authored-by: Muaz Kadan <32595066+muazkadan@users.noreply.github.com>
Co-authored-by: kingokksa <47888451+kingokksa@users.noreply.github.com>
Co-authored-by: Jamie Schenker <jam1esch3nk3r@gmail.com>
Co-authored-by: Jagger15 <jagger2333@live.com>

* Glasses: keep a pending 2D restore and the SteamVR recovery offer, show Play SteamVR only with a picture

* Translate the glasses calibration, XREAL and SteamVR recovery strings

* SteamVR recovery: give a finished teardown time to collect logs before restarting

* Credit the projects the XR glasses support builds on

* Joy-Cons: SteamVR hands from their gyro while SteamVR runs, one combined pad otherwise

* Headsets: settings badge on Play SteamVR instead of a top-right button, both shown only once connected

* Hand tracking: camera-tracked Joy-Cons and bare hands in SteamVR, drawn as white gloves

- Joy-Con movement from the accelerometer (report reordering, per-Joy-Con calibration, zero-velocity updates)
- MediaPipe pose and hand landmarks from the device camera, placement check, per-hand Kalman filter with the Joy-Con
- No controller connected: bare camera hands with Quest-style gestures, 1€ filtered, rendered as a white glove model built from a distance field
- SteamVR safe mode cleared before each start so one crash no longer blocks the driver

---------

Co-authored-by: MaxsTechReview <6025299+maxjivi05@users.noreply.github.com>
Co-authored-by: informalTechCode <informaltechyt@gmail.com>
Co-authored-by: connor g <connorsspam1992@gmail.com>
Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com>
Co-authored-by: Yassine Gherbi <y.gherbi.dev@gmail.com>
Co-authored-by: Resul Övür <92828257+ovurrsl@users.noreply.github.com>
Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com>
Co-authored-by: Kirk <kirtoy.junsay@gmail.com>
Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com>
Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com>
Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com>
Co-authored-by: asdf17128 <asdf17128@gmail.com>
Co-authored-by: siyuan.qsy <siyuan.qsy@alibaba-inc.com>
Co-authored-by: Dantalian <wsu091227@qq.com>
Co-authored-by: Dantalian <Dantalian-dev@users.noreply.github.com>
Co-authored-by: The412Banner <the412banner@users.noreply.github.com>
Co-authored-by: The412Banner <205237651+The412Banner@users.noreply.github.com>
Co-authored-by: FX1922 <120828496+FX192@users.noreply.github.com>
Co-authored-by: Muaz Kadan <32595066+muazkadan@users.noreply.github.com>
Co-authored-by: kingokksa <47888451+kingokksa@users.noreply.github.com>
Co-authored-by: Jamie Schenker <jam1esch3nk3r@gmail.com>
Co-authored-by: Jagger15 <jagger2333@live.com>
This commit is contained in:
221 files changed
+24369 -491

No files matched your search

+120
View File
@@ -0,0 +1,120 @@
// Arm model ported from Google's GvrArmModel.cs (Google VR SDK for Unity, Apache-2.0), see docs/third-party/joycon.
#pragma once
#include <algorithm>
#include <cmath>
namespace droiddeck {
struct Vec3 {
double x = 0.0, y = 0.0, z = 0.0;
};
struct Quat {
double w = 1.0, x = 0.0, y = 0.0, z = 0.0;
};
inline Vec3 operator+(const Vec3 &a, const Vec3 &b) { return {a.x + b.x, a.y + b.y, a.z + b.z}; }
inline Vec3 operator-(const Vec3 &a, const Vec3 &b) { return {a.x - b.x, a.y - b.y, a.z - b.z}; }
inline Vec3 operator*(const Vec3 &a, double s) { return {a.x * s, a.y * s, a.z * s}; }
inline double Dot(const Vec3 &a, const Vec3 &b) { return a.x * b.x + a.y * b.y + a.z * b.z; }
inline Vec3 Cross(const Vec3 &a, const Vec3 &b) { return {a.y * b.z - a.z * b.y, a.z * b.x - a.x * b.z, a.x * b.y - a.y * b.x}; }
inline double Length(const Vec3 &a) { return std::sqrt(Dot(a, a)); }
inline Quat operator*(const Quat &a, const Quat &b) {
return {a.w * b.w - a.x * b.x - a.y * b.y - a.z * b.z,
a.w * b.x + a.x * b.w + a.y * b.z - a.z * b.y,
a.w * b.y - a.x * b.z + a.y * b.w + a.z * b.x,
a.w * b.z + a.x * b.y - a.y * b.x + a.z * b.w};
}
inline Quat Conjugate(const Quat &q) { return {q.w, -q.x, -q.y, -q.z}; }
inline Quat Normalize(const Quat &q) {
const double n = std::sqrt(q.w * q.w + q.x * q.x + q.y * q.y + q.z * q.z);
if (!(n > 1e-9)) return {};
return {q.w / n, q.x / n, q.y / n, q.z / n};
}
inline Vec3 Rotate(const Quat &q, const Vec3 &v) {
const Vec3 u{q.x, q.y, q.z};
const Vec3 t = Cross(u, v) * 2.0;
return v + t * q.w + Cross(u, t);
}
inline Quat AboutX(double radians) { return {std::cos(radians / 2), std::sin(radians / 2), 0.0, 0.0}; }
inline Quat AboutY(double radians) { return {std::cos(radians / 2), 0.0, std::sin(radians / 2), 0.0}; }
inline Quat FromTo(const Vec3 &from, const Vec3 &to) {
const double d = Dot(from, to);
if (d < -0.999999) {
Vec3 axis = Cross({1.0, 0.0, 0.0}, from);
if (Length(axis) < 1e-6) axis = Cross({0.0, 1.0, 0.0}, from);
axis = axis * (1.0 / Length(axis));
return {0.0, axis.x, axis.y, axis.z};
}
const Vec3 c = Cross(from, to);
return Normalize({1.0 + d, c.x, c.y, c.z});
}
inline Quat Nlerp(const Quat &a, const Quat &b, double t) {
const double sign = a.w * b.w + a.x * b.x + a.y * b.y + a.z * b.z < 0.0 ? -1.0 : 1.0;
return Normalize({a.w + (sign * b.w - a.w) * t, a.x + (sign * b.x - a.x) * t,
a.y + (sign * b.y - a.y) * t, a.z + (sign * b.z - a.z) * t});
}
inline double AngleDegrees(const Quat &q) { return 2.0 * std::acos(std::min(std::fabs(q.w), 1.0)) * 180.0 / M_PI; }
inline bool Heading(const Quat &q, double *radians) {
const Vec3 forward = Rotate(q, {0.0, 0.0, -1.0});
if (std::hypot(forward.x, forward.z) < 0.25) return false;
*radians = std::atan2(-forward.x, -forward.z);
return true;
}
class ArmModel {
public:
explicit ArmModel(bool right) : side_(right ? 1.0 : -1.0) {}
void Reset() { torsoSet_ = false; }
Vec3 Update(const Vec3 &head, const Quat &headRotation, const Quat &controller, double angularSpeed, double dt) {
Vec3 gaze = Rotate(headRotation, {0.0, 0.0, -1.0});
gaze.y = 0.0;
const double length = Length(gaze);
if (length > 1e-3) {
gaze = gaze * (1.0 / length);
if (!torsoSet_) {
torso_ = gaze;
torsoSet_ = true;
} else {
const double strength = std::clamp((angularSpeed - 0.2) / 45.0, 0.0, 0.1);
const double follow = 1.0 - std::pow(1.0 - strength, std::max(dt, 0.0) * 60.0);
const double angle = std::atan2(Cross(torso_, gaze).y, Dot(torso_, gaze));
torso_ = Rotate(AboutY(angle * follow), torso_);
}
}
const Quat torso = AboutY(std::atan2(-torso_.x, -torso_.z));
const Vec3 forward = Rotate(Conjugate(torso) * controller, {0.0, 0.0, -1.0});
const double elevation = std::asin(std::clamp(forward.y, -1.0, 1.0)) * 180.0 / M_PI;
const Quat swing = FromTo({0.0, 0.0, -1.0}, forward);
const double extension = std::clamp((elevation - kMinExtension) / (kMaxExtension - kMinExtension), 0.0, 1.0);
const Vec3 elbow = Vec3{0.195 * side_, -0.5, -0.005} + Vec3{-0.13 * side_, 0.14, -0.08} * extension;
const double suppression = 1.0 - std::pow(AngleDegrees(swing) / 180.0, 6.0);
const double share = ((1.0 - kElbowBend) + kElbowBend * extension * kExtensionWeight) * suppression;
const Quat lerp = Nlerp({}, swing, share);
const Quat elbowRotation = torso * Conjugate(lerp) * swing;
const Quat wristRotation = elbowRotation * lerp;
const Vec3 elbowAt = head + Rotate(torso, elbow);
const Vec3 wristAt = elbowAt + Rotate(elbowRotation, {0.0, 0.0, -0.25});
return wristAt + Rotate(wristRotation, {0.0, 0.0, -0.05});
}
private:
static constexpr double kMinExtension = 7.0, kMaxExtension = 60.0, kElbowBend = 0.6, kExtensionWeight = 0.4;
double side_;
bool torsoSet_ = false;
Vec3 torso_{0.0, 0.0, -1.0};
};
}
File diff suppressed because it is too large. Load diff
+104
View File
@@ -0,0 +1,104 @@
#pragma once
#include <stdint.h>
#include <stdlib.h>
#include <vulkan/vulkan.h>
/* All pool operations require the caller's tracker lock. Reserved fences are
* invisible to the completer until vkQueueSubmit has returned successfully. */
struct tracked {
struct tracked *next;
VkDevice dev;
VkFence fence;
PFN_vkWaitForFences wait;
PFN_vkResetFences reset;
PFN_vkDestroyFence destroy;
uint32_t mark;
enum { TRACK_IDLE, TRACK_RESERVED, TRACK_SUBMITTED } state;
};
static struct tracked *track_pool_take(struct tracked **pool, VkDevice dev, PFN_vkGetDeviceProcAddr gdpa) {
struct tracked *idle = NULL;
for (struct tracked *t = *pool; t; t = t->next) {
if (t->state != TRACK_IDLE) continue;
if (t->dev == dev) {
t->state = TRACK_RESERVED;
return t;
}
if (!idle) idle = t;
}
struct tracked t = {.dev = dev, .state = TRACK_RESERVED};
t.wait = (PFN_vkWaitForFences)gdpa(dev, "vkWaitForFences");
t.reset = (PFN_vkResetFences)gdpa(dev, "vkResetFences");
t.destroy = (PFN_vkDestroyFence)gdpa(dev, "vkDestroyFence");
PFN_vkCreateFence create = (PFN_vkCreateFence)gdpa(dev, "vkCreateFence");
if (!t.wait || !t.reset || !t.destroy || !create) return NULL;
/* Never wait for a slot: the next submission may signal a semaphore that
* every outstanding fence is waiting on. Grow only at the high-water mark. */
struct tracked *slot = idle ? idle : calloc(1, sizeof(*slot));
if (!slot) return NULL;
VkFenceCreateInfo fci = {VK_STRUCTURE_TYPE_FENCE_CREATE_INFO, NULL, 0};
if (create(dev, &fci, NULL, &t.fence) != VK_SUCCESS) {
if (!idle) free(slot);
return NULL;
}
if (idle) {
idle->destroy(idle->dev, idle->fence, NULL);
t.next = idle->next;
} else {
t.next = *pool;
*pool = slot;
}
*slot = t;
return slot;
}
static uint32_t track_pool_submitted(struct tracked *t, VkResult result, uint32_t issued) {
if (result == VK_SUCCESS) {
t->mark = ++issued;
t->state = TRACK_SUBMITTED;
} else {
t->reset(t->dev, 1, &t->fence);
t->state = TRACK_IDLE;
}
return issued;
}
static int track_pool_reap(struct tracked *pool, uint32_t issued, uint32_t *done) {
int pending = 0;
uint32_t through = issued;
for (struct tracked *t = pool; t; t = t->next) {
if (t->state != TRACK_SUBMITTED) continue;
/* Scan all fences without blocking behind an earlier submission on a
* different queue. Device loss retires fences as in the previous path. */
if (t->wait(t->dev, 1, &t->fence, VK_TRUE, 0) == VK_TIMEOUT) {
pending = 1;
if ((int32_t)(t->mark - through) <= 0) through = t->mark - 1;
} else {
t->reset(t->dev, 1, &t->fence);
t->state = TRACK_IDLE;
}
}
/* done is a contiguous watermark, not a count of completed fences. Later
* work may free its slot but cannot release an unfinished presenter frame.
* Signed differences also handle the uint32_t counter wrapping. */
*done = through;
return pending;
}
static int track_pool_device_busy(struct tracked *pool, VkDevice dev) {
for (struct tracked *t = pool; t; t = t->next)
if (t->dev == dev && t->state != TRACK_IDLE) return 1;
return 0;
}
/* The caller must first wait for all this device's reservations and work. */
static void track_pool_forget(struct tracked **pool, VkDevice dev) {
for (struct tracked **p = pool; *p;) {
struct tracked *t = *p;
if (t->dev != dev) { p = &t->next; continue; }
t->destroy(t->dev, t->fence, NULL);
*p = t->next;
free(t);
}
}
+463
View File
@@ -0,0 +1,463 @@
#pragma once
#include <algorithm>
#include <array>
#include <cmath>
#include <cstdint>
#include <cstdio>
#include <string>
#include <vector>
#include "hand_pose.h"
namespace droiddeck {
struct HandMesh {
std::vector<Vec3> points, normals;
std::vector<double> tints, shades;
std::vector<std::array<int, 3>> faces;
int Add(const Vec3 &p, const Vec3 &n, double tint = 0.0, double shade = 1.0) {
points.push_back(p);
normals.push_back(n);
tints.push_back(tint);
shades.push_back(shade);
return (int)points.size() - 1;
}
void Face(int a, int b, int c, const Vec3 &outward) {
const Vec3 n = Cross(points[b] - points[a], points[c] - points[a]);
if (Dot(n, outward) < 0.0) faces.push_back({a, c, b});
else faces.push_back({a, b, c});
}
};
inline FingerCurls RelaxedCurls() {
FingerCurls c;
c.thumb = 0.15;
c.index = 0.1;
c.middle = c.ring = c.pinky = 0.15;
return c;
}
inline void HandShape(bool right, bool pinch, bool fist, Vec3 out[kHandJoints]) {
using namespace hand;
FingerCurls c = RelaxedCurls();
if (pinch) {
c.thumb = 0.7;
c.index = 0.55;
}
if (fist) c.middle = c.ring = c.pinky = 1.0;
HandJoints(right, c, out);
if (pinch) {
const Vec3 meet = out[kIndexTip] + (out[kThumbIp] - out[kIndexTip]) * 0.15;
const Vec3 shift = meet - out[kThumbTip];
out[kThumbTip] = meet;
out[kThumbIp] = out[kThumbIp] + shift * 0.5;
out[kThumbMcp] = out[kThumbMcp] + shift * 0.15;
}
}
enum class HandPart { Hand, ThumbIndex };
namespace glove {
constexpr double kStep = 0.0026;
constexpr double kPalmHalf = 0.0105;
constexpr double kPalmRound = 0.0085;
constexpr double kWristSide = 0.02;
constexpr double kWristSpan = 0.017;
constexpr double kWristRadius = 0.0165;
constexpr double kCuffStart = 0.006;
constexpr double kCuffLength = 0.016;
constexpr double kCuffWidth[2] = {0.0295, 0.029};
constexpr double kCuffDepth[2] = {0.021, 0.0205};
constexpr double kCuffRound = 0.006;
constexpr double kJointBlend = 0.001;
constexpr double kFingerBlend = 0.010;
constexpr double kThumbBlend = 0.016;
constexpr double kWristBlend = 0.012;
constexpr double kCuffBlend = 0.005;
constexpr double kTintEdge = 0.0015;
constexpr double kTouchBlend = 0.008;
constexpr double kCutBlend = 0.004;
constexpr double kDome[3] = {0.030, 0.040, 0.0135};
constexpr double kDomeAt = 0.55;
constexpr double kDomeBlend = 0.022;
constexpr double kFistRadius = 0.0095;
constexpr double kFistBlend = 0.006;
constexpr double kOcclusionStep = 0.004;
constexpr double kOcclusion = 3.0;
constexpr double kDarkest = 0.4;
constexpr double kInset = 0.0003;
constexpr double kReach = 0.03;
constexpr double kRadii[5][4] = {{0.0135, 0.0122, 0.0110, 0.0096},
{0.0106, 0.0098, 0.0092, 0.0087},
{0.0110, 0.0102, 0.0096, 0.0090},
{0.0105, 0.0097, 0.0091, 0.0086},
{0.0094, 0.0087, 0.0082, 0.0077}};
inline Vec3 Unit(const Vec3 &v) {
const double l = Length(v);
return l > 1e-12 ? v * (1.0 / l) : Vec3{0.0, 1.0, 0.0};
}
inline double Smin(double a, double b, double k) {
const double h = std::max(k - std::fabs(a - b), 0.0) / k;
return std::min(a, b) - h * h * k * 0.25;
}
inline double Cone(const Vec3 &p, const Vec3 &a, const Vec3 &b, double ra, double rb) {
const Vec3 ab = b - a;
const double t = std::min(1.0, std::max(0.0, Dot(p - a, ab) / std::max(Dot(ab, ab), 1e-12)));
return Length(p - (a + ab * t)) - (ra + (rb - ra) * t);
}
inline double Polygon(const std::vector<std::array<double, 2>> &v, double x, double y) {
double d = (x - v[0][0]) * (x - v[0][0]) + (y - v[0][1]) * (y - v[0][1]);
double sign = 1.0;
for (size_t i = 0, j = v.size() - 1; i < v.size(); j = i++) {
const double ex = v[j][0] - v[i][0], ey = v[j][1] - v[i][1];
const double wx = x - v[i][0], wy = y - v[i][1];
const double t = std::min(1.0, std::max(0.0, (wx * ex + wy * ey) / std::max(ex * ex + ey * ey, 1e-18)));
const double bx = wx - ex * t, by = wy - ey * t;
d = std::min(d, bx * bx + by * by);
const bool a = y >= v[i][1], b = y < v[j][1], c = ex * wy > ey * wx;
if ((a && b && c) || (!a && !b && !c)) sign = -sign;
}
return sign * std::sqrt(d);
}
inline double Rounded(double a, double b, double round) {
return std::min(std::max(a, b), 0.0) + std::hypot(std::max(a, 0.0), std::max(b, 0.0)) - round;
}
struct Finger {
Vec3 joints[4];
double radii[4];
double Distance(const Vec3 &p) const {
double d = Cone(p, joints[0], joints[1], radii[0], radii[1]);
for (int i = 1; i < 3; i++) d = Smin(d, Cone(p, joints[i], joints[i + 1], radii[i], radii[i + 1]), kJointBlend);
return d;
}
};
class Glove {
public:
Glove(bool right, HandPart part, bool closed) : part_(part) {
using namespace hand;
Vec3 open[kHandJoints], j[kHandJoints];
HandShape(right, false, false, open);
HandShape(right, part == HandPart::ThumbIndex && closed, part == HandPart::Hand && closed, j);
origin_ = open[kWrist];
along_ = Unit(open[kMiddleMcp] - open[kWrist]);
normal_ = Unit(Cross(open[kIndexMcp] - open[kWrist], open[kPinkyMcp] - open[kWrist]));
across_ = Unit(Cross(along_, normal_));
const Vec3 side = across_ * (Dot(open[kIndexMcp] - open[kPinkyMcp], across_) > 0.0 ? 1.0 : -1.0);
for (const Vec3 &q : {open[kWrist] + side * kWristSide, open[kThumbCmc], open[kIndexMcp], open[kMiddleMcp], open[kRingMcp],
open[kPinkyMcp], open[kWrist] - side * kWristSide})
outline_.push_back({Dot(q - origin_, across_), Dot(q - origin_, along_)});
wristA_ = open[kWrist] + across_ * kWristSpan;
wristB_ = open[kWrist] - across_ * kWristSpan;
const int first = part == HandPart::Hand ? 2 : 0, last = part == HandPart::Hand ? 4 : 1;
for (int f = first; f <= last; f++) {
Finger finger;
const int root = f == 0 ? kThumbCmc : kIndexMcp + 4 * (f - 1);
for (int i = 0; i < 4; i++) {
finger.joints[i] = j[root + i];
finger.radii[i] = kRadii[f][i];
}
finger.joints[3] = finger.joints[3] - Unit(finger.joints[3] - finger.joints[2]) * (finger.radii[3] * 0.7);
fingers_.push_back(finger);
}
dome_ = open[kWrist] + (open[kMiddleMcp] - open[kWrist]) * kDomeAt;
back_ = normal_ * (Dot(normal_, open[kThumbTip] - dome_) > 0.0 ? -1.0 : 1.0);
fist_ = part == HandPart::Hand && closed;
for (int f = 0; f < 3 && fist_; f++) {
fistLine_[f] = Vec3{};
for (int i = 0; i < 4; i++) fistLine_[f] = fistLine_[f] + j[kMiddleMcp + 4 * f + i] * 0.25;
}
lo_ = hi_ = origin_;
const auto grow = [&](const Vec3 &q, double r) {
lo_ = Vec3{std::min(lo_.x, q.x - r), std::min(lo_.y, q.y - r), std::min(lo_.z, q.z - r)};
hi_ = Vec3{std::max(hi_.x, q.x + r), std::max(hi_.y, q.y + r), std::max(hi_.z, q.z + r)};
};
for (int i : {kWrist, kThumbCmc, kIndexMcp, kMiddleMcp, kRingMcp, kPinkyMcp}) grow(open[i], kPalmHalf + kPalmRound + kWristSide);
grow(origin_ - along_ * kCuffLength, kCuffWidth[1] + kCuffRound);
for (const Finger &f : fingers_)
for (int i = 0; i < 4; i++) grow(f.joints[i], f.radii[i] + kThumbBlend);
}
double Distance(const Vec3 &p) const {
double palm = Smin(Smin(Slab(p), Cone(p, wristA_, wristB_, kWristRadius, kWristRadius), kWristBlend), Cuff(p), kCuffBlend);
if (part_ == HandPart::ThumbIndex) {
double reach = 1e9;
for (const Finger &f : fingers_) reach = std::min(reach, f.Distance(p));
palm = -Smin(-(palm + kInset), kReach - reach, kCutBlend);
}
double fingers = fingers_[0].Distance(p);
for (size_t i = 1; i < fingers_.size(); i++) fingers = Smin(fingers, fingers_[i].Distance(p), kTouchBlend);
if (fist_)
fingers = Smin(fingers, std::min(Cone(p, fistLine_[0], fistLine_[1], kFistRadius, kFistRadius),
Cone(p, fistLine_[1], fistLine_[2], kFistRadius, kFistRadius)), kFistBlend);
return Smin(palm, fingers, part_ == HandPart::ThumbIndex ? kThumbBlend : kFingerBlend);
}
double Tint(const Vec3 &p) const {
double rest = Smin(Slab(p), Cone(p, wristA_, wristB_, kWristRadius, kWristRadius), kWristBlend);
for (const Finger &f : fingers_) rest = std::min(rest, f.Distance(p));
return std::min(1.0, std::max(0.0, 0.5 + (rest - Cuff(p)) / (2.0 * kTintEdge)));
}
double Shade(const Vec3 &p, const Vec3 &n) const {
double hidden = 0.0, weight = 1.0;
for (int i = 1; i <= 5; i++) {
const double h = kOcclusionStep * i;
hidden += (h - Distance(p + n * h)) * weight;
weight *= 0.6;
}
const double open = std::min(1.0, std::max(0.0, 1.0 - kOcclusion * hidden / kOcclusionStep / 5.0));
const double light = 0.88 + 0.12 * Dot(n, back_);
return std::min(1.0, std::max(0.0, open * light));
}
Vec3 Low() const { return lo_ - Vec3{2.0, 2.0, 2.0} * kStep; }
Vec3 High() const { return hi_ + Vec3{2.0, 2.0, 2.0} * kStep; }
private:
double Slab(const Vec3 &p) const {
const Vec3 q = p - origin_;
const double slab = Rounded(Polygon(outline_, Dot(q, across_), Dot(q, along_)), std::fabs(Dot(q, normal_)) - (kPalmHalf - kPalmRound), kPalmRound);
const Vec3 d = p - dome_;
const double x = Dot(d, across_), y = Dot(d, along_), z = Dot(d, normal_);
const double k0 = std::sqrt(x * x / (kDome[0] * kDome[0]) + y * y / (kDome[1] * kDome[1]) + z * z / (kDome[2] * kDome[2]));
const double k1 = std::sqrt(x * x / std::pow(kDome[0], 4) + y * y / std::pow(kDome[1], 4) + z * z / std::pow(kDome[2], 4));
return Smin(slab, k1 > 1e-12 ? k0 * (k0 - 1.0) / k1 : -kDome[2], kDomeBlend);
}
double Cuff(const Vec3 &p) const {
const Vec3 q = p - origin_;
const double t = -Dot(q, along_) + kCuffStart;
const double s = std::min(1.0, std::max(0.0, t / kCuffLength));
const double a = kCuffWidth[0] + (kCuffWidth[1] - kCuffWidth[0]) * s - kCuffRound;
const double b = kCuffDepth[0] + (kCuffDepth[1] - kCuffDepth[0]) * s - kCuffRound;
const double x = Dot(q, across_), y = Dot(q, normal_);
const double k0 = std::hypot(x / a, y / b), k1 = std::hypot(x / (a * a), y / (b * b));
const double radial = k1 > 1e-12 ? k0 * (k0 - 1.0) / k1 : -std::min(a, b);
return Rounded(radial, std::fabs(t - kCuffLength * 0.5) - (kCuffLength * 0.5 - kCuffRound), kCuffRound);
}
HandPart part_;
bool fist_ = false;
Vec3 origin_, along_, across_, normal_, wristA_, wristB_, lo_, hi_, dome_, back_, fistLine_[3];
std::vector<std::array<double, 2>> outline_;
std::vector<Finger> fingers_;
};
template <typename Field> Vec3 Gradient(const Field &f, const Vec3 &p) {
constexpr double e = 1e-5;
return Vec3{f(p + Vec3{e, 0.0, 0.0}) - f(p - Vec3{e, 0.0, 0.0}), f(p + Vec3{0.0, e, 0.0}) - f(p - Vec3{0.0, e, 0.0}),
f(p + Vec3{0.0, 0.0, e}) - f(p - Vec3{0.0, 0.0, e})} *
(0.5 / e);
}
template <typename Field, typename Painter>
HandMesh SurfaceNet(const Field &f, const Painter &paint, const Vec3 &lo, const Vec3 &hi, double step) {
const int nx = (int)std::ceil((hi.x - lo.x) / step) + 1, ny = (int)std::ceil((hi.y - lo.y) / step) + 1,
nz = (int)std::ceil((hi.z - lo.z) / step) + 1;
const auto at = [&](int i, int j, int k) { return (size_t(k) * ny + j) * nx + i; };
const auto corner = [&](int i, int j, int k) { return Vec3{lo.x + i * step, lo.y + j * step, lo.z + k * step}; };
std::vector<double> value(size_t(nx) * ny * nz);
for (int k = 0; k < nz; k++)
for (int j = 0; j < ny; j++)
for (int i = 0; i < nx; i++) value[at(i, j, k)] = f(corner(i, j, k));
std::vector<int> vertex(value.size(), -1);
HandMesh m;
for (int k = 0; k + 1 < nz; k++)
for (int j = 0; j + 1 < ny; j++)
for (int i = 0; i + 1 < nx; i++) {
double c[8];
int inside = 0;
for (int n = 0; n < 8; n++) inside += (c[n] = value[at(i + (n & 1), j + ((n >> 1) & 1), k + (n >> 2))]) < 0.0;
if (inside == 0 || inside == 8) continue;
Vec3 sum;
int count = 0;
for (int a = 0; a < 8; a++)
for (int bit = 1; bit < 8; bit <<= 1) {
const int b = a | bit;
if (b == a || (c[a] < 0.0) == (c[b] < 0.0)) continue;
const Vec3 pa = corner(i + (a & 1), j + ((a >> 1) & 1), k + (a >> 2));
const Vec3 pb = corner(i + (b & 1), j + ((b >> 1) & 1), k + (b >> 2));
sum = sum + pa + (pb - pa) * (c[a] / (c[a] - c[b]));
count++;
}
Vec3 p = sum * (1.0 / count);
for (int it = 0; it < 3; it++) {
const Vec3 g = Gradient(f, p);
const double gg = Dot(g, g);
if (gg < 1e-12) break;
p = p - g * (f(p) / gg);
}
const Vec3 low = corner(i, j, k), high = corner(i + 1, j + 1, k + 1);
p = Vec3{std::clamp(p.x, low.x, high.x), std::clamp(p.y, low.y, high.y), std::clamp(p.z, low.z, high.z)};
const Vec3 n = Unit(Gradient(f, p));
const std::array<double, 2> colour = paint(p, n);
vertex[at(i, j, k)] = m.Add(p, n, colour[0], colour[1]);
}
for (int k = 1; k + 1 < nz; k++)
for (int j = 1; j + 1 < ny; j++)
for (int i = 1; i + 1 < nx; i++) {
const bool in = value[at(i, j, k)] < 0.0;
for (int axis = 0; axis < 3; axis++) {
const int di = axis == 0, dj = axis == 1, dk = axis == 2;
if ((value[at(i + di, j + dj, k + dk)] < 0.0) == in) continue;
int q[4];
if (axis == 0) q[0] = vertex[at(i, j - 1, k - 1)], q[1] = vertex[at(i, j, k - 1)], q[2] = vertex[at(i, j, k)], q[3] = vertex[at(i, j - 1, k)];
else if (axis == 1) q[0] = vertex[at(i - 1, j, k - 1)], q[1] = vertex[at(i, j, k - 1)], q[2] = vertex[at(i, j, k)], q[3] = vertex[at(i - 1, j, k)];
else q[0] = vertex[at(i - 1, j - 1, k)], q[1] = vertex[at(i, j - 1, k)], q[2] = vertex[at(i, j, k)], q[3] = vertex[at(i - 1, j, k)];
if (q[0] < 0 || q[1] < 0 || q[2] < 0 || q[3] < 0) continue;
if (in == (axis == 1)) std::swap(q[1], q[3]);
if (Length(m.points[q[0]] - m.points[q[2]]) < Length(m.points[q[1]] - m.points[q[3]])) {
m.faces.push_back({q[0], q[1], q[2]});
m.faces.push_back({q[0], q[2], q[3]});
} else {
m.faces.push_back({q[0], q[1], q[3]});
m.faces.push_back({q[1], q[2], q[3]});
}
}
}
return m;
}
}
inline HandMesh BuildHandPart(bool right, HandPart part, bool closed) {
const glove::Glove g(right, part, closed);
return glove::SurfaceNet([&](const Vec3 &p) { return g.Distance(p); }, [&](const Vec3 &p, const Vec3 &n) { return std::array<double, 2>{g.Tint(p), g.Shade(p, n)}; }, g.Low(), g.High(),
glove::kStep);
}
inline std::string HandObj(const HandMesh &m, const char *material) {
std::string out = "mtllib hand.mtl\n";
char line[160];
for (const Vec3 &p : m.points) {
snprintf(line, sizeof(line), "v %.5f %.5f %.5f\n", p.x, p.y, p.z);
out += line;
}
for (size_t i = 0; i < m.tints.size(); i++) {
snprintf(line, sizeof(line), "vt %.3f %.3f\n", 0.02 + 0.96 * m.shades[i], 0.02 + 0.46 * m.tints[i]);
out += line;
}
for (const Vec3 &n : m.normals) {
snprintf(line, sizeof(line), "vn %.4f %.4f %.4f\n", n.x, n.y, n.z);
out += line;
}
out += "usemtl ";
out += material;
out += "\n";
for (const auto &f : m.faces) {
snprintf(line, sizeof(line), "f %d/%d/%d %d/%d/%d %d/%d/%d\n", f[0] + 1, f[0] + 1, f[0] + 1, f[1] + 1, f[1] + 1, f[1] + 1, f[2] + 1,
f[2] + 1, f[2] + 1);
out += line;
}
return out;
}
inline std::string HandMtl(const char *material, const char *texture) {
std::string out = "newmtl ";
out += material;
out += "\nNs 12.0\nd 1.0\nillum 2\nKa 1.0 1.0 1.0\nKd 1.0 1.0 1.0\nKs 0.1 0.1 0.1\nmap_Kd ";
out += texture;
out += "\n";
return out;
}
inline std::string HandModelJson(bool right) {
const auto part = [&](const char *name, const char *path, int button, bool shown) {
char text[512];
snprintf(text, sizeof(text),
" \"%s\": {\n"
" \"filename\": \"%s.obj\",\n"
" \"motion\": {\n"
" \"type\": \"translate\",\n"
" \"pressed_path\": \"%s\",\n"
" \"controller_button\": %d,\n"
" \"value_mapping\": [0.0, 0.0],\n"
" \"axis\": [0.0, 0.0, 1.0]\n"
" },\n"
" \"visibility\": { \"default\": %s, \"press\": %s }\n"
" },\n",
name, name, path, button, shown ? "false" : "true", shown ? "true" : "false");
return std::string(text);
};
const auto pose = [](const char *name, const char *origin, const char *rotate) {
char text[256];
snprintf(text, sizeof(text), " \"%s\": { \"component_local\": { \"origin\": [%s], \"rotate_xyz\": [%s] } }", name, origin, rotate);
return std::string(text);
};
std::string out = "{\n \"components\": {\n";
out += part("hand_open", "/input/grip/touch", 2, false);
out += part("hand_fist", "/input/grip/touch", 2, true);
out += part("thumb_index_open", "/input/trigger/click", 33, false);
out += part("thumb_index_pinch", "/input/trigger/click", 33, true);
out += pose("tip", right ? "-0.006, -0.015, 0.02" : "0.006, -0.015, 0.02", right ? "-40.0, 5.0, 0.0" : "-40.0, -5.0, 0.0") + ",\n";
out += pose("base", right ? "-0.004758, -0.037977, 0.200466" : "0.004758, -0.037977, 0.200466",
right ? "-155.4, -0.427, -7.081" : "-155.4, -0.427, 7.081") + ",\n";
out += pose("handgrip", right ? "0.003851, 0.003715, 0.075948" : "-0.003851, 0.003715, 0.075948",
right ? "15.392, 2.071, -0.303" : "15.392, -2.071, 0.303") + ",\n";
out += pose("grip", "0.0, -0.015, 0.13", right ? "15.392, 2.071, -0.303" : "15.392, -2.071, 0.303") + ",\n";
out += pose("gdc2015", "0.0, 0.0, 0.0", "0, 0, 0") + "\n";
out += " }\n}\n";
return out;
}
inline uint32_t Crc32(const uint8_t *data, size_t size, uint32_t crc = 0) {
crc = ~crc;
for (size_t i = 0; i < size; i++) {
crc ^= data[i];
for (int k = 0; k < 8; k++) crc = (crc >> 1) ^ (0xEDB88320u & (0u - (crc & 1u)));
}
return ~crc;
}
inline std::vector<uint8_t> GlovePng(int size, const std::array<uint8_t, 3> &glove, const std::array<uint8_t, 3> &cuff, double darkest) {
std::vector<uint8_t> raw;
for (int y = 0; y < size; y++) {
raw.push_back(0);
const double t = 1.0 - std::fabs(2.0 * (y + 0.5) / size - 1.0);
for (int x = 0; x < size; x++) {
const double light = darkest + (1.0 - darkest) * x / double(size - 1);
for (int c = 0; c < 3; c++) raw.push_back(uint8_t(std::lround((glove[c] + (cuff[c] - glove[c]) * t) * light)));
}
}
std::vector<uint8_t> zlib = {0x78, 0x01, 0x01, uint8_t(raw.size() & 0xff), uint8_t(raw.size() >> 8),
uint8_t(~raw.size() & 0xff), uint8_t((~raw.size() >> 8) & 0xff)};
zlib.insert(zlib.end(), raw.begin(), raw.end());
uint32_t a = 1, s = 0;
for (uint8_t v : raw) {
a = (a + v) % 65521;
s = (s + a) % 65521;
}
const uint32_t adler = (s << 16) | a;
for (int i = 3; i >= 0; i--) zlib.push_back(uint8_t(adler >> (8 * i)));
std::vector<uint8_t> png = {0x89, 'P', 'N', 'G', 0x0d, 0x0a, 0x1a, 0x0a};
const auto chunk = [&](const char *type, const std::vector<uint8_t> &data) {
for (int i = 3; i >= 0; i--) png.push_back(uint8_t(data.size() >> (8 * i)));
std::vector<uint8_t> body(type, type + 4);
body.insert(body.end(), data.begin(), data.end());
png.insert(png.end(), body.begin(), body.end());
const uint32_t crc = Crc32(body.data(), body.size());
for (int i = 3; i >= 0; i--) png.push_back(uint8_t(crc >> (8 * i)));
};
std::vector<uint8_t> header;
for (int v : {size, size})
for (int i = 3; i >= 0; i--) header.push_back(uint8_t(v >> (8 * i)));
header.insert(header.end(), {8, 2, 0, 0, 0});
chunk("IHDR", header);
chunk("IDAT", zlib);
chunk("IEND", {});
return png;
}
}
+322
View File
@@ -0,0 +1,322 @@
#pragma once
#include <algorithm>
#include <cmath>
#include <cstdint>
#include "arm_model.h"
namespace droiddeck {
class HandFollower {
public:
void Reset() { placed_ = false; }
void Place(const Vec3 &at, const Vec3 &moved) {
at_ = at;
lastMoved_ = moved;
placed_ = true;
}
Vec3 Update(const Vec3 &anchor, const Vec3 &moved, double dt, double anchorSeconds, double maxOffset) {
if (!placed_) {
at_ = anchor;
placed_ = true;
} else {
at_ = at_ + (moved - lastMoved_);
const double pull = 1.0 - std::exp(-std::max(dt, 0.0) / anchorSeconds);
at_ = at_ + (anchor - at_) * pull;
const Vec3 off = at_ - anchor;
const double length = Length(off);
if (length > maxOffset) at_ = anchor + off * (maxOffset / length);
}
lastMoved_ = moved;
return at_;
}
private:
bool placed_ = false;
Vec3 at_, lastMoved_;
};
class TimeHistory {
public:
void Clear() { count_ = next_ = 0; }
void Add(int64_t timeNs, const Vec3 &value) {
if (count_ && timeNs <= time_[Last()]) {
if (timeNs == time_[Last()]) value_[Last()] = value;
return;
}
time_[next_] = timeNs;
value_[next_] = value;
next_ = (next_ + 1) % kSize;
count_ = std::min(count_ + 1, kSize);
}
bool At(int64_t timeNs, Vec3 *out) const {
if (!count_) return false;
int newer = Last();
if (timeNs >= time_[newer]) {
*out = value_[newer];
return true;
}
for (int i = 1; i < count_; i++) {
const int older = (newer - 1 + kSize) % kSize;
if (time_[older] <= timeNs) {
const double t = double(timeNs - time_[older]) / double(time_[newer] - time_[older]);
*out = value_[older] + (value_[newer] - value_[older]) * t;
return true;
}
newer = older;
}
*out = value_[newer];
return true;
}
private:
int Last() const { return (next_ - 1 + kSize) % kSize; }
static constexpr int kSize = 256;
int64_t time_[kSize] = {};
Vec3 value_[kSize];
int count_ = 0, next_ = 0;
};
class CameraYaw {
public:
void Reset() {
c_ = s_ = weight_ = 0.0;
outliers_ = 0;
}
void Add(double yaw, double dt) {
double current = 0.0;
if (Get(&current) && std::fabs(std::remainder(yaw - current, 2.0 * M_PI)) > kJump) {
if (++outliers_ < kOutlierFrames) return;
Reset();
}
outliers_ = 0;
const double keep = std::exp(-std::max(dt, 0.0) / kSeconds);
c_ = c_ * keep + std::cos(yaw);
s_ = s_ * keep + std::sin(yaw);
weight_ = weight_ * keep + 1.0;
}
bool Get(double *yaw) const {
if (weight_ < kMinWeight || std::hypot(c_, s_) < kMinAgreement * weight_) return false;
*yaw = std::atan2(s_, c_);
return true;
}
private:
static constexpr double kSeconds = 3.0, kMinWeight = 4.0, kMinAgreement = 0.7, kJump = 0.6;
static constexpr int kOutlierFrames = 8;
double c_ = 0.0, s_ = 0.0, weight_ = 0.0;
int outliers_ = 0;
};
class HandKalman {
public:
void Reset() {
ready_ = false;
haveInput_ = false;
count_ = 0;
outliers_ = 0;
seenNs_ = 0;
stillSeconds_ = 0.0;
}
void Forget() {
ready_ = false;
outliers_ = 0;
}
bool Ready() const { return ready_; }
int64_t SeenNs() const { return seenNs_; }
int64_t TimeNs() const { return lastNs_; }
Vec3 Position() const { return {axes_[0].x[0], axes_[1].x[0], axes_[2].x[0]}; }
Vec3 Velocity() const { return {axes_[0].x[1], axes_[1].x[1], axes_[2].x[1]}; }
bool Still() const { return stillSeconds_ > kStillSeconds; }
void Input(int64_t timeNs, const Vec3 &inertial, double turnRate) {
if (haveInput_ && timeNs <= lastNs_) return;
if (!haveInput_ || timeNs - lastNs_ > kGapNs) {
Reset();
haveInput_ = true;
lastNs_ = timeNs;
lastInertial_ = inertial;
return;
}
Step step;
step.timeNs = timeNs;
step.dt = (timeNs - lastNs_) * 1e-9;
step.du = inertial - lastInertial_;
lastNs_ = timeNs;
lastInertial_ = inertial;
const bool calm = turnRate < kStillTurn && Length(step.du) < kStillAccel * step.dt;
stillSeconds_ = calm ? stillSeconds_ + step.dt : 0.0;
if (ready_) {
Propagate(axes_, step.du, step.dt, Fade(timeNs - seenNs_));
if (Still()) Stop(axes_);
for (int a = 0; a < 3; a++) step.after[a] = axes_[a];
}
steps_[next_] = step;
next_ = (next_ + 1) % kSteps;
count_ = std::min(count_ + 1, kSteps);
}
bool Measure(int64_t timeNs, const Vec3 &at, double sigma) {
if (!haveInput_ || timeNs > lastNs_ + kFutureNs || (count_ == 0 && timeNs < lastNs_)) return false;
int k = -1;
for (int i = 0; i < count_; i++) {
const Step &s = steps_[Index(i)];
if (s.timeNs >= timeNs) {
k = i;
break;
}
}
const double z[3] = {at.x, at.y, at.z};
if (k < 0) {
if (!ready_) Start(axes_, z, sigma);
else if (!Accept(axes_, z, sigma)) return false;
else Correct(axes_, z, sigma);
ready_ = true;
seenNs_ = timeNs;
return true;
}
const Step &first = steps_[Index(k)];
if (timeNs < first.timeNs - int64_t(first.dt * 1e9)) return false;
Axis axes[3];
if (ready_) {
if (k == 0) return false;
for (int a = 0; a < 3; a++) axes[a] = steps_[Index(k - 1)].after[a];
}
const double frac = std::clamp((timeNs - (first.timeNs - first.dt * 1e9)) / (first.dt * 1e9), 0.0, 1.0);
if (ready_) {
Propagate(axes, first.du * frac, first.dt * frac, Fade(timeNs - seenNs_));
if (!Accept(axes, z, sigma)) return false;
Correct(axes, z, sigma);
} else {
Start(axes, z, sigma);
}
Propagate(axes, first.du * (1.0 - frac), first.dt * (1.0 - frac), kDampingSeconds);
for (int a = 0; a < 3; a++) steps_[Index(k)].after[a] = axes[a];
for (int i = k + 1; i < count_; i++) {
Step &s = steps_[Index(i)];
Propagate(axes, s.du, s.dt, Fade(s.timeNs - timeNs));
for (int a = 0; a < 3; a++) s.after[a] = axes[a];
}
for (int a = 0; a < 3; a++) axes_[a] = axes[a];
ready_ = true;
seenNs_ = timeNs;
return true;
}
private:
struct Axis {
double x[2] = {0.0, 0.0};
double P[2][2] = {};
};
struct Step {
int64_t timeNs = 0;
double dt = 0.0;
Vec3 du;
Axis after[3];
};
int Index(int i) const { return (next_ - count_ + i + kSteps) % kSteps; }
static void Start(Axis *axes, const double *z, double sigma) {
for (int a = 0; a < 3; a++) {
axes[a] = Axis();
axes[a].x[0] = z[a];
axes[a].P[0][0] = sigma * sigma;
axes[a].P[1][1] = kStartSpeed * kStartSpeed;
}
}
bool Accept(const Axis *axes, const double *z, double sigma) {
double d2 = 0.0;
for (int a = 0; a < 3; a++) {
const double y = z[a] - axes[a].x[0];
d2 += y * y / (axes[a].P[0][0] + sigma * sigma);
}
if (d2 <= kGate) {
outliers_ = 0;
return true;
}
if (++outliers_ < kOutliers) return false;
outliers_ = 0;
ready_ = false;
return true;
}
void Correct(Axis *axes, const double *z, double sigma) const {
if (!ready_) {
Start(axes, z, sigma);
return;
}
for (int a = 0; a < 3; a++) Update(axes[a], 0, z[a], sigma);
}
static void Update(Axis &s, int state, double z, double sigma) {
const double S = s.P[state][state] + sigma * sigma;
const double K[2] = {s.P[0][state] / S, s.P[1][state] / S};
const double y = z - s.x[state];
s.x[0] += K[0] * y;
s.x[1] += K[1] * y;
const double p00 = s.P[0][0] - K[0] * s.P[state][0];
const double p01 = s.P[0][1] - K[0] * s.P[state][1];
const double p11 = s.P[1][1] - K[1] * s.P[state][1];
s.P[0][0] = p00;
s.P[0][1] = s.P[1][0] = p01;
s.P[1][1] = p11;
}
static void Stop(Axis *axes) {
for (int a = 0; a < 3; a++) Update(axes[a], 1, 0.0, kStillSpeed);
}
static double Fade(int64_t sinceSeenNs) { return sinceSeenNs > kStaleNs ? kStaleDampingSeconds : kDampingSeconds; }
static void Propagate(Axis *axes, const Vec3 &du, double dt, double damping) {
if (dt <= 0.0) return;
const double motion = Length(du);
const double g = kMotionError * kMotionError * motion * motion;
const double d[3] = {du.x, du.y, du.z};
const double keep = std::exp(-dt / damping);
const double q00 = kNoise * dt * dt * dt / 3.0 + g * dt * dt / 4.0;
const double q01 = kNoise * dt * dt / 2.0 + g * dt / 2.0;
const double q11 = kNoise * dt + g;
for (int a = 0; a < 3; a++) {
Axis &s = axes[a];
s.x[0] += s.x[1] * dt + 0.5 * d[a] * dt;
s.x[1] = (s.x[1] + d[a]) * keep;
const double p00 = s.P[0][0] + 2.0 * dt * s.P[0][1] + dt * dt * s.P[1][1] + q00;
const double p01 = keep * (s.P[0][1] + dt * s.P[1][1]) + q01;
const double p11 = keep * keep * s.P[1][1] + q11;
s.P[0][0] = p00;
s.P[0][1] = s.P[1][0] = p01;
s.P[1][1] = p11;
}
}
static constexpr int kSteps = 192;
static constexpr int64_t kGapNs = 250000000;
static constexpr int64_t kFutureNs = 50000000, kStaleNs = 200000000;
static constexpr double kNoise = 2.0, kMotionError = 0.3, kDampingSeconds = 0.3, kStaleDampingSeconds = 0.1;
static constexpr double kStartSpeed = 0.5;
static constexpr double kStillTurn = 0.3, kStillAccel = 0.6, kStillSeconds = 0.1, kStillSpeed = 0.05;
static constexpr double kGate = 25.0;
static constexpr int kOutliers = 3;
Step steps_[kSteps];
Axis axes_[3];
Vec3 lastInertial_;
int64_t lastNs_ = 0, seenNs_ = 0;
double stillSeconds_ = 0.0;
int count_ = 0, next_ = 0, outliers_ = 0;
bool ready_ = false, haveInput_ = false;
};
}
+443
View File
@@ -0,0 +1,443 @@
#pragma once
#include <algorithm>
#include <cmath>
#include <cstdint>
#include "hand_skeleton.h"
namespace droiddeck {
constexpr int kHandJoints = 21;
namespace hand {
enum : int {
kWrist = 0, kThumbCmc = 1, kThumbMcp = 2, kThumbIp = 3, kThumbTip = 4,
kIndexMcp = 5, kIndexPip = 6, kIndexDip = 7, kIndexTip = 8,
kMiddleMcp = 9, kMiddleTip = 12, kRingMcp = 13, kRingTip = 16, kPinkyMcp = 17, kPinkyTip = 20,
};
constexpr int kPalm[5] = {kWrist, kIndexMcp, kMiddleMcp, kRingMcp, kPinkyMcp};
constexpr int kCurlSteps = 20;
inline double Angle(const Vec3 &a, const Vec3 &b) {
const double la = Length(a), lb = Length(b);
if (la < 1e-9 || lb < 1e-9) return 0.0;
return std::acos(std::clamp(Dot(a, b) / (la * lb), -1.0, 1.0));
}
inline double Bend(const Vec3 joints[kHandJoints], int finger) {
if (finger == 0)
return Angle(joints[kThumbMcp] - joints[kThumbCmc], joints[kThumbIp] - joints[kThumbMcp]) +
Angle(joints[kThumbIp] - joints[kThumbMcp], joints[kThumbTip] - joints[kThumbIp]);
const int mcp = kIndexMcp + (finger - 1) * 4;
return Angle(joints[mcp] - joints[kWrist], joints[mcp + 1] - joints[mcp]) +
Angle(joints[mcp + 1] - joints[mcp], joints[mcp + 2] - joints[mcp + 1]) +
Angle(joints[mcp + 2] - joints[mcp + 1], joints[mcp + 3] - joints[mcp + 2]);
}
inline double &CurlOf(FingerCurls &c, int finger) {
switch (finger) {
case 0: return c.thumb;
case 1: return c.index;
case 2: return c.middle;
case 3: return c.ring;
default: return c.pinky;
}
}
}
inline void HandJoints(bool right, const FingerCurls &curls, Vec3 out[kHandJoints]) {
using namespace hand;
vr::VRBoneTransform_t bones[kHandBones];
HandSkeleton(right, curls, bones);
const Quat wristRotation = ToQuat(bones[1].orientation);
const Vec3 wrist = ToVec(bones[1].position);
out[kWrist] = wrist;
const auto chain = [&](int first, int count, Vec3 *joints) {
Quat r = wristRotation;
Vec3 p = wrist;
for (int b = 0; b < count; b++) {
p = p + Rotate(r, ToVec(bones[first + b].position));
r = r * ToQuat(bones[first + b].orientation);
joints[b] = p;
}
};
chain(2, 4, &out[kThumbCmc]);
for (int f = 0; f < 4; f++) {
Vec3 joints[5];
chain(6 + f * 5, 5, joints);
for (int j = 0; j < 4; j++) out[kIndexMcp + f * 4 + j] = joints[j + 1];
}
}
class HandModel {
public:
explicit HandModel(bool right) : right_(right) {
Vec3 open[kHandJoints], closed[kHandJoints];
HandJoints(right, FingerCurls{}, open);
FingerCurls fist;
fist.thumb = fist.index = fist.middle = fist.ring = fist.pinky = 1.0;
HandJoints(right, fist, closed);
for (int i = 0; i < 5; i++) center_ = center_ + open[hand::kPalm[i]] * 0.2;
const Vec3 normal = Cross(open[hand::kIndexMcp] - open[hand::kWrist], open[hand::kPinkyMcp] - open[hand::kWrist]);
normalSign_ = Dot(normal, closed[hand::kMiddleTip] - open[hand::kMiddleTip]) >= 0.0 ? 1.0 : -1.0;
for (int s = 0; s <= hand::kCurlSteps; s++) {
FingerCurls c;
c.thumb = c.index = c.middle = c.ring = c.pinky = double(s) / hand::kCurlSteps;
Vec3 joints[kHandJoints];
HandJoints(right, c, joints);
for (int f = 0; f < 5; f++) bends_[f][s] = hand::Bend(joints, f);
}
}
bool Right() const { return right_; }
Vec3 PalmCenter() const { return center_; }
bool Fit(const Vec3 joints[kHandJoints], const FingerCurls &curls, Quat *rotation, Vec3 *center, double *rms) const {
Vec3 shape[kHandJoints], palm[5], model;
HandJoints(right_, curls, shape);
for (int i = 0; i < 5; i++) model = model + shape[hand::kPalm[i]] * 0.2;
for (int i = 0; i < 5; i++) palm[i] = shape[hand::kPalm[i]] - model;
Vec3 observed[5], mean;
for (int i = 0; i < 5; i++) mean = mean + joints[hand::kPalm[i]] * 0.2;
for (int i = 0; i < 5; i++) observed[i] = joints[hand::kPalm[i]] - mean;
double S[3][3] = {};
for (int i = 0; i < 5; i++) {
const double m[3] = {palm[i].x, palm[i].y, palm[i].z};
const double o[3] = {observed[i].x, observed[i].y, observed[i].z};
for (int r = 0; r < 3; r++)
for (int c = 0; c < 3; c++) S[r][c] += m[r] * o[c];
}
double N[4][4] = {
{S[0][0] + S[1][1] + S[2][2], S[1][2] - S[2][1], S[2][0] - S[0][2], S[0][1] - S[1][0]},
{S[1][2] - S[2][1], S[0][0] - S[1][1] - S[2][2], S[0][1] + S[1][0], S[2][0] + S[0][2]},
{S[2][0] - S[0][2], S[0][1] + S[1][0], -S[0][0] + S[1][1] - S[2][2], S[1][2] + S[2][1]},
{S[0][1] - S[1][0], S[2][0] + S[0][2], S[1][2] + S[2][1], -S[0][0] - S[1][1] + S[2][2]},
};
double q[4];
if (!LargestEigenvector(N, q)) return false;
const Quat fitted = Normalize({q[0], q[1], q[2], q[3]});
double squares = 0.0;
for (int i = 0; i < 5; i++) {
const Vec3 d = Rotate(fitted, palm[i]) - observed[i];
squares += Dot(d, d);
}
*rotation = fitted;
if (center) *center = model;
if (rms) *rms = std::sqrt(squares / 5.0);
return true;
}
Vec3 PalmNormal(const Vec3 joints[kHandJoints]) const {
const Vec3 n = Cross(joints[hand::kIndexMcp] - joints[hand::kWrist], joints[hand::kPinkyMcp] - joints[hand::kWrist]);
const double length = Length(n);
return length > 1e-9 ? n * (normalSign_ / length) : Vec3{};
}
FingerCurls Curls(const Vec3 joints[kHandJoints]) const {
FingerCurls out;
for (int f = 0; f < 5; f++) {
const double bend = hand::Bend(joints, f);
const double *table = bends_[f];
double curl = bend <= table[0] ? 0.0 : 1.0;
for (int s = 0; s < hand::kCurlSteps; s++) {
if (bend < table[s] || bend > table[s + 1]) continue;
const double span = table[s + 1] - table[s];
curl = (s + (span > 1e-9 ? (bend - table[s]) / span : 0.0)) / hand::kCurlSteps;
break;
}
hand::CurlOf(out, f) = curl;
}
return out;
}
private:
static bool LargestEigenvector(double A[4][4], double out[4]) {
double V[4][4] = {{1, 0, 0, 0}, {0, 1, 0, 0}, {0, 0, 1, 0}, {0, 0, 0, 1}};
for (int sweep = 0; sweep < 50; sweep++) {
double off = 0.0;
for (int p = 0; p < 4; p++)
for (int r = p + 1; r < 4; r++) off += A[p][r] * A[p][r];
if (off < 1e-24) break;
for (int p = 0; p < 4; p++)
for (int r = p + 1; r < 4; r++) {
if (std::fabs(A[p][r]) < 1e-30) continue;
const double theta = (A[r][r] - A[p][p]) / (2.0 * A[p][r]);
const double t = (theta >= 0.0 ? 1.0 : -1.0) / (std::fabs(theta) + std::sqrt(theta * theta + 1.0));
const double c = 1.0 / std::sqrt(t * t + 1.0), s = t * c;
for (int k = 0; k < 4; k++) {
const double akp = A[k][p], akr = A[k][r];
A[k][p] = c * akp - s * akr;
A[k][r] = s * akp + c * akr;
}
for (int k = 0; k < 4; k++) {
const double apk = A[p][k], ark = A[r][k];
A[p][k] = c * apk - s * ark;
A[r][k] = s * apk + c * ark;
}
for (int k = 0; k < 4; k++) {
const double vkp = V[k][p], vkr = V[k][r];
V[k][p] = c * vkp - s * vkr;
V[k][r] = s * vkp + c * vkr;
}
}
}
int best = 0;
for (int i = 1; i < 4; i++)
if (A[i][i] > A[best][best]) best = i;
double length = 0.0;
for (int k = 0; k < 4; k++) length += V[k][best] * V[k][best];
if (!(length > 1e-12)) return false;
for (int k = 0; k < 4; k++) out[k] = V[k][best] / std::sqrt(length);
return true;
}
bool right_;
Vec3 center_;
double normalSign_ = 1.0;
double bends_[5][hand::kCurlSteps + 1] = {};
};
class Debounce {
public:
bool Update(bool in, int64_t nowNs, int64_t onNs, int64_t offNs) {
if (in == on_) {
pending_ = false;
return on_;
}
if (!pending_) {
pending_ = true;
sinceNs_ = nowNs;
}
if (nowNs - sinceNs_ >= (in ? onNs : offNs)) {
on_ = in;
pending_ = false;
}
return on_;
}
void Reset() { on_ = pending_ = false; }
bool On() const { return on_; }
private:
bool on_ = false, pending_ = false;
int64_t sinceNs_ = 0;
};
struct HandButtons {
float trigger = 0.0f, grip = 0.0f;
bool triggerClick = false, triggerTouch = false, gripClick = false;
bool a = false, b = false, system = false, facing = false;
};
class HandGestures {
public:
explicit HandGestures(bool right) : right_(right) {}
HandButtons Update(const HandModel &model, const Vec3 joints[kHandJoints], const FingerCurls &curls, const Vec3 &palm,
const Vec3 &head, int64_t nowNs) {
using namespace hand;
const double index = Length(joints[kThumbTip] - joints[kIndexTip]);
const double middle = Length(joints[kThumbTip] - joints[kMiddleTip]);
const double ring = Length(joints[kThumbTip] - joints[kRingTip]);
const Vec3 toHead = head - palm;
const double distance = Length(toHead);
const bool towards = distance > 1e-6 && Dot(model.PalmNormal(joints), toHead * (1.0 / distance)) > kFacing;
HandButtons out;
out.facing = facing_.Update(towards, nowNs, kFacingOnNs, kOffNs);
const double fist = (curls.middle + curls.ring + curls.pinky) / 3.0;
out.grip = (float)std::clamp((fist - kGripStart) / (kGripFull - kGripStart), 0.0, 1.0);
out.gripClick = grip_.Update(gripClick_ ? out.grip > kGripRelease : out.grip >= 1.0f, nowNs, kOnNs, kOffNs);
gripClick_ = out.gripClick;
const bool pinch = pinch_.Update(index < (pinchHeld_ ? kPinchOff : kPinchOn), nowNs, kOnNs, kOffNs);
pinchHeld_ = pinch;
const bool loose = out.grip < 0.5f;
const bool aPinch = a_.Update(loose && !pinch && Touched(ring, a_.On()), nowNs, kOnNs, kOffNs);
const bool bPinch = b_.Update(loose && !pinch && Touched(middle, b_.On()), nowNs, kOnNs, kOffNs);
if (out.facing) {
if (right_) out.system = pinch;
else out.b = pinch;
return out;
}
out.trigger = (float)std::clamp((kPinchFar - index) / (kPinchFar - kPinchNear), 0.0, 1.0);
out.triggerClick = pinch;
out.triggerTouch = out.trigger > 0.0f;
out.a = aPinch;
out.b = bPinch;
return out;
}
void Reset() {
pinch_.Reset();
a_.Reset();
b_.Reset();
grip_.Reset();
facing_.Reset();
pinchHeld_ = gripClick_ = false;
}
static constexpr double kPinchOn = 0.035, kPinchOff = 0.045, kPinchNear = 0.03, kPinchFar = 0.06;
static constexpr double kTouchOn = 0.025, kTouchOff = 0.035;
static constexpr double kGripStart = 0.35, kGripFull = 0.75, kGripRelease = 0.7;
static constexpr double kFacing = 0.6;
static constexpr int64_t kOnNs = 50000000, kOffNs = 100000000, kFacingOnNs = 300000000;
private:
static bool Touched(double distance, bool held) { return distance < (held ? kTouchOff : kTouchOn); }
bool right_;
Debounce pinch_, a_, b_, grip_, facing_;
bool pinchHeld_ = false, gripClick_ = false;
};
class SmoothTurn {
public:
void Reset() { ready_ = false; }
Quat Update(const Quat &target, double dt) {
if (!ready_) {
at_ = last_ = target;
speed_ = 0.0;
ready_ = true;
return at_;
}
if (!(dt > 0.0)) return at_;
const double moved = AngleDegrees(Conjugate(last_) * target) * M_PI / 180.0;
last_ = target;
speed_ += (moved / dt - speed_) * Alpha(dt, kSpeedCutoff);
at_ = Nlerp(at_, target, Alpha(dt, kMinCutoff + kBeta * speed_));
return at_;
}
static constexpr double kMinCutoff = 1.0, kBeta = 0.2, kSpeedCutoff = 1.0;
private:
static double Alpha(double dt, double hz) { return 1.0 / (1.0 + 1.0 / (2.0 * M_PI * hz * dt)); }
bool ready_ = false;
Quat at_, last_;
double speed_ = 0.0;
};
class SmoothPoint {
public:
void Reset() { ready_ = false; }
Vec3 Update(const Vec3 &target, double dt) {
if (!ready_) {
at_ = target;
speed_ = 0.0;
velocity_ = Vec3{};
ready_ = true;
return at_;
}
if (!(dt > 0.0)) return at_;
const Vec3 before = at_;
speed_ += (Length(target - at_) / dt - speed_) * Alpha(dt, kSpeedCutoff);
at_ = at_ + (target - at_) * Alpha(dt, kMinCutoff + kBeta * speed_);
velocity_ = velocity_ + ((at_ - before) * (1.0 / dt) - velocity_) * Alpha(dt, kSpeedCutoff);
return at_;
}
Vec3 Velocity() const { return velocity_; }
static constexpr double kMinCutoff = 1.0, kBeta = 0.5, kSpeedCutoff = 1.0;
private:
static double Alpha(double dt, double hz) { return 1.0 / (1.0 + 1.0 / (2.0 * M_PI * hz * dt)); }
bool ready_ = false;
Vec3 at_, velocity_;
double speed_ = 0.0;
};
struct CameraHandPose {
bool visible = false;
Quat rotation;
Vec3 position, velocity;
FingerCurls curls;
HandButtons buttons;
};
class CameraHand {
public:
explicit CameraHand(bool right) : model_(right), gestures_(right) {}
void Reset() {
Hide();
shown_ = false;
}
void See(int64_t timeNs, const Vec3 &palm, const Vec3 *joints, const Vec3 &head, int64_t nowNs) {
if (!joints) return;
const FingerCurls curls = model_.Curls(joints);
Quat rotation;
Vec3 center;
double rms = 1.0;
if (!model_.Fit(joints, curls, &rotation, &center, &rms) || rms > kMaxRms) return;
palm_ = palm;
seenNs_ = timeNs;
rms_ = rms;
if (seen_ && AngleDegrees(Conjugate(target_) * rotation) > kMaxTurnDegrees) {
const bool agrees = pending_ && AngleDegrees(Conjugate(pendingTurn_) * rotation) < kAgreeDegrees;
pending_ = !agrees;
pendingTurn_ = rotation;
if (!agrees) return;
}
pending_ = false;
seen_ = true;
target_ = rotation;
center_ = center;
targetCurls_ = curls;
buttons_ = gestures_.Update(model_, joints, curls, palm, head, nowNs);
}
CameraHandPose Update(int64_t nowNs, double dt) {
CameraHandPose out;
if (!seen_ || nowNs - seenNs_ > kHoldNs) {
if (shown_) Hide();
shown_ = false;
return out;
}
if (!shown_) curls_ = targetCurls_;
shown_ = true;
const double follow = 1.0 - std::exp(-std::max(dt, 0.0) / kCurlSeconds);
for (int f = 0; f < 5; f++) hand::CurlOf(curls_, f) += (hand::CurlOf(targetCurls_, f) - hand::CurlOf(curls_, f)) * follow;
out.visible = true;
out.rotation = turn_.Update(target_, dt);
out.position = place_.Update(palm_, dt) - Rotate(out.rotation, center_);
out.velocity = place_.Velocity();
out.curls = curls_;
out.buttons = buttons_;
return out;
}
double Rms() const { return rms_; }
static constexpr double kMaxRms = 0.02, kCurlSeconds = 0.08, kMaxTurnDegrees = 45.0, kAgreeDegrees = 25.0;
static constexpr int64_t kHoldNs = 500000000;
private:
void Hide() {
place_.Reset();
turn_.Reset();
gestures_.Reset();
buttons_ = HandButtons();
seen_ = pending_ = false;
}
HandModel model_;
HandGestures gestures_;
SmoothPoint place_;
SmoothTurn turn_;
Quat target_, pendingTurn_;
Vec3 center_, palm_;
FingerCurls targetCurls_, curls_;
HandButtons buttons_;
int64_t seenNs_ = 0;
double rms_ = 0.0;
bool seen_ = false, pending_ = false, shown_ = false;
};
}
+128
View File
@@ -0,0 +1,128 @@
// Adapted from Valve's OpenVR handskeletonsimulation driver sample (BSD-3-Clause, see OPENVR_LICENSE),
// which follows Moshi Turner's hand simulation in Monado.
#pragma once
#include "arm_model.h"
#include "openvr_driver.h"
namespace droiddeck {
constexpr uint32_t kHandBones = 31;
struct FingerCurls {
double thumb = 0.0, index = 0.0, middle = 0.0, ring = 0.0, pinky = 0.0;
};
namespace hand {
constexpr double kDegree = M_PI / 180.0;
constexpr double kLengths[5][5] = {
{0.05, 0.05, 0.035, 0.025, 0.0},
{0.03, 0.073, 0.045, 0.025, 0.02},
{0.01, 0.091, 0.049, 0.03, 0.02},
{0.02, 0.073, 0.045, 0.03, 0.03},
{0.03, 0.067, 0.03, 0.025, 0.02},
};
struct Joint {
double swing[2] = {0.0, 0.0};
double twist = 0.0;
};
struct Finger {
Joint metacarpal, proximal;
double intermediate = 0.0, distal = 0.0;
};
inline Quat SwingTwist(const Joint &j) {
const double squared = j.swing[0] * j.swing[0] + j.swing[1] * j.swing[1];
const double cosTwist = std::cos(j.twist * 0.5), sinTwist = std::sin(j.twist * 0.5);
double cosSwing = 1.0, sinOver = 0.5;
if (squared > 0.0) {
const double theta = std::sqrt(squared);
cosSwing = std::cos(theta * 0.5);
sinOver = std::sin(theta * 0.5) / theta;
}
return {cosSwing * cosTwist, cosSwing * sinTwist,
j.swing[1] * cosTwist * sinOver - j.swing[0] * sinTwist * sinOver,
j.swing[0] * cosTwist * sinOver + j.swing[1] * sinTwist * sinOver};
}
inline Quat AboutZ(double radians) { return {std::cos(radians * 0.5), 0.0, 0.0, std::sin(radians * 0.5)}; }
inline vr::VRBoneTransform_t Bone(bool right, const Quat &q, Vec3 position) {
if (right) position.x = -position.x;
vr::VRBoneTransform_t t;
t.position = {{(float)position.x, (float)position.y, (float)position.z, 1.0f}};
t.orientation = {(float)q.w, (float)q.x, (float)q.y, (float)q.z};
return t;
}
inline vr::VRBoneTransform_t Metacarpal(bool right, const Quat &orientation, double length) {
Quat q = Quat{0.5, 0.5, -0.5, 0.5} * orientation;
const Vec3 position = Rotate(q, {length, 0.0, 0.0});
if (right) q = {q.x, -q.w, q.z, -q.y};
return Bone(right, q, position);
}
inline Quat ToQuat(const vr::HmdQuaternionf_t &q) { return {q.w, q.x, q.y, q.z}; }
inline Vec3 ToVec(const vr::HmdVector4_t &v) { return {v.v[0], v.v[1], v.v[2]}; }
}
inline void HandSkeleton(bool right, const FingerCurls &curls, vr::VRBoneTransform_t out[kHandBones]) {
using namespace hand;
out[0] = {{{0.0f, 0.0f, 0.0f, 1.0f}}, {1.0f, 0.0f, 0.0f, 0.0f}};
out[1] = {{{-0.034038f, 0.036503f, 0.164722f, 1.0f}}, {-0.055147f, -0.078608f, -0.920279f, 0.379296f}};
if (right) {
out[1].position.v[0] = -out[1].position.v[0];
out[1].orientation.y = -out[1].orientation.y;
out[1].orientation.z = -out[1].orientation.z;
}
Joint thumbMetacarpal, thumbProximal;
thumbMetacarpal.swing[0] = 10 * kDegree + curls.thumb * 5 * kDegree;
thumbMetacarpal.swing[1] = 40 * kDegree;
thumbProximal.swing[0] = curls.thumb * 90 * kDegree;
const double thumbDistal = curls.thumb * 90 * kDegree;
out[2] = Metacarpal(right, SwingTwist(thumbMetacarpal), kLengths[0][0]);
out[3] = Bone(right, SwingTwist(thumbProximal), {kLengths[0][1], 0.0, 0.0});
out[4] = Bone(right, AboutZ(thumbDistal), {kLengths[0][2], 0.0, 0.0});
out[5] = Bone(right, Quat{}, {kLengths[0][3], 0.0, 0.0});
const double metacarpalSplay[4] = {13.0, 0.0, -15.0, -27.0};
const double proximalSplay[4] = {3.0, 0.0, -1.0, -2.0};
const double fingerCurls[4] = {curls.index, curls.middle, curls.ring, curls.pinky};
for (int f = 0; f < 4; f++) {
Finger finger;
finger.metacarpal.swing[0] = fingerCurls[f] * 5 * kDegree;
finger.metacarpal.swing[1] = metacarpalSplay[f] * kDegree;
finger.proximal.swing[0] = fingerCurls[f] * 90 * kDegree;
finger.proximal.swing[1] = proximalSplay[f] * kDegree;
finger.intermediate = 5 * kDegree + fingerCurls[f] * 80 * kDegree;
finger.distal = 5 * kDegree + fingerCurls[f] * 80 * kDegree;
const int base = 6 + f * 5;
const double *length = kLengths[f + 1];
out[base] = Metacarpal(right, SwingTwist(finger.metacarpal), length[0]);
out[base + 1] = Bone(right, SwingTwist(finger.proximal), {length[1], 0.0, 0.0});
out[base + 2] = Bone(right, AboutZ(finger.intermediate), {length[2], 0.0, 0.0});
out[base + 3] = Bone(right, AboutZ(finger.distal), {length[3], 0.0, 0.0});
out[base + 4] = Bone(right, Quat{}, {length[4], 0.0, 0.0});
}
const int distal[5] = {4, 9, 14, 19, 24};
for (int f = 0; f < 5; f++) {
const int first = f == 0 ? 2 : 6 + (f - 1) * 5;
Quat rotation = ToQuat(out[1].orientation);
Vec3 position = ToVec(out[1].position);
for (int b = first; b <= distal[f]; b++) {
position = position + Rotate(rotation, ToVec(out[b].position));
rotation = rotation * ToQuat(out[b].orientation);
}
out[26 + f] = {{{(float)position.x, (float)position.y, (float)position.z, 1.0f}},
{(float)rotation.w, (float)rotation.x, (float)rotation.y, (float)rotation.z}};
}
}
}
+51
View File
@@ -0,0 +1,51 @@
#ifndef HANDCAM_SHM_H
#define HANDCAM_SHM_H
#include <stddef.h>
#include <stdint.h>
#define DROIDDECK_HANDCAM_MAGIC 0x4D434844u
#define DROIDDECK_HANDCAM_VERSION 2u
#define DROIDDECK_HANDCAM_JOINT_COUNT 21
#define DROIDDECK_HANDCAM_FILE "droiddeck-handcam"
enum {
DROIDDECK_HANDCAM_RUNNING = 1u << 0,
DROIDDECK_HANDCAM_FACE = 1u << 1,
DROIDDECK_HANDCAM_BARE_HANDS = 1u << 2,
};
enum {
DROIDDECK_HANDCAM_SEEN = 1u << 0,
DROIDDECK_HANDCAM_ESTIMATED = 1u << 1,
DROIDDECK_HANDCAM_JOINTS = 1u << 2,
};
struct droiddeck_handcam_hand {
uint32_t flags;
float position[3];
float joints[DROIDDECK_HANDCAM_JOINT_COUNT][3];
};
struct droiddeck_handcam {
uint32_t magic;
uint32_t version;
uint32_t seq;
uint32_t flags;
int64_t time_ns;
uint32_t frame;
float face_yaw;
struct droiddeck_handcam_hand hands[2];
};
#ifdef __cplusplus
#define DROIDDECK_HANDCAM_ASSERT static_assert
#else
#define DROIDDECK_HANDCAM_ASSERT _Static_assert
#endif
DROIDDECK_HANDCAM_ASSERT(offsetof(struct droiddeck_handcam_hand, joints) == 16, "handcam hand layout");
DROIDDECK_HANDCAM_ASSERT(sizeof(struct droiddeck_handcam_hand) == 268, "handcam hand layout");
DROIDDECK_HANDCAM_ASSERT(offsetof(struct droiddeck_handcam, time_ns) == 16, "handcam layout");
DROIDDECK_HANDCAM_ASSERT(offsetof(struct droiddeck_handcam, hands) == 32, "handcam layout");
DROIDDECK_HANDCAM_ASSERT(sizeof(struct droiddeck_handcam) == 568, "handcam layout");
#endif
+17
View File
@@ -0,0 +1,17 @@
// Index controller tip and grip offsets in its raw frame, as Monado's vive_poses.c (BSL-1.0) lists them; see docs/third-party/joycon.
#pragma once
#include "arm_model.h"
namespace droiddeck {
inline Quat IndexTipRotation(bool right) {
return AboutY((right ? 5.0 : -5.0) * M_PI / 180.0) * AboutX(-40.0 * M_PI / 180.0);
}
inline void IndexRawPose(bool right, const Quat &aim, const Vec3 &grip, Quat *raw, Vec3 *position) {
*raw = Normalize(aim * Conjugate(IndexTipRotation(right)));
*position = grip - Rotate(*raw, {0.0, -0.015, 0.13});
}
}
+61
View File
@@ -0,0 +1,61 @@
#ifndef JOYCON_SHM_H
#define JOYCON_SHM_H
#include <stddef.h>
#include <stdint.h>
#define DROIDDECK_JOYCON_MAGIC 0x4E434A44u
#define DROIDDECK_JOYCON_VERSION 3u
#define DROIDDECK_JOYCON_FILE "droiddeck-joycon"
#define DROIDDECK_JOYCON_RUMBLE_SLOT 0x100u
enum {
DROIDDECK_JOYCON_CONNECTED = 1u << 0,
DROIDDECK_JOYCON_CHARGING = 1u << 1,
};
enum {
DROIDDECK_JOYCON_TRIGGER = 1u << 0,
DROIDDECK_JOYCON_GRIP = 1u << 1,
DROIDDECK_JOYCON_A = 1u << 2,
DROIDDECK_JOYCON_B = 1u << 3,
DROIDDECK_JOYCON_STICK = 1u << 4,
DROIDDECK_JOYCON_SYSTEM = 1u << 5,
};
struct droiddeck_joycon_hand {
uint32_t flags;
uint32_t buttons;
float rotation[4];
float angular_velocity[3];
float stick[2];
float battery;
uint32_t recenter;
uint32_t pad;
int64_t time_ns;
float displacement[3];
float velocity[3];
float inertial_velocity[3];
uint32_t reserved;
};
struct droiddeck_joycon {
uint32_t magic;
uint32_t version;
uint32_t seq;
uint32_t reserved;
struct droiddeck_joycon_hand hands[2];
};
#ifdef __cplusplus
#define DROIDDECK_JOYCON_ASSERT static_assert
#else
#define DROIDDECK_JOYCON_ASSERT _Static_assert
#endif
DROIDDECK_JOYCON_ASSERT(sizeof(struct droiddeck_joycon_hand) == 104, "joycon hand layout");
DROIDDECK_JOYCON_ASSERT(offsetof(struct droiddeck_joycon_hand, time_ns) == 56, "joycon hand layout");
DROIDDECK_JOYCON_ASSERT(offsetof(struct droiddeck_joycon_hand, displacement) == 64, "joycon hand layout");
DROIDDECK_JOYCON_ASSERT(offsetof(struct droiddeck_joycon_hand, inertial_velocity) == 88, "joycon hand layout");
DROIDDECK_JOYCON_ASSERT(sizeof(struct droiddeck_joycon) == 224, "joycon layout");
DROIDDECK_JOYCON_ASSERT(offsetof(struct droiddeck_joycon, hands) == 16, "joycon layout");
#endif
+10
View File
@@ -0,0 +1,10 @@
#pragma once
#include <stdint.h>
#define DROIDDECK_PRESENT_SYNC_NAME "present-sync"
#define DROIDDECK_PRESENT_SYNC_SIZE 4096
struct droiddeck_present_sync {
uint32_t issued;
uint32_t done;
};
+134 -18
View File
@@ -8,11 +8,14 @@ typedef unsigned long VisualID;
#include <vulkan/vulkan_wayland.h>
#include "xdg-shell-client-protocol.h"
#include "../../../app/src/main/cpp/waylandcomp/src/headpose_shm.h"
#include "presenter.h"
#include "present_sync.h"
#include "xr_tracking_file.h"
#include "xr_frame_channel.h"
#include "producer_lease.h"
#include <algorithm>
#include <atomic>
#include <chrono>
#include <condition_variable>
@@ -22,7 +25,11 @@ typedef unsigned long VisualID;
#include <cstring>
#include <dlfcn.h>
#include <fcntl.h>
#include <linux/futex.h>
#include <mutex>
#include <string>
#include <sys/mman.h>
#include <sys/syscall.h>
#include <thread>
#include <unistd.h>
#include <vector>
@@ -137,10 +144,12 @@ VkFormat UnormOf(uint32_t f) {
class Impl {
public:
static Impl &Get() {
static Impl impl;
return impl;
static Impl *impl = made_ = new Impl;
return *impl;
}
static Impl *Made() { return made_; }
void Add(const uint64_t handles[3], const int fds[3], uint32_t format, uint32_t w, uint32_t h, uint32_t producerPid) {
std::lock_guard<std::mutex> lock(mutex_);
if (failed_ || exit_) return;
@@ -168,6 +177,11 @@ public:
EnsureThread();
frame_ = f;
frameAt_ = std::chrono::steady_clock::now();
const droiddeck_present_sync *sync = MapSync(frameAt_);
const uint32_t issued = sync ? __atomic_load_n(&sync->issued, __ATOMIC_ACQUIRE) : 0;
frameTracked_ = sync && issued != lastIssued_;
frameMark_ = issued;
lastIssued_ = issued;
haveFrame_ = true;
submitted_++;
uint64_t ticket = ++ticket_;
@@ -220,12 +234,16 @@ private:
Log("presenter: unavailable, frames are not shown");
Fail(); return;
}
uint64_t shown = 0;
if (const char *g = getenv("DROIDDECK_VR_PRESENT_GUARD_MS")) guard_ = std::chrono::microseconds((long)(atof(g) * 1000));
uint64_t shown = 0, waited = 0, late = 0;
std::chrono::steady_clock::duration longest{};
auto lastLog = std::chrono::steady_clock::now();
while (true) {
EyeFrame f;
uint64_t ticket = 0;
std::chrono::steady_clock::time_point at;
uint32_t mark = 0;
bool tracked = false;
std::vector<PendingImport> imports;
std::vector<uint64_t> removals;
{
@@ -236,6 +254,8 @@ private:
removals.swap(removals_);
f = frame_;
at = frameAt_;
mark = frameMark_;
tracked = frameTracked_;
bool draw = haveFrame_;
haveFrame_ = false;
inFlight_ = draw; ticket = ticket_;
@@ -244,7 +264,19 @@ private:
if (wl_) { wl_display_dispatch_pending(wl_); wl_display_flush(wl_); }
for (auto &p : imports) Import(p);
for (auto h : removals) Forget(h);
if (f.texture[0] && !frameChannel_) std::this_thread::sleep_until(at + guard_);
if (f.texture[0] && !frameChannel_ && tracked) {
const auto start = std::chrono::steady_clock::now();
const int r = WaitDrawn(mark, start + kDrawnTimeout);
if (r > 0) {
waited++;
longest = std::max(longest, std::chrono::steady_clock::now() - start);
} else if (r < 0) {
late++;
}
} else if (f.texture[0] && !frameChannel_) {
std::this_thread::sleep_until(at + guard_);
}
if (f.texture[0] && wayland_) FollowPanel();
if (f.texture[0] && (frameChannel_ ? Capture(f) : Draw(f))) shown++;
{
std::lock_guard<std::mutex> lock(mutex_);
@@ -253,9 +285,11 @@ private:
}
auto now = std::chrono::steady_clock::now();
if (now - lastLog > std::chrono::seconds(10)) {
Log("presenter: %llu frames shown, %llu submitted (last 10 s)", (unsigned long long)shown,
(unsigned long long)submitted_.exchange(0));
shown = 0;
Log("presenter: %llu frames shown, %llu submitted, %llu waited for the GPU (longest %.1f ms), %llu late (last 10 s)",
(unsigned long long)shown, (unsigned long long)submitted_.exchange(0), (unsigned long long)waited,
std::chrono::duration<double, std::milli>(longest).count(), (unsigned long long)late);
shown = waited = late = 0;
longest = {};
lastLog = now;
}
}
@@ -278,7 +312,79 @@ private:
surface_ = VK_NULL_HANDLE; inst_ = VK_NULL_HANDLE;
}
bool PanelSize(uint32_t *w, uint32_t *h) {
if (!pose_) {
int fd = open("/run/droiddeck/" DROIDDECK_HEADPOSE_FILE, O_RDONLY | O_CLOEXEC);
if (fd < 0) return false;
void *m = mmap(nullptr, sizeof(droiddeck_headpose), PROT_READ, MAP_SHARED, fd, 0);
close(fd);
if (m == MAP_FAILED) return false;
pose_ = (const volatile droiddeck_headpose *)m;
}
for (int tries = 0; tries < 8; tries++) {
const uint32_t a = __atomic_load_n(&pose_->seq, __ATOMIC_ACQUIRE);
if (a & 1u) continue;
const uint32_t ew = pose_->eye_w, eh = pose_->eye_h;
__atomic_thread_fence(__ATOMIC_ACQUIRE);
if (__atomic_load_n(&pose_->seq, __ATOMIC_RELAXED) != a) continue;
if (!ew || !eh || pose_->magic != DROIDDECK_HEADPOSE_MAGIC) return false;
*w = ew * 2;
*h = eh;
return true;
}
return false;
}
void FollowPanel() {
uint32_t w = 0, h = 0;
if (!PanelSize(&w, &h)) return;
const bool resized = w != width_ || h != height_;
const auto now = std::chrono::steady_clock::now();
if (!resized && (swapchain_ || now < retryAt_)) return;
vk_.DeviceWaitIdle(dev_);
if (swapchain_) vk_.DestroySwapchainKHR(dev_, swapchain_, nullptr);
swapchain_ = VK_NULL_HANDLE;
images_.clear();
if (resized) Log("presenter: panel now %ux%u (was %ux%u)", w, h, width_, height_);
width_ = w;
height_ = h;
if (!CreateSwapchain()) retryAt_ = now + std::chrono::seconds(1);
}
std::chrono::microseconds guard_{4000};
std::chrono::steady_clock::time_point retryAt_{};
static constexpr std::chrono::milliseconds kDrawnTimeout{100};
const droiddeck_present_sync *MapSync(std::chrono::steady_clock::time_point now) {
if (sync_) return sync_;
if (syncTried_.time_since_epoch().count() && now - syncTried_ < std::chrono::seconds(1)) return nullptr;
syncTried_ = now;
const char *rt = getenv("XDG_RUNTIME_DIR");
std::string path = std::string(rt && *rt ? rt : "/run/droiddeck") + "/droiddeck-steamvr/" DROIDDECK_PRESENT_SYNC_NAME;
int fd = open(path.c_str(), O_RDONLY | O_CLOEXEC);
if (fd < 0) return nullptr;
void *m = mmap(nullptr, DROIDDECK_PRESENT_SYNC_SIZE, PROT_READ, MAP_SHARED, fd, 0);
close(fd);
if (m == MAP_FAILED) return nullptr;
sync_ = (const droiddeck_present_sync *)m;
Log("presenter: frames wait for the compositor's GPU work (%s)", path.c_str());
return sync_;
}
int WaitDrawn(uint32_t mark, std::chrono::steady_clock::time_point until) {
const droiddeck_present_sync *s = sync_;
int slept = 0;
while (true) {
const uint32_t done = __atomic_load_n(&s->done, __ATOMIC_ACQUIRE);
if ((int32_t)(done - mark) >= 0) return slept;
const auto left = until - std::chrono::steady_clock::now();
if (left <= std::chrono::steady_clock::duration::zero()) return -1;
const auto ns = std::chrono::duration_cast<std::chrono::nanoseconds>(left).count();
struct timespec ts = {(time_t)(ns / 1000000000), (long)(ns % 1000000000)};
syscall(SYS_futex, const_cast<uint32_t *>(&s->done), FUTEX_WAIT, done, &ts, nullptr, 0);
slept = 1;
}
}
struct wl_display *wl_ = nullptr;
struct wl_compositor *wlComp_ = nullptr;
@@ -333,12 +439,7 @@ private:
wl_surface_commit(wlSurf_);
for (int i = 0; i < 20 && !configured_; i++) wl_display_roundtrip(wl_);
width_ = height_ = 0;
int fd = open("/run/droiddeck/droiddeck-headpose", O_RDONLY | O_CLOEXEC);
if (fd >= 0) {
uint32_t eye[2] = {0, 0};
if (pread(fd, eye, sizeof(eye), 72) == (ssize_t)sizeof(eye) && eye[0] && eye[1]) { width_ = eye[0] * 2; height_ = eye[1]; }
close(fd);
}
if (!PanelSize(&width_, &height_)) width_ = height_ = 0;
XrTrackingFile xrFile;
DroidDeckXrTracking tracking;
if (xrFile.Read(&tracking) && XrTrackingFile::Fresh(tracking)) {
@@ -351,7 +452,6 @@ private:
}
bool InitWindow() {
if (const char *g = getenv("DROIDDECK_VR_PRESENT_GUARD_MS")) guard_ = std::chrono::microseconds((long)(atof(g) * 1000));
if (!x_.Load()) { Log("presenter: no libX11"); return false; }
const char *name = getenv("DROIDDECK_VR_DISPLAY");
if (!name || !*name) name = ":0";
@@ -439,7 +539,7 @@ private:
VkSemaphoreCreateInfo semci = {VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO, nullptr, 0};
vk_.CreateSemaphore(dev_, &semci, nullptr, &acquired_);
vk_.CreateSemaphore(dev_, &semci, nullptr, &rendered_);
if (!frameChannel_ && !CreateSwapchain()) return false;
if (!frameChannel_ && !CreateSwapchain() && !wayland_) return false;
Log("presenter: Vulkan on %s, queue family %u", props.deviceName, qfam_);
return true;
}
@@ -480,7 +580,11 @@ private:
sci.presentMode = pm;
sci.clipped = VK_TRUE;
VkResult r = vk_.CreateSwapchainKHR(dev_, &sci, nullptr, &swapchain_);
if (r != VK_SUCCESS) { Log("presenter: vkCreateSwapchainKHR %d", r); return false; }
if (r != VK_SUCCESS) {
swapchain_ = VK_NULL_HANDLE;
Log("presenter: vkCreateSwapchainKHR %d", r);
return false;
}
uint32_t ni = 0;
vk_.GetSwapchainImagesKHR(dev_, swapchain_, &ni, nullptr);
images_.resize(ni);
@@ -693,7 +797,7 @@ private:
bool Draw(const EyeFrame &f) {
const Imported *eye[2] = {Find(f.texture[0]), Find(f.texture[1])};
if (!eye[0] || !eye[1]) return false;
if (!eye[0] || !eye[1] || !swapchain_) return false;
vk_.WaitForFences(dev_, 1, &fence_, VK_TRUE, UINT64_MAX);
uint32_t idx = 0;
VkResult r = vk_.AcquireNextImageKHR(dev_, swapchain_, 100000000ull, acquired_, VK_NULL_HANDLE, &idx);
@@ -745,6 +849,10 @@ private:
VkPipelineStageFlags wait = VK_PIPELINE_STAGE_TRANSFER_BIT;
VkSubmitInfo si = {VK_STRUCTURE_TYPE_SUBMIT_INFO, nullptr, 1, &acquired_, &wait, 1, &cmd_, 1, &rendered_};
if ((r = vk_.QueueSubmit(queue_, 1, &si, fence_)) != VK_SUCCESS) { Log("presenter: submit %d", r); return false; }
if (vk_.WaitForFences(dev_, 1, &fence_, VK_TRUE, (uint64_t)std::chrono::nanoseconds(kDrawnTimeout).count()) == VK_TIMEOUT) {
static int said;
if (said++ < 3) Log("presenter: the copy took over %lld ms; showing it anyway", (long long)kDrawnTimeout.count());
}
VkPresentInfoKHR pi = {VK_STRUCTURE_TYPE_PRESENT_INFO_KHR, nullptr, 1, &rendered_, 1, &swapchain_, &idx, nullptr};
r = vk_.QueuePresentKHR(queue_, &pi);
if (r != VK_SUCCESS && r != VK_SUBOPTIMAL_KHR) {
@@ -760,6 +868,11 @@ private:
std::vector<uint64_t> removals_;
EyeFrame frame_{};
std::chrono::steady_clock::time_point frameAt_{};
uint32_t frameMark_ = 0, lastIssued_ = 0;
bool frameTracked_ = false;
const droiddeck_present_sync *sync_ = nullptr;
const volatile droiddeck_headpose *pose_ = nullptr;
std::chrono::steady_clock::time_point syncTried_{};
bool haveFrame_ = false, exit_ = false, started_ = false, inFlight_ = false, failed_ = false;
uint64_t ticket_ = 0, completed_ = 0;
bool frameChannel_ = false;
@@ -770,6 +883,7 @@ private:
uint32_t captureWidth_ = 0, captureHeight_ = 0;
std::atomic<uint64_t> submitted_{0};
std::thread thread_;
static inline std::atomic<Impl *> made_{nullptr};
X11 x_;
Display *dpy_ = nullptr;
@@ -798,6 +912,8 @@ private:
void Presenter::AddTextures(const uint64_t h[3], const int fds[3], uint32_t f, uint32_t w, uint32_t hh, uint32_t pid) { Impl::Get().Add(h, fds, f, w, hh, pid); }
void Presenter::RemoveTexture(uint64_t h) { Impl::Get().Remove(h); }
void Presenter::Submit(const EyeFrame &f) { Impl::Get().Submit(f); }
void Presenter::Shutdown() { Impl::Get().Shutdown(); }
void Presenter::Shutdown() {
if (Impl *impl = Impl::Made()) impl->Shutdown();
}
}
+21
View File
@@ -0,0 +1,21 @@
#ifndef DROIDDECK_RENDER_TARGET_H
#define DROIDDECK_RENDER_TARGET_H
#include <algorithm>
#include <cmath>
#include <cstdint>
namespace droiddeck {
struct RenderTargetSize { uint32_t width, height; };
inline RenderTargetSize RenderTarget(float aspect, uint32_t selectedHeight, uint32_t panelHeight) {
uint32_t height = selectedHeight > 0 ? std::min(selectedHeight, 8192u) : 720u;
if (panelHeight > 0) height = std::min(height, panelHeight);
if (!std::isfinite(aspect) || aspect <= 0.0f) aspect = 1.0f;
const float width = std::clamp(height * aspect, 1.0f, 8192.0f);
return {static_cast<uint32_t>(std::lround(width)), height};
}
}
#endif
+353 -10
View File
@@ -291,6 +291,7 @@ static VkResult VKAPI_CALL CreateDevice(VkPhysicalDevice pd, const VkDeviceCreat
}
static void forget_device_semaphores(VkDevice dev);
static void track_forget_device(VkDevice dev);
static void producer_destroy_device(struct dev *device, const VkAllocationCallbacks *allocator);
static void VKAPI_CALL DestroyDevice(VkDevice device, const VkAllocationCallbacks *alloc) {
@@ -301,6 +302,7 @@ static void VKAPI_CALL DestroyDevice(VkDevice device, const VkAllocationCallback
#endif
if (d->producer) producer_destroy_device(d, alloc);
forget_device_semaphores(device);
track_forget_device(device);
PFN_vkDestroyDevice f = d->destroy;
pthread_mutex_lock(&g_lock);
memset(d, 0, sizeof(*d));
@@ -309,19 +311,23 @@ static void VKAPI_CALL DestroyDevice(VkDevice device, const VkAllocationCallback
}
#include <fcntl.h>
#include <stdatomic.h>
#include <time.h>
#include "../../../app/src/main/cpp/waylandcomp/src/headpose_shm.h"
#define FAKE_DISPLAY ((VkDisplayKHR)(uintptr_t)0xdd0001)
#define FAKE_MODE ((VkDisplayModeKHR)(uintptr_t)0xdd0002)
static VkSurfaceKHR g_hmd_surface;
static uint32_t g_mode_w = 1920, g_mode_h = 1080, g_mode_mhz = 60000;
static int g_mode_fixed;
static void mode_init(void) {
static int done;
if (done++) return;
const char *m = getenv("DROIDDECK_VR_DISPLAY_MODE");
unsigned w, h, hz;
if (m && sscanf(m, "%ux%u@%u", &w, &h, &hz) == 3 && w && h && hz) { g_mode_w = w; g_mode_h = h; g_mode_mhz = hz * 1000; }
if (m && sscanf(m, "%ux%u@%u", &w, &h, &hz) == 3 && w && h && hz) { g_mode_w = w; g_mode_h = h; g_mode_mhz = hz * 1000; g_mode_fixed = 1; }
}
static uint64_t now_ns(void) {
@@ -329,7 +335,57 @@ static uint64_t now_ns(void) {
clock_gettime(CLOCK_MONOTONIC, &ts);
return (uint64_t)ts.tv_sec * 1000000000ull + (uint64_t)ts.tv_nsec;
}
static uint64_t frame_ns(void) { return 1000000000000ull / g_mode_mhz; }
static _Atomic(const struct droiddeck_headpose *) g_pose;
static _Atomic uint64_t g_pose_tried;
static const struct droiddeck_headpose *pose_map(void) {
const struct droiddeck_headpose *p = atomic_load(&g_pose);
if (p) return p;
uint64_t t = now_ns(), last = atomic_load(&g_pose_tried);
if (last && t - last < 1000000000ull) return NULL;
atomic_store(&g_pose_tried, t);
int fd = open("/run/droiddeck/" DROIDDECK_HEADPOSE_FILE, O_RDONLY | O_CLOEXEC);
if (fd < 0) return NULL;
void *m = mmap(NULL, sizeof(struct droiddeck_headpose), PROT_READ, MAP_SHARED, fd, 0);
close(fd);
if (m == MAP_FAILED) return NULL;
const struct droiddeck_headpose *expected = NULL;
if (!atomic_compare_exchange_strong(&g_pose, &expected, (const struct droiddeck_headpose *)m)) {
munmap(m, sizeof(struct droiddeck_headpose));
return expected;
}
return m;
}
static uint32_t mode_mhz(void) {
mode_init();
if (g_mode_fixed) return g_mode_mhz;
const struct droiddeck_headpose *p = pose_map();
if (!p || __atomic_load_n(&p->magic, __ATOMIC_RELAXED) != DROIDDECK_HEADPOSE_MAGIC) return g_mode_mhz;
uint32_t bits = __atomic_load_n((const uint32_t *)&p->display_hz, __ATOMIC_RELAXED);
float hz;
memcpy(&hz, &bits, sizeof(hz));
return hz >= 24.0f && hz <= 240.0f ? (uint32_t)(hz * 1000.0f + 0.5f) : g_mode_mhz;
}
static uint64_t frame_ns(void) { return 1000000000000ull / mode_mhz(); }
static pthread_mutex_t g_vblank_lock = PTHREAD_MUTEX_INITIALIZER;
static uint64_t g_vblank_base, g_vblank_at, g_vblank_frame;
static uint64_t vblank_count(void) {
uint64_t f = frame_ns();
pthread_mutex_lock(&g_vblank_lock);
uint64_t t = now_ns();
if (f != g_vblank_frame) {
g_vblank_base = g_vblank_frame ? g_vblank_base + (t - g_vblank_at) / g_vblank_frame : t / f;
g_vblank_at = t;
g_vblank_frame = f;
}
uint64_t n = g_vblank_base + (t - g_vblank_at) / f;
pthread_mutex_unlock(&g_vblank_lock);
return n;
}
static VkDisplayPropertiesKHR display_props(void) {
mode_init();
@@ -379,7 +435,7 @@ static VkResult VKAPI_CALL GetDisplayPlaneSupportedDisplaysKHR(VkPhysicalDevice
}
static VkDisplayModePropertiesKHR mode_props(void) {
mode_init();
VkDisplayModePropertiesKHR m = {FAKE_MODE, {{g_mode_w, g_mode_h}, g_mode_mhz}};
VkDisplayModePropertiesKHR m = {FAKE_MODE, {{g_mode_w, g_mode_h}, mode_mhz()}};
return m;
}
static VkResult VKAPI_CALL GetDisplayModePropertiesKHR(VkPhysicalDevice pd, VkDisplayKHR d, uint32_t *count, VkDisplayModePropertiesKHR *p) {
@@ -467,7 +523,7 @@ static VkResult VKAPI_CALL RegisterDisplayEventEXT(VkDevice d, VkDisplayKHR disp
}
static VkResult VKAPI_CALL GetSwapchainCounterEXT(VkDevice d, VkSwapchainKHR sc, VkSurfaceCounterFlagBitsEXT counter, uint64_t *value) {
mode_init();
*value = now_ns() / frame_ns();
*value = vblank_count();
return VK_SUCCESS;
}
@@ -661,6 +717,9 @@ struct bridged {
int fd;
int timeline;
unsigned long ino;
uint64_t submitted;
uint64_t seen, seen_us;
int imported;
};
static pthread_mutex_t g_br_lock = PTHREAD_MUTEX_INITIALIZER;
static struct bridged g_br[MAX_SEMS];
@@ -705,6 +764,12 @@ static int sem_trace_on(void) {
return on;
}
static uint64_t now_us(void) {
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
return (uint64_t)ts.tv_sec * 1000000u + (uint64_t)ts.tv_nsec / 1000u;
}
static void *poller(void *arg) {
unsigned ticks = 0;
for (;;) {
@@ -722,10 +787,11 @@ static void *poller(void *arg) {
uint64_t local = 0;
if (!get || get(b->dev, b->sem, &local) != VK_SUCCESS) continue;
uint64_t shared = atomic_load(b->counter);
if (shared != b->seen) { b->seen = shared; b->seen_us = now_us(); }
if (trace) {
char line[192];
snprintf(line, sizeof(line), "%s sem %p %s counter %lu local %llu shared %llu", program_invocation_short_name,
(void *)b->sem, b->fd >= 0 ? "made here" : "imported", b->ino, (unsigned long long)local, (unsigned long long)shared);
(void *)b->sem, b->imported ? "imported" : "made here", b->ino, (unsigned long long)local, (unsigned long long)shared);
say("%s", line);
}
if (local > shared) {
@@ -808,7 +874,7 @@ static VkResult VKAPI_CALL CreateSemaphore(VkDevice d, const VkSemaphoreCreateIn
atomic_store(counter, timeline ? type_info->initialValue : 0);
pthread_mutex_lock(&g_br_lock);
struct bridged *slot = bridged_find(VK_NULL_HANDLE);
if (slot) *slot = (struct bridged){*out, d, counter, fd, timeline, fd_ino(fd)};
if (slot) *slot = (struct bridged){*out, d, counter, fd, timeline, fd_ino(fd), 0, atomic_load(counter), now_us(), 0};
pthread_mutex_unlock(&g_br_lock);
if (!slot) {
munmap((void *)counter, 4096);
@@ -869,7 +935,7 @@ static VkResult VKAPI_CALL ImportSemaphoreFdKHR(VkDevice d, const VkImportSemaph
struct bridged *b = bridged_find(info->semaphore);
if (b) bridged_drop(b);
b = bridged_find(VK_NULL_HANDLE);
if (b) *b = (struct bridged){info->semaphore, d, counter, info->fd, timeline, ino};
if (b) *b = (struct bridged){info->semaphore, d, counter, info->fd, timeline, ino, 0, atomic_load(counter), now_us(), 1};
pthread_mutex_unlock(&g_br_lock);
if (!b) {
munmap((void *)counter, 4096);
@@ -877,10 +943,98 @@ static VkResult VKAPI_CALL ImportSemaphoreFdKHR(VkDevice d, const VkImportSemaph
}
if (timeline) poller_start();
static int said;
if (said++ < 4) say("imported a shared timeline semaphore (%s): waits retained", program_invocation_short_name);
if (said++ < 4) say("imported a shared timeline semaphore (%s): bridged", program_invocation_short_name);
return VK_SUCCESS;
}
#define SUBMIT_GRACE_US 3000
#define SHARED_IDLE_US 100000
static int quest_frames(void) {
static _Atomic uint64_t checked_us;
static _Atomic int present;
const uint64_t t = now_us();
const uint64_t last = atomic_load(&checked_us);
if (!last || t - last > 1000000) {
atomic_store(&present, access("/run/droiddeck/droiddeck-xr-frames", F_OK) == 0);
atomic_store(&checked_us, t);
}
return atomic_load(&present);
}
static int submit_drop_wait(VkSemaphore s, uint64_t value) {
pthread_mutex_lock(&g_br_lock);
struct bridged *b = bridged_find(s);
if (!b || !b->timeline || !b->imported) { pthread_mutex_unlock(&g_br_lock); return 0; }
const uint64_t queued = b->submitted;
const uint64_t seen_us = b->seen_us;
VkDevice dev = b->dev;
_Atomic uint64_t *counter = b->counter;
pthread_mutex_unlock(&g_br_lock);
if (!value || value <= queued) return 0;
struct dev *d = dev_for(dev);
if (!d || quest_frames()) return 0;
PFN_vkGetSemaphoreCounterValue get = (PFN_vkGetSemaphoreCounterValue)d->gdpa(dev, "vkGetSemaphoreCounterValue");
if (!get) get = (PFN_vkGetSemaphoreCounterValue)d->gdpa(dev, "vkGetSemaphoreCounterValueKHR");
PFN_vkSignalSemaphore sig = (PFN_vkSignalSemaphore)d->gdpa(dev, "vkSignalSemaphore");
if (!sig) sig = (PFN_vkSignalSemaphore)d->gdpa(dev, "vkSignalSemaphoreKHR");
if (!get) return 0;
const uint64_t start = now_us();
const uint64_t until = start + (start - seen_us > SHARED_IDLE_US ? 0 : SUBMIT_GRACE_US);
for (;;) {
uint64_t local = 0;
if (get(dev, s, &local) == VK_SUCCESS && local >= value) return 0;
uint64_t shared = atomic_load(counter);
if (shared >= value && sig) {
VkSemaphoreSignalInfo si = {VK_STRUCTURE_TYPE_SEMAPHORE_SIGNAL_INFO, NULL, s, shared};
if (sig(dev, &si) == VK_SUCCESS) return 0;
}
if (now_us() >= until) break;
usleep(100);
}
static _Atomic int dropped;
if (atomic_fetch_add(&dropped, 1) == 0)
say("%s: dropping a submit's wait on a shared value nobody has signalled yet (Turnip would block the submit)",
program_invocation_short_name);
return 1;
}
static void note_signal(VkSemaphore s, uint64_t value) {
pthread_mutex_lock(&g_br_lock);
struct bridged *b = bridged_find(s);
if (b && b->timeline && value > b->submitted) b->submitted = value;
pthread_mutex_unlock(&g_br_lock);
}
static void note_submit_signals(uint32_t count, const VkSubmitInfo *submits) {
for (uint32_t i = 0; i < count; i++) {
const VkTimelineSemaphoreSubmitInfo *t = NULL;
for (const VkBaseInStructure *p = submits[i].pNext; p; p = p->pNext)
if (p->sType == VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO) t = (const void *)p;
for (uint32_t j = 0; t && j < submits[i].signalSemaphoreCount && j < t->signalSemaphoreValueCount; j++)
note_signal(submits[i].pSignalSemaphores[j], t->pSignalSemaphoreValues[j]);
}
}
static void note_submit2_signals(uint32_t count, const VkSubmitInfo2 *submits) {
for (uint32_t i = 0; i < count; i++)
for (uint32_t j = 0; j < submits[i].signalSemaphoreInfoCount; j++)
note_signal(submits[i].pSignalSemaphoreInfos[j].semaphore, submits[i].pSignalSemaphoreInfos[j].value);
}
static uint32_t keep_waits(const VkSemaphore *in, const VkPipelineStageFlags *stages, const uint64_t *values, uint32_t n,
VkSemaphore *out, VkPipelineStageFlags *out_stages, uint64_t *out_values) {
uint32_t k = 0;
for (uint32_t i = 0; i < n; i++) {
if (submit_drop_wait(in[i], values ? values[i] : 0)) continue;
out[k] = in[i];
if (stages) out_stages[k] = stages[i];
if (values) out_values[k] = values[i];
k++;
}
return k;
}
static void trace_sem(const char *what, VkSemaphore sem, uint64_t value) {
if (!sem_trace_on() || value > 3) return;
pthread_mutex_lock(&g_br_lock);
@@ -894,6 +1048,113 @@ static void trace_sem(const char *what, VkSemaphore sem, uint64_t value) {
say("%s", line);
}
#include <limits.h>
#include <linux/futex.h>
#include <sys/syscall.h>
#include "present_sync.h"
#include "fence_tracker.h"
#include <time.h>
static pthread_mutex_t g_trk_lock = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t g_trk_cond = PTHREAD_COND_INITIALIZER;
static struct tracked *g_trk;
static struct droiddeck_present_sync *g_sync;
static pthread_once_t g_sync_once = PTHREAD_ONCE_INIT;
static void sync_wake(struct droiddeck_present_sync *s) {
syscall(SYS_futex, &s->done, FUTEX_WAKE, INT_MAX, NULL, NULL, 0);
}
static void *completer(void *arg) {
struct droiddeck_present_sync *s = arg;
pthread_mutex_lock(&g_trk_lock);
for (;;) {
uint32_t done;
int pending = track_pool_reap(g_trk, __atomic_load_n(&s->issued, __ATOMIC_ACQUIRE), &done);
if (done != __atomic_load_n(&s->done, __ATOMIC_RELAXED)) {
__atomic_store_n(&s->done, done, __ATOMIC_RELEASE);
sync_wake(s);
}
pthread_cond_broadcast(&g_trk_cond);
if (pending) {
struct timespec until;
clock_gettime(CLOCK_REALTIME, &until);
until.tv_nsec += 1000000; /* Poll pending GPU work at most every 1 ms. */
if (until.tv_nsec >= 1000000000) { until.tv_sec++; until.tv_nsec -= 1000000000; }
pthread_cond_timedwait(&g_trk_cond, &g_trk_lock, &until);
} else {
pthread_cond_wait(&g_trk_cond, &g_trk_lock);
}
}
return NULL;
}
static void sync_init(void) {
if (strcmp(program_invocation_short_name, "vrcompositor")) return;
const char *rt = getenv("XDG_RUNTIME_DIR");
char path[512];
snprintf(path, sizeof(path), "%s/droiddeck-steamvr/%s", rt && *rt ? rt : "/run/droiddeck", DROIDDECK_PRESENT_SYNC_NAME);
int fd = open(path, O_RDWR | O_CREAT | O_CLOEXEC, 0600);
if (fd < 0) return;
void *m = ftruncate(fd, DROIDDECK_PRESENT_SYNC_SIZE) == 0
? mmap(NULL, DROIDDECK_PRESENT_SYNC_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0) : MAP_FAILED;
close(fd);
if (m == MAP_FAILED) return;
struct droiddeck_present_sync *s = m;
__atomic_store_n(&s->done, __atomic_load_n(&s->issued, __ATOMIC_ACQUIRE), __ATOMIC_RELEASE);
sync_wake(s);
pthread_t t;
if (pthread_create(&t, NULL, completer, s) != 0) { munmap(m, DROIDDECK_PRESENT_SYNC_SIZE); return; }
pthread_detach(t);
g_sync = s;
say("%s", "frames go to the presenter once their GPU work is done");
}
static struct tracked *track_take(VkQueue q) {
pthread_once(&g_sync_once, sync_init);
if (!g_sync) return NULL;
struct dev *d = dev_for(q);
if (!d) return NULL;
pthread_mutex_lock(&g_trk_lock);
struct tracked *t = track_pool_take(&g_trk, d->handle, d->gdpa);
pthread_mutex_unlock(&g_trk_lock);
return t;
}
static void track_submitted(struct tracked *t, VkResult r) {
pthread_mutex_lock(&g_trk_lock);
uint32_t issued = track_pool_submitted(t, r, __atomic_load_n(&g_sync->issued, __ATOMIC_RELAXED));
__atomic_store_n(&g_sync->issued, issued, __ATOMIC_RELEASE);
pthread_cond_broadcast(&g_trk_cond);
pthread_mutex_unlock(&g_trk_lock);
}
static void track_forget_device(VkDevice dev) {
pthread_mutex_lock(&g_trk_lock);
while (track_pool_device_busy(g_trk, dev))
pthread_cond_wait(&g_trk_cond, &g_trk_lock);
track_pool_forget(&g_trk, dev);
pthread_mutex_unlock(&g_trk_lock);
}
static VkResult submit_tracked(PFN_vkQueueSubmit next, VkQueue q, uint32_t count, const VkSubmitInfo *submits, VkFence fence) {
struct tracked *slot = track_take(q);
if (!slot) return next(q, count, submits, fence);
const VkFence mine = slot->fence;
VkResult r = next(q, count, submits, fence ? fence : mine);
track_submitted(slot, r == VK_SUCCESS && fence ? next(q, 0, NULL, mine) : r);
return r;
}
static VkResult submit2_tracked(PFN_vkQueueSubmit2 next, VkQueue q, uint32_t count, const VkSubmitInfo2 *submits, VkFence fence) {
struct tracked *slot = track_take(q);
if (!slot) return next(q, count, submits, fence);
const VkFence mine = slot->fence;
VkResult r = next(q, count, submits, fence ? fence : mine);
track_submitted(slot, r == VK_SUCCESS && fence ? next(q, 0, NULL, mine) : r);
return r;
}
static VkResult VKAPI_CALL QueueSubmit(VkQueue q, uint32_t count, const VkSubmitInfo *submits, VkFence fence) {
PFN_vkQueueSubmit next = (PFN_vkQueueSubmit)dev_for(q)->gdpa(dev_for(q)->handle, "vkQueueSubmit");
if (sem_trace_on())
@@ -906,10 +1167,61 @@ static VkResult VKAPI_CALL QueueSubmit(VkQueue q, uint32_t count, const VkSubmit
for (uint32_t j = 0; j < submits[i].signalSemaphoreCount; j++)
trace_sem("signals", submits[i].pSignalSemaphores[j], t && j < t->signalSemaphoreValueCount ? t->pSignalSemaphoreValues[j] : 0);
}
int any = 0;
for (uint32_t i = 0; i < count && !any; i++) {
const VkTimelineSemaphoreSubmitInfo *t = NULL;
for (const VkBaseInStructure *p = submits[i].pNext; p; p = p->pNext)
if (p->sType == VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO) t = (const void *)p;
for (uint32_t j = 0; submits[i].pWaitSemaphores && j < submits[i].waitSemaphoreCount && !any; j++)
any = submit_drop_wait(submits[i].pWaitSemaphores[j], t && j < t->waitSemaphoreValueCount ? t->pWaitSemaphoreValues[j] : 0);
}
VkSubmitInfo *copy = NULL;
VkTimelineSemaphoreSubmitInfo *tl = NULL;
void **bufs = NULL;
if (any) {
copy = calloc(count, sizeof(*copy));
tl = calloc(count, sizeof(*tl));
bufs = calloc(count * 3, sizeof(void *));
if (!copy || !tl || !bufs) { free(copy); free(tl); free(bufs); return VK_ERROR_OUT_OF_HOST_MEMORY; }
for (uint32_t i = 0; i < count; i++) {
copy[i] = submits[i];
uint32_t n = submits[i].waitSemaphoreCount;
if (!n) continue;
VkSemaphore *sems = calloc(n, sizeof(*sems));
VkPipelineStageFlags *stages = calloc(n, sizeof(*stages));
uint64_t *vals = calloc(n, sizeof(*vals));
bufs[i * 3] = sems; bufs[i * 3 + 1] = stages; bufs[i * 3 + 2] = vals;
if (!sems || !stages || !vals) {
for (uint32_t k = 0; k < count * 3; k++) free(bufs[k]);
free(bufs); free(tl); free(copy);
return VK_ERROR_OUT_OF_HOST_MEMORY;
}
const VkTimelineSemaphoreSubmitInfo *t = NULL;
for (const VkBaseInStructure *p = submits[i].pNext; p; p = p->pNext)
if (p->sType == VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO) t = (const void *)p;
uint32_t k = keep_waits(submits[i].pWaitSemaphores, submits[i].pWaitDstStageMask,
t && t->waitSemaphoreValueCount ? t->pWaitSemaphoreValues : NULL, n, sems, stages, vals);
copy[i].waitSemaphoreCount = k;
copy[i].pWaitSemaphores = sems;
copy[i].pWaitDstStageMask = stages;
if (t) {
tl[i] = *t;
tl[i].waitSemaphoreValueCount = t->waitSemaphoreValueCount ? k : 0;
tl[i].pWaitSemaphoreValues = vals;
tl[i].pNext = t->pNext;
copy[i].pNext = &tl[i];
}
}
}
struct dev *device = dev_for(q);
if (device->producer) { pthread_mutex_lock(&g_submission_lock); producer_record(device, q); }
VkResult result = next(q, count, submits, fence);
VkResult result = submit_tracked(next, q, count, any ? copy : submits, fence);
if (device->producer) pthread_mutex_unlock(&g_submission_lock);
note_submit_signals(count, submits);
if (any) {
for (uint32_t i = 0; i < count * 3; i++) free(bufs[i]);
free(bufs); free(tl); free(copy);
}
return result;
}
@@ -923,10 +1235,41 @@ static VkResult VKAPI_CALL QueueSubmit2(VkQueue q, uint32_t count, const VkSubmi
for (uint32_t j = 0; j < submits[i].signalSemaphoreInfoCount; j++)
trace_sem("signals", submits[i].pSignalSemaphoreInfos[j].semaphore, submits[i].pSignalSemaphoreInfos[j].value);
}
int any = 0;
for (uint32_t i = 0; i < count && !any; i++)
for (uint32_t j = 0; submits[i].pWaitSemaphoreInfos && j < submits[i].waitSemaphoreInfoCount && !any; j++)
any = submit_drop_wait(submits[i].pWaitSemaphoreInfos[j].semaphore, submits[i].pWaitSemaphoreInfos[j].value);
VkSubmitInfo2 *copy = NULL;
VkSemaphoreSubmitInfo **waits = NULL;
if (any) {
copy = calloc(count, sizeof(*copy));
waits = calloc(count, sizeof(*waits));
if (!copy || !waits) { free(copy); free(waits); return VK_ERROR_OUT_OF_HOST_MEMORY; }
for (uint32_t i = 0; i < count; i++) {
copy[i] = submits[i];
uint32_t n = submits[i].waitSemaphoreInfoCount, k = 0;
waits[i] = calloc(n + 1, sizeof(VkSemaphoreSubmitInfo));
if (!waits[i]) {
for (uint32_t j = 0; j < i; j++) free(waits[j]);
free(waits); free(copy);
return VK_ERROR_OUT_OF_HOST_MEMORY;
}
for (uint32_t j = 0; j < n; j++)
if (!submit_drop_wait(submits[i].pWaitSemaphoreInfos[j].semaphore, submits[i].pWaitSemaphoreInfos[j].value))
waits[i][k++] = submits[i].pWaitSemaphoreInfos[j];
copy[i].waitSemaphoreInfoCount = k;
copy[i].pWaitSemaphoreInfos = waits[i];
}
}
struct dev *device = dev_for(q);
if (device->producer) { pthread_mutex_lock(&g_submission_lock); producer_record(device, q); }
VkResult result = next(q, count, submits, fence);
VkResult result = submit2_tracked(next, q, count, any ? copy : submits, fence);
if (device->producer) pthread_mutex_unlock(&g_submission_lock);
note_submit2_signals(count, submits);
if (any) {
for (uint32_t i = 0; i < count; i++) free(waits[i]);
free(waits); free(copy);
}
return result;
}