mirror of
https://github.com/Droid-Deck/DroidDeck.git
synced 2026-11-18 20:36:04 +00:00
Add VITURE XR controls, head tracking, and SteamVR (#732)
* Add VITURE glasses support: hardware control, 3D, head tracking Glasses get their own controller over the MCU (permission, 120 Hz default, brightness, volume, sunblock, 2D/3D) and the game moves onto them when plugged in. The IMU drives Cardboard head tracking and the SteamVR driver. 3D falls back to one eye's view on phones that can't send the 3840x1080 signal. Options open from a glasses button in game and on the Steam page. * Drop the glasses settings like Steam's: Play-style button, Settings lip, tabs * Hang the glasses Settings lip over the top of the button * Space glasses MCU commands so the IMU rate sticks, fit fixed resolutions to the glasses * Keep Steam alive with SteamVR registered, run SteamVR at the glasses' rate, keep the glasses lip on top * Fit VR games on the glasses: full view, seated origin at the head, pad hands reach with the d-pad * Let the pad hands reach a metre out so buttons with deep colliders press * Show every VR frame in order at 120 fps: wait for SteamVR's GPU work, present only finished copies, skip the grace on idle shared counters * Glasses: SteamVR starts and stops with them, closer view with a field of view setting, Recenter on the pad * Glasses: SteamVR field of view defaults to 90° * VR pad hands: sticks move and aim the hands, left stick click switches the sticks to the game, d-pad pulls a held hand * Glasses: Play SteamVR beside the glasses button instead of starting SteamVR with them, pad hands drawn as Index controllers and off while other VR controllers are connected, glasses sheet tabs sized like Steam's * Track glasses mode, refresh-aware glasses window, align Play SteamVR with Play Steam * Fix 3D on the glasses: full-res SteamVR presenter, revert VR 3D at session end, coalesce vsync ticks, no session restart after end * Address review: glasses output after installs and session end, VR revert retry, display fallback, presenter swapchain retry, agent hex checks, drawer bumpers, glasses translations, docs * Harden VITURE stereo negotiation and output refresh tracking * Localize stereo recovery feedback and record Astra reconnect tests * Improve test failure diagnostics and refresh Astra firmware findings * Handle VITURE docks separately and reject truncated USB frames * Glasses: remember a 3D fallback per model, docks as display only, VR timing fixes - A 3D attempt that ends at the 640x480 fallback is remembered for that glasses model, so Play SteamVR no longer blacks out the glasses after every restart; Retry clears it and "3D for VR games" stays reachable - A VITURE Mobile Dock on its own no longer counts as glasses; through a dock Play SteamVR is hidden and the sheet shows the output and why there is no control - HDR is probed on the glasses when the session starts there - Glasses frames with a mismatched checksum are marked in the status frame log - SteamVR layer and driver vblank counters stay monotonic across refresh changes; one-eye render scale applies from activation and on mono/3D switches - Compositor binds announce the refresh clients were last told; the presenter retries a failed first swapchain - Dead strings and helpers removed; docs updated * Glasses: drop USB frames with a bad checksum * Glasses: count frames dropped for a bad checksum * Feat/viture xr (#907) * Add RayNeo Air and GT support to the XR glasses pipeline * Add RayNeo Air and GT support to the XR glasses pipeline * Checkpoint glasses tracking, calibration, and SteamVR launch integration * Checkpoint glasses tracking, calibration, and SteamVR launch integration * Add XREAL 1S support and correct GT and Pro 2 tracking * Add XREAL 1S support and correct GT and Pro 2 tracking * Fix mirrored glasses tracking and recover SteamVR after device loss * Fix mirrored glasses tracking and recover SteamVR after device loss * Document confirmed XREAL 1S brightness control * Document confirmed XREAL 1S brightness control * Fix SteamVR glasses defaults and rendering controls * Fix SteamVR glasses defaults and rendering controls * chore: keep only the 3 newest session log folders (#852) * chore: add agent comment rules (#748) Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com> * Fix SteamVR polling, VITURE checksums, and fence tracking Keep manual SteamVR launches polled, discard corrupt VITURE packets, and retire Vulkan fences out of order without blocking submissions at a fixed capacity. Preserve the contiguous completion watermark used by the presenter. Fix optional Windows launcher configuration and audited helper staging. Add Android, Windows, and native tracker regression coverage. * Fix SteamVR polling, VITURE checksums, and fence tracking Keep manual SteamVR launches polled, discard corrupt VITURE packets, and retire Vulkan fences out of order without blocking submissions at a fixed capacity. Preserve the contiguous completion watermark used by the presenter. Fix optional Windows launcher configuration and audited helper staging. Add Android, Windows, and native tracker regression coverage. * fix: wireless ADB page names the child-process switch the way Android shows it (#848) * l10n: refresh stale and untranslated strings in es, fr, ko, ru (#850) * l10n: refresh stale and untranslated strings in es, fr, ko, ru - mode_added_art_hint: now mentions Steam then SteamGridDB (all four locales). - wincomp_needs_installer / wincomp_installers_note: es and ko still described the old "coming soon" behaviour; align with the current English text. - summary_remove_game: was still English in all four locales. Machine-assisted translation; native-speaker review is welcome. * l10n(es): say registro, not grabación, for the missing installer record --------- Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com> * fix: a failed cloud save recovery no longer ends the app (#846) The cloud-recovery thread started at every app start had nothing to catch an error, so one went to CrashHandler, which halts the process with status 2. In unit tests the thread can outlive the test that started it, fail on the torn-down preferences, and take the whole test run down. The error is now logged and the thread ends. Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com> * fix: an interrupted Linux runtime download resumes instead of starting again (#847) * fix: an interrupted Linux runtime download resumes instead of starting again installOnce() asked the downloader to resume but always deleted the archive in its finally block, so the next attempt began at zero. The archive is now named after the release version and checksum, a failed download keeps what it fetched, and archives of other releases (and the old fixed name) are removed. A checksum or extraction failure still deletes the file. * test: Downloader resumes a cut-off download from the bytes it kept Starts a socket server that promises the whole file, hangs up part way and then answers a Range request, and checks that the second downloadFile call resumes with the right offset and ends with identical bytes. * fix: a resumed download discards a 206 that does not start where the kept bytes end Downloader appended every 206 response to the partial file without looking at Content-Range. A server or proxy that answers a Range request with a different offset would leave a corrupt file. Check that the range starts at the kept length; otherwise delete the partial file and start again without a Range header. Adds a test with a server that answers the Range request from offset 0. --------- Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com> * l10n: German, French, Spanish, Italian, Portuguese and Russian for the zRAM, reset-settings and store free-space strings (#849) * l10n: German, French, Spanish, Italian, Portuguese and Russian for the zRAM, reset-settings and store free-space strings Adds the strings values-de, values-it and values-pt were missing (14 each) and the six that values-fr, values-es and values-ru were missing. Machine-assisted drafts that follow each file's existing register and terms; native review is welcome. * l10n: correct missing active swap translations --------- Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com> Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com> * fix: droiddeckctl reaches variant-package builds (recover #842) (#854) fix: droiddeckctl reaches builds shipped under a variant package The provider authority and the start Activity's component were fixed to com.droiddeck.launcher, so droiddeckctl could not talk to the antutu, ludashi or pubg builds from tools/release/variants.txt. DROIDDECK_PACKAGE now picks the package; the Activity's class name stays the same in every variant. Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com> * feat: droiddeckctl reports on-screen pacing (recover #843) (#855) feat: droiddeckctl pacing reports the frames that reached the screen Reads SurfaceFlinger's timestats for the game's layer (or the session's SurfaceView) over a capture: frames on screen, fps on screen, the present-to-present histogram and how many intervals were off the usual cadence. Records split by a refresh-rate change are summed. A game's own fps counter counts frames handed over, so it can't show dropped or doubled frames. Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com> * fix: Steam can restart more than twice in a session (#754) * fix: Steam can restart more than twice in a session Every exit 42 counted toward the cap that stops an update which will not install, and the count was never cleared, so the third restart of a session ended it however far apart they were. A client that ran for a minute before asking is not that loop: the count starts again. * fix: measure the client's run time as it exits The time spent collecting logs and checking FEX after the client exits counted toward its run, so a slow cleanup could make a quick exit 42 look like a long one and clear the count. --------- Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com> Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com> * fix: the front end keeps its page when the Fold is folded or unfolded (#832) * fix: the front end keeps its page when the Fold is folded or unfolded MainActivity, FileManagerActivity, FilePickerActivity and StoreLoginActivity did not declare the screen-size configuration changes, so folding or unfolding a foldable recreated them and dropped their in-memory state. They now handle orientation, screenSize, screenLayout, smallestScreenSize and density themselves, like SessionActivity already does. uiMode is left out on purpose. * fix: store login keeps rebuilding its toolbar when the display density changes StoreLoginActivity builds its toolbar in code with dp-to-pixel values taken at creation time and has no onConfigurationChanged. It no longer declares density in configChanges, so a density change recreates it as before this PR. Size and layout changes are still handled in place. --------- Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com> * Release pointer buttons without replaying stale motion (#803) * Fix cursor warping on native touch down * Preserve input cleanup without replaying stale pointer motion --------- Co-authored-by: siyuan.qsy <siyuan.qsy@alibaba-inc.com> * l10n: translate the strings added since #284 into zh-CN, zh-HK, zh-TW and ja (#539) * l10n: translate the 184 strings added since #284 (zh-CN/HK/TW, ja) The stores, cloud saves, Windows components, Steam touch, desktop removal and background-downloads strings landed after #284's translation snapshot, so every one of them fell back to English. Translate all 184 into Simplified Chinese, Traditional Chinese (Taiwan and Hong Kong) and Japanese, keeping each locale file's own vocabulary (工作階段/會話, 解除安裝/卸載, 元件/組件, 設定). Also fix four Hong Kong strings that kept the simplified 着色器. * l10n(zh): fix stale and ambiguous strings found in review Severe: summary_remove_game was untranslated English in all three files; mode_added_art_hint, wincomp_installers_note and wincomp_needs_installer still described the pre-#480/#492 English. Also: distinguish the Setup wizard from Android system settings in zh-HK/zh-TW (初始設定 / 系統設定 / Android 設定), unify library as 收藏庫, component as 組件 in zh-HK, session as 會話 in zh-HK, account as 帳戶 in zh-HK, Trending as 趋势/趨勢, Forget as 清除, reword the re-applied-at-launch, action-failed and already-in-Games strings, and play as 執行/開啟遊戲 in zh-TW/zh-HK. * l10n(ja): fix stale and ambiguous strings found in review --------- Co-authored-by: Dantalian <Dantalian-dev@users.noreply.github.com> Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com> * Rebuild and verify Windows VR runtime components * Rebuild and verify Windows VR runtime components * Add top, bottom, and centered picture positions for Steam sessions (#871) * fix: dependency scan ignores folders above the game folder (#859) * DroidDeck-Proton: get releases from the Droid-Deck org repo DroidDeck-Proton is now built and released on Droid-Deck/DroidDeck-Proton (11-9.1 is Latest there), so the Components page, the update check and the guest installer read releases from the org repo instead of The412Banner's. * feat: low-latency controller and touch toggles in the session drawer (#808) * fix: SteamGridDB lookups no longer stick as "no match" after a failure; add a search to the art chooser (#876) * feat: Steam home wall fills from the signed-in library instead of store picks (#878) * test: cover GPU family and support classification (#880) * Fix local builds on Linux failing after the first Comet build (#866) * feat: sort the store library and installed tabs, and switch between card and list views (#798) * feat: three-finger tap toggles Steam's on-screen keyboard in game mode (#782) * fix: keep Quest-patched Turnip as the auto default (#853) * Fix deploy_local.sh and droiddeckctl not finding a phone connected over wireless debugging (#864) * fix: keep proot sessions alive when the kernel loses fork events (#457) * fix(quest): correct desktop and Steam display corruption across sessions (#733) fix(quest): stabilize desktop and Steam layer presentation Keep Quest SDR scenes and the black backdrop off the Android Vulkan swapchain regardless of session order or HDR preferences. Only report the first frame after client content is submitted. Add backdrop lifecycle and first-frame regression coverage. Record the user's successful Quest 3 Test 2 result. Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com> * fix: keep SteamVR syscall hooks and report Steam crashes (#905) * fix: preserve SteamVR syscall hooks * fix: report Steam crashes through gamescope * fix: honor glasses pacing and resolution in SteamVR launches --------- Co-authored-by: connor g <connorsspam1992@gmail.com> Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com> Co-authored-by: Yassine Gherbi <y.gherbi.dev@gmail.com> Co-authored-by: Resul Övür <92828257+ovurrsl@users.noreply.github.com> Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com> Co-authored-by: Kirk <kirtoy.junsay@gmail.com> Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com> Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com> Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com> Co-authored-by: asdf17128 <asdf17128@gmail.com> Co-authored-by: siyuan.qsy <siyuan.qsy@alibaba-inc.com> Co-authored-by: Dantalian <wsu091227@qq.com> Co-authored-by: Dantalian <Dantalian-dev@users.noreply.github.com> Co-authored-by: The412Banner <the412banner@users.noreply.github.com> Co-authored-by: The412Banner <205237651+The412Banner@users.noreply.github.com> Co-authored-by: FX1922 <120828496+FX192@users.noreply.github.com> Co-authored-by: Muaz Kadan <32595066+muazkadan@users.noreply.github.com> Co-authored-by: kingokksa <47888451+kingokksa@users.noreply.github.com> Co-authored-by: Jamie Schenker <jam1esch3nk3r@gmail.com> Co-authored-by: Jagger15 <jagger2333@live.com> * Glasses: keep a pending 2D restore and the SteamVR recovery offer, show Play SteamVR only with a picture * Translate the glasses calibration, XREAL and SteamVR recovery strings * SteamVR recovery: give a finished teardown time to collect logs before restarting * Credit the projects the XR glasses support builds on * Joy-Cons: SteamVR hands from their gyro while SteamVR runs, one combined pad otherwise * Headsets: settings badge on Play SteamVR instead of a top-right button, both shown only once connected * Hand tracking: camera-tracked Joy-Cons and bare hands in SteamVR, drawn as white gloves - Joy-Con movement from the accelerometer (report reordering, per-Joy-Con calibration, zero-velocity updates) - MediaPipe pose and hand landmarks from the device camera, placement check, per-hand Kalman filter with the Joy-Con - No controller connected: bare camera hands with Quest-style gestures, 1€ filtered, rendered as a white glove model built from a distance field - SteamVR safe mode cleared before each start so one crash no longer blocks the driver --------- Co-authored-by: MaxsTechReview <6025299+maxjivi05@users.noreply.github.com> Co-authored-by: informalTechCode <informaltechyt@gmail.com> Co-authored-by: connor g <connorsspam1992@gmail.com> Co-authored-by: Kurt Himebauch <136133082+xXJSONDeruloXx@users.noreply.github.com> Co-authored-by: Yassine Gherbi <y.gherbi.dev@gmail.com> Co-authored-by: Resul Övür <92828257+ovurrsl@users.noreply.github.com> Co-authored-by: ovurrsl <ovurrsl@users.noreply.github.com> Co-authored-by: Kirk <kirtoy.junsay@gmail.com> Co-authored-by: Kirk <12691869+KirkPatrickJunsay@users.noreply.github.com> Co-authored-by: xXJsonDeruloXx <danielhimebauch@gmail.com> Co-authored-by: angelrenefm <17342133+angelrenefm@users.noreply.github.com> Co-authored-by: asdf17128 <asdf17128@gmail.com> Co-authored-by: siyuan.qsy <siyuan.qsy@alibaba-inc.com> Co-authored-by: Dantalian <wsu091227@qq.com> Co-authored-by: Dantalian <Dantalian-dev@users.noreply.github.com> Co-authored-by: The412Banner <the412banner@users.noreply.github.com> Co-authored-by: The412Banner <205237651+The412Banner@users.noreply.github.com> Co-authored-by: FX1922 <120828496+FX192@users.noreply.github.com> Co-authored-by: Muaz Kadan <32595066+muazkadan@users.noreply.github.com> Co-authored-by: kingokksa <47888451+kingokksa@users.noreply.github.com> Co-authored-by: Jamie Schenker <jam1esch3nk3r@gmail.com> Co-authored-by: Jagger15 <jagger2333@live.com>
This commit is contained in:
+13








authored and
GitHub committed 2026-10-17 13:17:05 -04:00
1 parent
8acf3037fd
commit
16f9916825
221 files changed
+24369
-491
No files matched your search
@@ -16,12 +16,17 @@ class App : Application() {
|
||||
|
||||
override fun onCreate() {
|
||||
super.onCreate()
|
||||
// The restart screen must not initialize USB, guest processes, or another graphics context.
|
||||
val process = (getSystemService(ACTIVITY_SERVICE) as android.app.ActivityManager)
|
||||
.runningAppProcesses?.firstOrNull { it.pid == android.os.Process.myPid() }?.processName
|
||||
if (process == "$packageName:vr_recovery") return
|
||||
CrashHandler.install(this)
|
||||
com.droiddeck.launcher.session.QuestExperimental.initialize(this)
|
||||
com.droiddeck.launcher.session.SessionPrefs.settleGpuDriverMode(this)
|
||||
com.droiddeck.launcher.update.AppUpdates.init(this)
|
||||
com.droiddeck.launcher.core.DownloadSource.init(this)
|
||||
GpuClockPin.clearLeftover(this)
|
||||
com.droiddeck.launcher.glasses.Glasses.init(this)
|
||||
BwrapSpawner.start(this)
|
||||
com.droiddeck.launcher.frontend.GameFileSync.start(this)
|
||||
com.droiddeck.launcher.stores.StoresState.init(this)
|
||||
|
||||
@@ -192,6 +192,10 @@ class MainActivity : ComponentActivity() {
|
||||
SessionPrefs.setWifiDiscoveryEnabled(this, WifiDiscovery.permissionGranted(this))
|
||||
refreshWifiDiscovery()
|
||||
}
|
||||
private val cameraPermission = registerForActivityResult(ActivityResultContracts.RequestPermission()) {
|
||||
com.droiddeck.launcher.handtracking.HandTracking.refreshPermission(this)
|
||||
}
|
||||
private var placementCheck by mutableStateOf<Boolean?>(null)
|
||||
private val wifiPermissionSettings = registerForActivityResult(ActivityResultContracts.StartActivityForResult()) {
|
||||
SessionPrefs.setWifiDiscoveryEnabled(this, WifiDiscovery.permissionGranted(this))
|
||||
refreshWifiDiscovery()
|
||||
@@ -507,19 +511,27 @@ class MainActivity : ComponentActivity() {
|
||||
android.widget.Toast.makeText(this, R.string.game_link_session_busy, android.widget.Toast.LENGTH_LONG).show()
|
||||
return true
|
||||
}
|
||||
if (!SessionService.launchGame(this, game.gameIdString)) {
|
||||
val steamVr = game.gameIdString == com.droiddeck.launcher.gpu.SteamVr.APP_ID
|
||||
if (!steamVr && !SessionService.launchGame(this, game.gameIdString)) {
|
||||
android.widget.Toast.makeText(this, R.string.game_link_failed, android.widget.Toast.LENGTH_LONG).show()
|
||||
return false
|
||||
}
|
||||
startActivity(Intent(this, SessionActivity::class.java).setAction(SessionService.ACTION_RESUME)
|
||||
.putExtra(SessionActivity.EXTRA_STEAMVR, steamVr)
|
||||
.putExtra(SessionActivity.EXTRA_RETURN_HOME, returnHome)
|
||||
.addFlags(Intent.FLAG_ACTIVITY_CLEAR_TOP or Intent.FLAG_ACTIVITY_SINGLE_TOP))
|
||||
return true
|
||||
}
|
||||
return startSession(Intent(this, SessionActivity::class.java)
|
||||
.putExtra(SessionActivity.EXTRA_RETURN_HOME, returnHome)
|
||||
.putExtra(SessionService.EXTRA_STEAM_URL,
|
||||
com.droiddeck.launcher.frontend.GameLaunchLink.steamUrl(game.gameIdString)), steamSession = true)
|
||||
.apply {
|
||||
if (game.gameIdString == com.droiddeck.launcher.gpu.SteamVr.APP_ID) {
|
||||
putExtra(SessionActivity.EXTRA_STEAMVR, true)
|
||||
} else {
|
||||
putExtra(SessionService.EXTRA_STEAM_URL,
|
||||
com.droiddeck.launcher.frontend.GameLaunchLink.steamUrl(game.gameIdString))
|
||||
}
|
||||
}, steamSession = true)
|
||||
}
|
||||
|
||||
private fun handleGameLink(games: List<Library.SteamGame>) {
|
||||
@@ -636,6 +648,7 @@ class MainActivity : ComponentActivity() {
|
||||
),
|
||||
FrontEndActions(
|
||||
onPlay = { playSteam() },
|
||||
onPlaySteamVr = { playSteamVr() },
|
||||
onSteamUrl = { url ->
|
||||
if (!SessionState.running) startSession(
|
||||
Intent(this, SessionActivity::class.java).putExtra(SessionService.EXTRA_STEAM_URL, url), steamSession = true,
|
||||
@@ -886,6 +899,7 @@ class MainActivity : ComponentActivity() {
|
||||
onQamButton = { on -> ControllerPrefs.setQamButton(this, on); refreshController() },
|
||||
onKeyboardButton = { on -> ControllerPrefs.setKeyboardButton(this, on); refreshController() },
|
||||
onSelectSteam = { on -> ControllerPrefs.setSelectSteam(this, on); refreshController() },
|
||||
onJoyConVr = { on -> ControllerPrefs.setJoyConVr(this, on); refreshController() },
|
||||
onEditLayout = { startActivity(Intent(this, ControllerEditorActivity::class.java)) },
|
||||
onResetLayout = { ControllerPrefs.resetAllLayouts(this); refreshController() },
|
||||
onMapping = { showPerformance = false; showComponents = false; showMapping = true },
|
||||
@@ -928,6 +942,12 @@ class MainActivity : ComponentActivity() {
|
||||
onDismiss = { lowSpace = null },
|
||||
)
|
||||
}
|
||||
com.droiddeck.launcher.ui.HandCameraHost(
|
||||
check = placementCheck,
|
||||
onAskPermission = { cameraPermission.launch(Manifest.permission.CAMERA) },
|
||||
onPlay = { placementCheck = null; startSteamSession(steamVr = true) },
|
||||
onClose = { placementCheck = null },
|
||||
)
|
||||
if (showResetSettings) com.droiddeck.launcher.ui.ResetSettingsDialog(
|
||||
onConfirm = { gameProfiles -> resetSettings(gameProfiles) },
|
||||
onDismiss = { showResetSettings = false },
|
||||
@@ -988,8 +1008,13 @@ class MainActivity : ComponentActivity() {
|
||||
} else startSteamSession()
|
||||
}
|
||||
|
||||
private fun startSteamSession() {
|
||||
startSession(Intent(this, SessionActivity::class.java), steamSession = true)
|
||||
private fun playSteamVr() {
|
||||
if (com.droiddeck.launcher.handtracking.HandTracking.wanted(this)) placementCheck = true
|
||||
else startSteamSession(steamVr = true)
|
||||
}
|
||||
|
||||
private fun startSteamSession(steamVr: Boolean = false) {
|
||||
startSession(Intent(this, SessionActivity::class.java).putExtra(SessionActivity.EXTRA_STEAMVR, steamVr), steamSession = true)
|
||||
}
|
||||
|
||||
override fun onWindowFocusChanged(hasFocus: Boolean) {
|
||||
|
||||
@@ -26,7 +26,7 @@ class QuestXrActivity : NativeActivity() {
|
||||
if (active) {
|
||||
check(CompositorHost.isStarted) { "DroidDeck compositor is not running" }
|
||||
SessionService.resume(this)
|
||||
WaylandCompositor.nativeSetCardboard(false, 1, 0.064f, 3f, 4f, 0f, 0f, 90f, 0f, 0f)
|
||||
WaylandCompositor.nativeSetCardboard(false, 1, 0.064f, 3f, 4f, 0f, 0f, 90f, 0f, 0f, false, 0f)
|
||||
WaylandCompositor.nativeSetFrameGenArmed(false, 0, 0)
|
||||
WaylandCompositor.nativeSetZeroCopy(false)
|
||||
CompositorHost.attachXr()
|
||||
|
||||
@@ -52,6 +52,11 @@ import com.droiddeck.launcher.ui.SessionAchievements
|
||||
import com.droiddeck.launcher.gpu.Cardboard
|
||||
import com.droiddeck.launcher.gpu.FrameGen
|
||||
import com.droiddeck.launcher.gpu.HeadTracker
|
||||
import com.droiddeck.launcher.glasses.Glasses
|
||||
import com.droiddeck.launcher.glasses.GlassesScreen
|
||||
import com.droiddeck.launcher.glasses.GlassesRendering
|
||||
import kotlinx.coroutines.cancel
|
||||
import kotlinx.coroutines.launch
|
||||
import com.droiddeck.launcher.gpu.SteamVr
|
||||
import com.droiddeck.launcher.gpu.ScreenEffectLooks
|
||||
import com.droiddeck.launcher.gpu.ScreenEffects
|
||||
@@ -141,6 +146,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
private lateinit var loading: LoadingState
|
||||
private lateinit var hud: PerfHud
|
||||
private var padBridge: PadBridge? = null
|
||||
private var joyConVr: com.droiddeck.launcher.input.JoyConVr? = null
|
||||
private var handCamera: com.droiddeck.launcher.handtracking.HandCamera? = null
|
||||
private var padMotion: com.droiddeck.launcher.input.PadMotion? = null
|
||||
private var resumed = false
|
||||
private val deckPadListener: () -> Unit = { uiHandler.post { updatePadMotion(); openDeckRing("deck pad changed"); applyUnfoldedControls() } }
|
||||
@@ -278,6 +285,23 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
private var steamVrStatus by mutableStateOf<SteamVr.Status?>(null)
|
||||
private var steamVrSwitchedPicture by mutableStateOf(false)
|
||||
private var steamVrSeenRunning = false
|
||||
private val steamVrRecovery by lazy { com.droiddeck.launcher.gpu.SteamVrRecovery(this) {
|
||||
resumed && !ended && SessionState.running && !isInPictureInPictureMode
|
||||
} }
|
||||
private var activeCardboard = Cardboard.OFF
|
||||
private var glasses by mutableStateOf(Glasses.state.value)
|
||||
private var glassesSettings by mutableStateOf(Glasses.settings.value)
|
||||
private var glassesScreen: GlassesScreen? = null
|
||||
private var glassesOutputActive by mutableStateOf(false)
|
||||
private var glasses3dForVr = false
|
||||
private var ended = false
|
||||
private var glassesPresent = false
|
||||
private var steamVrAutoTried = false
|
||||
private var steamVrInstallTried = false
|
||||
private var headTracking by mutableStateOf(false)
|
||||
private var glassesSlot by mutableStateOf(0)
|
||||
private var glassesKey: List<Any?> = emptyList()
|
||||
private val glassesScope = kotlinx.coroutines.MainScope()
|
||||
private val hasRotationSensor by lazy {
|
||||
(getSystemService(SENSOR_SERVICE) as android.hardware.SensorManager).let {
|
||||
it.getDefaultSensor(android.hardware.Sensor.TYPE_GAME_ROTATION_VECTOR)
|
||||
@@ -318,10 +342,10 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
|
||||
private fun followPanelRefresh(displayId: Int) {
|
||||
if (!CompositorHost.isStarted) return
|
||||
val display = (if (Build.VERSION.SDK_INT >= 30) display else windowManager.defaultDisplay) ?: return
|
||||
if (display.displayId != displayId) return
|
||||
val hz = display.refreshRate
|
||||
if (hz > 1f) WaylandCompositor.nativeSetOutputRefreshRate(hz)
|
||||
val output = glassesScreen?.display
|
||||
?: (if (Build.VERSION.SDK_INT >= 30) display else windowManager.defaultDisplay) ?: return
|
||||
if (output.displayId != displayId) return
|
||||
applyOutputPacing()
|
||||
}
|
||||
private var isHomeApp by mutableStateOf(false)
|
||||
private var androidApps by mutableStateOf<List<HomeApp.LaunchableApp>>(emptyList())
|
||||
@@ -329,17 +353,24 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
private val deviceListener = object : InputManager.InputDeviceListener {
|
||||
override fun onInputDeviceAdded(deviceId: Int) {
|
||||
logInputDevice("connected", deviceId)
|
||||
joyConVr?.refresh()
|
||||
padBridge?.syncConnectedControllers()
|
||||
refreshControls()
|
||||
updatePointerCapture()
|
||||
updateHandCamera()
|
||||
}
|
||||
override fun onInputDeviceRemoved(deviceId: Int) {
|
||||
Log.i(TAG, "input device $deviceId disconnected")
|
||||
joyConVr?.refresh()
|
||||
padBridge?.onDeviceRemoved(deviceId)
|
||||
refreshControls()
|
||||
updatePointerCapture()
|
||||
updateHandCamera()
|
||||
}
|
||||
override fun onInputDeviceChanged(deviceId: Int) {
|
||||
joyConVr?.refresh()
|
||||
refreshControls()
|
||||
}
|
||||
override fun onInputDeviceChanged(deviceId: Int) = refreshControls()
|
||||
}
|
||||
|
||||
private var runtimeRemovalBlocked = false
|
||||
@@ -363,17 +394,21 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
SessionState.programArgs = intent.getStringArrayExtra(SessionService.EXTRA_PROGRAM_ARGS)?.toList().orEmpty()
|
||||
SessionState.steamUi = intent.getStringExtra(SessionService.EXTRA_STEAM_UI)
|
||||
SessionState.steamUrl = intent.getStringExtra(SessionService.EXTRA_STEAM_URL)
|
||||
SessionState.steamVr = intent.getBooleanExtra(EXTRA_STEAMVR, false)
|
||||
} else if (intent.getBooleanExtra(EXTRA_STEAMVR, false) && SessionState.mode == SessionService.MODE_STEAM) {
|
||||
SessionState.steamVr = true
|
||||
}
|
||||
if (intent.action == SessionService.ACTION_AGENT_START) {
|
||||
SessionEvents.record("agent.start_requested", mapOf("mode" to SessionState.mode))
|
||||
}
|
||||
com.droiddeck.launcher.gpu.SteamVrHealth.begin(this, SessionState.sessionId)
|
||||
window.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
|
||||
if (!SessionState.running) {
|
||||
SessionState.fpsLimit = SessionPrefs.fpsLimit(
|
||||
this, SessionPrefs.prefMode(intent.getStringExtra(SessionService.EXTRA_MODE) ?: SessionService.MODE_STEAM),
|
||||
)
|
||||
}
|
||||
Log.i(TAG, "perf: " + PerfMode.apply(this, SessionState.fpsLimit))
|
||||
applyOutputPacing()
|
||||
if (!SessionState.running) SessionState.followScreen = followChoice()
|
||||
applyFollowOrientation(resources.configuration)
|
||||
goFullscreen()
|
||||
@@ -403,6 +438,16 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
bridge.setSelectSteam(ControllerPrefs.read(this).selectSteam)
|
||||
padBridge = bridge
|
||||
SessionState.padBridge = bridge
|
||||
if (Build.VERSION.SDK_INT >= 31) {
|
||||
val vr = com.droiddeck.launcher.input.JoyConVr(this, File(filesDir, ".wayland-rt/${com.droiddeck.launcher.input.JoyConVr.FILE}")) { taken, released ->
|
||||
if (joyConVr == null) return@JoyConVr
|
||||
taken.forEach(bridge::onDeviceRemoved)
|
||||
if (released.isNotEmpty()) bridge.syncConnectedControllers()
|
||||
updateHandCamera()
|
||||
}
|
||||
bridge.setExcluded(vr::owns)
|
||||
joyConVr = vr
|
||||
}
|
||||
SteamControllerBle.listener = { uiHandler.post { updatePadMotion(); refreshControls() } }
|
||||
steamController = SteamControllerBle(this).also { it.start() }
|
||||
padMotion = com.droiddeck.launcher.input.PadMotion(this) {
|
||||
@@ -450,6 +495,21 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
sessionOverlay = createSessionOverlay()
|
||||
root.addView(sessionOverlay)
|
||||
setContentView(root)
|
||||
glassesScope.launch { Glasses.state.collect { glasses = it; onGlassesChanged() } }
|
||||
glassesScope.launch { Glasses.settings.collect { glassesSettings = it; onGlassesChanged() } }
|
||||
glassesScope.launch { com.droiddeck.launcher.handtracking.HandTracking.settings.collect { updateHandCamera() } }
|
||||
glassesScope.launch {
|
||||
while (true) {
|
||||
if (SessionState.running && SessionState.mode == SessionService.MODE_STEAM &&
|
||||
(SessionState.steamVr || steamVrStatus == SteamVr.Status.RUNNING)) {
|
||||
val lost = kotlinx.coroutines.withContext(kotlinx.coroutines.Dispatchers.IO) {
|
||||
com.droiddeck.launcher.gpu.SteamVrHealth.deviceLost()
|
||||
}
|
||||
if (lost) steamVrRecovery.detected()
|
||||
}
|
||||
kotlinx.coroutines.delay(2000)
|
||||
}
|
||||
}
|
||||
pip = com.droiddeck.launcher.session.SessionPipController(this, ::pipBounds, ::showPipUi)
|
||||
surfaceView.addOnLayoutChangeListener { _, _, _, _, _, _, _, _, _ -> pip.refresh() }
|
||||
pip.refresh()
|
||||
@@ -496,6 +556,15 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
CursorOverlay(cursorPos, cursorVisible, resources.displayMetrics.density,
|
||||
cursorImage, cursorHotX, cursorHotY, cursorImageScale)
|
||||
if (hud.text.isNotEmpty()) HudText(hud.text)
|
||||
if (glassesOutputActive && !loading.visible) com.droiddeck.launcher.ui.GlassesPlayingHint(glasses.model)
|
||||
if (GlassesRendering.usesGlasses(glasses, glassesOutputActive, steamVrStatus == SteamVr.Status.RUNNING) &&
|
||||
headTracking && !drawerOpen && !pcKeyboardOpen && !loading.visible) com.droiddeck.launcher.ui.GlassesRecenterButton(
|
||||
controllerSettings?.tint ?: android.graphics.Color.rgb(90, 160, 255), controllerSettings?.opacity ?: 100) { headTracker?.recenter() }
|
||||
if (glasses.present && !drawerOpen && !pcKeyboardOpen && !loading.visible) com.droiddeck.launcher.ui.GlassesCornerButton(glassesSlot,
|
||||
controllerSettings?.tint ?: android.graphics.Color.rgb(90, 160, 255), controllerSettings?.opacity ?: 100) {
|
||||
drawerPage = com.droiddeck.launcher.ui.SessionDrawerPage.GLASSES
|
||||
drawerOpen = true
|
||||
}
|
||||
if (loading.visible) LoadingOverlay(
|
||||
loading.step, loading.percent, loading.elapsed, loading.hint, loading.ended,
|
||||
topic = loading.topic, readable = loading.readable,
|
||||
@@ -514,7 +583,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
androidx.compose.runtime.LaunchedEffect(drawerOpen) {
|
||||
if (drawerOpen) {
|
||||
syncLossless()
|
||||
padBridge?.releaseAll()
|
||||
releasePads()
|
||||
androidx.compose.runtime.withFrameNanos { }
|
||||
val requested = sessionOverlay.requestFocus()
|
||||
Log.i(TAG, "drawer root focus requested=$requested focused=${sessionOverlay.hasFocus()}")
|
||||
@@ -522,8 +591,12 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
releaseDrawerDirection()
|
||||
}
|
||||
}
|
||||
LaunchedEffect(cardboard.enabled, steamVrSwitchedPicture, drawerOpen, drawerPage) {
|
||||
while (cardboard.enabled || steamVrSwitchedPicture || (drawerOpen && drawerPage == SessionDrawerPage.VR)) {
|
||||
val glassesVr = glasses.usable && glasses.canControl
|
||||
val steamVrRequested = SessionState.steamVr
|
||||
val pollSteamVr = SteamVr.shouldPoll(steamVrStatus, cardboard.enabled,
|
||||
steamVrSwitchedPicture, glassesVr, steamVrRequested) || (drawerOpen && drawerPage == SessionDrawerPage.VR)
|
||||
LaunchedEffect(pollSteamVr) {
|
||||
while (pollSteamVr) {
|
||||
val now = kotlinx.coroutines.withContext(kotlinx.coroutines.Dispatchers.IO) {
|
||||
SteamVr.status(this@SessionActivity)
|
||||
}
|
||||
@@ -540,7 +613,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
achievementsInFront = found
|
||||
if (found == null) achievementsOpen = false
|
||||
}
|
||||
androidx.compose.runtime.LaunchedEffect(pcKeyboardOpen) { if (pcKeyboardOpen) padBridge?.releaseAll() }
|
||||
androidx.compose.runtime.LaunchedEffect(pcKeyboardOpen) { if (pcKeyboardOpen) releasePads() }
|
||||
if (pcKeyboardOpen) com.droiddeck.launcher.ui.PcKeyboard(
|
||||
sendKey = { code, down -> if (CompositorHost.isStarted) sendGuestKey(code, down) },
|
||||
onAndroidKeyboard = { pcKeyboardOpen = false; keyboard?.toggle() },
|
||||
@@ -559,6 +632,9 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
cardboardSensor = hasRotationSensor,
|
||||
onCardboard = { c -> SessionPrefs.setCardboard(this@SessionActivity, c); applyCardboard(c) },
|
||||
onCardboardRecenter = { headTracker?.recenter() },
|
||||
glassesPresent = glasses.present,
|
||||
glassesOutput = glassesOutputActive,
|
||||
onGlassesRecenter = { headTracker?.recenter() },
|
||||
steamVr = steamVrStatus,
|
||||
immersiveSteamVr = com.droiddeck.launcher.session.QuestExperimental.isSupported(),
|
||||
onSteamVr = { action -> steamVrAction(action) },
|
||||
@@ -747,7 +823,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
))
|
||||
LaunchedEffect(achievementsOpen) {
|
||||
if (achievementsOpen) {
|
||||
padBridge?.releaseAll()
|
||||
releasePads()
|
||||
withFrameNanos { }
|
||||
sessionOverlay.requestFocus()
|
||||
}
|
||||
@@ -1224,7 +1300,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
}
|
||||
loading.percent = -1
|
||||
loading.say(getString(R.string.session_starting_session), LoadingState.Topic.SESSION, readable = false)
|
||||
if (surfaceView.holder.surface?.isValid == true) surfaceCreated(surfaceView.holder)
|
||||
outputHolder().let { if (it.surface?.isValid == true) surfaceCreated(it) }
|
||||
}
|
||||
}, "runtime-install").start()
|
||||
}
|
||||
@@ -1288,7 +1364,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
}
|
||||
|
||||
override fun surfaceCreated(holder: SurfaceHolder) {
|
||||
if (installingRuntime || SessionState.stopRequested) return
|
||||
if (holder !== outputHolder()) return
|
||||
if (installingRuntime || SessionState.stopRequested || ended) return
|
||||
if (!SessionState.running) {
|
||||
SessionEvents.transition(SessionPhase.STARTING_COMPOSITOR, "compositor.starting")
|
||||
}
|
||||
@@ -1304,7 +1381,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
val live = if (SessionState.followScreen && !isInPictureInPictureMode)
|
||||
com.droiddeck.launcher.session.SessionDisplay.followSize(holder.surfaceFrame.width(), holder.surfaceFrame.height())
|
||||
else null
|
||||
val size = live ?: if (SessionState.running) previous else outputSize()
|
||||
val size = live ?: if (SessionState.running) previous else glassesOutputSize() ?: outputSize()
|
||||
SessionState.outputSize = size
|
||||
if (!SessionState.running) SessionState.refreshHz = refreshHz()
|
||||
val panel = panelBounds()
|
||||
@@ -1317,7 +1394,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
WaylandCompositor.nativeSetUpscaler(SessionPrefs.upscaler(this))
|
||||
WaylandCompositor.nativeSetUpscaleSharpness(SessionPrefs.upscaleSharpness(this))
|
||||
SessionPrefs.screenEffects(this).push(SessionPrefs.upscaler(this))
|
||||
panelSizeMm().let { (w, h) -> SessionPrefs.cardboard(this).push(w, h) }
|
||||
activeCardboard = effectiveCardboard(SessionPrefs.cardboard(this))
|
||||
pushCardboard(activeCardboard)
|
||||
WaylandCompositor.nativeSetCardboardPoseFile(File(filesDir, ".wayland-rt/droiddeck-headpose").path)
|
||||
if (!SessionState.running) {
|
||||
val waylandLog = File(SessionPaths.beginOrCurrent(this), "wayland.log")
|
||||
@@ -1330,7 +1408,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
}
|
||||
if (!CompositorHost.isStarted) {
|
||||
val mode = SessionPrefs.prefMode(intent.getStringExtra(SessionService.EXTRA_MODE) ?: SessionService.MODE_STEAM)
|
||||
val probe = HdrSupport.probe(this)
|
||||
val probe = HdrSupport.probe(this, glassesScreen?.display)
|
||||
WaylandCompositor.nativeSetHdrDisplay(
|
||||
probe.displayId, probe.name, probe.formats, probe.hdr10,
|
||||
probe.maxLuminance, probe.maxAverageLuminance, probe.minLuminance,
|
||||
@@ -1354,8 +1432,9 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
CompositorHost.startOrAttach(
|
||||
holder.surface, runtimeDir.path,
|
||||
driverId?.let { turnip.driverPath(it) }, driverId?.let { turnip.libraryName(it) },
|
||||
applicationInfo.nativeLibraryDir, size.first, size.second, refreshHz(), SessionState.fpsLimit,
|
||||
applicationInfo.nativeLibraryDir, size.first, size.second, refreshHz(), outputFpsLimit(),
|
||||
)
|
||||
onOutputAttached(holder)
|
||||
if (SessionState.running && size != previous) {
|
||||
Log.i(TAG, "follow screen: ${previous.first}x${previous.second} -> ${size.first}x${size.second} on a new surface")
|
||||
WaylandCompositor.nativeResizeOutput(size.first, size.second)
|
||||
@@ -1379,6 +1458,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
private var surfaceH = 0
|
||||
|
||||
override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {
|
||||
if (holder !== outputHolder()) return
|
||||
val resized = surfaceW != 0 && (width != surfaceW || height != surfaceH)
|
||||
surfaceW = width
|
||||
surfaceH = height
|
||||
@@ -1422,8 +1502,12 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
}
|
||||
|
||||
override fun surfaceDestroyed(holder: SurfaceHolder) {
|
||||
// Hiding the phone surface after a handoff delivers a late destruction
|
||||
// callback. It must not reset the new output's size or attachment.
|
||||
if (holder !== outputHolder()) return
|
||||
surfaceW = 0
|
||||
surfaceH = 0
|
||||
glassesOutputActive = false
|
||||
CompositorHost.detach(holder.surface)
|
||||
}
|
||||
|
||||
@@ -1448,14 +1532,32 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
}
|
||||
|
||||
private fun refreshHz(): Float {
|
||||
glassesScreen?.hz?.takeIf { it > 1f }?.let { return it }
|
||||
val display = if (Build.VERSION.SDK_INT >= 30) display else windowManager.defaultDisplay
|
||||
val wanted = window.attributes.preferredDisplayModeId
|
||||
val hz = display?.supportedModes?.firstOrNull { wanted != 0 && it.modeId == wanted }?.refreshRate
|
||||
?: display?.refreshRate ?: 60f
|
||||
val capped = minOf(hz, PerfMode.peakRefreshCap(this))
|
||||
val mirrorCap = mirrorRefreshLimit().takeIf { it > 0 }?.toFloat() ?: Float.POSITIVE_INFINITY
|
||||
val capped = minOf(hz, PerfMode.peakRefreshCap(this), mirrorCap)
|
||||
return if (capped > 1f) capped else 60f
|
||||
}
|
||||
|
||||
private fun mirrorRefreshLimit(): Int =
|
||||
GlassesRendering.mirrorRefreshLimit(glassesSettings, glasses, glassesScreen != null)
|
||||
|
||||
private fun outputFpsLimit(): Int = GlassesRendering.fpsLimit(SessionState.fpsLimit, mirrorRefreshLimit())
|
||||
|
||||
private fun applyOutputPacing() {
|
||||
Log.i(TAG, "perf: " + PerfMode.apply(this, outputFpsLimit(), mirrorRefreshLimit()))
|
||||
if (CompositorHost.isStarted) {
|
||||
val hz = refreshHz()
|
||||
val changed = kotlin.math.abs(SessionState.refreshHz - hz) > 0.1f
|
||||
CompositorHost.updatePacing(hz, outputFpsLimit())
|
||||
pushCardboard(activeCardboard)
|
||||
if (changed && ::surfaceView.isInitialized) applyFrameGen()
|
||||
}
|
||||
}
|
||||
|
||||
private fun applyFrameGen() {
|
||||
val hz = refreshHz()
|
||||
Thread({
|
||||
@@ -1514,11 +1616,14 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
}
|
||||
|
||||
private fun onSessionEnded(status: Int) {
|
||||
ended = true
|
||||
if (isInPictureInPictureMode) {
|
||||
runOnUiThread { finish() }
|
||||
return
|
||||
}
|
||||
closeSecondScreen(reset = true)
|
||||
updateGlassesScreen()
|
||||
updateHeadTracker()
|
||||
if (status == 0) {
|
||||
runOnUiThread {
|
||||
if (isFinishing || isDestroyed) return@runOnUiThread
|
||||
@@ -1560,9 +1665,14 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
focusEndedScreen()
|
||||
}
|
||||
|
||||
private fun releasePads() {
|
||||
padBridge?.releaseAll()
|
||||
joyConVr?.releaseButtons()
|
||||
}
|
||||
|
||||
private fun focusEndedScreen() {
|
||||
drawerOpen = false
|
||||
padBridge?.releaseAll()
|
||||
releasePads()
|
||||
uiHandler.post { sessionOverlay.requestFocus() }
|
||||
}
|
||||
|
||||
@@ -1610,6 +1720,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
return super.dispatchKeyEvent(event)
|
||||
}
|
||||
if (pipUi) return true
|
||||
val pad = com.droiddeck.launcher.input.JoyCon.normalize(event)
|
||||
if (pad !== event) return dispatchKeyEvent(pad)
|
||||
val fromController = PadBridge.isControllerKeyEvent(event)
|
||||
if (fromController && event.action == KeyEvent.ACTION_DOWN) {
|
||||
if (drawerOpen && !drawerControllerActive) sessionOverlay.requestFocus()
|
||||
@@ -1632,7 +1744,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
if (drawerOpen) {
|
||||
if (fromController && (event.keyCode == KeyEvent.KEYCODE_BUTTON_L1 || event.keyCode == KeyEvent.KEYCODE_BUTTON_R1)) {
|
||||
if (event.action == KeyEvent.ACTION_DOWN && event.repeatCount == 0) {
|
||||
drawerPage = sessionDrawerPages(SessionState.mode == SessionService.MODE_STEAM && secondScreenDisplays.isNotEmpty())
|
||||
drawerPage = sessionDrawerPages(SessionState.mode == SessionService.MODE_STEAM && secondScreenDisplays.isNotEmpty(), glasses.present)
|
||||
.step(drawerPage, if (event.keyCode == KeyEvent.KEYCODE_BUTTON_R1) 1 else -1)
|
||||
releaseDrawerDirection()
|
||||
}
|
||||
@@ -1661,6 +1773,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
return true
|
||||
}
|
||||
if ((pcKeyboardOpen || achievementsOpen) && fromController) return super.dispatchKeyEvent(event)
|
||||
if (joyConVr?.onKeyEvent(event) == true) return true
|
||||
if (event.keyCode != KeyEvent.KEYCODE_BACK && padBridge?.onKeyEvent(event) == true) return true
|
||||
if (CompositorHost.isStarted && event.keyCode != KeyEvent.KEYCODE_BACK && !fromController) {
|
||||
val down = event.action == KeyEvent.ACTION_DOWN
|
||||
@@ -1704,6 +1817,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
}
|
||||
if (drawerDirectionKey != KeyEvent.KEYCODE_UNKNOWN) releaseDrawerDirection()
|
||||
if ((pcKeyboardOpen || achievementsOpen) && PadBridge.isControllerMotionEvent(event)) return super.dispatchGenericMotionEvent(event)
|
||||
if (joyConVr?.onMotionEvent(event) == true) return true
|
||||
if (padBridge?.onMotionEvent(event) == true) return true
|
||||
if (event.isFromSource(android.view.InputDevice.SOURCE_MOUSE) && !drawerOpen && onMouse(event)) return true
|
||||
return super.dispatchGenericMotionEvent(event)
|
||||
@@ -1868,6 +1982,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
if (!::displayManager.isInitialized) return
|
||||
val primaryId = display?.displayId ?: windowManager.defaultDisplay.displayId
|
||||
val candidates = SecondScreenDisplays.available(displayManager, primaryId)
|
||||
.filter { glassesScreen == null || it.id != glasses.displayId }
|
||||
val oldIds = secondScreenDisplays.map { it.id }.toSet()
|
||||
secondScreenDisplays = candidates
|
||||
val newIds = candidates.map { it.id }.toSet()
|
||||
@@ -2219,6 +2334,17 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
}
|
||||
|
||||
private fun updateOnScreenControls() {
|
||||
updateOnScreenPad()
|
||||
val c = onScreenControls
|
||||
glassesSlot = when {
|
||||
c == null || c.visibility != View.VISIBLE -> 0
|
||||
c.isButtonsOnly -> -1
|
||||
controllerSettings?.keyboardButton != false -> 2
|
||||
else -> 1
|
||||
}
|
||||
}
|
||||
|
||||
private fun updateOnScreenPad() {
|
||||
val forced = File(Environment.getExternalStorageDirectory(), "Download/droiddeck-osc")
|
||||
.takeIf { it.isFile }
|
||||
?.let { FileUtils.readString(it)?.trim()?.lowercase() }
|
||||
@@ -2301,7 +2427,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
pendingBackAction = null
|
||||
releaseDrawerDirection()
|
||||
onScreenControls?.releaseAll()
|
||||
padBridge?.releaseAll()
|
||||
releasePads()
|
||||
touchpad.cancel()
|
||||
guestKeysDown.toList().forEach { sendGuestKey(it, false) }
|
||||
keyboard?.hide()
|
||||
@@ -2337,6 +2463,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
if (::pip.isInitialized) pip.changed(inPip)
|
||||
if (!inPip && surfaceW > 0) followWindow(surfaceW, surfaceH)
|
||||
applyUnfoldedControls()
|
||||
updateGlassesScreen()
|
||||
}
|
||||
|
||||
override fun startActivityForResult(intent: Intent, requestCode: Int, options: Bundle?) {
|
||||
@@ -2355,13 +2482,18 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
refreshSecondScreenDisplays()
|
||||
if (secondScreenMode != SecondScreenMode.NONE && secondScreenPresentation == null) showSecondScreen(secondScreenMode)
|
||||
SessionService.setActivityVisible(this, true)
|
||||
Glasses.rescan()
|
||||
updateGlassesScreen()
|
||||
updateHandCamera()
|
||||
}
|
||||
|
||||
override fun onStop() {
|
||||
if (runtimeRemovalBlocked) { super.onStop(); return }
|
||||
started = false
|
||||
updateHandCamera()
|
||||
displayManager.unregisterDisplayListener(displayListener)
|
||||
closeSecondScreen(reset = false)
|
||||
updateGlassesScreen()
|
||||
SessionService.setActivityVisible(this, false)
|
||||
super.onStop()
|
||||
}
|
||||
@@ -2379,6 +2511,10 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
}
|
||||
if (this.intent.getBooleanExtra(EXTRA_RETURN_HOME, false)) intent.putExtra(EXTRA_RETURN_HOME, true)
|
||||
setIntent(intent)
|
||||
if (intent.getBooleanExtra(EXTRA_STEAMVR, false) && SessionState.mode == SessionService.MODE_STEAM) {
|
||||
SessionState.steamVr = true
|
||||
steamVrAutoTried = false
|
||||
}
|
||||
if (intent.action == SessionService.ACTION_HOME_GUIDE) {
|
||||
handleHomeGuideIntent(intent)
|
||||
} else if (intent.action == SessionService.ACTION_RESUME) {
|
||||
@@ -2400,6 +2536,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
(getSystemService(INPUT_SERVICE) as? InputManager)
|
||||
?.registerInputDeviceListener(deviceListener, Handler(Looper.getMainLooper()))
|
||||
if (SessionState.guestPid > 1) padBridge?.syncConnectedControllers()
|
||||
joyConVr?.refresh()
|
||||
onScreenControls?.reload()
|
||||
controllerSettings = ControllerPrefs.read(this)
|
||||
resumed = true
|
||||
@@ -2436,13 +2573,32 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
startActivity(android.content.Intent(this, QuestXrActivity::class.java))
|
||||
return
|
||||
}
|
||||
if (action == "start") {
|
||||
SessionState.steamVr = true
|
||||
steamVrAutoTried = true
|
||||
}
|
||||
if (action == "stop") SessionState.steamVr = false
|
||||
if (glasses.present) {
|
||||
// Optional stereo starts only after RUNNING is observed. READY can
|
||||
// arrive repeatedly while the launch worker is still starting VR.
|
||||
if (action == "install") steamVrStatus = SteamVr.Status.INSTALLING
|
||||
steamVrWorker.execute {
|
||||
val ok = SteamVr.run(this, action)
|
||||
val now = SteamVr.status(this)
|
||||
uiHandler.post {
|
||||
onSteamVrStatus(now)
|
||||
if (!ok) Toast.makeText(this, getString(R.string.cardboard_steamvr_failed), Toast.LENGTH_LONG).show()
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
if (action == "start") {
|
||||
switchPictureForSteamVr()
|
||||
steamVrSeenRunning = false
|
||||
}
|
||||
if (action == "stop") restoreSteamVrPicture()
|
||||
if (action == "install") steamVrStatus = SteamVr.Status.INSTALLING
|
||||
Thread({
|
||||
steamVrWorker.execute {
|
||||
val ok = SteamVr.run(this, action)
|
||||
val now = SteamVr.status(this)
|
||||
uiHandler.post {
|
||||
@@ -2450,7 +2606,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
onSteamVrStatus(now)
|
||||
if (!ok) Toast.makeText(this, getString(R.string.cardboard_steamvr_failed), Toast.LENGTH_LONG).show()
|
||||
}
|
||||
}, "steamvr-$action").start()
|
||||
}
|
||||
}
|
||||
|
||||
private fun switchPictureForSteamVr() {
|
||||
@@ -2463,8 +2619,41 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
steamVrSwitchedPicture = SessionPrefs.cardboardSourceBeforeSteamVr(this) >= 0
|
||||
}
|
||||
|
||||
private fun updateHandCamera() {
|
||||
val joyCons = joyConVr?.holding == true
|
||||
val bare = !joyCons && !controllerConnected()
|
||||
val want = started && steamVrStatus == SteamVr.Status.RUNNING && (joyCons || bare) &&
|
||||
com.droiddeck.launcher.handtracking.HandTracking.wanted(this)
|
||||
handCamera?.bareHands = bare
|
||||
if (want == (handCamera != null)) return
|
||||
if (want) {
|
||||
handCamera = com.droiddeck.launcher.handtracking.HandCamera(this,
|
||||
File(filesDir, ".wayland-rt/${com.droiddeck.launcher.handtracking.HandCameraShm.FILE}"), preview = false)
|
||||
.also {
|
||||
it.bareHands = bare
|
||||
it.start(com.droiddeck.launcher.handtracking.HandTracking.settings.value.side)
|
||||
}
|
||||
} else {
|
||||
handCamera?.stop()
|
||||
handCamera = null
|
||||
}
|
||||
}
|
||||
|
||||
private fun controllerConnected(): Boolean = InputDevice.getDeviceIds().any { id ->
|
||||
joyConVr?.owns(id) != true && com.droiddeck.launcher.input.PadBridge.isFromController(InputDevice.getDevice(id))
|
||||
}
|
||||
|
||||
private fun onSteamVrStatus(now: SteamVr.Status) {
|
||||
val was = steamVrStatus
|
||||
steamVrStatus = now
|
||||
joyConVr?.setActive(now == SteamVr.Status.RUNNING && ControllerPrefs.joyConVr(this))
|
||||
updateHandCamera()
|
||||
autoStartSteamVr(now)
|
||||
if ((was == SteamVr.Status.RUNNING) != (now == SteamVr.Status.RUNNING)) applyCardboard(cardboard)
|
||||
if (glasses.present) onGlassesSteamVr(now)
|
||||
if (GlassesRendering.usesGlasses(glasses, glassesScreen != null, now == SteamVr.Status.RUNNING)) {
|
||||
return
|
||||
}
|
||||
if (now == SteamVr.Status.RUNNING && cardboard.enabled && !steamVrSwitchedPicture &&
|
||||
cardboard.source != Cardboard.SOURCE_SBS) {
|
||||
switchPictureForSteamVr()
|
||||
@@ -2487,23 +2676,172 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
}
|
||||
|
||||
private fun applyCardboard(c: Cardboard) {
|
||||
val was = cardboard
|
||||
val was = activeCardboard
|
||||
cardboard = c
|
||||
if (CompositorHost.isStarted) panelSizeMm().let { (w, h) -> c.push(w, h) }
|
||||
if (c.enabled && !was.enabled) headTracker?.recenter()
|
||||
val e = effectiveCardboard(c)
|
||||
activeCardboard = e
|
||||
if (CompositorHost.isStarted) pushCardboard(e)
|
||||
if (e.enabled && !was.enabled) headTracker?.recenter()
|
||||
updateHeadTracker()
|
||||
}
|
||||
|
||||
private fun pushCardboard(c: Cardboard) {
|
||||
val fov = when {
|
||||
c.viewer != Cardboard.Viewer.VITURE.id -> 0f
|
||||
c.source == Cardboard.SOURCE_SBS -> glassesSettings.vrFovDeg.toFloat()
|
||||
else -> glasses.verticalFovDeg
|
||||
}
|
||||
panelSizeMm().let { (w, h) -> c.push(w, h, fov, refreshHz()) }
|
||||
}
|
||||
|
||||
private fun effectiveCardboard(c: Cardboard): Cardboard = GlassesRendering.cardboard(
|
||||
base = c,
|
||||
settings = glassesSettings,
|
||||
state = glasses,
|
||||
outputActive = glassesScreen != null,
|
||||
steamVrRunning = steamVrStatus == SteamVr.Status.RUNNING,
|
||||
)
|
||||
|
||||
private fun updateHeadTracker() {
|
||||
val want = resumed && cardboard.enabled && cardboard.headTracking
|
||||
val onGlasses = GlassesRendering.usesGlasses(glasses, glassesScreen != null,
|
||||
steamVrStatus == SteamVr.Status.RUNNING)
|
||||
val c = activeCardboard
|
||||
val want = !ended && (resumed || (glassesScreen != null && started)) && c.enabled && c.headTracking
|
||||
val current = headTracker
|
||||
if (current != null && (!want || current.usesGlasses != onGlasses)) {
|
||||
current.stop()
|
||||
headTracker = null
|
||||
}
|
||||
if (want) {
|
||||
val t = headTracker ?: HeadTracker(this) {
|
||||
val t = headTracker ?: HeadTracker(this, glasses = onGlasses) {
|
||||
@Suppress("DEPRECATION")
|
||||
(if (Build.VERSION.SDK_INT >= 30) display else windowManager.defaultDisplay)?.rotation ?: android.view.Surface.ROTATION_0
|
||||
}.also { headTracker = it }
|
||||
t.start()
|
||||
} else {
|
||||
headTracker?.stop()
|
||||
}
|
||||
headTracking = headTracker != null
|
||||
}
|
||||
|
||||
private fun outputHolder(): SurfaceHolder = glassesScreen?.holder ?: surfaceView.holder
|
||||
|
||||
private fun onOutputAttached(holder: SurfaceHolder) {
|
||||
if (holder !== outputHolder()) return
|
||||
glassesOutputActive = glassesScreen?.holder === holder
|
||||
// Keep the phone's layout for touch/controller coordinates, but remove
|
||||
// its SurfaceView so it cannot retain the last game frame under the hint.
|
||||
surfaceView.visibility = if (glassesOutputActive) View.INVISIBLE else View.VISIBLE
|
||||
Log.i(TAG, "compositor output attached: " +
|
||||
if (glassesOutputActive) "glasses display ${glassesScreen?.display?.displayId}" else "phone")
|
||||
}
|
||||
|
||||
private fun restorePhoneOutput(screen: GlassesScreen) {
|
||||
if (glassesScreen !== screen) return
|
||||
glassesScreen = null
|
||||
glassesOutputActive = false
|
||||
surfaceW = 0
|
||||
surfaceH = 0
|
||||
CompositorHost.detach(screen.holder.surface)
|
||||
surfaceView.visibility = View.VISIBLE
|
||||
applyOutputPacing()
|
||||
if (surfaceView.holder.surface?.isValid == true) surfaceCreated(surfaceView.holder)
|
||||
}
|
||||
|
||||
private fun glassesOutputSize(): Pair<Int, Int>? {
|
||||
if (glassesScreen == null || glasses.width <= 0 || glasses.height <= 0) return null
|
||||
val panel = (if (glasses.sbs) glasses.width / 2 else glasses.width) to glasses.height
|
||||
val mode = SessionPrefs.prefMode(intent.getStringExtra(SessionService.EXTRA_MODE) ?: SessionService.MODE_STEAM)
|
||||
val display = com.droiddeck.launcher.session.SessionDisplay
|
||||
val choice = SessionPrefs.resolutionChoice(this, mode, panel)
|
||||
val size = display.resolveChoice(panel, choice)
|
||||
val fixed = choice != display.MATCH_SCREEN && choice != display.FOLLOW_SCREEN && display.presetHeight(choice) == null
|
||||
return if (fixed) display.resolve(panel, minOf(size.first, size.second), SessionPrefs.SHAPE_AUTO) else size
|
||||
}
|
||||
|
||||
private fun onGlassesChanged() {
|
||||
val g = glasses
|
||||
val s = glassesSettings
|
||||
val key = listOf(g.present, g.usable, g.displayId, g.sbs, g.sbs90, g.switching3d, g.canControl, g.width, g.height, g.hz, s)
|
||||
if (key == glassesKey) return
|
||||
glassesKey = key
|
||||
updateGlassesScreen()
|
||||
applyCardboard(cardboard)
|
||||
if (!g.switching3d) {
|
||||
if (glassesPresent && !g.present) onGlassesUnplugged()
|
||||
glassesPresent = g.present
|
||||
}
|
||||
}
|
||||
|
||||
private fun onGlassesUnplugged() {
|
||||
val hadVr = steamVrAutoTried
|
||||
steamVrAutoTried = false
|
||||
glasses3dForVr = false
|
||||
if (hadVr && !cardboard.enabled) steamVrAction("stop")
|
||||
}
|
||||
|
||||
private fun autoStartSteamVr(now: SteamVr.Status) {
|
||||
if (now == SteamVr.Status.RUNNING) steamVrAutoTried = true
|
||||
val action = SteamVr.autoAction(now, SessionState.steamVr, loading.visible,
|
||||
SessionState.mode == SessionService.MODE_STEAM, steamVrAutoTried, steamVrInstallTried) ?: return
|
||||
if (action == "start") steamVrAutoTried = true else steamVrInstallTried = true
|
||||
Log.i(TAG, "requested SteamVR $action (glasses=${glasses.model}, output=$glassesOutputActive, control=${glasses.canControl})")
|
||||
steamVrAction(action)
|
||||
}
|
||||
|
||||
private fun updateGlassesScreen() {
|
||||
if (!::surfaceView.isInitialized) return
|
||||
val g = glasses
|
||||
val want = started && !ended && !isInPictureInPictureMode && !runtimeRemovalBlocked && g.usable && glassesSettings.gameOnGlasses
|
||||
val target = if (want) Glasses.glassesDisplay()?.takeIf { it.isValid } else null
|
||||
val current = glassesScreen
|
||||
if (current != null && current.display.displayId == target?.displayId) {
|
||||
current.prefer(Glasses.outputHz())
|
||||
applyOutputPacing()
|
||||
return
|
||||
}
|
||||
if (current != null) {
|
||||
restorePhoneOutput(current)
|
||||
runCatching { current.dismiss() }
|
||||
}
|
||||
if (target != null) {
|
||||
val screen = GlassesScreen(this, target, Glasses.outputHz(), this)
|
||||
// show() can create the surface synchronously. Select its holder
|
||||
// first, otherwise surfaceCreated() rejects the glasses callback.
|
||||
glassesScreen = screen
|
||||
applyOutputPacing()
|
||||
surfaceW = 0
|
||||
surfaceH = 0
|
||||
screen.setOnDismissListener {
|
||||
if (glassesScreen === screen) {
|
||||
restorePhoneOutput(screen)
|
||||
refreshSecondScreenDisplays()
|
||||
applyCardboard(cardboard)
|
||||
}
|
||||
}
|
||||
try {
|
||||
screen.show()
|
||||
} catch (e: Exception) {
|
||||
restorePhoneOutput(screen)
|
||||
runCatching { screen.dismiss() }
|
||||
Log.w(TAG, "could not show the game on ${target.name}", e)
|
||||
return
|
||||
}
|
||||
Log.i(TAG, "requested game output on ${g.model} (display ${target.displayId})")
|
||||
}
|
||||
applyOutputPacing()
|
||||
refreshSecondScreenDisplays()
|
||||
applyCardboard(cardboard)
|
||||
}
|
||||
|
||||
private fun onGlassesSteamVr(now: SteamVr.Status) {
|
||||
val g = glasses
|
||||
if (now == SteamVr.Status.RUNNING) {
|
||||
if (glassesSettings.auto3d && g.can3d && !g.sbs && !glasses3dForVr) {
|
||||
glasses3dForVr = true
|
||||
Glasses.set3dForVr(true)
|
||||
}
|
||||
} else if (glasses3dForVr && now != SteamVr.Status.UNAVAILABLE) {
|
||||
glasses3dForVr = false
|
||||
Glasses.set3dForVr(false)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2575,6 +2913,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
}
|
||||
|
||||
override fun onDestroy() {
|
||||
steamVrRecovery.close()
|
||||
uiHandler.removeCallbacks(steamTouchRetry)
|
||||
if (runtimeRemovalBlocked) { super.onDestroy(); return }
|
||||
watching = false
|
||||
@@ -2583,7 +2922,14 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
pendingBackAction = null
|
||||
if (::pip.isInitialized) pip.destroyed()
|
||||
closeSecondScreen(reset = false)
|
||||
glassesScope.cancel()
|
||||
glassesScreen?.let { screen -> glassesScreen = null; runCatching { screen.dismiss() } }
|
||||
headTracker?.stop()
|
||||
if (::hud.isInitialized) hud.stop()
|
||||
joyConVr?.close()
|
||||
joyConVr = null
|
||||
handCamera?.stop()
|
||||
handCamera = null
|
||||
padBridge?.stop()
|
||||
steamController?.stop()
|
||||
if (SessionState.padBridge === padBridge) SessionState.padBridge = null
|
||||
@@ -2637,6 +2983,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
|
||||
companion object {
|
||||
const val EXTRA_RETURN_HOME = "returnHome"
|
||||
const val EXTRA_STEAMVR = "steamVr"
|
||||
private const val LEAVE_TIMEOUT_MS = 1_500L
|
||||
private const val TAG = "SessionActivity"
|
||||
private const val FEX_MISSING = "droiddeck-fex: x86 Linux programs need"
|
||||
@@ -2650,6 +2997,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
|
||||
private const val DRAWER_SLOW_REPEAT_FLOOR_MS = 240L
|
||||
private const val DRAWER_FAST_REPEAT_INTERVAL_MS = 90L
|
||||
private const val DRAWER_REPEAT_ACCELERATION_MS = 1_400L
|
||||
private val steamVrWorker = java.util.concurrent.ThreadPoolExecutor(0, 1, 30, java.util.concurrent.TimeUnit.SECONDS,
|
||||
java.util.concurrent.LinkedBlockingQueue()) { Thread(it, "steamvr") }
|
||||
private const val SCALE_FIT = 1
|
||||
private const val ALIGN_CENTER = 0
|
||||
private const val ALIGN_PICTURE_TOP = 3
|
||||
|
||||
@@ -92,11 +92,75 @@ class AgentBridgeProvider : ContentProvider() {
|
||||
AgentEnv.override(context, request.optString("name")).put("ok", true).put("command", "override")
|
||||
}
|
||||
"prefs" -> prefs(context, request)
|
||||
"glasses" -> glasses(context, request)
|
||||
"access" -> JSONObject().put("ok", true).put("commands", AgentAccess.commandsAllowed(context))
|
||||
.put("debugBuild", BuildConfig.DEBUG).put("inbox", AgentEnv.inbox(context).absolutePath)
|
||||
else -> error("UNKNOWN_COMMAND", "Unknown agent command '$method'")
|
||||
}
|
||||
|
||||
private fun glasses(context: Context, request: JSONObject): JSONObject {
|
||||
val g = com.droiddeck.launcher.glasses.Glasses
|
||||
when (request.optString("op", "status")) {
|
||||
"status" -> {}
|
||||
"allow" -> g.requestControl()
|
||||
"set" -> {
|
||||
val key = request.optString("key")
|
||||
val value = request.optString("value")
|
||||
fun on() = value == "1" || value.equals("true", true) || value.equals("on", true)
|
||||
val number = value.toIntOrNull()
|
||||
val numeric = setOf("refreshHz", "sbsHz", "brightness", "volume", "picture3d", "screenWidthDm", "screenDistanceDm", "vrFovDeg", "sbsSupport")
|
||||
if (key in numeric && number == null) return error("INVALID_VALUE", "'$key' takes a number")
|
||||
when (key) {
|
||||
"refreshHz" -> g.setRefreshHz(number!!)
|
||||
"sbsHz" -> g.setSbsHz(number!!)
|
||||
"brightness" -> g.setBrightness(number!!)
|
||||
"volume" -> g.setVolume(number!!)
|
||||
"sunblock" -> g.setSunblock(on())
|
||||
"3d" -> g.set3d(on())
|
||||
"gameOnGlasses" -> g.setGameOnGlasses(on())
|
||||
"headTracking" -> g.setHeadTracking(on())
|
||||
"auto3d" -> g.setAuto3d(on())
|
||||
"picture3d" -> g.setPicture3d(number!!)
|
||||
"screenWidthDm" -> g.setScreenWidthDm(number!!)
|
||||
"screenDistanceDm" -> g.setScreenDistanceDm(number!!)
|
||||
"vrFovDeg" -> g.setVrFovDeg(number!!)
|
||||
"sbsSupport" -> g.setSbsSupport(number!!)
|
||||
else -> return error("INVALID_KEY", "Unknown glasses setting '$key'")
|
||||
}
|
||||
}
|
||||
"simulate" -> {
|
||||
fun angle(key: String) = if (request.has(key)) request.optDouble(key).takeIf { it.isFinite() }?.coerceIn(-90.0, 90.0) else null
|
||||
if (!g.simulate(request.optInt("seconds", 20).coerceIn(1, 600), angle("yaw"), angle("pitch"))) return error("BUSY", "Already simulating")
|
||||
}
|
||||
"stop-simulate" -> g.stopSimulation()
|
||||
"calibrate" -> g.startCalibration()
|
||||
"cancel-calibration" -> g.cancelCalibration()
|
||||
"reset-calibration" -> g.resetCalibration()
|
||||
"query" -> { g.query(); Thread.sleep(600) }
|
||||
"mode" -> {
|
||||
val mode = hex(request.optString("mode"), 0xFF) ?: return error("INVALID_VALUE", "mode must be one hex byte, e.g. 0x31")
|
||||
g.sendMode(mode)
|
||||
Thread.sleep(600)
|
||||
}
|
||||
"imu" -> return g.sampleImu(request.optLong("ms", 2000).coerceIn(200, 20_000)).put("ok", true).put("command", "glasses")
|
||||
"raw" -> {
|
||||
AgentAccess.requireCommands(context)
|
||||
val msgId = hex(request.optString("msgId"), 0xFFFF) ?: return error("INVALID_VALUE", "msgId must be a hex number up to 0xFFFF")
|
||||
val digits = request.optString("payload").removePrefix("0x")
|
||||
if (digits.length % 2 != 0 || digits.any { Character.digit(it, 16) < 0 }) return error("INVALID_VALUE", "payload must be whole hex bytes")
|
||||
val payload = digits.chunked(2).map { it.toInt(16).toByte() }.toByteArray()
|
||||
if (!g.sendRaw(msgId, payload)) return error("NOT_CONNECTED", "The glasses' control link is not open")
|
||||
Thread.sleep(300)
|
||||
}
|
||||
else -> return error("INVALID_OP", "Unknown glasses op")
|
||||
}
|
||||
if (request.optString("op") in setOf("set", "allow")) Thread.sleep(400)
|
||||
return g.status().put("ok", true).put("command", "glasses").put("frames", g.recentFrames())
|
||||
}
|
||||
|
||||
private fun hex(text: String, max: Int): Int? =
|
||||
text.removePrefix("0x").removePrefix("0X").takeIf { it.isNotEmpty() }?.toIntOrNull(16)?.takeIf { it in 0..max }
|
||||
|
||||
private fun state(context: Context): JSONObject {
|
||||
val runtimeVersion = LinuxRuntimeInstaller.installedVersion(context)
|
||||
val dir = SessionState.logDirectory ?: SessionState.logFile?.parentFile ?: SessionPaths.sessionFolders(context).lastOrNull()
|
||||
@@ -109,6 +173,7 @@ class AgentBridgeProvider : ContentProvider() {
|
||||
.put("steamUi", SessionState.steamUi ?: JSONObject.NULL)
|
||||
.put("steamUrl", SessionState.steamUrl ?: JSONObject.NULL)
|
||||
.put("suspended", SessionState.suspended)
|
||||
.put("steamVr", SessionState.steamVr)
|
||||
.put("pip", SessionState.pipActive)
|
||||
.put("firstFrame", SessionState.firstFrameSeen)
|
||||
.put("output", JSONArray().put(SessionState.outputSize.first).put(SessionState.outputSize.second))
|
||||
|
||||
@@ -97,6 +97,7 @@ class AgentStartActivity : Activity() {
|
||||
request.optString("steamUrl").takeIf { it.startsWith("steam://") }?.let {
|
||||
putExtra(SessionService.EXTRA_STEAM_URL, it)
|
||||
}
|
||||
if (request.optBoolean("steamVr")) putExtra(SessionActivity.EXTRA_STEAMVR, true)
|
||||
}
|
||||
}
|
||||
startActivity(sessionIntent)
|
||||
|
||||
@@ -0,0 +1,862 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import android.annotation.SuppressLint
|
||||
import android.app.PendingIntent
|
||||
import android.content.BroadcastReceiver
|
||||
import android.content.Context
|
||||
import android.content.Intent
|
||||
import android.content.IntentFilter
|
||||
import android.hardware.display.DisplayManager
|
||||
import android.hardware.usb.UsbDevice
|
||||
import android.hardware.usb.UsbManager
|
||||
import android.os.Build
|
||||
import android.os.Handler
|
||||
import android.os.HandlerThread
|
||||
import android.os.SystemClock
|
||||
import android.util.Log
|
||||
import android.view.Display
|
||||
import kotlinx.coroutines.flow.MutableStateFlow
|
||||
import kotlinx.coroutines.flow.StateFlow
|
||||
import kotlinx.coroutines.flow.asStateFlow
|
||||
import org.json.JSONArray
|
||||
import org.json.JSONObject
|
||||
import kotlin.math.abs
|
||||
import kotlin.math.sin
|
||||
import com.droiddeck.launcher.R
|
||||
import java.io.File
|
||||
|
||||
data class GlassesState(
|
||||
val present: Boolean = false,
|
||||
val model: String = "VITURE",
|
||||
val controllable: Boolean = false,
|
||||
val permission: Glasses.Permission = Glasses.Permission.NONE,
|
||||
val connected: Boolean = false,
|
||||
val displayId: Int = Display.INVALID_DISPLAY,
|
||||
val width: Int = 0,
|
||||
val height: Int = 0,
|
||||
val hz: Float = 0f,
|
||||
val sbs: Boolean = false,
|
||||
val switching3d: Boolean = false,
|
||||
val brightness: Int = -1,
|
||||
val volume: Int = -1,
|
||||
val brightnessMax: Int = 8,
|
||||
val volumeMax: Int = 8,
|
||||
val sunblock: Boolean = false,
|
||||
val hardwareControls: Boolean = true,
|
||||
val hasSunblock: Boolean = true,
|
||||
val controlError: String? = null,
|
||||
val tracking: Boolean = false,
|
||||
val diagonalFovDeg: Float = 46f,
|
||||
val noSignal: Boolean = false,
|
||||
val sbsSupport: Int = SBS_UNKNOWN,
|
||||
val sbs90: Boolean = false,
|
||||
val experimental: Boolean = false,
|
||||
) {
|
||||
val hasDisplay: Boolean get() = displayId != Display.INVALID_DISPLAY
|
||||
val usable: Boolean get() = hasDisplay && !noSignal
|
||||
val canControl: Boolean get() = connected && controllable
|
||||
val can3d: Boolean get() = canControl && !noSignal && sbsSupport != SBS_FAILS
|
||||
val verticalFovDeg: Float get() {
|
||||
val half = Math.toRadians(diagonalFovDeg / 2.0)
|
||||
val aspect = if (width > 0 && height > 0)
|
||||
width.toDouble() / (if (sbs) 2 else 1) / height else 16.0 / 9.0
|
||||
val v = kotlin.math.atan(kotlin.math.tan(half) / kotlin.math.sqrt(1 + aspect * aspect))
|
||||
return Math.toDegrees(2 * v).toFloat()
|
||||
}
|
||||
|
||||
companion object {
|
||||
const val SBS_UNKNOWN = 0
|
||||
const val SBS_WORKS = 1
|
||||
const val SBS_FAILS = -1
|
||||
}
|
||||
}
|
||||
|
||||
data class GlassesSettings(
|
||||
val refreshHz: Int = 120,
|
||||
val sbsHz: Int = 60,
|
||||
val sunblock: Boolean? = null,
|
||||
val gameOnGlasses: Boolean = true,
|
||||
val headTracking: Boolean = true,
|
||||
val auto3d: Boolean = false,
|
||||
val picture3d: Int = PICTURE_AUTO,
|
||||
val screenWidthDm: Int = 20,
|
||||
val screenDistanceDm: Int = 30,
|
||||
val vrFovDeg: Int = VR_FOV_DEFAULT,
|
||||
) {
|
||||
companion object {
|
||||
const val VR_FOV_MIN = 24
|
||||
const val VR_FOV_MAX = 90
|
||||
const val VR_FOV_DEFAULT = 90
|
||||
const val PICTURE_AUTO = 0
|
||||
const val PICTURE_SCREEN = 1
|
||||
const val PICTURE_SBS = 2
|
||||
}
|
||||
}
|
||||
|
||||
@SuppressLint("StaticFieldLeak")
|
||||
object Glasses {
|
||||
enum class Permission { NONE, ASKING, DENIED, GRANTED }
|
||||
|
||||
fun interface PoseSink {
|
||||
fun onPose(x: Float, y: Float, z: Float, w: Float, timeNs: Long)
|
||||
}
|
||||
|
||||
private const val TAG = "Glasses"
|
||||
private const val ACTION_PERMISSION = "com.droiddeck.launcher.glasses.USB_PERMISSION"
|
||||
private const val PREFS = "glasses"
|
||||
private const val KEY_SBS_FAILED = "sbsFailed"
|
||||
private const val SBS_CONFIRM_MS = 20_000L
|
||||
|
||||
private val _state = MutableStateFlow(GlassesState())
|
||||
val state: StateFlow<GlassesState> = _state.asStateFlow()
|
||||
private val _settings = MutableStateFlow(GlassesSettings())
|
||||
val settings: StateFlow<GlassesSettings> = _settings.asStateFlow()
|
||||
private val _calibration = MutableStateFlow(GlassesCalibrationState())
|
||||
val calibration: StateFlow<GlassesCalibrationState> = _calibration.asStateFlow()
|
||||
private lateinit var calibrationStore: RayNeoCalibrationStore
|
||||
private var calibrationKey: String? = null
|
||||
private var calibrationRevision = 0L
|
||||
|
||||
private lateinit var app: Context
|
||||
private lateinit var handler: Handler
|
||||
private lateinit var usb: UsbManager
|
||||
private lateinit var displays: DisplayManager
|
||||
@Volatile private var link: GlassesLink? = null
|
||||
@Volatile private var linkGeneration = 0L
|
||||
private var linkDevice: UsbDevice? = null
|
||||
private var model: GlassesModel? = null
|
||||
private var asked = false
|
||||
private var lastModeSentAt = 0L
|
||||
private val modeState = GlassesModeState(SBS_CONFIRM_MS)
|
||||
private val confirmMode = Runnable { refresh() }
|
||||
private var glassesMode = 0
|
||||
private var autoModeSent = false
|
||||
|
||||
@Volatile private var sink: PoseSink? = null
|
||||
@Volatile private var probe: ((ByteArray?, FloatArray?) -> Unit)? = null
|
||||
@Volatile private var latest: FloatArray? = null
|
||||
@Volatile private var imuPackets = 0L
|
||||
private val simulating = java.util.concurrent.atomic.AtomicReference<Thread?>(null)
|
||||
private val frames = ArrayDeque<String>()
|
||||
|
||||
fun init(context: Context) {
|
||||
if (::handler.isInitialized) return
|
||||
app = context.applicationContext
|
||||
usb = app.getSystemService(Context.USB_SERVICE) as UsbManager
|
||||
displays = app.getSystemService(Context.DISPLAY_SERVICE) as DisplayManager
|
||||
_settings.value = load()
|
||||
calibrationStore = RayNeoCalibrationStore(app.getSharedPreferences(PREFS, Context.MODE_PRIVATE))
|
||||
handler = Handler(HandlerThread("droiddeck-glasses").apply { start() }.looper)
|
||||
handler.post {
|
||||
val filter = IntentFilter(ACTION_PERMISSION).apply {
|
||||
addAction(UsbManager.ACTION_USB_DEVICE_ATTACHED)
|
||||
addAction(UsbManager.ACTION_USB_DEVICE_DETACHED)
|
||||
}
|
||||
if (Build.VERSION.SDK_INT >= 33) app.registerReceiver(receiver, filter, null, handler, Context.RECEIVER_NOT_EXPORTED)
|
||||
else app.registerReceiver(receiver, filter, null, handler)
|
||||
displays.registerDisplayListener(displayListener, handler)
|
||||
refresh()
|
||||
}
|
||||
}
|
||||
|
||||
fun rescan() = post { refresh() }
|
||||
|
||||
fun startCalibration() = post {
|
||||
val l = link as? RayNeoLink ?: return@post
|
||||
if (!l.ready || _calibration.value.running || simulating.get() != null) return@post
|
||||
_calibration.value = _calibration.value.copy(phase = CalibrationPhase.SETTLING, progress = 0)
|
||||
l.startCalibration(++calibrationRevision)
|
||||
updateImu()
|
||||
}
|
||||
|
||||
fun cancelCalibration() = post {
|
||||
(link as? RayNeoLink)?.cancelCalibration(++calibrationRevision, calibrationKey?.let(calibrationStore::load))
|
||||
_calibration.value = _calibration.value.copy(phase = CalibrationPhase.IDLE, progress = 0)
|
||||
updateImu()
|
||||
}
|
||||
|
||||
fun resetCalibration() = post {
|
||||
(link as? RayNeoLink)?.resetCalibration(++calibrationRevision)
|
||||
calibrationKey?.let { calibrationStore.save(it, null) }
|
||||
_calibration.value = _calibration.value.copy(saved = false, phase = CalibrationPhase.IDLE, progress = 0)
|
||||
updateImu()
|
||||
}
|
||||
|
||||
fun query() = post {
|
||||
val l = link ?: return@post
|
||||
listOf(VitureProtocol.MSG_GET_DISPLAY, VitureProtocol.MSG_GET_BRIGHTNESS, VitureProtocol.MSG_GET_VOLUME, VitureProtocol.MSG_GET_IMU).forEachIndexed { i, msg ->
|
||||
handler.postDelayed({ if (link === l) l.send(msg) }, VitureLink.COMMAND_GAP_MS * i)
|
||||
}
|
||||
}
|
||||
|
||||
fun sendMode(mode: Int) = post {
|
||||
if (!_state.value.canControl) return@post
|
||||
if (link?.send(VitureProtocol.MSG_DISPLAY, byteArrayOf(mode.toByte())) == true) lastModeSentAt = SystemClock.uptimeMillis()
|
||||
}
|
||||
|
||||
fun requestControl() = post {
|
||||
asked = false
|
||||
refresh(ask = true)
|
||||
}
|
||||
|
||||
fun setRefreshHz(hz: Int) = post {
|
||||
update { it.copy(refreshHz = hz.coerceIn(60, 120)) }
|
||||
if (!_state.value.sbs) applyMode(force = true)
|
||||
}
|
||||
|
||||
fun setSbsHz(hz: Int) = post {
|
||||
update { it.copy(sbsHz = if (hz >= 90) 90 else 60) }
|
||||
if (_state.value.sbs && _state.value.can3d) sendMode(true)
|
||||
}
|
||||
|
||||
fun set3d(on: Boolean) = post {
|
||||
val s = _state.value
|
||||
if (!s.canControl || (on && s.noSignal)) return@post
|
||||
modeState.manual()
|
||||
if (on) {
|
||||
markSbsFailed(linkDevice?.productId, false)
|
||||
rememberSbs(GlassesState.SBS_UNKNOWN)
|
||||
}
|
||||
sendMode(on)
|
||||
}
|
||||
|
||||
fun set3dForVr(on: Boolean) = post {
|
||||
val s = _state.value
|
||||
if (on) {
|
||||
if (!s.can3d || s.sbs || modeState.pendingStereo) return@post
|
||||
if (sendMode(true)) modeState.startVr()
|
||||
} else if (modeState.stopVr() && s.canControl) {
|
||||
sendMode(false)
|
||||
}
|
||||
}
|
||||
|
||||
fun setBrightness(level: Int) = post {
|
||||
val s = _state.value
|
||||
if (!s.canControl || s.brightnessMax == 0) return@post
|
||||
val v = level.coerceIn(0, s.brightnessMax)
|
||||
if (link?.send(VitureProtocol.MSG_BRIGHTNESS, byteArrayOf(v.toByte(), 0)) == true && link !is XrealLink)
|
||||
publish { it.copy(brightness = v) }
|
||||
}
|
||||
|
||||
fun setVolume(level: Int) = post {
|
||||
val s = _state.value
|
||||
if (!s.canControl || s.volumeMax == 0) return@post
|
||||
val v = level.coerceIn(0, s.volumeMax)
|
||||
if (link?.send(VitureProtocol.MSG_VOLUME, byteArrayOf(v.toByte(), 0)) == true) publish { it.copy(volume = v) }
|
||||
}
|
||||
|
||||
fun setSunblock(on: Boolean) = post {
|
||||
update { it.copy(sunblock = on) }
|
||||
sendFilm(on)
|
||||
}
|
||||
|
||||
fun setSbsSupport(support: Int) = post {
|
||||
val s = support.coerceIn(GlassesState.SBS_FAILS, GlassesState.SBS_WORKS)
|
||||
markSbsFailed(linkDevice?.productId, s == GlassesState.SBS_FAILS)
|
||||
rememberSbs(s)
|
||||
}
|
||||
fun setGameOnGlasses(on: Boolean) = post { update { it.copy(gameOnGlasses = on) } }
|
||||
fun setHeadTracking(on: Boolean) = post { update { it.copy(headTracking = on) } }
|
||||
fun setAuto3d(on: Boolean) = post { update { it.copy(auto3d = on) } }
|
||||
fun setPicture3d(picture: Int) = post { update { it.copy(picture3d = picture.coerceIn(0, 2)) } }
|
||||
fun setScreenWidthDm(dm: Int) = post { update { it.copy(screenWidthDm = dm.coerceIn(5, 200)) } }
|
||||
fun setScreenDistanceDm(dm: Int) = post { update { it.copy(screenDistanceDm = dm.coerceIn(5, 200)) } }
|
||||
fun setVrFovDeg(deg: Int) = post { update { it.copy(vrFovDeg = deg.coerceIn(GlassesSettings.VR_FOV_MIN, GlassesSettings.VR_FOV_MAX)) } }
|
||||
|
||||
fun outputHz(): Int = GlassesRendering.outputHz(_settings.value, _state.value)
|
||||
|
||||
fun glassesDisplay(): Display? = _state.value.displayId.takeIf { it != Display.INVALID_DISPLAY }?.let { displays.getDisplay(it) }
|
||||
|
||||
fun setPoseSink(s: PoseSink) {
|
||||
synchronized(this) { sink = s }
|
||||
post { updateImu() }
|
||||
}
|
||||
|
||||
fun clearPoseSink(s: PoseSink) {
|
||||
synchronized(this) { if (sink === s) sink = null }
|
||||
post { updateImu() }
|
||||
}
|
||||
|
||||
fun simulate(seconds: Int, holdYaw: Double? = null, holdPitch: Double? = null): Boolean {
|
||||
val hold = holdYaw != null || holdPitch != null
|
||||
val t = Thread({
|
||||
val start = SystemClock.elapsedRealtimeNanos()
|
||||
val end = start + seconds * 1_000_000_000L
|
||||
while (SystemClock.elapsedRealtimeNanos() < end && simulating.get() === Thread.currentThread()) {
|
||||
val now = SystemClock.elapsedRealtimeNanos()
|
||||
val t = (now - start) / 1e9
|
||||
val yaw = if (hold) holdYaw ?: 0.0 else 40.0 * sin(t * 2 * Math.PI / 6.0)
|
||||
val pitch = if (hold) holdPitch ?: 0.0 else 15.0 * sin(t * 2 * Math.PI / 4.0)
|
||||
val roll = if (hold) 0.0 else 5.0 * sin(t * 2 * Math.PI / 5.0)
|
||||
val q = VitureProtocol.eulerZxy(roll.toFloat(), pitch.toFloat(), yaw.toFloat())
|
||||
deliver(q, now)
|
||||
SystemClock.sleep(4)
|
||||
}
|
||||
simulating.compareAndSet(Thread.currentThread(), null)
|
||||
}, "glasses-simulate")
|
||||
if (!simulating.compareAndSet(null, t)) return false
|
||||
t.start()
|
||||
return true
|
||||
}
|
||||
|
||||
fun stopSimulation() {
|
||||
simulating.set(null)
|
||||
}
|
||||
|
||||
fun status(): JSONObject {
|
||||
val s = _state.value
|
||||
val g = _settings.value
|
||||
val q = latest
|
||||
return JSONObject()
|
||||
.put("present", s.present).put("model", s.model).put("controllable", s.controllable)
|
||||
.put("permission", s.permission.name).put("connected", s.connected)
|
||||
.put("controlError", s.controlError ?: JSONObject.NULL)
|
||||
.put("experimental", s.experimental)
|
||||
.put("display", if (s.hasDisplay) JSONObject().put("id", s.displayId).put("width", s.width).put("height", s.height).put("hz", s.hz.toDouble()) else JSONObject.NULL)
|
||||
.put("sbs", s.sbs).put("deviceMode", glassesMode).put("switching3d", s.switching3d)
|
||||
.put("brightness", s.brightness).put("volume", s.volume)
|
||||
.put("brightnessMax", s.brightnessMax).put("volumeMax", s.volumeMax).put("sunblock", s.sunblock)
|
||||
.put("tracking", s.tracking).put("simulating", simulating.get() != null).put("imuPackets", imuPackets)
|
||||
.put("calibration", JSONObject().put("supported", _calibration.value.supported)
|
||||
.put("saved", _calibration.value.saved).put("phase", _calibration.value.phase.name)
|
||||
.put("progress", _calibration.value.progress))
|
||||
.put("badFrames", link?.badFrames ?: 0L)
|
||||
.put("pose", q?.let { JSONArray().put(it[0].toDouble()).put(it[1].toDouble()).put(it[2].toDouble()).put(it[3].toDouble()) } ?: JSONObject.NULL)
|
||||
.put("fovDeg", s.diagonalFovDeg.toDouble()).put("noSignal", s.noSignal).put("sbsSupport", s.sbsSupport)
|
||||
.put("settings", JSONObject().put("refreshHz", g.refreshHz).put("sbsHz", g.sbsHz).put("sunblock", g.sunblock ?: JSONObject.NULL)
|
||||
.put("gameOnGlasses", g.gameOnGlasses).put("headTracking", g.headTracking).put("auto3d", g.auto3d)
|
||||
.put("picture3d", g.picture3d).put("screenWidthDm", g.screenWidthDm).put("screenDistanceDm", g.screenDistanceDm)
|
||||
.put("vrFovDeg", g.vrFovDeg))
|
||||
}
|
||||
|
||||
fun recentFrames(): JSONArray = synchronized(frames) { JSONArray(frames.toList()) }
|
||||
|
||||
fun sendRaw(msgId: Int, payload: ByteArray): Boolean {
|
||||
val l = link ?: return false
|
||||
return l.send(msgId, payload)
|
||||
}
|
||||
|
||||
private fun post(block: () -> Unit) {
|
||||
if (!::handler.isInitialized) return
|
||||
handler.post(block)
|
||||
}
|
||||
|
||||
private fun publish(transform: (GlassesState) -> GlassesState) {
|
||||
_state.value = transform(_state.value)
|
||||
}
|
||||
|
||||
private fun update(transform: (GlassesSettings) -> GlassesSettings) {
|
||||
val next = transform(_settings.value)
|
||||
_settings.value = next
|
||||
save(next)
|
||||
}
|
||||
|
||||
private val receiver = object : BroadcastReceiver() {
|
||||
override fun onReceive(context: Context, intent: Intent) {
|
||||
when (intent.action) {
|
||||
ACTION_PERMISSION -> {
|
||||
val granted = intent.getBooleanExtra(UsbManager.EXTRA_PERMISSION_GRANTED, false)
|
||||
Log.i(TAG, "USB permission ${if (granted) "granted" else "denied"}")
|
||||
if (!granted) publish { it.copy(permission = Permission.DENIED) }
|
||||
refresh()
|
||||
}
|
||||
UsbManager.ACTION_USB_DEVICE_ATTACHED -> {
|
||||
if (intent.device()?.let { isGlassesVendor(it.vendorId) } == true) asked = false
|
||||
refresh(ask = true)
|
||||
}
|
||||
UsbManager.ACTION_USB_DEVICE_DETACHED -> if (intent.device()?.let { isGlassesVendor(it.vendorId) } == true) {
|
||||
refresh()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private val displayListener = object : DisplayManager.DisplayListener {
|
||||
override fun onDisplayAdded(displayId: Int) = refresh()
|
||||
override fun onDisplayRemoved(displayId: Int) = refresh()
|
||||
override fun onDisplayChanged(displayId: Int) {
|
||||
if (displayId == _state.value.displayId || displayId != Display.DEFAULT_DISPLAY) refresh()
|
||||
}
|
||||
}
|
||||
|
||||
private fun refresh(ask: Boolean = false) {
|
||||
val devices = usb.deviceList.values.filter {
|
||||
it.vendorId == VitureProtocol.VENDOR_ID || deviceModel(it) != null
|
||||
}
|
||||
val known = devices.mapNotNull { device ->
|
||||
deviceModel(device)?.let { device to it }
|
||||
}
|
||||
val selected = known.firstOrNull { canOpenModel(it.second) } ?: known.firstOrNull()
|
||||
val device = selected?.first
|
||||
val m = selected?.second
|
||||
val controllable = m?.let(::canOpenModel) == true
|
||||
// XREAL uses Android's Ethernet sockets, without opening or claiming USB interfaces.
|
||||
val permitted = controllable && device != null && (m === XrealModel || usb.hasPermission(device))
|
||||
val identity = device?.let {
|
||||
GlassesModeState.Device(it.productId, if (permitted) runCatching { it.serialNumber }.getOrNull() else null)
|
||||
}
|
||||
if (modeState.updateDevice(identity, SystemClock.uptimeMillis())) {
|
||||
rememberSbs(if (identity != null && sbsFailed(identity.productId)) GlassesState.SBS_FAILS else GlassesState.SBS_UNKNOWN)
|
||||
}
|
||||
if (device == null || (linkDevice != null && linkDevice?.deviceName != device.deviceName)) closeLink()
|
||||
model = m
|
||||
val display = findDisplay(devices.isNotEmpty())
|
||||
val mode = display?.mode
|
||||
val width = mode?.physicalWidth ?: 0
|
||||
val height = mode?.physicalHeight ?: 0
|
||||
val sbsByDisplay = width >= 3000
|
||||
val noSignal = display != null && width in 1 until 1280
|
||||
when (modeState.observe(sbsByDisplay, SystemClock.uptimeMillis())) {
|
||||
GlassesModeState.Confirmation.FAILED -> {
|
||||
Log.w(TAG, "3D signal was not confirmed at ${width}x$height; restoring 2D")
|
||||
if (noSignal) markSbsFailed(device?.productId, true)
|
||||
rememberSbs(GlassesState.SBS_FAILS)
|
||||
}
|
||||
else -> Unit
|
||||
}
|
||||
if (sbsByDisplay) {
|
||||
markSbsFailed(device?.productId, false)
|
||||
rememberSbs(GlassesState.SBS_WORKS)
|
||||
}
|
||||
if (controllable && device != null && !permitted && (ask || !asked) && _state.value.permission != Permission.ASKING) askPermission(device)
|
||||
if (device != null && permitted && link == null) openLink(device)
|
||||
publish {
|
||||
it.copy(
|
||||
present = device != null || display != null,
|
||||
model = m?.name ?: display?.let(::displayName) ?: "VITURE",
|
||||
controllable = controllable,
|
||||
permission = when {
|
||||
!controllable -> Permission.NONE
|
||||
permitted -> Permission.GRANTED
|
||||
it.permission == Permission.ASKING || it.permission == Permission.DENIED -> it.permission
|
||||
else -> Permission.NONE
|
||||
},
|
||||
connected = link?.ready == true,
|
||||
controlError = when {
|
||||
m is VitureProtocol.Model && m.needsSdk && !VitureSdk.available -> app.getString(R.string.glasses_driver_missing)
|
||||
device == null || link?.ready == true -> null
|
||||
else -> it.controlError
|
||||
},
|
||||
displayId = display?.displayId ?: Display.INVALID_DISPLAY,
|
||||
width = width, height = height, hz = mode?.refreshRate ?: 0f,
|
||||
sbs = sbsByDisplay,
|
||||
switching3d = modeState.pendingStereo,
|
||||
brightnessMax = m?.brightnessMax ?: 8, volumeMax = m?.volumeMax ?: 8,
|
||||
sunblock = if (m === XrealModel) it.sunblock else _settings.value.sunblock ?: it.sunblock,
|
||||
hardwareControls = device?.vendorId == VitureProtocol.VENDOR_ID || m === XrealModel,
|
||||
hasSunblock = m?.hasSunblock == true,
|
||||
diagonalFovDeg = m?.diagonalFovDeg ?: 46f,
|
||||
tracking = it.tracking && link != null,
|
||||
noSignal = noSignal,
|
||||
sbs90 = m?.sbs90 == true,
|
||||
experimental = m === XrealModel,
|
||||
)
|
||||
}
|
||||
if (link != null) applyMode(force = false)
|
||||
scheduleModeCheck()
|
||||
}
|
||||
|
||||
internal fun isGlassesVendor(vid: Int) = vid == VitureProtocol.VENDOR_ID || RayNeoModel.isVendor(vid) || vid == XrealModel.VENDOR_ID
|
||||
|
||||
private fun canOpenModel(m: GlassesModel): Boolean = m.controllable &&
|
||||
(m !is VitureProtocol.Model || !m.needsSdk || VitureSdk.available)
|
||||
|
||||
internal fun deviceModel(device: UsbDevice): GlassesModel? {
|
||||
val name = runCatching { device.productName }.getOrNull()
|
||||
return if (device.vendorId == VitureProtocol.VENDOR_ID) {
|
||||
VitureSdk.marketName(device.productId)?.let(VitureProtocol::sdkModel)
|
||||
?: VitureProtocol.model(device.productId, name)
|
||||
}
|
||||
else if (XrealModel.matches(device.vendorId, device.productId)) XrealModel
|
||||
else RayNeoModel.model(device.vendorId, device.productId, name)
|
||||
}
|
||||
|
||||
internal fun isGlassesDisplayName(name: String) =
|
||||
name.contains("VITURE", ignoreCase = true) || RayNeoModel.isDisplay(name) || XrealModel.isDisplay(name)
|
||||
|
||||
private fun findDisplay(usbPresent: Boolean): Display? {
|
||||
return selectGlassesDisplay(displays.getDisplays(DisplayManager.DISPLAY_CATEGORY_PRESENTATION).toList(), usbPresent)
|
||||
}
|
||||
|
||||
internal fun selectGlassesDisplay(displays: List<Display>, usbPresent: Boolean): Display? {
|
||||
val candidates = displays
|
||||
.filter { it.displayId != Display.DEFAULT_DISPLAY && it.flags and Display.FLAG_PRESENTATION != 0 }
|
||||
// Android can expose the glasses name on either the logical display or
|
||||
// its product metadata. A generic label on one must not hide the other.
|
||||
return candidates.firstOrNull { display ->
|
||||
isGlassesDisplayName(display.name) ||
|
||||
(Build.VERSION.SDK_INT >= 31 && display.deviceProductInfo?.let { product ->
|
||||
product.name?.let(::isGlassesDisplayName) == true ||
|
||||
RayNeoModel.isDisplayProduct(product.name, product.manufacturerPnpId, product.productId)
|
||||
} == true)
|
||||
}
|
||||
?: candidates.singleOrNull()?.takeIf { usbPresent && unnamedPanel(it) }
|
||||
}
|
||||
|
||||
private fun unnamedPanel(d: Display): Boolean {
|
||||
if (Build.VERSION.SDK_INT < 31) return true
|
||||
return d.deviceProductInfo?.name.isNullOrBlank()
|
||||
}
|
||||
|
||||
private fun displayName(d: Display): String? {
|
||||
if (Build.VERSION.SDK_INT >= 31) d.deviceProductInfo?.name?.takeIf { it.isNotBlank() }?.let { return it }
|
||||
return d.name
|
||||
}
|
||||
|
||||
private fun askPermission(device: UsbDevice) {
|
||||
asked = true
|
||||
publish { it.copy(permission = Permission.ASKING) }
|
||||
val flags = if (Build.VERSION.SDK_INT >= 31) PendingIntent.FLAG_MUTABLE else 0
|
||||
val pi = PendingIntent.getBroadcast(app, 0, Intent(ACTION_PERMISSION).setPackage(app.packageName), flags)
|
||||
runCatching { usb.requestPermission(device, pi) }.onFailure { Log.w(TAG, "requestPermission", it) }
|
||||
}
|
||||
|
||||
private fun openLink(device: UsbDevice) {
|
||||
val generation = ++linkGeneration
|
||||
calibrationKey = calibrationStore.key(device.vendorId, device.productId,
|
||||
runCatching { device.serialNumber }.getOrNull(), model?.name.orEmpty())
|
||||
val l = if (device.vendorId == VitureProtocol.VENDOR_ID) {
|
||||
if ((model as? VitureProtocol.Model)?.needsSdk == true) {
|
||||
VitureSdkLink.open(usb, device, File(app.cacheDir, "viture"), sdkListener(generation),
|
||||
(model as? VitureProtocol.Model)?.family)
|
||||
} else VitureLink.open(usb, device, linkListener(generation))
|
||||
} else if (model === XrealModel) {
|
||||
XrealLink.open(app, xrealListener(generation))
|
||||
} else {
|
||||
val baseKey = calibrationKey!!
|
||||
val gtUsb = RayNeoProtocol.isGt(device.vendorId, device.productId)
|
||||
RayNeoLink.open(usb, device, rayNeoListener(generation)) { board ->
|
||||
calibrationStore.load(RayNeoCalibrationStore.profileKey(baseKey, gtUsb, board))
|
||||
}
|
||||
}
|
||||
val opened = l ?: run {
|
||||
linkGeneration++
|
||||
Log.w(TAG, "could not open ${device.deviceName}")
|
||||
publish { it.copy(controlError = app.getString(R.string.glasses_connection_failed)) }
|
||||
return
|
||||
}
|
||||
link = opened
|
||||
_calibration.value = GlassesCalibrationState(supported = opened is RayNeoLink,
|
||||
saved = opened is RayNeoLink && calibrationKey?.let(calibrationStore::load) != null)
|
||||
linkDevice = device
|
||||
glassesMode = 0
|
||||
autoModeSent = false
|
||||
Log.i(TAG, "opened ${model?.name} pid=0x%04X imu=%s".format(device.productId, opened.hasImu))
|
||||
publish { it.copy(connected = opened.ready, controlError = null) }
|
||||
listOf(VitureProtocol.MSG_GET_BRIGHTNESS, VitureProtocol.MSG_GET_VOLUME, VitureProtocol.MSG_GET_DISPLAY).forEachIndexed { i, msg ->
|
||||
handler.postDelayed({ if (link === opened) opened.send(msg) }, VitureLink.COMMAND_GAP_MS * (i + 1))
|
||||
}
|
||||
handler.postDelayed({
|
||||
if (link !== opened) return@postDelayed
|
||||
_settings.value.sunblock?.let { sendFilm(it) }
|
||||
updateImu()
|
||||
applyMode(force = false)
|
||||
}, 300)
|
||||
}
|
||||
|
||||
private fun xrealListener(generation: Long) = object : XrealLink.Listener {
|
||||
override fun onReady() = post {
|
||||
if (generation != linkGeneration) return@post
|
||||
publish { it.copy(connected = true, controlError = null) }
|
||||
autoModeSent = false
|
||||
updateImu()
|
||||
applyMode(force = false)
|
||||
}
|
||||
override fun onValue(message: Int, value: Int) = post {
|
||||
if (generation != linkGeneration) return@post
|
||||
when (message) {
|
||||
VitureProtocol.MSG_GET_DISPLAY -> glassesMode = value
|
||||
VitureProtocol.MSG_BRIGHTNESS -> publish { it.copy(brightness = value) }
|
||||
VitureProtocol.MSG_FILM -> publish { it.copy(sunblock = value != 0) }
|
||||
}
|
||||
}
|
||||
override fun onPose(raw: ByteArray, pose: FloatArray, timeNs: Long) {
|
||||
if (generation != linkGeneration) return
|
||||
imuPackets++
|
||||
probe?.invoke(raw, pose)
|
||||
if (simulating.get() == null) deliver(pose, timeNs)
|
||||
}
|
||||
override fun onUnavailable(networkMissing: Boolean) = post {
|
||||
if (generation != linkGeneration) return@post
|
||||
publish { it.copy(connected = false, tracking = false,
|
||||
controlError = app.getString(if (networkMissing) R.string.glasses_xreal_network_missing else R.string.glasses_xreal_connection_failed)) }
|
||||
}
|
||||
}
|
||||
|
||||
private fun sdkListener(generation: Long) = object : VitureSdkLink.Listener {
|
||||
override fun onValue(msgId: Int, value: Int) = post {
|
||||
if (generation != linkGeneration) return@post
|
||||
when (msgId) {
|
||||
VitureProtocol.MSG_GET_DISPLAY -> glassesMode = value
|
||||
VitureProtocol.MSG_GET_BRIGHTNESS -> publish { it.copy(brightness = value) }
|
||||
VitureProtocol.MSG_GET_VOLUME -> publish { it.copy(volume = value) }
|
||||
VitureProtocol.MSG_FILM -> publish { it.copy(sunblock = value != 0) }
|
||||
}
|
||||
}
|
||||
override fun onPose(q: FloatArray, timeNs: Long) {
|
||||
if (generation != linkGeneration) return
|
||||
imuPackets++
|
||||
probe?.invoke(null, q)
|
||||
if (simulating.get() == null) deliver(q, timeNs)
|
||||
}
|
||||
override fun onLost() = post {
|
||||
if (generation != linkGeneration) return@post
|
||||
closeLink()
|
||||
publish { it.copy(controlError = app.getString(R.string.glasses_connection_failed)) }
|
||||
// Let a reconnect or explicit Allow retry restart the SDK, not an endless failure loop.
|
||||
}
|
||||
}
|
||||
|
||||
private fun rayNeoListener(generation: Long) = object : RayNeoLink.Listener {
|
||||
override fun onDeviceInfo(boardId: Int, calibrationSaved: Boolean) = post {
|
||||
if (generation != linkGeneration) return@post
|
||||
val device = linkDevice ?: return@post
|
||||
val base = calibrationStore.key(device.vendorId, device.productId,
|
||||
runCatching { device.serialNumber }.getOrNull(), model?.name.orEmpty())
|
||||
calibrationKey = RayNeoCalibrationStore.profileKey(base,
|
||||
RayNeoProtocol.isGt(device.vendorId, device.productId), boardId)
|
||||
_calibration.value = _calibration.value.copy(saved = calibrationSaved)
|
||||
}
|
||||
override fun onCalibration(revision: Long, phase: CalibrationPhase, progress: Int) = post {
|
||||
if (generation != linkGeneration || revision != calibrationRevision) return@post
|
||||
if (phase != CalibrationPhase.IDLE && !_calibration.value.running) return@post
|
||||
_calibration.value = _calibration.value.copy(phase = phase, progress = progress)
|
||||
if (!_calibration.value.running) updateImu()
|
||||
}
|
||||
override fun onCalibrationSaved(revision: Long, value: RayNeoCalibration?) = post {
|
||||
if (generation != linkGeneration || revision != calibrationRevision) return@post
|
||||
if (value != null && !_calibration.value.running) return@post
|
||||
calibrationKey?.let { calibrationStore.save(it, value) }
|
||||
_calibration.value = _calibration.value.copy(saved = value != null)
|
||||
}
|
||||
override fun onReady() = post {
|
||||
if (generation == linkGeneration) {
|
||||
publish { it.copy(connected = true) }
|
||||
updateImu()
|
||||
applyMode(force = false)
|
||||
}
|
||||
}
|
||||
override fun onPose(raw: ByteArray, xyzw: FloatArray, timeNs: Long) {
|
||||
if (generation != linkGeneration) return
|
||||
imuPackets++
|
||||
probe?.invoke(raw, xyzw)
|
||||
if (simulating.get() == null) deliver(xyzw, timeNs)
|
||||
}
|
||||
override fun onLost() = post {
|
||||
if (generation != linkGeneration) return@post
|
||||
closeLink()
|
||||
refresh()
|
||||
}
|
||||
}
|
||||
|
||||
private fun closeLink() {
|
||||
val l = link ?: return
|
||||
linkGeneration++
|
||||
link = null
|
||||
linkDevice = null
|
||||
calibrationKey = null
|
||||
_calibration.value = GlassesCalibrationState(phase = if (_calibration.value.running) CalibrationPhase.FAILED else CalibrationPhase.IDLE)
|
||||
glassesMode = 0
|
||||
l.close()
|
||||
publish { it.copy(connected = false, tracking = false) }
|
||||
Log.i(TAG, "link closed")
|
||||
}
|
||||
|
||||
private fun rememberSbs(support: Int) {
|
||||
if (_state.value.sbsSupport == support) return
|
||||
publish { it.copy(sbsSupport = support) }
|
||||
}
|
||||
|
||||
private fun sbsFailed(productId: Int): Boolean =
|
||||
app.getSharedPreferences(PREFS, Context.MODE_PRIVATE).getStringSet(KEY_SBS_FAILED, null)?.contains(productId.toString(16)) == true
|
||||
|
||||
private fun markSbsFailed(productId: Int?, failed: Boolean) {
|
||||
if (productId == null || sbsFailed(productId) == failed) return
|
||||
val prefs = app.getSharedPreferences(PREFS, Context.MODE_PRIVATE)
|
||||
val ids = prefs.getStringSet(KEY_SBS_FAILED, null).orEmpty()
|
||||
val id = productId.toString(16)
|
||||
prefs.edit().putStringSet(KEY_SBS_FAILED, if (failed) ids + id else ids - id).apply()
|
||||
}
|
||||
|
||||
private fun applyMode(force: Boolean) {
|
||||
val s = _state.value
|
||||
val m = model ?: return
|
||||
if (!s.canControl) return
|
||||
if (modeState.restorePending) {
|
||||
sendMode(false)
|
||||
return
|
||||
}
|
||||
if (!s.usable || s.sbs || modeState.pendingStereo) return
|
||||
val hz = _settings.value.refreshHz
|
||||
if (glassesMode == m.mode(hz, false)) return
|
||||
if (!force) {
|
||||
if (autoModeSent || SystemClock.uptimeMillis() - lastModeSentAt < 8_000) return
|
||||
if (glassesMode == 0 && abs(s.hz - hz) < 1.5f) return
|
||||
autoModeSent = true
|
||||
}
|
||||
sendMode(false)
|
||||
}
|
||||
|
||||
private fun sendMode(sbs: Boolean): Boolean {
|
||||
val m = model ?: return false
|
||||
val g = _settings.value
|
||||
val mode = m.mode(if (sbs) g.sbsHz else g.refreshHz, sbs)
|
||||
if (link?.send(VitureProtocol.MSG_DISPLAY, byteArrayOf(mode.toByte())) == true) {
|
||||
if (link is RayNeoLink) glassesMode = mode
|
||||
lastModeSentAt = SystemClock.uptimeMillis()
|
||||
modeState.sent(sbs, lastModeSentAt)
|
||||
publish { it.copy(switching3d = modeState.pendingStereo) }
|
||||
scheduleModeCheck()
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
private fun scheduleModeCheck() {
|
||||
handler.removeCallbacks(confirmMode)
|
||||
modeState.nextCheckAt?.let { at ->
|
||||
handler.postDelayed(confirmMode, (at - SystemClock.uptimeMillis()).coerceAtLeast(0))
|
||||
}
|
||||
}
|
||||
|
||||
private fun sendFilm(on: Boolean) {
|
||||
if (!_state.value.canControl || !_state.value.hardwareControls || !_state.value.hasSunblock) return
|
||||
if (link?.send(VitureProtocol.MSG_FILM, byteArrayOf(if (on) 1 else 0)) == true && link !is XrealLink)
|
||||
publish { it.copy(sunblock = on) }
|
||||
}
|
||||
|
||||
private fun updateImu() {
|
||||
val l = link ?: return
|
||||
if (!l.ready) return
|
||||
val want = (sink != null || probe != null || _calibration.value.running) && l.hasImu
|
||||
if (want == _state.value.tracking) return
|
||||
val accepted = l.send(VitureProtocol.MSG_IMU, byteArrayOf(if (want) 1 else 0))
|
||||
if (want && !accepted) return
|
||||
if (want) l.send(VitureProtocol.MSG_IMU_RATE, byteArrayOf(VitureProtocol.IMU_RATE_240.toByte()))
|
||||
publish { it.copy(tracking = want) }
|
||||
}
|
||||
|
||||
private fun linkListener(generation: Long) = object : VitureLink.Listener {
|
||||
override fun onFrame(frame: VitureProtocol.Frame) = post {
|
||||
if (generation != linkGeneration) return@post
|
||||
val line = "%s 0x%04X %s".format(
|
||||
when (frame.magic) { VitureProtocol.MAGIC_REPORT -> "FD"; VitureProtocol.MAGIC_COMMAND -> "FE"; else -> "FC" },
|
||||
frame.msgId, frame.payload.joinToString("") { "%02X".format(it) },
|
||||
)
|
||||
synchronized(frames) {
|
||||
frames.addLast(line)
|
||||
while (frames.size > 64) frames.removeFirst()
|
||||
}
|
||||
Log.i(TAG, "frame $line")
|
||||
onReport(frame)
|
||||
}
|
||||
|
||||
override fun onImu(payload: ByteArray, timeNs: Long) {
|
||||
if (generation != linkGeneration) return
|
||||
imuPackets++
|
||||
probe?.invoke(payload, null)
|
||||
if (simulating.get() != null) return
|
||||
VitureProtocol.imuQuaternion(payload)?.let { deliver(it, timeNs) }
|
||||
}
|
||||
|
||||
override fun onLost() = post {
|
||||
if (generation != linkGeneration) return@post
|
||||
closeLink()
|
||||
refresh()
|
||||
}
|
||||
}
|
||||
|
||||
private fun deliver(q: FloatArray, timeNs: Long) {
|
||||
latest = q
|
||||
sink?.onPose(q[0], q[1], q[2], q[3], timeNs)
|
||||
}
|
||||
|
||||
private fun onReport(frame: VitureProtocol.Frame) {
|
||||
val p = frame.payload
|
||||
val event = p.firstOrNull()?.toInt()?.and(0xFF) ?: return
|
||||
val answer = if (p.size >= 2 && event == 0) p[1].toInt() and 0xFF else null
|
||||
when (frame.msgId) {
|
||||
VitureProtocol.EVENT_DISPLAY -> {
|
||||
Log.i(TAG, "glasses switched to mode 0x%02X".format(event))
|
||||
glassesMode = event
|
||||
}
|
||||
VitureProtocol.EVENT_DISPLAY_STEP -> model?.modeForStep(event)?.let { mode ->
|
||||
Log.i(TAG, "glasses report mode 0x%02X".format(mode))
|
||||
glassesMode = mode
|
||||
}
|
||||
VitureProtocol.EVENT_BRIGHTNESS -> publish { it.copy(brightness = event) }
|
||||
VitureProtocol.EVENT_VOLUME -> publish { it.copy(volume = event) }
|
||||
VitureProtocol.MSG_GET_BRIGHTNESS, VitureProtocol.MSG_BRIGHTNESS -> answer?.let { v -> publish { it.copy(brightness = v) } }
|
||||
VitureProtocol.MSG_GET_VOLUME, VitureProtocol.MSG_VOLUME -> answer?.let { v -> publish { it.copy(volume = v) } }
|
||||
VitureProtocol.MSG_DISPLAY -> if (event != 0) {
|
||||
Log.w(TAG, "glasses refused display mode 0x%02X".format(p.getOrNull(1)?.toInt()?.and(0xFF) ?: 0))
|
||||
modeState.cancelPending()
|
||||
publish { it.copy(switching3d = false) }
|
||||
scheduleModeCheck()
|
||||
} else answer?.let { glassesMode = it }
|
||||
VitureProtocol.MSG_GET_DISPLAY -> answer?.let { v ->
|
||||
glassesMode = v
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fun sampleImu(ms: Long): JSONObject {
|
||||
val raw = java.util.Collections.synchronizedList(mutableListOf<Pair<ByteArray?, FloatArray?>>())
|
||||
val counter = java.util.concurrent.atomic.AtomicInteger()
|
||||
val began = SystemClock.elapsedRealtime()
|
||||
probe = { payload, pose -> val n = counter.incrementAndGet(); if (raw.size < 4 || n % 60 == 0) raw.add(payload?.copyOf() to pose?.copyOf()) }
|
||||
post { updateImu() }
|
||||
SystemClock.sleep(ms)
|
||||
probe = null
|
||||
post { updateImu() }
|
||||
val seconds = (SystemClock.elapsedRealtime() - began) / 1000.0
|
||||
val count = counter.get()
|
||||
val samples = JSONArray()
|
||||
synchronized(raw) {
|
||||
raw.forEach { (b, pose) ->
|
||||
val rayneo = b != null && RayNeoProtocol.classifyPacket(b) is RayNeoPacket.Sensor
|
||||
val q = pose ?: if (b == null || rayneo) null else VitureProtocol.imuQuaternion(b)
|
||||
samples.put(JSONObject()
|
||||
.put("hex", b?.joinToString("") { "%02X".format(it) } ?: JSONObject.NULL)
|
||||
.put("euler", if (b == null || b.size < 12 || rayneo || pose != null) JSONObject.NULL else JSONArray().put(VitureProtocol.beFloat(b, 0).toDouble()).put(VitureProtocol.beFloat(b, 4).toDouble()).put(VitureProtocol.beFloat(b, 8).toDouble()))
|
||||
.put("quat", q?.let { JSONArray().put(it[0].toDouble()).put(it[1].toDouble()).put(it[2].toDouble()).put(it[3].toDouble()) } ?: JSONObject.NULL))
|
||||
}
|
||||
}
|
||||
return JSONObject().put("packets", count).put("rateHz", count / seconds).put("samples", samples)
|
||||
}
|
||||
|
||||
private fun Intent.device(): UsbDevice? =
|
||||
if (Build.VERSION.SDK_INT >= 33) getParcelableExtra(UsbManager.EXTRA_DEVICE, UsbDevice::class.java)
|
||||
else @Suppress("DEPRECATION") getParcelableExtra(UsbManager.EXTRA_DEVICE)
|
||||
|
||||
private fun load(): GlassesSettings {
|
||||
val p = app.getSharedPreferences(PREFS, Context.MODE_PRIVATE)
|
||||
return GlassesSettings(
|
||||
refreshHz = p.getInt("refreshHz", 120),
|
||||
sbsHz = p.getInt("sbsHz", 60),
|
||||
sunblock = if (p.contains("sunblock")) p.getBoolean("sunblock", false) else null,
|
||||
gameOnGlasses = p.getBoolean("gameOnGlasses", true),
|
||||
headTracking = p.getBoolean("headTracking", true),
|
||||
auto3d = p.getBoolean("auto3d", GlassesSettings().auto3d),
|
||||
picture3d = p.getInt("picture3d", GlassesSettings.PICTURE_AUTO),
|
||||
screenWidthDm = p.getInt("screenWidthDm", 20),
|
||||
screenDistanceDm = p.getInt("screenDistanceDm", 30),
|
||||
vrFovDeg = p.getInt("vrFovDeg", GlassesSettings.VR_FOV_DEFAULT).coerceIn(GlassesSettings.VR_FOV_MIN, GlassesSettings.VR_FOV_MAX),
|
||||
)
|
||||
}
|
||||
|
||||
private fun save(s: GlassesSettings) {
|
||||
app.getSharedPreferences(PREFS, Context.MODE_PRIVATE).edit().apply {
|
||||
putInt("refreshHz", s.refreshHz)
|
||||
putInt("sbsHz", s.sbsHz)
|
||||
s.sunblock?.let { putBoolean("sunblock", it) } ?: remove("sunblock")
|
||||
putBoolean("gameOnGlasses", s.gameOnGlasses)
|
||||
putBoolean("headTracking", s.headTracking)
|
||||
putBoolean("auto3d", s.auto3d)
|
||||
putInt("picture3d", s.picture3d)
|
||||
putInt("screenWidthDm", s.screenWidthDm)
|
||||
putInt("screenDistanceDm", s.screenDistanceDm)
|
||||
putInt("vrFovDeg", s.vrFovDeg)
|
||||
}.apply()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
/** Hardware properties; display refresh is selected from Android's advertised modes. */
|
||||
interface GlassesModel {
|
||||
val name: String
|
||||
val controllable: Boolean
|
||||
val brightnessMax: Int
|
||||
val volumeMax: Int
|
||||
val diagonalFovDeg: Float
|
||||
val sbs90: Boolean
|
||||
val hasSunblock: Boolean get() = true
|
||||
fun mode(hz: Int, sbs: Boolean): Int
|
||||
fun modeForStep(step: Int): Int?
|
||||
}
|
||||
|
||||
/** The command IDs here are the existing glasses controller's internal vocabulary.
|
||||
* Each transport translates them to its own wire protocol; unsupported controls return false.
|
||||
*/
|
||||
internal interface GlassesLink {
|
||||
val hasImu: Boolean
|
||||
val badFrames: Long get() = 0L
|
||||
val ready: Boolean get() = true
|
||||
fun send(msgId: Int, payload: ByteArray = ByteArray(0)): Boolean
|
||||
fun close()
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
internal class GlassesModeState(private val timeoutMs: Long) {
|
||||
enum class Confirmation { NONE, WAITING, CONFIRMED, FAILED }
|
||||
data class Device(val productId: Int, val serial: String?)
|
||||
|
||||
private var deadline: Long? = null
|
||||
private var device: Device? = null
|
||||
private var attached = false
|
||||
private var reconnectDeadline: Long? = null
|
||||
private var vrOwned = false
|
||||
var restorePending = false
|
||||
private set
|
||||
val pendingStereo: Boolean get() = deadline != null
|
||||
val nextCheckAt: Long? get() = listOfNotNull(deadline, reconnectDeadline).minOrNull()
|
||||
|
||||
fun updateDevice(current: Device?, now: Long): Boolean {
|
||||
if (reconnectDeadline?.let { now >= it } == true) abandon()
|
||||
if (current == null) {
|
||||
if (attached) reconnectDeadline = now + timeoutMs
|
||||
attached = false
|
||||
return false
|
||||
}
|
||||
val previous = device
|
||||
val changed = previous == null || previous.productId != current.productId ||
|
||||
(previous.serial != null && current.serial != null && previous.serial != current.serial)
|
||||
if (changed) abandon()
|
||||
device = if (!changed && current.serial == null) previous else current
|
||||
attached = true
|
||||
reconnectDeadline = null
|
||||
return changed
|
||||
}
|
||||
|
||||
private fun abandon() {
|
||||
deadline = null
|
||||
reconnectDeadline = null
|
||||
vrOwned = false
|
||||
restorePending = false
|
||||
}
|
||||
|
||||
fun manual() {
|
||||
vrOwned = false
|
||||
restorePending = false
|
||||
}
|
||||
|
||||
fun startVr() {
|
||||
vrOwned = true
|
||||
restorePending = false
|
||||
}
|
||||
|
||||
fun stopVr(): Boolean {
|
||||
if (vrOwned) {
|
||||
vrOwned = false
|
||||
restorePending = true
|
||||
}
|
||||
return restorePending
|
||||
}
|
||||
|
||||
fun sent(stereo: Boolean, now: Long) {
|
||||
deadline = if (stereo) now + timeoutMs else null
|
||||
if (!stereo) restorePending = false
|
||||
}
|
||||
|
||||
fun cancelPending() {
|
||||
deadline = null
|
||||
}
|
||||
|
||||
fun observe(stereo: Boolean, now: Long): Confirmation {
|
||||
val until = deadline ?: return Confirmation.NONE
|
||||
if (stereo) {
|
||||
deadline = null
|
||||
return Confirmation.CONFIRMED
|
||||
}
|
||||
if (now < until) return Confirmation.WAITING
|
||||
if (!attached) {
|
||||
abandon()
|
||||
return Confirmation.NONE
|
||||
}
|
||||
deadline = null
|
||||
vrOwned = false
|
||||
restorePending = true
|
||||
return Confirmation.FAILED
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import com.droiddeck.launcher.gpu.Cardboard
|
||||
|
||||
/** Share the actual picture mode with settings so controls match the rendered view. */
|
||||
internal object GlassesRendering {
|
||||
enum class Picture { FLAT, VIRTUAL_SCREEN, STEREO }
|
||||
|
||||
fun outputHz(settings: GlassesSettings, state: GlassesState): Int =
|
||||
if (state.sbs) if (state.sbs90) settings.sbsHz else 60 else settings.refreshHz
|
||||
|
||||
fun mirrorRefreshLimit(settings: GlassesSettings, state: GlassesState, outputActive: Boolean): Int =
|
||||
if (state.usable && !outputActive) outputHz(settings, state) else 0
|
||||
|
||||
fun fpsLimit(requested: Int, mirrorRefreshLimit: Int): Int = when {
|
||||
mirrorRefreshLimit <= 0 -> requested
|
||||
requested <= 0 -> mirrorRefreshLimit
|
||||
else -> minOf(requested, mirrorRefreshLimit)
|
||||
}
|
||||
|
||||
fun usesGlasses(state: GlassesState, outputActive: Boolean, steamVrRunning: Boolean): Boolean =
|
||||
outputActive || (steamVrRunning && state.usable)
|
||||
|
||||
fun picture(settings: GlassesSettings, stereoDisplay: Boolean, steamVrRunning: Boolean): Picture =
|
||||
when (settings.picture3d) {
|
||||
GlassesSettings.PICTURE_SCREEN -> Picture.VIRTUAL_SCREEN
|
||||
GlassesSettings.PICTURE_SBS -> Picture.STEREO
|
||||
else -> when {
|
||||
steamVrRunning -> Picture.STEREO
|
||||
stereoDisplay -> Picture.VIRTUAL_SCREEN
|
||||
else -> Picture.FLAT
|
||||
}
|
||||
}
|
||||
|
||||
fun cardboard(
|
||||
base: Cardboard,
|
||||
settings: GlassesSettings,
|
||||
state: GlassesState,
|
||||
outputActive: Boolean,
|
||||
steamVrRunning: Boolean,
|
||||
): Cardboard {
|
||||
// Mirror output still uses the glasses' picture and sensors during VR.
|
||||
if (!usesGlasses(state, outputActive, steamVrRunning)) {
|
||||
// SteamVR uses the renderer's enabled flag as headset proximity.
|
||||
// A flat phone preview still needs an active HMD and one SBS eye;
|
||||
// otherwise SteamVR enters standby and leaves a frozen frame.
|
||||
return if (steamVrRunning && !base.enabled) base.copy(
|
||||
enabled = true,
|
||||
source = Cardboard.SOURCE_SBS,
|
||||
viewer = Cardboard.Viewer.NONE.id,
|
||||
mono = true,
|
||||
) else base
|
||||
}
|
||||
val picture = picture(settings, state.sbs, steamVrRunning)
|
||||
if (picture == Picture.FLAT) return Cardboard.OFF
|
||||
return Cardboard(
|
||||
enabled = true,
|
||||
source = if (picture == Picture.STEREO) Cardboard.SOURCE_SBS else Cardboard.SOURCE_SCREEN,
|
||||
viewer = Cardboard.Viewer.VITURE.id,
|
||||
ipdMm = base.ipdMm,
|
||||
screenWidthDm = settings.screenWidthDm,
|
||||
screenDistanceDm = settings.screenDistanceDm,
|
||||
headTracking = settings.headTracking && state.canControl,
|
||||
mono = !state.sbs,
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import android.app.Presentation
|
||||
import android.content.Context
|
||||
import android.graphics.Color
|
||||
import android.os.Bundle
|
||||
import android.util.Log
|
||||
import android.view.Display
|
||||
import android.view.SurfaceHolder
|
||||
import android.view.SurfaceView
|
||||
import android.view.ViewGroup
|
||||
import android.view.WindowManager
|
||||
import android.widget.FrameLayout
|
||||
import kotlin.math.abs
|
||||
|
||||
class GlassesScreen(context: Context, display: Display, refreshHz: Int, private val callback: SurfaceHolder.Callback) : Presentation(context, display) {
|
||||
val surfaceView = SurfaceView(getContext())
|
||||
val holder: SurfaceHolder get() = surfaceView.holder
|
||||
private var wantHz = refreshHz
|
||||
|
||||
val hz: Float get() = display.mode.refreshRate
|
||||
|
||||
override fun onCreate(savedInstanceState: Bundle?) {
|
||||
super.onCreate(savedInstanceState)
|
||||
val root = FrameLayout(context).apply { setBackgroundColor(Color.BLACK) }
|
||||
root.addView(surfaceView, FrameLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT))
|
||||
setContentView(root)
|
||||
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
|
||||
applyMode()
|
||||
surfaceView.holder.addCallback(callback)
|
||||
}
|
||||
|
||||
fun prefer(hz: Int) {
|
||||
wantHz = hz
|
||||
// A 2D/SBS transition can replace the active mode without changing Hz.
|
||||
applyMode()
|
||||
}
|
||||
|
||||
private fun applyMode() {
|
||||
val w = window ?: return
|
||||
val current = display.mode
|
||||
val best = display.supportedModes.filter {
|
||||
it.physicalWidth == current.physicalWidth && it.physicalHeight == current.physicalHeight
|
||||
}.minByOrNull { abs(it.refreshRate - wantHz) } ?: return
|
||||
if (w.attributes.preferredDisplayModeId != best.modeId) {
|
||||
Log.i("GlassesScreen", "display ${display.displayId}: prefer mode ${best.modeId} " +
|
||||
"${best.physicalWidth}x${best.physicalHeight}@${best.refreshRate} (requested $wantHz Hz)")
|
||||
w.attributes = w.attributes.apply { preferredDisplayModeId = best.modeId }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import android.content.SharedPreferences
|
||||
import org.json.JSONArray
|
||||
import org.json.JSONObject
|
||||
|
||||
enum class CalibrationPhase { IDLE, SETTLING, COLLECTING, MOVING, TURN_OVER, SAVED, FAILED }
|
||||
|
||||
data class GlassesCalibrationState(
|
||||
val supported: Boolean = false,
|
||||
val saved: Boolean = false,
|
||||
val phase: CalibrationPhase = CalibrationPhase.IDLE,
|
||||
val progress: Int = 0,
|
||||
) {
|
||||
val running: Boolean get() = phase in setOf(CalibrationPhase.SETTLING, CalibrationPhase.COLLECTING,
|
||||
CalibrationPhase.MOVING, CalibrationPhase.TURN_OVER)
|
||||
}
|
||||
|
||||
internal data class RayNeoCalibration(val gyroBiasDps: Vec3, val level: Quaternion) {
|
||||
fun encode(): String = JSONObject().put("version", 1)
|
||||
.put("bias", JSONArray(listOf(gyroBiasDps.x, gyroBiasDps.y, gyroBiasDps.z)))
|
||||
.put("level", JSONArray(listOf(level.w, level.x, level.y, level.z))).toString()
|
||||
|
||||
companion object {
|
||||
fun decode(json: String?): RayNeoCalibration? = runCatching {
|
||||
val obj = JSONObject(json ?: return null)
|
||||
require(obj.getInt("version") == 1)
|
||||
val b = obj.getJSONArray("bias"); val q = obj.getJSONArray("level")
|
||||
require(b.length() == 3 && q.length() == 4)
|
||||
val bias = Vec3(b.getDouble(0).toFloat(), b.getDouble(1).toFloat(), b.getDouble(2).toFloat())
|
||||
val level = Quaternion(q.getDouble(0).toFloat(), q.getDouble(1).toFloat(), q.getDouble(2).toFloat(), q.getDouble(3).toFloat())
|
||||
require(listOf(bias.x, bias.y, bias.z, level.w, level.x, level.y, level.z).all { it.isFinite() })
|
||||
require(bias.length() <= 6f && level.w in 0.96f..1.001f)
|
||||
val norm = level.w * level.w + level.x * level.x + level.y * level.y + level.z * level.z
|
||||
require(norm in 0.99f..1.01f)
|
||||
RayNeoCalibration(bias, level.normalized())
|
||||
}.getOrNull()
|
||||
}
|
||||
}
|
||||
|
||||
internal class RayNeoCalibrationStore(private val prefs: SharedPreferences) {
|
||||
companion object {
|
||||
// GT and GT Max can share USB identity fields. Keep the non-Max mounting
|
||||
// correction separate from existing Max profiles and their legacy keys.
|
||||
fun profileKey(base: String, gtUsb: Boolean, boardId: Int): String =
|
||||
if (gtUsb && boardId == RayNeoProtocol.BOARD_GT) "$base:gt-standard-v1" else base
|
||||
}
|
||||
// A serial isolates physical units. Devices without one share their model profile.
|
||||
fun key(vid: Int, pid: Int, serial: String?, modelName: String = ""): String =
|
||||
"rayneo-cal-v1:$vid:$pid:${serial?.takeIf { it.isNotBlank() } ?: "model:$modelName"}"
|
||||
fun load(key: String): RayNeoCalibration? = RayNeoCalibration.decode(prefs.getString(key, null))
|
||||
fun save(key: String, value: RayNeoCalibration?) {
|
||||
if (value == null) prefs.edit().remove(key).apply()
|
||||
else prefs.edit().putString(key, value.encode()).apply()
|
||||
}
|
||||
}
|
||||
|
||||
/** One upside-down stationary pose estimates gyro bias and tilt, never yaw or accelerometer scale. */
|
||||
internal class RayNeoCalibrationCollector(private val axes: RayNeoSensorAxes, private val beganMs: Long) {
|
||||
var phase = CalibrationPhase.SETTLING; private set
|
||||
var progress = 0; private set
|
||||
private var windowStart = -1L
|
||||
private var lastMs = -1L
|
||||
private var count = 0
|
||||
private var meanA = Vec3(0f, 0f, 0f)
|
||||
private var meanG = Vec3(0f, 0f, 0f)
|
||||
private var aVariance = 0f
|
||||
private var gVariance = 0f
|
||||
|
||||
private fun clear(next: CalibrationPhase) {
|
||||
phase = next; progress = 0; windowStart = -1; count = 0
|
||||
meanA = Vec3(0f, 0f, 0f); meanG = meanA; aVariance = 0f; gVariance = 0f
|
||||
}
|
||||
|
||||
fun update(sample: RayNeoSensorSample, nowMs: Long): RayNeoCalibration? {
|
||||
if (phase == CalibrationPhase.FAILED || phase == CalibrationPhase.SAVED) return null
|
||||
if (nowMs - beganMs >= 30_000) { clear(CalibrationPhase.FAILED); return null }
|
||||
val gap = lastMs >= 0 && nowMs - lastMs > 250
|
||||
lastMs = nowMs
|
||||
if (nowMs - beganMs < 3000) return null
|
||||
val a = axes.map(Vec3(sample.accelMps2[0], sample.accelMps2[1], sample.accelMps2[2]))
|
||||
val g = Vec3(sample.gyroDps[0], sample.gyroDps[1], sample.gyroDps[2])
|
||||
if (a.length() !in 8.5f..11f || !g.length().isFinite() || g.length() > 6f || gap) {
|
||||
clear(CalibrationPhase.MOVING); return null
|
||||
}
|
||||
if (a.normalized().y > -0.94f) { clear(CalibrationPhase.TURN_OVER); return null }
|
||||
if (windowStart < 0) windowStart = nowMs
|
||||
phase = CalibrationPhase.COLLECTING
|
||||
count++
|
||||
val da = a - meanA; val dg = g - meanG
|
||||
meanA += da * (1f / count); meanG += dg * (1f / count)
|
||||
fun dot(x: Vec3, y: Vec3) = x.x * y.x + x.y * y.y + x.z * y.z
|
||||
aVariance += dot(da, a - meanA); gVariance += dot(dg, g - meanG)
|
||||
if (count > 25 && (aVariance / count > 0.04f || gVariance / count > 2.25f)) {
|
||||
clear(CalibrationPhase.MOVING); return null
|
||||
}
|
||||
progress = ((nowMs - windowStart) * 100 / 5000).toInt().coerceIn(0, 99)
|
||||
if (nowMs - windowStart < 5000 || count < 200) return null
|
||||
val down = meanA.normalized()
|
||||
val axis = down.cross(Vec3(0f, -1f, 0f))
|
||||
val result = RayNeoCalibration(meanG, Quaternion(1f - down.y, axis.x, axis.y, axis.z).normalized())
|
||||
phase = CalibrationPhase.SAVED; progress = 100
|
||||
return result
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,202 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import android.hardware.usb.UsbDevice
|
||||
import android.hardware.usb.UsbDeviceConnection
|
||||
import android.hardware.usb.UsbManager
|
||||
import android.os.SystemClock
|
||||
import android.util.Log
|
||||
|
||||
/** RayNeo command/response and IMU share one endpoint. Keep reads on a single worker. */
|
||||
internal class RayNeoLink private constructor(
|
||||
private val connection: UsbDeviceConnection,
|
||||
private val endpoints: RayNeoEndpointSelection,
|
||||
private val listener: Listener,
|
||||
private val gt: Boolean,
|
||||
private val loadCalibration: (Int) -> RayNeoCalibration?,
|
||||
) : GlassesLink {
|
||||
interface Listener {
|
||||
fun onReady()
|
||||
fun onPose(raw: ByteArray, xyzw: FloatArray, timeNs: Long)
|
||||
fun onLost()
|
||||
fun onCalibration(revision: Long, phase: CalibrationPhase, progress: Int) {}
|
||||
fun onCalibrationSaved(revision: Long, value: RayNeoCalibration?) {}
|
||||
fun onDeviceInfo(boardId: Int, calibrationSaved: Boolean) {}
|
||||
}
|
||||
|
||||
@Volatile private var running = true
|
||||
@Volatile override var ready = false
|
||||
private set
|
||||
@Volatile private var wantImu = false
|
||||
private val writeLock = Any()
|
||||
private val assembler = RayNeoPacketAssembler()
|
||||
private val readBuffer = ByteArray(maxOf(endpoints.inEndpoint.maxPacketSize, 64))
|
||||
private var failures = 0
|
||||
private lateinit var worker: Thread
|
||||
private var calibration: RayNeoCalibration? = null
|
||||
private data class CalibrationCommand(val revision: Long, val action: Int, val saved: RayNeoCalibration? = null)
|
||||
private val calibrationCommand = java.util.concurrent.atomic.AtomicReference<CalibrationCommand?>(null)
|
||||
fun startCalibration(revision: Long) { calibrationCommand.set(CalibrationCommand(revision, 1)) }
|
||||
fun cancelCalibration(revision: Long, saved: RayNeoCalibration?) { calibrationCommand.set(CalibrationCommand(revision, 2, saved)) }
|
||||
fun resetCalibration(revision: Long) { calibrationCommand.set(CalibrationCommand(revision, 3)) }
|
||||
override val hasImu = true
|
||||
|
||||
override fun send(msgId: Int, payload: ByteArray): Boolean {
|
||||
if (!running) return false
|
||||
return when (msgId) {
|
||||
VitureProtocol.MSG_IMU -> {
|
||||
wantImu = payload.firstOrNull() == 1.toByte()
|
||||
true
|
||||
}
|
||||
VitureProtocol.MSG_DISPLAY -> {
|
||||
val mode = payload.firstOrNull()?.toInt()?.and(0xff)
|
||||
ready && (mode == RayNeoProtocol.CMD_SWITCH_TO_3D || mode == RayNeoProtocol.CMD_SWITCH_TO_2D) &&
|
||||
command(mode)
|
||||
}
|
||||
// No VITURE brightness, volume, film, rate or raw command is sent to RayNeo.
|
||||
else -> false
|
||||
}
|
||||
}
|
||||
|
||||
private fun command(cmd: Int): Boolean = synchronized(writeLock) {
|
||||
if (!running) return false
|
||||
val packet = RayNeoProtocol.buildCommandPacket(cmd, 0, endpoints.outEndpoint.maxPacketSize)
|
||||
connection.bulkTransfer(endpoints.outEndpoint, packet, packet.size, 250) == packet.size
|
||||
}
|
||||
|
||||
private fun read(timeoutMs: Int): ByteArray? {
|
||||
val deadline = SystemClock.elapsedRealtime() + timeoutMs
|
||||
while (running && SystemClock.elapsedRealtime() < deadline) {
|
||||
assembler.nextPacket()?.let { return it }
|
||||
val began = SystemClock.elapsedRealtime()
|
||||
val count = connection.bulkTransfer(endpoints.inEndpoint, readBuffer, readBuffer.size, 50)
|
||||
if (count > 0) {
|
||||
failures = 0
|
||||
assembler.append(readBuffer, count)
|
||||
} else if (count < 0 && SystemClock.elapsedRealtime() - began < 25) {
|
||||
if (++failures >= 50) error("RayNeo USB link lost")
|
||||
SystemClock.sleep(5)
|
||||
}
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
private fun response(cmd: Int, timeoutMs: Int): RayNeoPacket.Response? {
|
||||
val deadline = SystemClock.elapsedRealtime() + timeoutMs
|
||||
while (running && SystemClock.elapsedRealtime() < deadline) {
|
||||
val packet = read(100)?.let(RayNeoProtocol::classifyPacket)
|
||||
if (packet is RayNeoPacket.Response && packet.cmd == cmd) return packet
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
private fun run() {
|
||||
var streaming = false
|
||||
try {
|
||||
check(command(RayNeoProtocol.CMD_ACQUIRE_DEVICE_INFO)) { "RayNeo device-info write failed" }
|
||||
val info = response(RayNeoProtocol.CMD_ACQUIRE_DEVICE_INFO, 2500)
|
||||
?.let { RayNeoProtocol.parseDeviceInfo(it.raw) } ?: error("RayNeo device-info timed out")
|
||||
val profile = RayNeoSensorProfile.forDevice(gt, info.boardId)
|
||||
val rotation = profile.rotationXDeg
|
||||
val gtMaxAxes = profile.gtMaxAxes
|
||||
calibration = loadCalibration(info.boardId)
|
||||
listener.onDeviceInfo(info.boardId, calibration != null)
|
||||
Log.i("RayNeoLink", "Device board=0x%02x gt=%s".format(info.boardId, gt))
|
||||
var filter = RayNeoOrientationFilter(rotation, gtMaxAxes, calibration)
|
||||
var collector: RayNeoCalibrationCollector? = null
|
||||
var calibrationRevision = 0L
|
||||
var lastCalibrationUpdate = 0L
|
||||
var warmupUntil = 0L
|
||||
var lastSensor = 0L
|
||||
ready = true
|
||||
listener.onReady()
|
||||
while (running) {
|
||||
val request = calibrationCommand.getAndSet(null)
|
||||
if (request != null) calibrationRevision = request.revision
|
||||
when (request?.action) {
|
||||
1 -> collector = RayNeoCalibrationCollector(RayNeoSensorAxes(rotation, gtMaxAxes), SystemClock.elapsedRealtime())
|
||||
2 -> {
|
||||
collector = null
|
||||
if (calibration != request.saved) {
|
||||
calibration = request.saved
|
||||
filter = RayNeoOrientationFilter(rotation, gtMaxAxes, calibration)
|
||||
}
|
||||
listener.onCalibration(calibrationRevision, CalibrationPhase.IDLE, 0)
|
||||
}
|
||||
3 -> {
|
||||
collector = null; calibration = null
|
||||
filter = RayNeoOrientationFilter(rotation, gtMaxAxes)
|
||||
listener.onCalibrationSaved(calibrationRevision, null)
|
||||
listener.onCalibration(calibrationRevision, CalibrationPhase.IDLE, 0)
|
||||
}
|
||||
}
|
||||
if (wantImu != streaming) {
|
||||
val cmd = if (wantImu) RayNeoProtocol.CMD_OPEN_IMU else RayNeoProtocol.CMD_CLOSE_IMU
|
||||
check(command(cmd) && response(cmd, 1500) != null) { "RayNeo IMU command timed out" }
|
||||
streaming = cmd == RayNeoProtocol.CMD_OPEN_IMU
|
||||
filter = RayNeoOrientationFilter(rotation, gtMaxAxes, calibration)
|
||||
lastSensor = SystemClock.elapsedRealtime()
|
||||
warmupUntil = lastSensor + 2000
|
||||
}
|
||||
val raw = read(100)
|
||||
if (streaming && SystemClock.elapsedRealtime() - lastSensor > 4000) error("RayNeo IMU stream stalled")
|
||||
if (raw == null) continue
|
||||
val packet = RayNeoProtocol.classifyPacket(raw)
|
||||
if (streaming && packet is RayNeoPacket.Sensor) {
|
||||
lastSensor = SystemClock.elapsedRealtime()
|
||||
collector?.let { c ->
|
||||
val result = c.update(packet.sample, lastSensor)
|
||||
if (result != null && calibrationCommand.get() == null) {
|
||||
calibration = result
|
||||
filter = RayNeoOrientationFilter(rotation, gtMaxAxes, result)
|
||||
warmupUntil = lastSensor + 2000
|
||||
listener.onCalibrationSaved(calibrationRevision, result)
|
||||
}
|
||||
if (lastSensor - lastCalibrationUpdate >= 200 || result != null || c.phase == CalibrationPhase.FAILED) {
|
||||
listener.onCalibration(calibrationRevision, c.phase, c.progress)
|
||||
lastCalibrationUpdate = lastSensor
|
||||
}
|
||||
if (result != null || c.phase == CalibrationPhase.FAILED) collector = null
|
||||
}
|
||||
val wxyz = filter.update(packet.sample)
|
||||
if (wantImu && collector == null && lastSensor >= warmupUntil) {
|
||||
// Normalize basis and component order at the hardware boundary.
|
||||
listener.onPose(raw, RayNeoOrientationFilter.glassesPose(wxyz),
|
||||
SystemClock.elapsedRealtimeNanos())
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
if (running) Log.w("RayNeoLink", "RayNeo stream failed", e)
|
||||
} finally {
|
||||
ready = false
|
||||
if (running) listener.onLost()
|
||||
}
|
||||
}
|
||||
|
||||
override fun close() {
|
||||
synchronized(writeLock) {
|
||||
if (!running) return
|
||||
// Best effort: stop the sensor before releasing the claimed interface.
|
||||
if (ready) runCatching { command(RayNeoProtocol.CMD_CLOSE_IMU) }
|
||||
running = false
|
||||
}
|
||||
if (Thread.currentThread() !== worker) worker.join(500)
|
||||
runCatching { connection.releaseInterface(endpoints.usbInterface) }
|
||||
connection.close()
|
||||
}
|
||||
|
||||
companion object {
|
||||
fun open(usb: UsbManager, device: UsbDevice, listener: Listener, loadCalibration: (Int) -> RayNeoCalibration? = { null }): RayNeoLink? {
|
||||
val endpoints = RayNeoProtocol.selectEndpoints(device) ?: return null
|
||||
val conn = usb.openDevice(device) ?: return null
|
||||
if (!conn.claimInterface(endpoints.usbInterface, true)) {
|
||||
conn.close()
|
||||
return null
|
||||
}
|
||||
return RayNeoLink(conn, endpoints, listener, RayNeoProtocol.isGt(device.vendorId, device.productId), loadCalibration).also { link ->
|
||||
link.worker = Thread({ link.run() }, "rayneo-imu").apply { isDaemon = true }
|
||||
link.worker.start()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
object RayNeoModel {
|
||||
private val modelName = Regex("(?i)(ray\\s*neo|tcl).*\\b(air|gt)(?=\\b|[0-9])")
|
||||
|
||||
fun isVendor(vid: Int) = vid == RayNeoProtocol.VENDOR_ID ||
|
||||
vid == RayNeoProtocol.GT_VENDOR_ID || vid == 0x22d9
|
||||
|
||||
fun model(vid: Int, pid: Int, name: String?): GlassesModel? {
|
||||
if (!isVendor(vid)) return null
|
||||
// GT's identity is confirmed by verncat's SDK; do not claim other products
|
||||
// from that vendor just because their name resembles a supported model.
|
||||
val gt = RayNeoProtocol.isGt(vid, pid)
|
||||
if (vid == RayNeoProtocol.GT_VENDOR_ID && !gt) return null
|
||||
if (!(vid == RayNeoProtocol.VENDOR_ID && pid == RayNeoProtocol.PRODUCT_ID) &&
|
||||
!gt &&
|
||||
!modelName.containsMatchIn(name.orEmpty())) return null
|
||||
return Model(name?.trim()?.takeIf { modelName.containsMatchIn(it) }
|
||||
?: if (gt) "RayNeo GT series" else "RayNeo Air")
|
||||
}
|
||||
|
||||
fun isDisplay(name: String): Boolean =
|
||||
name.contains("rayneo", ignoreCase = true) || name.contains("ray neo", ignoreCase = true) ||
|
||||
modelName.containsMatchIn(name)
|
||||
|
||||
// Observed on GT Max: Android exposes an unbranded EDID monitor name.
|
||||
// Match its product metadata, not every display called SmartGlasses/TCL.
|
||||
fun isDisplayProduct(name: String?, manufacturer: String?, productId: String?): Boolean =
|
||||
name?.trim().equals("SmartGlasses", ignoreCase = true) &&
|
||||
manufacturer.equals("TCL", ignoreCase = true) && productId == "980"
|
||||
|
||||
private class Model(override val name: String) : GlassesModel {
|
||||
override val controllable = true
|
||||
override val brightnessMax = 0
|
||||
override val volumeMax = 0
|
||||
override val diagonalFovDeg = 46f // Conservative existing default, not a calibrated model profile.
|
||||
override val sbs90 = false
|
||||
override fun mode(hz: Int, sbs: Boolean) =
|
||||
if (sbs) RayNeoProtocol.CMD_SWITCH_TO_3D else RayNeoProtocol.CMD_SWITCH_TO_2D
|
||||
override fun modeForStep(step: Int): Int? = null
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,190 @@
|
||||
// Adapted from informalTechCode/RayNeo-Air-Series-3DoF-Scaffold (Apache-2.0).
|
||||
// See docs/third-party/rayneo/. Returns scaffold quaternions in (w, x, y, z) order.
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import kotlin.math.cos
|
||||
import kotlin.math.sin
|
||||
import kotlin.math.sqrt
|
||||
|
||||
internal data class Vec3(val x: Float, val y: Float, val z: Float) {
|
||||
fun length(): Float = sqrt(x * x + y * y + z * z)
|
||||
|
||||
fun normalized(): Vec3 {
|
||||
val n = length()
|
||||
if (n < 1e-6f) {
|
||||
return Vec3(0f, 0f, 0f)
|
||||
}
|
||||
return Vec3(x / n, y / n, z / n)
|
||||
}
|
||||
|
||||
fun cross(other: Vec3): Vec3 {
|
||||
return Vec3(
|
||||
y * other.z - z * other.y,
|
||||
z * other.x - x * other.z,
|
||||
x * other.y - y * other.x
|
||||
)
|
||||
}
|
||||
|
||||
operator fun plus(other: Vec3): Vec3 = Vec3(x + other.x, y + other.y, z + other.z)
|
||||
operator fun minus(other: Vec3): Vec3 = Vec3(x - other.x, y - other.y, z - other.z)
|
||||
operator fun times(scale: Float): Vec3 = Vec3(x * scale, y * scale, z * scale)
|
||||
}
|
||||
|
||||
internal data class Quaternion(val w: Float, val x: Float, val y: Float, val z: Float) {
|
||||
operator fun times(other: Quaternion): Quaternion {
|
||||
return Quaternion(
|
||||
w = w * other.w - x * other.x - y * other.y - z * other.z,
|
||||
x = w * other.x + x * other.w + y * other.z - z * other.y,
|
||||
y = w * other.y - x * other.z + y * other.w + z * other.x,
|
||||
z = w * other.z + x * other.y - y * other.x + z * other.w
|
||||
)
|
||||
}
|
||||
|
||||
operator fun times(scale: Float): Quaternion {
|
||||
return Quaternion(w * scale, x * scale, y * scale, z * scale)
|
||||
}
|
||||
|
||||
fun normalized(): Quaternion {
|
||||
val n = sqrt(w * w + x * x + y * y + z * z)
|
||||
if (n < 1e-6f) {
|
||||
return IDENTITY
|
||||
}
|
||||
return Quaternion(w / n, x / n, y / n, z / n)
|
||||
}
|
||||
|
||||
fun conjugate(): Quaternion = Quaternion(w, -x, -y, -z)
|
||||
|
||||
fun rotate(v: Vec3): Vec3 {
|
||||
val qv = Quaternion(0f, v.x, v.y, v.z)
|
||||
val r = this * qv * conjugate()
|
||||
return Vec3(r.x, r.y, r.z)
|
||||
}
|
||||
|
||||
fun integrateBodyRate(omegaRadPerSec: Vec3, dtSec: Float): Quaternion {
|
||||
val omega = Quaternion(0f, omegaRadPerSec.x, omegaRadPerSec.y, omegaRadPerSec.z)
|
||||
val qDot = (this * omega) * 0.5f
|
||||
return Quaternion(
|
||||
w + qDot.w * dtSec,
|
||||
x + qDot.x * dtSec,
|
||||
y + qDot.y * dtSec,
|
||||
z + qDot.z * dtSec
|
||||
)
|
||||
}
|
||||
|
||||
companion object {
|
||||
val IDENTITY = Quaternion(1f, 0f, 0f, 0f)
|
||||
|
||||
fun fromAxisAngle(axis: Vec3, angleRad: Float): Quaternion {
|
||||
val nAxis = axis.normalized()
|
||||
val half = angleRad * 0.5f
|
||||
val s = sin(half)
|
||||
val c = cos(half)
|
||||
return Quaternion(c, nAxis.x * s, nAxis.y * s, nAxis.z * s).normalized()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
internal data class RayNeoSensorProfile(val rotationXDeg: Float, val gtMaxAxes: Boolean) {
|
||||
companion object {
|
||||
fun forDevice(gtUsb: Boolean, boardId: Int) = when {
|
||||
gtUsb && boardId == RayNeoProtocol.BOARD_GT -> RayNeoSensorProfile(-25f, false)
|
||||
gtUsb -> RayNeoSensorProfile(35f, true)
|
||||
boardId == RayNeoProtocol.BOARD_AIR_4_PRO -> RayNeoSensorProfile(-20f, false)
|
||||
else -> RayNeoSensorProfile(0f, false)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
internal class RayNeoSensorAxes(imuRotationXDeg: Float, private val gt: Boolean) {
|
||||
private val rotation = Quaternion.fromAxisAngle(Vec3(1f, 0f, 0f),
|
||||
Math.toRadians((if (gt) 35f else imuRotationXDeg).toDouble()).toFloat())
|
||||
|
||||
fun map(v: Vec3): Vec3 = rotation.rotate(if (gt) Vec3(-v.y, v.z, -v.x) else v)
|
||||
}
|
||||
|
||||
internal class RayNeoOrientationFilter(
|
||||
imuRotationXDeg: Float,
|
||||
gt: Boolean = false,
|
||||
private val calibration: RayNeoCalibration? = null,
|
||||
) {
|
||||
private val worldUp = Vec3(0f, 1f, 0f)
|
||||
private val accelGain = 1.5f
|
||||
private val dpsToRad = (Math.PI.toFloat() / 180f)
|
||||
private val gravityMps2 = 9.81f
|
||||
private val stationaryAccelTol = 1.25f
|
||||
private val stationaryGyroRadPerSec = 0.18f
|
||||
private val gyroBiasUpdateHz = 0.5f
|
||||
private val axes = RayNeoSensorAxes(imuRotationXDeg, gt)
|
||||
|
||||
private var q = Quaternion.IDENTITY
|
||||
private var gravityInitialized = false
|
||||
private var gyroBias = Vec3(0f, 0f, 0f)
|
||||
private var lastTick100us: Long = -1L
|
||||
private var lastRealtimeNs: Long = 0L
|
||||
|
||||
companion object {
|
||||
/** Scaffold: right +X, up +Y, back +Z. VITURE: right -Y, up +Z, back -X.
|
||||
* Conjugate by HeadPose.GLASSES_AXES before entering the unchanged VITURE pose path.
|
||||
* This is a basis conversion, not a projection or seated-origin adjustment.
|
||||
*/
|
||||
fun glassesPose(wxyz: FloatArray): FloatArray =
|
||||
floatArrayOf(-wxyz[3], -wxyz[1], wxyz[2], wxyz[0])
|
||||
}
|
||||
|
||||
fun update(sample: RayNeoSensorSample): FloatArray {
|
||||
var accel = Vec3(sample.accelMps2[0], sample.accelMps2[1], sample.accelMps2[2])
|
||||
var gyro = Vec3(sample.gyroDps[0], sample.gyroDps[1], sample.gyroDps[2])
|
||||
calibration?.let { gyro -= it.gyroBiasDps }
|
||||
accel = axes.map(accel)
|
||||
gyro = axes.map(gyro) * dpsToRad
|
||||
calibration?.let {
|
||||
accel = it.level.rotate(accel)
|
||||
gyro = it.level.rotate(gyro)
|
||||
}
|
||||
|
||||
val nowNs = System.nanoTime()
|
||||
var dtSec = 0.01f
|
||||
if (lastTick100us >= 0L && sample.deviceTick100us != lastTick100us) {
|
||||
dtSec = (((sample.deviceTick100us - lastTick100us) and 0xffffffffL).toFloat() * 1e-4f)
|
||||
} else if (lastRealtimeNs > 0L && nowNs > lastRealtimeNs) {
|
||||
dtSec = ((nowNs - lastRealtimeNs).toFloat() * 1e-9f)
|
||||
}
|
||||
if (dtSec < 0.001f || dtSec > 0.1f) {
|
||||
dtSec = 0.01f
|
||||
}
|
||||
lastTick100us = sample.deviceTick100us
|
||||
lastRealtimeNs = nowNs
|
||||
|
||||
val accelNorm = accel.length()
|
||||
val stationary =
|
||||
kotlin.math.abs(accelNorm - gravityMps2) < stationaryAccelTol &&
|
||||
gyro.length() < stationaryGyroRadPerSec
|
||||
if (stationary) {
|
||||
val alpha = minOf(1f, dtSec * gyroBiasUpdateHz)
|
||||
gyroBias = gyroBias * (1f - alpha) + gyro * alpha
|
||||
}
|
||||
gyro -= gyroBias
|
||||
|
||||
if (accelNorm > 1e-3f) {
|
||||
val measuredUp = accel.normalized()
|
||||
if (!gravityInitialized) {
|
||||
// Align gravity before publishing the first pose. Starting at
|
||||
// identity and settling after recenter can rotate the horizon.
|
||||
val axis = measuredUp.cross(worldUp)
|
||||
q = if (measuredUp.y < -0.9999f) {
|
||||
Quaternion.fromAxisAngle(Vec3(1f, 0f, 0f), Math.PI.toFloat())
|
||||
} else {
|
||||
Quaternion(1f + measuredUp.y, axis.x, axis.y, axis.z).normalized()
|
||||
}
|
||||
gravityInitialized = true
|
||||
}
|
||||
val predictedUp = q.conjugate().rotate(worldUp).normalized()
|
||||
// q maps body to world: measured x predicted reduces gravity error.
|
||||
val correction = measuredUp.cross(predictedUp)
|
||||
gyro += correction * accelGain
|
||||
}
|
||||
|
||||
q = q.integrateBodyRate(gyro, dtSec).normalized()
|
||||
return floatArrayOf(q.w, q.x, q.y, q.z)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,270 @@
|
||||
// Adapted from informalTechCode/RayNeo-Air-Series-3DoF-Scaffold (Apache-2.0).
|
||||
// See docs/third-party/rayneo/ for license and protocol attribution.
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import android.hardware.usb.UsbConstants
|
||||
import android.hardware.usb.UsbDevice
|
||||
import android.hardware.usb.UsbEndpoint
|
||||
import android.hardware.usb.UsbInterface
|
||||
import java.util.Arrays
|
||||
|
||||
private fun Byte.u(): Int = toInt() and 0xFF
|
||||
|
||||
data class RayNeoSensorSample(
|
||||
val accelMps2: FloatArray,
|
||||
val gyroDps: FloatArray,
|
||||
val temperatureC: Float,
|
||||
val magnet: FloatArray,
|
||||
val proximity: Float,
|
||||
val light: Float,
|
||||
val deviceTick100us: Long
|
||||
)
|
||||
|
||||
data class RayNeoDeviceInfo(
|
||||
val boardId: Int,
|
||||
val sideBySideEnabled: Boolean
|
||||
)
|
||||
|
||||
sealed interface RayNeoPacket {
|
||||
data class Sensor(val sample: RayNeoSensorSample) : RayNeoPacket
|
||||
data class Response(val cmd: Int, val raw: ByteArray) : RayNeoPacket
|
||||
data object Unknown : RayNeoPacket
|
||||
}
|
||||
|
||||
data class RayNeoEndpointSelection(
|
||||
val usbInterface: UsbInterface,
|
||||
val inEndpoint: UsbEndpoint,
|
||||
val outEndpoint: UsbEndpoint
|
||||
)
|
||||
|
||||
object RayNeoProtocol {
|
||||
const val VENDOR_ID = 0x1BBB
|
||||
const val PRODUCT_ID = 0xAF50
|
||||
// verncat/RayNeo-Air-3S-Pro-OpenVR PR #3 (MIT), tested on GT hardware.
|
||||
const val GT_VENDOR_ID = 0x3941
|
||||
private const val GT_CONTROL_INTERFACE = 5
|
||||
|
||||
fun isGt(vid: Int, pid: Int) = vid == GT_VENDOR_ID && pid == PRODUCT_ID
|
||||
|
||||
const val BOARD_AIR_4_PRO = 0x3A
|
||||
const val BOARD_GT = 0x40 // Observed on the user's non-Max GT.
|
||||
|
||||
const val PKT_MAGIC = 0x99
|
||||
const val PKT_SENSOR = 0x65
|
||||
const val PKT_RESPONSE = 0xC8
|
||||
|
||||
const val CMD_ACQUIRE_DEVICE_INFO = 0
|
||||
const val CMD_OPEN_IMU = 1
|
||||
const val CMD_CLOSE_IMU = 2
|
||||
const val CMD_SWITCH_TO_3D = 6
|
||||
const val CMD_SWITCH_TO_2D = 7
|
||||
|
||||
private const val RESPONSE_CMD_OFFSET = 8
|
||||
|
||||
private const val ACCEL_OFFSET = 4
|
||||
private const val GYRO_OFFSET = 16
|
||||
private const val TEMP_OFFSET = 28
|
||||
private const val MAG_X_OFFSET = 32
|
||||
private const val TICK_OFFSET = 40
|
||||
private const val PSENSOR_OFFSET = 44
|
||||
private const val LSENSOR_OFFSET = 48
|
||||
private const val MAG_Z_OFFSET = 52
|
||||
|
||||
private const val DEVINFO_BOARD_ID_OFFSET = 21
|
||||
private const val DEVINFO_SIDE_BY_SIDE_OFFSET = 43
|
||||
|
||||
fun buildCommandPacket(cmd: Int, arg: Int, packetSize: Int): ByteArray {
|
||||
val len = maxOf(packetSize, 64)
|
||||
val pkt = ByteArray(len)
|
||||
pkt[0] = 0x66
|
||||
pkt[1] = cmd.toByte()
|
||||
pkt[2] = arg.toByte()
|
||||
return pkt
|
||||
}
|
||||
|
||||
fun classifyPacket(raw: ByteArray): RayNeoPacket {
|
||||
if (raw.size < 4 || raw[0].u() != PKT_MAGIC || raw[2].u() != raw.size) {
|
||||
return RayNeoPacket.Unknown
|
||||
}
|
||||
|
||||
return when (raw[1].u()) {
|
||||
PKT_SENSOR -> {
|
||||
val sample = parseSensorSample(raw)
|
||||
if (sample != null) RayNeoPacket.Sensor(sample) else RayNeoPacket.Unknown
|
||||
}
|
||||
PKT_RESPONSE -> {
|
||||
if (raw.size <= RESPONSE_CMD_OFFSET) {
|
||||
RayNeoPacket.Unknown
|
||||
} else {
|
||||
RayNeoPacket.Response(raw[RESPONSE_CMD_OFFSET].u(), raw)
|
||||
}
|
||||
}
|
||||
else -> RayNeoPacket.Unknown
|
||||
}
|
||||
}
|
||||
|
||||
fun parseDeviceInfo(raw: ByteArray): RayNeoDeviceInfo? {
|
||||
if (raw.size <= DEVINFO_SIDE_BY_SIDE_OFFSET || raw[2].u() != raw.size || raw[0].u() != PKT_MAGIC ||
|
||||
raw[1].u() != PKT_RESPONSE || raw[RESPONSE_CMD_OFFSET].u() != CMD_ACQUIRE_DEVICE_INFO) {
|
||||
return null
|
||||
}
|
||||
|
||||
return RayNeoDeviceInfo(
|
||||
boardId = raw[DEVINFO_BOARD_ID_OFFSET].u(),
|
||||
sideBySideEnabled = raw[DEVINFO_SIDE_BY_SIDE_OFFSET].u() != 0
|
||||
)
|
||||
}
|
||||
|
||||
fun selectEndpoints(device: UsbDevice): RayNeoEndpointSelection? {
|
||||
val gt = isGt(device.vendorId, device.productId)
|
||||
val gtCandidates = mutableListOf<RayNeoEndpointSelection>()
|
||||
var fallback: RayNeoEndpointSelection? = null
|
||||
|
||||
for (i in 0 until device.interfaceCount) {
|
||||
val usbInterface = device.getInterface(i)
|
||||
var inEndpoint: UsbEndpoint? = null
|
||||
var outEndpoint: UsbEndpoint? = null
|
||||
|
||||
for (j in 0 until usbInterface.endpointCount) {
|
||||
val endpoint = usbInterface.getEndpoint(j)
|
||||
if (endpoint.type != UsbConstants.USB_ENDPOINT_XFER_INT &&
|
||||
endpoint.type != UsbConstants.USB_ENDPOINT_XFER_BULK) {
|
||||
continue
|
||||
}
|
||||
// Exclude small button reports from GT's protocol transport.
|
||||
if (gt && endpoint.maxPacketSize < 64) continue
|
||||
|
||||
if (endpoint.direction == UsbConstants.USB_DIR_IN && inEndpoint == null) {
|
||||
inEndpoint = endpoint
|
||||
} else if (endpoint.direction == UsbConstants.USB_DIR_OUT && outEndpoint == null) {
|
||||
outEndpoint = endpoint
|
||||
}
|
||||
}
|
||||
|
||||
if (inEndpoint != null && outEndpoint != null) {
|
||||
val selection = RayNeoEndpointSelection(usbInterface, inEndpoint, outEndpoint)
|
||||
if (gt) {
|
||||
if (usbInterface.id == GT_CONTROL_INTERFACE) return selection
|
||||
gtCandidates += selection
|
||||
continue
|
||||
}
|
||||
if (inEndpoint.type == UsbConstants.USB_ENDPOINT_XFER_INT &&
|
||||
outEndpoint.type == UsbConstants.USB_ENDPOINT_XFER_INT
|
||||
) {
|
||||
return selection
|
||||
}
|
||||
if (fallback == null) {
|
||||
fallback = selection
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Interface numbers can change with firmware. Use a unique protocol-sized
|
||||
// pair when interface 5 is absent, never an arbitrary composite interface.
|
||||
return if (gt) gtCandidates.singleOrNull() else fallback
|
||||
}
|
||||
|
||||
private fun parseSensorSample(raw: ByteArray): RayNeoSensorSample? {
|
||||
if (raw.size < MAG_Z_OFFSET + 4 ||
|
||||
(ACCEL_OFFSET until TEMP_OFFSET step 4).any { !readF32LE(raw, it).isFinite() }) {
|
||||
return null
|
||||
}
|
||||
|
||||
return RayNeoSensorSample(
|
||||
accelMps2 = floatArrayOf(
|
||||
readF32LE(raw, ACCEL_OFFSET + 0),
|
||||
readF32LE(raw, ACCEL_OFFSET + 4),
|
||||
readF32LE(raw, ACCEL_OFFSET + 8)
|
||||
),
|
||||
gyroDps = floatArrayOf(
|
||||
readF32LE(raw, GYRO_OFFSET + 0),
|
||||
readF32LE(raw, GYRO_OFFSET + 4),
|
||||
readF32LE(raw, GYRO_OFFSET + 8)
|
||||
),
|
||||
temperatureC = readF32LE(raw, TEMP_OFFSET),
|
||||
magnet = floatArrayOf(
|
||||
readF32LE(raw, MAG_X_OFFSET),
|
||||
readF32LE(raw, MAG_X_OFFSET + 4),
|
||||
readF32LE(raw, MAG_Z_OFFSET)
|
||||
),
|
||||
proximity = readF32LE(raw, PSENSOR_OFFSET),
|
||||
light = readF32LE(raw, LSENSOR_OFFSET),
|
||||
deviceTick100us = readU32LE(raw, TICK_OFFSET).toLong() and 0xffffffffL
|
||||
)
|
||||
}
|
||||
|
||||
private fun readU32LE(raw: ByteArray, offset: Int): Int {
|
||||
val b0 = raw[offset].u()
|
||||
val b1 = raw[offset + 1].u()
|
||||
val b2 = raw[offset + 2].u()
|
||||
val b3 = raw[offset + 3].u()
|
||||
return b0 or (b1 shl 8) or (b2 shl 16) or (b3 shl 24)
|
||||
}
|
||||
|
||||
private fun readF32LE(raw: ByteArray, offset: Int): Float {
|
||||
return Float.fromBits(readU32LE(raw, offset))
|
||||
}
|
||||
}
|
||||
|
||||
class RayNeoPacketAssembler {
|
||||
private val stash = ByteArray(4096)
|
||||
private var len = 0
|
||||
|
||||
fun append(data: ByteArray, count: Int) {
|
||||
require(count in 0..data.size)
|
||||
if (count <= 0) {
|
||||
return
|
||||
}
|
||||
|
||||
if (count >= stash.size || len + count > stash.size) {
|
||||
len = 0
|
||||
}
|
||||
|
||||
val copyLen = minOf(count, stash.size - len)
|
||||
System.arraycopy(data, 0, stash, len, copyLen)
|
||||
len += copyLen
|
||||
}
|
||||
|
||||
fun nextPacket(): ByteArray? {
|
||||
var offset = 0
|
||||
while (offset + 4 <= len) {
|
||||
if (stash[offset].u() != RayNeoProtocol.PKT_MAGIC) {
|
||||
offset++
|
||||
continue
|
||||
}
|
||||
|
||||
val packetLen = stash[offset + 2].u()
|
||||
if (packetLen < 4 || packetLen > 64 ||
|
||||
stash[offset + 1].u() !in listOf(RayNeoProtocol.PKT_SENSOR, RayNeoProtocol.PKT_RESPONSE)) {
|
||||
offset++
|
||||
continue
|
||||
}
|
||||
if (offset + packetLen > len) {
|
||||
break
|
||||
}
|
||||
|
||||
val packet = Arrays.copyOfRange(stash, offset, offset + packetLen)
|
||||
compact(offset + packetLen)
|
||||
return packet
|
||||
}
|
||||
|
||||
if (offset > 0) {
|
||||
compact(offset)
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
private fun compact(consumed: Int) {
|
||||
if (consumed <= 0) {
|
||||
return
|
||||
}
|
||||
|
||||
if (consumed >= len) {
|
||||
len = 0
|
||||
return
|
||||
}
|
||||
|
||||
System.arraycopy(stash, consumed, stash, 0, len - consumed)
|
||||
len -= consumed
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,134 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import android.hardware.usb.UsbConstants
|
||||
import android.hardware.usb.UsbDevice
|
||||
import android.hardware.usb.UsbDeviceConnection
|
||||
import android.hardware.usb.UsbEndpoint
|
||||
import android.hardware.usb.UsbInterface
|
||||
import android.hardware.usb.UsbManager
|
||||
import android.os.SystemClock
|
||||
import android.util.Log
|
||||
import java.util.concurrent.atomic.AtomicLong
|
||||
|
||||
internal class VitureLink private constructor(
|
||||
private val connection: UsbDeviceConnection,
|
||||
private val mcu: UsbInterface,
|
||||
private val mcuOut: UsbEndpoint,
|
||||
private val mcuIn: UsbEndpoint?,
|
||||
private val imu: UsbInterface?,
|
||||
private val imuIn: UsbEndpoint?,
|
||||
private val listener: Listener,
|
||||
) : GlassesLink {
|
||||
interface Listener {
|
||||
fun onFrame(frame: VitureProtocol.Frame)
|
||||
fun onImu(payload: ByteArray, timeNs: Long)
|
||||
fun onLost()
|
||||
}
|
||||
|
||||
@Volatile private var running = true
|
||||
private val writeLock = Any()
|
||||
private var lastSendAt = 0L
|
||||
private val readers = mutableListOf<Thread>()
|
||||
private val bad = AtomicLong()
|
||||
|
||||
override val hasImu: Boolean get() = imuIn != null
|
||||
override val badFrames: Long get() = bad.get()
|
||||
|
||||
private fun startReaders() {
|
||||
mcuIn?.let { ep -> readers += reader("viture-mcu", ep) { buf, n -> onPacket(buf, n, false) } }
|
||||
imuIn?.let { ep ->
|
||||
readers += reader("viture-imu", ep) { buf, n -> onPacket(buf, n, true) }
|
||||
}
|
||||
}
|
||||
|
||||
private fun onPacket(buf: ByteArray, size: Int, imuEndpoint: Boolean) {
|
||||
val frame = VitureProtocol.parse(buf, size) ?: return
|
||||
if (!frame.crcOk) {
|
||||
if (bad.incrementAndGet() == 1L) Log.w(TAG, "dropping frames with a bad checksum")
|
||||
return
|
||||
}
|
||||
if (imuEndpoint && frame.magic == VitureProtocol.MAGIC_IMU)
|
||||
listener.onImu(frame.payload, SystemClock.elapsedRealtimeNanos())
|
||||
else listener.onFrame(frame)
|
||||
}
|
||||
|
||||
private fun reader(name: String, ep: UsbEndpoint, onPacket: (ByteArray, Int) -> Unit): Thread =
|
||||
Thread({
|
||||
val buf = ByteArray(VitureProtocol.PACKET_SIZE)
|
||||
var failures = 0
|
||||
while (running) {
|
||||
val started = SystemClock.uptimeMillis()
|
||||
val n = connection.bulkTransfer(ep, buf, buf.size, READ_TIMEOUT_MS)
|
||||
if (!running) break
|
||||
if (n > 0) {
|
||||
failures = 0
|
||||
runCatching { onPacket(buf, n) }.onFailure { Log.w(TAG, "$name packet", it) }
|
||||
} else if (n < 0 && SystemClock.uptimeMillis() - started < READ_TIMEOUT_MS / 2) {
|
||||
failures++
|
||||
SystemClock.sleep(2)
|
||||
if (failures >= LOST_AFTER) {
|
||||
Log.w(TAG, "$name: reads keep failing, link lost")
|
||||
if (running) listener.onLost()
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}, name).apply { isDaemon = true; priority = Thread.MAX_PRIORITY; start() }
|
||||
|
||||
override fun send(msgId: Int, payload: ByteArray): Boolean {
|
||||
val pkt = VitureProtocol.packet(msgId, payload)
|
||||
val n = synchronized(writeLock) {
|
||||
if (!running) return false
|
||||
val wait = lastSendAt + COMMAND_GAP_MS - SystemClock.uptimeMillis()
|
||||
if (wait > 0) SystemClock.sleep(wait)
|
||||
connection.bulkTransfer(mcuOut, pkt, pkt.size, WRITE_TIMEOUT_MS).also { lastSendAt = SystemClock.uptimeMillis() }
|
||||
}
|
||||
Log.i(TAG, "send 0x%04X [%s] -> %d".format(msgId, payload.joinToString("") { "%02X".format(it) }, n))
|
||||
return n == pkt.size
|
||||
}
|
||||
|
||||
override fun close() {
|
||||
synchronized(writeLock) { running = false }
|
||||
val self = Thread.currentThread()
|
||||
readers.forEach { if (it !== self) runCatching { it.join(READ_TIMEOUT_MS * 4L) } }
|
||||
runCatching { connection.releaseInterface(mcu) }
|
||||
imu?.let { runCatching { connection.releaseInterface(it) } }
|
||||
runCatching { connection.close() }
|
||||
}
|
||||
|
||||
companion object {
|
||||
private const val TAG = "VitureLink"
|
||||
private const val READ_TIMEOUT_MS = 100
|
||||
private const val WRITE_TIMEOUT_MS = 250
|
||||
private const val LOST_AFTER = 50
|
||||
const val COMMAND_GAP_MS = 60L
|
||||
|
||||
fun open(usb: UsbManager, device: UsbDevice, listener: Listener): VitureLink? {
|
||||
val hid = (0 until device.interfaceCount).map(device::getInterface)
|
||||
.filter { it.interfaceClass == UsbConstants.USB_CLASS_HID }
|
||||
.sortedBy { it.id }
|
||||
for (intf in hid) Log.i(TAG, "iface ${intf.id} eps " + (0 until intf.endpointCount).joinToString {
|
||||
val ep = intf.getEndpoint(it)
|
||||
"0x%02X".format(ep.address)
|
||||
})
|
||||
val mcu = hid.lastOrNull { it.outEndpoint() != null } ?: return null
|
||||
val imu = hid.firstOrNull { it.id != mcu.id && it.inEndpoint() != null }
|
||||
val conn = usb.openDevice(device) ?: return null
|
||||
if (!conn.claimInterface(mcu, true)) {
|
||||
conn.close()
|
||||
return null
|
||||
}
|
||||
val imuClaimed = imu != null && conn.claimInterface(imu, true)
|
||||
val link = VitureLink(conn, mcu, mcu.outEndpoint()!!, mcu.inEndpoint(),
|
||||
imu.takeIf { imuClaimed }, imu.takeIf { imuClaimed }?.inEndpoint(), listener)
|
||||
link.startReaders()
|
||||
return link
|
||||
}
|
||||
|
||||
private fun UsbInterface.outEndpoint(): UsbEndpoint? = (0 until endpointCount).map(::getEndpoint)
|
||||
.firstOrNull { it.direction == UsbConstants.USB_DIR_OUT && it.type == UsbConstants.USB_ENDPOINT_XFER_INT }
|
||||
|
||||
private fun UsbInterface.inEndpoint(): UsbEndpoint? = (0 until endpointCount).map(::getEndpoint)
|
||||
.firstOrNull { it.direction == UsbConstants.USB_DIR_IN && it.type == UsbConstants.USB_ENDPOINT_XFER_INT }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,186 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import kotlin.math.cos
|
||||
import kotlin.math.sin
|
||||
import kotlin.math.sqrt
|
||||
|
||||
object VitureProtocol {
|
||||
const val VENDOR_ID = 0x35CA
|
||||
const val PACKET_SIZE = 64
|
||||
private const val HEADER = 0x12
|
||||
|
||||
const val MAGIC_COMMAND = 0xFE
|
||||
const val MAGIC_REPORT = 0xFD
|
||||
const val MAGIC_IMU = 0xFC
|
||||
|
||||
const val MSG_GET_BRIGHTNESS = 0x0005
|
||||
const val MSG_BRIGHTNESS = 0x0006
|
||||
const val MSG_GET_DISPLAY = 0x0007
|
||||
const val MSG_DISPLAY = 0x0008
|
||||
const val MSG_FILM = 0x000E
|
||||
const val MSG_IMU = 0x0015
|
||||
const val MSG_GET_IMU = 0x0017
|
||||
const val MSG_IMU_RATE = 0x0018
|
||||
const val MSG_GET_VOLUME = 0x0032
|
||||
const val MSG_VOLUME = 0x0033
|
||||
|
||||
const val EVENT_BRIGHTNESS = 0x0301
|
||||
const val EVENT_DISPLAY = 0x0302
|
||||
const val EVENT_VOLUME = 0x0304
|
||||
const val EVENT_DISPLAY_STEP = 0x030D
|
||||
|
||||
const val IMU_RATE_240 = 3
|
||||
|
||||
enum class Family { ONE, PRO, LUMA, ULTRA, BEAST, PRO2 }
|
||||
|
||||
class Model(override val name: String, val family: Family) : GlassesModel {
|
||||
override val controllable: Boolean get() = true
|
||||
val needsSdk: Boolean get() = family == Family.ULTRA || family == Family.BEAST || family == Family.PRO2
|
||||
val tall: Boolean get() = family == Family.LUMA || family == Family.ULTRA || family == Family.BEAST
|
||||
override val sbs90: Boolean get() = family == Family.ULTRA
|
||||
override val brightnessMax: Int get() = 8
|
||||
override val volumeMax: Int get() = if (family == Family.BEAST) 15 else 8
|
||||
override val hasSunblock: Boolean get() = family != Family.PRO2
|
||||
override val diagonalFovDeg: Float get() = when (family) {
|
||||
Family.ONE -> 43f
|
||||
Family.PRO -> 46f
|
||||
Family.PRO2, Family.LUMA, Family.ULTRA -> 50f
|
||||
Family.BEAST -> 58f
|
||||
}
|
||||
|
||||
override fun mode(hz: Int, sbs: Boolean): Int = when {
|
||||
tall && sbs -> if (hz >= 90 && sbs90) 0x45 else 0x42
|
||||
tall -> when { hz >= 120 -> 0x44; hz >= 90 -> 0x43; else -> 0x41 }
|
||||
sbs -> if (hz >= 90 && sbs90) 0x35 else 0x32
|
||||
else -> when { hz >= 120 -> 0x34; hz >= 90 -> 0x33; else -> 0x31 }
|
||||
}
|
||||
|
||||
override fun modeForStep(step: Int): Int? = if (tall) null else when (step) {
|
||||
1 -> 0x31
|
||||
2 -> 0x33
|
||||
3 -> 0x34
|
||||
4 -> 0x32
|
||||
else -> null
|
||||
}
|
||||
}
|
||||
|
||||
/** SDK validates the PID before returning this market name. Never guess a new PID. */
|
||||
fun sdkModel(marketName: String): Model? {
|
||||
val family = when (marketName) {
|
||||
"One", "Lite" -> Family.ONE
|
||||
"Pro" -> Family.PRO
|
||||
"Pro 2" -> Family.PRO2
|
||||
"Luma", "Luma Pro", "Luma Cyber" -> Family.LUMA
|
||||
"Luma Ultra" -> Family.ULTRA
|
||||
"Beast" -> Family.BEAST
|
||||
else -> return null
|
||||
}
|
||||
return Model("VITURE $marketName", family)
|
||||
}
|
||||
|
||||
fun model(productId: Int, productName: String?): Model? {
|
||||
val family = when (productId) {
|
||||
0x1011, 0x1013, 0x1015, 0x1017, 0x101b -> Family.ONE
|
||||
0x1019, 0x101d -> Family.PRO
|
||||
0x1121, 0x1131, 0x1141, 0x1151 -> Family.LUMA
|
||||
0x1101, 0x1104 -> Family.ULTRA
|
||||
0x1201, 0x1211 -> Family.BEAST
|
||||
else -> return null
|
||||
}
|
||||
val fallback = when (productId) {
|
||||
0x1015, 0x101b -> "VITURE One Lite"
|
||||
0x1011, 0x1013, 0x1017 -> "VITURE One"
|
||||
0x1019 -> "VITURE Pro"
|
||||
0x101d -> "VITURE Pro XR"
|
||||
0x1131 -> "VITURE Luma"
|
||||
0x1121, 0x1141, 0x1151 -> "VITURE Luma Pro"
|
||||
0x1101, 0x1104 -> "VITURE Luma Ultra"
|
||||
else -> "VITURE Beast"
|
||||
}
|
||||
val name = productName?.trim()?.removeSuffix(" Glasses")?.takeIf { it.startsWith("VITURE", ignoreCase = true) } ?: fallback
|
||||
return Model(name, family)
|
||||
}
|
||||
|
||||
private val CRC_TABLE = IntArray(256) { i ->
|
||||
var c = i shl 8
|
||||
repeat(8) { c = if (c and 0x8000 != 0) (c shl 1) xor 0x1021 else c shl 1 }
|
||||
c and 0xFFFF
|
||||
}
|
||||
|
||||
fun crc16(data: ByteArray, offset: Int, length: Int): Int {
|
||||
var crc = 0
|
||||
for (i in offset until offset + length) {
|
||||
crc = (CRC_TABLE[((data[i].toInt() and 0xFF) xor (crc ushr 8)) and 0xFF] xor (crc shl 8)) and 0xFFFF
|
||||
}
|
||||
return crc
|
||||
}
|
||||
|
||||
fun packet(msgId: Int, payload: ByteArray = ByteArray(0)): ByteArray {
|
||||
require(payload.size <= PACKET_SIZE - HEADER)
|
||||
val buf = ByteArray(PACKET_SIZE)
|
||||
buf[0] = 0xFF.toByte()
|
||||
buf[1] = MAGIC_COMMAND.toByte()
|
||||
val len = payload.size + 0x0C
|
||||
buf[4] = len.toByte()
|
||||
buf[5] = (len shr 8).toByte()
|
||||
buf[14] = msgId.toByte()
|
||||
buf[15] = (msgId shr 8).toByte()
|
||||
payload.copyInto(buf, HEADER)
|
||||
val crc = crc16(buf, 4, len + 2)
|
||||
buf[2] = crc.toByte()
|
||||
buf[3] = (crc shr 8).toByte()
|
||||
return buf
|
||||
}
|
||||
|
||||
class Frame(val magic: Int, val msgId: Int, val payload: ByteArray, val crcOk: Boolean)
|
||||
|
||||
fun parse(buf: ByteArray, size: Int): Frame? {
|
||||
if (size < HEADER || size > buf.size || (buf[0].toInt() and 0xFF) != 0xFF) return null
|
||||
val magic = buf[1].toInt() and 0xFF
|
||||
if (magic != MAGIC_COMMAND && magic != MAGIC_REPORT && magic != MAGIC_IMU) return null
|
||||
val len = u16(buf, 4)
|
||||
if (len < 0x0C || len + 6 > size) return null
|
||||
val dataLen = len - 0x0C
|
||||
val crcOk = crc16(buf, 4, len + 2) == u16(buf, 2)
|
||||
return Frame(magic, u16(buf, 14), buf.copyOfRange(HEADER, HEADER + dataLen), crcOk)
|
||||
}
|
||||
|
||||
private fun u16(buf: ByteArray, at: Int) = (buf[at].toInt() and 0xFF) or ((buf[at + 1].toInt() and 0xFF) shl 8)
|
||||
|
||||
fun beFloat(b: ByteArray, at: Int): Float = java.lang.Float.intBitsToFloat(
|
||||
((b[at].toInt() and 0xFF) shl 24) or ((b[at + 1].toInt() and 0xFF) shl 16) or
|
||||
((b[at + 2].toInt() and 0xFF) shl 8) or (b[at + 3].toInt() and 0xFF),
|
||||
)
|
||||
|
||||
fun imuQuaternion(payload: ByteArray): FloatArray? {
|
||||
if (payload.size >= 36) {
|
||||
val w = beFloat(payload, 20)
|
||||
val x = beFloat(payload, 24)
|
||||
val y = beFloat(payload, 28)
|
||||
val z = beFloat(payload, 32)
|
||||
val n = sqrt(w * w + x * x + y * y + z * z)
|
||||
if (n.isFinite() && n > 0.9f && n < 1.1f) return floatArrayOf(x / n, y / n, z / n, w / n)
|
||||
}
|
||||
if (payload.size < 12) return null
|
||||
val roll = beFloat(payload, 0)
|
||||
val pitch = beFloat(payload, 4)
|
||||
val yaw = beFloat(payload, 8)
|
||||
if (!roll.isFinite() || !pitch.isFinite() || !yaw.isFinite()) return null
|
||||
return eulerZxy(roll, pitch, yaw)
|
||||
}
|
||||
|
||||
fun eulerZxy(rollDeg: Float, pitchDeg: Float, yawDeg: Float): FloatArray {
|
||||
val r = Math.toRadians(rollDeg.toDouble()) / 2
|
||||
val p = Math.toRadians(pitchDeg.toDouble()) / 2
|
||||
val y = Math.toRadians(yawDeg.toDouble()) / 2
|
||||
val cx = cos(r); val sx = sin(r)
|
||||
val cy = cos(p); val sy = sin(p)
|
||||
val cz = cos(y); val sz = sin(y)
|
||||
return floatArrayOf(
|
||||
(sx * cy * cz - cx * sy * sz).toFloat(),
|
||||
(cx * sy * cz + sx * cy * sz).toFloat(),
|
||||
(cx * cy * sz + sx * sy * cz).toFloat(),
|
||||
(cx * cy * cz - sx * sy * sz).toFloat(),
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import android.util.Log
|
||||
import kotlin.math.sqrt
|
||||
|
||||
/** JNI adapter to VITURE's Android C SDK. The vendor binary is a local build input. */
|
||||
internal object VitureSdk {
|
||||
val available: Boolean by lazy {
|
||||
runCatching {
|
||||
System.loadLibrary("droiddeck_viture")
|
||||
nativeAvailable()
|
||||
}.onFailure { Log.w("VitureSdk", "VITURE driver unavailable", it) }.getOrDefault(false)
|
||||
}
|
||||
|
||||
fun marketName(pid: Int): String? = if (available) nativeMarketName(pid) else null
|
||||
external fun nativeAvailable(): Boolean
|
||||
external fun nativeMarketName(pid: Int): String?
|
||||
external fun nativeOpen(pid: Int, fd: Int, cacheDirectory: String): Long
|
||||
external fun nativeType(token: Long): Int
|
||||
external fun nativeTracking(token: Long, enabled: Boolean): Int
|
||||
external fun nativeCommand(token: Long, command: Int, value: Int): Int
|
||||
/** Returns CLOCK_BOOTTIME receipt time, zero for no new pose, or a negative SDK error. */
|
||||
external fun nativePose(token: Long, wxyz: FloatArray): Long
|
||||
external fun nativeNextState(token: Long, event: IntArray): Int
|
||||
external fun nativeClose(token: Long)
|
||||
|
||||
const val CARINA = 2
|
||||
const val GET_DISPLAY = 1
|
||||
const val SET_DISPLAY = 2
|
||||
const val GET_BRIGHTNESS = 3
|
||||
const val SET_BRIGHTNESS = 4
|
||||
const val GET_VOLUME = 5
|
||||
const val SET_VOLUME = 6
|
||||
const val SET_FILM = 7
|
||||
}
|
||||
|
||||
internal object VitureSdkPose {
|
||||
// Pro 2 hardware captures use OpenGL axes despite the generic Gen2 callback's
|
||||
// NWU documentation. Keep that model-specific exception out of other Gen2 devices.
|
||||
fun usesOpenGlAxes(carina: Boolean, family: VitureProtocol.Family?) =
|
||||
carina || family == VitureProtocol.Family.PRO2
|
||||
|
||||
/** SDK quaternions are wxyz; normalize their coordinate basis at the boundary. */
|
||||
fun toGlasses(wxyz: FloatArray, openGlAxes: Boolean): FloatArray? {
|
||||
if (wxyz.size != 4 || wxyz.any { !it.isFinite() }) return null
|
||||
val length = sqrt(wxyz.sumOf { it.toDouble() * it }.toFloat())
|
||||
if (!length.isFinite() || length < 0.5f || length > 1.5f) return null
|
||||
val q = if (openGlAxes) floatArrayOf(-wxyz[3], -wxyz[1], wxyz[2], wxyz[0])
|
||||
else floatArrayOf(wxyz[1], wxyz[2], wxyz[3], wxyz[0])
|
||||
return q.map { it / length }.toFloatArray()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import android.hardware.usb.UsbDevice
|
||||
import android.hardware.usb.UsbDeviceConnection
|
||||
import android.hardware.usb.UsbManager
|
||||
import android.os.SystemClock
|
||||
import android.util.Log
|
||||
import java.io.File
|
||||
|
||||
/** SDK owns interface selection (including Ultra bulk endpoints) and all USB I/O. */
|
||||
internal class VitureSdkLink private constructor(
|
||||
private val connection: UsbDeviceConnection,
|
||||
private val token: Long,
|
||||
private val listener: Listener,
|
||||
family: VitureProtocol.Family?,
|
||||
) : GlassesLink {
|
||||
interface Listener {
|
||||
fun onValue(msgId: Int, value: Int)
|
||||
fun onPose(q: FloatArray, timeNs: Long)
|
||||
fun onLost()
|
||||
}
|
||||
|
||||
private val lock = Any()
|
||||
@Volatile private var running = true
|
||||
@Volatile private var tracking = false
|
||||
override val ready: Boolean get() = running
|
||||
override val hasImu = true
|
||||
private val openGlAxes = VitureSdkPose.usesOpenGlAxes(VitureSdk.nativeType(token) == VitureSdk.CARINA, family)
|
||||
private val worker = Thread({ poll() }, "viture-sdk-pose").apply { isDaemon = true; start() }
|
||||
|
||||
override fun send(msgId: Int, payload: ByteArray): Boolean = synchronized(lock) {
|
||||
if (!running) return false
|
||||
val value = payload.firstOrNull()?.toInt()?.and(0xff) ?: 0
|
||||
when (msgId) {
|
||||
VitureProtocol.MSG_IMU -> {
|
||||
val wanted = value != 0
|
||||
if (wanted == tracking) return true
|
||||
val rc = VitureSdk.nativeTracking(token, wanted)
|
||||
if (rc == 0 || !wanted) tracking = wanted
|
||||
return result("tracking", rc)
|
||||
}
|
||||
// Frequency is negotiated against SDK capabilities, including Pro 2's pose limit.
|
||||
VitureProtocol.MSG_IMU_RATE -> return tracking
|
||||
VitureProtocol.MSG_GET_IMU -> return true
|
||||
}
|
||||
val operation = when (msgId) {
|
||||
VitureProtocol.MSG_GET_DISPLAY -> VitureSdk.GET_DISPLAY
|
||||
VitureProtocol.MSG_DISPLAY -> VitureSdk.SET_DISPLAY
|
||||
VitureProtocol.MSG_GET_BRIGHTNESS -> VitureSdk.GET_BRIGHTNESS
|
||||
VitureProtocol.MSG_BRIGHTNESS -> VitureSdk.SET_BRIGHTNESS
|
||||
VitureProtocol.MSG_GET_VOLUME -> VitureSdk.GET_VOLUME
|
||||
VitureProtocol.MSG_VOLUME -> VitureSdk.SET_VOLUME
|
||||
VitureProtocol.MSG_FILM -> VitureSdk.SET_FILM
|
||||
else -> return false
|
||||
}
|
||||
val rc = VitureSdk.nativeCommand(token, operation, value)
|
||||
if (rc < 0) return result("command $operation", rc)
|
||||
if (operation == VitureSdk.GET_DISPLAY || operation == VitureSdk.GET_BRIGHTNESS || operation == VitureSdk.GET_VOLUME)
|
||||
listener.onValue(msgId, rc)
|
||||
else if (operation == VitureSdk.SET_DISPLAY) {
|
||||
val actual = VitureSdk.nativeCommand(token, VitureSdk.GET_DISPLAY, 0)
|
||||
if (actual >= 0) listener.onValue(VitureProtocol.MSG_GET_DISPLAY, actual)
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
private fun result(operation: String, rc: Int): Boolean {
|
||||
if (rc < 0) Log.w("VitureSdkLink", "$operation failed: $rc")
|
||||
return rc == 0
|
||||
}
|
||||
|
||||
private fun poll() {
|
||||
val pose = FloatArray(4)
|
||||
val event = IntArray(2)
|
||||
var failures = 0
|
||||
while (running) {
|
||||
val timestamp = synchronized(lock) {
|
||||
if (running) repeat(6) {
|
||||
if (VitureSdk.nativeNextState(token, event) == 1) when (event[0]) {
|
||||
0 -> listener.onValue(VitureProtocol.MSG_GET_BRIGHTNESS, event[1])
|
||||
1 -> listener.onValue(VitureProtocol.MSG_GET_VOLUME, event[1])
|
||||
// A device event can use a different display-mode namespace.
|
||||
2 -> send(VitureProtocol.MSG_GET_DISPLAY)
|
||||
3 -> listener.onValue(VitureProtocol.MSG_FILM, event[1])
|
||||
}
|
||||
}
|
||||
if (!running || !tracking) 0L else VitureSdk.nativePose(token, pose)
|
||||
}
|
||||
if (timestamp > 0) {
|
||||
failures = 0
|
||||
VitureSdkPose.toGlasses(pose, openGlAxes)?.let { listener.onPose(it, timestamp) }
|
||||
} else if (timestamp == -2L || timestamp == -3L) {
|
||||
if (++failures >= 10) {
|
||||
Log.w("VitureSdkLink", "closing after $failures pose USB errors; last SDK result: $timestamp")
|
||||
listener.onLost()
|
||||
break
|
||||
}
|
||||
}
|
||||
SystemClock.sleep(if (tracking) 4 else 25)
|
||||
}
|
||||
}
|
||||
|
||||
override fun close() {
|
||||
synchronized(lock) {
|
||||
if (!running) return
|
||||
running = false
|
||||
VitureSdk.nativeClose(token)
|
||||
// Keep Java's descriptor alive until the SDK and its worker threads have stopped.
|
||||
connection.close()
|
||||
}
|
||||
if (Thread.currentThread() !== worker) worker.join(1000)
|
||||
}
|
||||
|
||||
companion object {
|
||||
fun open(usb: UsbManager, device: UsbDevice, cacheDirectory: File, listener: Listener,
|
||||
family: VitureProtocol.Family? = null): VitureSdkLink? {
|
||||
if (!VitureSdk.available) return null
|
||||
val conn = usb.openDevice(device) ?: return null
|
||||
val token = runCatching {
|
||||
cacheDirectory.mkdirs()
|
||||
VitureSdk.nativeOpen(device.productId, conn.fileDescriptor, cacheDirectory.absolutePath)
|
||||
}.onFailure { Log.w("VitureSdkLink", "SDK initialization failed", it) }.getOrDefault(0)
|
||||
if (token <= 0) { conn.close(); return null }
|
||||
return runCatching { VitureSdkLink(conn, token, listener, family) }.getOrElse {
|
||||
VitureSdk.nativeClose(token)
|
||||
conn.close()
|
||||
Log.w("VitureSdkLink", "SDK worker could not start", it)
|
||||
null
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import android.content.Context
|
||||
import android.net.ConnectivityManager
|
||||
import android.net.LinkProperties
|
||||
import android.net.NetworkCapabilities
|
||||
import java.io.Closeable
|
||||
import java.io.IOException
|
||||
import java.io.InputStream
|
||||
import java.io.OutputStream
|
||||
import java.net.InetAddress
|
||||
import java.net.InetSocketAddress
|
||||
import java.net.Socket
|
||||
|
||||
internal interface XrealChannels : Closeable {
|
||||
val controlInput: InputStream
|
||||
val controlOutput: OutputStream
|
||||
val reports: InputStream
|
||||
}
|
||||
|
||||
internal interface XrealConnector : Closeable {
|
||||
fun connect(): XrealChannels
|
||||
}
|
||||
|
||||
internal class XrealNetworkMissing : IOException("XREAL USB Ethernet network is not available")
|
||||
|
||||
/** Bind individual sockets, never the process: Steam must retain its Internet network. */
|
||||
internal class AndroidXrealConnector(context: Context) : XrealConnector {
|
||||
private val manager = context.getSystemService(ConnectivityManager::class.java)
|
||||
private val address = InetAddress.getByName(XrealProtocol.HOST)
|
||||
private val lock = Any()
|
||||
private val sockets = mutableSetOf<Socket>()
|
||||
private var closed = false
|
||||
|
||||
companion object {
|
||||
internal fun hasDirectRoute(properties: LinkProperties, address: InetAddress): Boolean =
|
||||
properties.routes.any { !it.hasGateway() && it.destination.prefixLength > 0 && it.matches(address) }
|
||||
}
|
||||
|
||||
override fun connect(): XrealChannels {
|
||||
val network = manager.allNetworks.firstOrNull { network ->
|
||||
manager.getNetworkCapabilities(network)?.let {
|
||||
it.hasTransport(NetworkCapabilities.TRANSPORT_ETHERNET) &&
|
||||
!it.hasTransport(NetworkCapabilities.TRANSPORT_VPN)
|
||||
} == true && manager.getLinkProperties(network)?.let { hasDirectRoute(it, address) } == true
|
||||
} ?: throw XrealNetworkMissing()
|
||||
fun open(port: Int): Socket {
|
||||
val socket = Socket()
|
||||
synchronized(lock) {
|
||||
if (closed) { socket.close(); throw IOException("XREAL connection closed") }
|
||||
sockets.add(socket)
|
||||
}
|
||||
try {
|
||||
network.bindSocket(socket)
|
||||
socket.soTimeout = 2000
|
||||
socket.tcpNoDelay = true
|
||||
socket.connect(InetSocketAddress(address, port), 1500)
|
||||
return socket
|
||||
} catch (e: Exception) {
|
||||
release(socket)
|
||||
throw e
|
||||
}
|
||||
}
|
||||
val control = open(XrealProtocol.CONTROL_PORT)
|
||||
val reportSocket = try { open(XrealProtocol.REPORT_PORT) } catch (e: Exception) {
|
||||
release(control)
|
||||
throw e
|
||||
}
|
||||
return object : XrealChannels {
|
||||
override val controlInput = control.getInputStream()
|
||||
override val controlOutput = control.getOutputStream()
|
||||
override val reports = reportSocket.getInputStream()
|
||||
override fun close() { release(control); release(reportSocket) }
|
||||
}
|
||||
}
|
||||
|
||||
private fun release(socket: Socket) = synchronized(lock) {
|
||||
runCatching { socket.close() }
|
||||
sockets.remove(socket)
|
||||
Unit
|
||||
}
|
||||
|
||||
override fun close() = synchronized(lock) {
|
||||
closed = true
|
||||
sockets.forEach { runCatching { it.close() } }
|
||||
sockets.clear()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,201 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import android.content.Context
|
||||
import android.util.Log
|
||||
import java.io.IOException
|
||||
import java.util.concurrent.ArrayBlockingQueue
|
||||
import java.util.concurrent.TimeUnit
|
||||
import java.util.concurrent.atomic.AtomicReference
|
||||
|
||||
/** One connection owns two TCP channels. All control requests are serialized off the UI thread. */
|
||||
internal class XrealLink internal constructor(
|
||||
private val connector: XrealConnector,
|
||||
private val listener: Listener,
|
||||
) : GlassesLink {
|
||||
interface Listener {
|
||||
fun onReady()
|
||||
fun onValue(message: Int, value: Int)
|
||||
fun onPose(raw: ByteArray, pose: FloatArray, timeNs: Long)
|
||||
fun onUnavailable(networkMissing: Boolean)
|
||||
}
|
||||
|
||||
private data class Command(val message: Int, val value: Int)
|
||||
private val commands = ArrayBlockingQueue<Command>(16)
|
||||
private val connection = AtomicReference<XrealChannels?>()
|
||||
@Volatile private var running = true
|
||||
@Volatile private var tracking = false
|
||||
@Volatile override var ready = false
|
||||
private set
|
||||
override val hasImu = true
|
||||
private val worker = Thread(::run, "xreal-control").apply { isDaemon = true; start() }
|
||||
|
||||
companion object {
|
||||
fun open(context: Context, listener: Listener) = XrealLink(AndroidXrealConnector(context), listener)
|
||||
}
|
||||
|
||||
override fun send(msgId: Int, payload: ByteArray): Boolean {
|
||||
if (!running || !ready) return false
|
||||
val value = payload.firstOrNull()?.toInt()?.and(255) ?: -1
|
||||
if (msgId == VitureProtocol.MSG_IMU) {
|
||||
if (value !in 0..1) return false
|
||||
tracking = value == 1
|
||||
return true
|
||||
}
|
||||
if (msgId == VitureProtocol.MSG_GET_IMU) return true
|
||||
val valid = when (msgId) {
|
||||
VitureProtocol.MSG_GET_DISPLAY -> true
|
||||
VitureProtocol.MSG_DISPLAY -> value in 2..5 || value == XrealModel.NATIVE_60HZ
|
||||
VitureProtocol.MSG_BRIGHTNESS -> value in 0..9
|
||||
VitureProtocol.MSG_FILM -> value in 0..1
|
||||
else -> false // No guessed volume, sensor-rate, or VITURE wire commands.
|
||||
}
|
||||
return valid && commands.offer(Command(msgId, value))
|
||||
}
|
||||
|
||||
private fun run() {
|
||||
var lastError: String? = null
|
||||
while (running) {
|
||||
var channels: XrealChannels? = null
|
||||
var reports: Thread? = null
|
||||
val failure = AtomicReference<Exception?>()
|
||||
try {
|
||||
channels = connector.connect()
|
||||
connection.set(channels)
|
||||
if (!running) break
|
||||
val control = XrealControl(channels)
|
||||
listener.onValue(VitureProtocol.MSG_GET_DISPLAY, control.display())
|
||||
val current = channels
|
||||
reports = Thread({
|
||||
try {
|
||||
var filter = XrealOrientationFilter()
|
||||
var wasTracking = false
|
||||
var lastSensor = System.nanoTime()
|
||||
while (running && connection.get() === current) {
|
||||
val frame = XrealProtocol.read(current.reports)
|
||||
val sample = XrealProtocol.sample(frame)
|
||||
if (sample != null) {
|
||||
lastSensor = System.nanoTime()
|
||||
val enabled = tracking
|
||||
if (enabled && !wasTracking) filter = XrealOrientationFilter()
|
||||
wasTracking = enabled
|
||||
if (enabled) filter.update(sample)?.let {
|
||||
if (running && ready && tracking && connection.get() === current)
|
||||
listener.onPose(frame.body, it, System.nanoTime())
|
||||
}
|
||||
} else if (System.nanoTime() - lastSensor > 3_000_000_000L) {
|
||||
throw IOException("No valid XREAL IMU reports")
|
||||
}
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
failure.set(e)
|
||||
current.close() // Wake a control read immediately.
|
||||
}
|
||||
}, "xreal-reports").apply { isDaemon = true; start() }
|
||||
ready = true
|
||||
lastError = null
|
||||
listener.onReady()
|
||||
var heartbeat = System.nanoTime()
|
||||
while (running && failure.get() == null) {
|
||||
val command = commands.poll(250, TimeUnit.MILLISECONDS)
|
||||
if (command != null) {
|
||||
when (command.message) {
|
||||
VitureProtocol.MSG_GET_DISPLAY ->
|
||||
listener.onValue(command.message, control.display())
|
||||
VitureProtocol.MSG_DISPLAY -> {
|
||||
listener.onValue(VitureProtocol.MSG_GET_DISPLAY, control.changeDisplay(command.value))
|
||||
}
|
||||
VitureProtocol.MSG_BRIGHTNESS -> {
|
||||
control.set(XrealProtocol.SET_BRIGHTNESS, command.value)
|
||||
listener.onValue(command.message, command.value)
|
||||
}
|
||||
VitureProtocol.MSG_FILM -> {
|
||||
control.set(XrealProtocol.SET_DIMMER, if (command.value == 1) 2 else 0)
|
||||
listener.onValue(command.message, command.value)
|
||||
}
|
||||
}
|
||||
}
|
||||
if (System.nanoTime() - heartbeat > 5_000_000_000L) {
|
||||
listener.onValue(VitureProtocol.MSG_GET_DISPLAY, control.display())
|
||||
heartbeat = System.nanoTime()
|
||||
}
|
||||
}
|
||||
failure.get()?.let { throw it }
|
||||
} catch (e: Exception) {
|
||||
ready = false
|
||||
if (running && lastError != e.message) {
|
||||
lastError = e.message
|
||||
Log.w("XrealLink", "Connection unavailable: ${e.message}")
|
||||
listener.onUnavailable(e is XrealNetworkMissing)
|
||||
}
|
||||
} finally {
|
||||
ready = false
|
||||
tracking = false
|
||||
commands.clear()
|
||||
connection.compareAndSet(channels, null)
|
||||
channels?.close()
|
||||
try { reports?.join(2500) } catch (_: InterruptedException) { }
|
||||
}
|
||||
if (running) try { Thread.sleep(2000) } catch (_: InterruptedException) { }
|
||||
}
|
||||
}
|
||||
|
||||
override fun close() {
|
||||
running = false
|
||||
ready = false
|
||||
tracking = false
|
||||
commands.clear()
|
||||
connection.getAndSet(null)?.close()
|
||||
connector.close() // Includes sockets still inside connect().
|
||||
worker.interrupt()
|
||||
}
|
||||
}
|
||||
|
||||
/** Kept independent of Android so the real framing/acknowledgement path is testable. */
|
||||
internal class XrealControl(private val channels: XrealChannels, private val settleDelayMs: Long = 150) {
|
||||
private var transaction = 0
|
||||
private fun exchange(command: Int, value: Int? = null): ByteArray {
|
||||
transaction = if (transaction == Int.MAX_VALUE) 1 else transaction + 1
|
||||
channels.controlOutput.write(XrealProtocol.request(command, transaction, value))
|
||||
channels.controlOutput.flush()
|
||||
val deadline = System.nanoTime() + 3_000_000_000L
|
||||
repeat(32) { // Bound unsolicited button/status packets and stale responses.
|
||||
if (System.nanoTime() > deadline) throw IOException("XREAL control timed out")
|
||||
XrealProtocol.response(XrealProtocol.read(channels.controlInput), command, transaction)?.let { return it }
|
||||
}
|
||||
throw IOException("XREAL control response was not received")
|
||||
}
|
||||
fun display(): Int {
|
||||
val reply = exchange(XrealProtocol.GET_DISPLAY)
|
||||
return XrealProtocol.numeric(reply)?.takeIf { it in 2..5 || it == XrealModel.NATIVE_60HZ }
|
||||
?: throw IOException("Unsupported XREAL display response: ${reply.take(32).joinToString("") { "%02x".format(it) }}")
|
||||
}
|
||||
fun changeDisplay(mode: Int): Int {
|
||||
val stereo = mode == 5
|
||||
set(XrealProtocol.SET_INPUT, if (stereo) 1 else 0)
|
||||
// On 1S, SET_INPUT itself changes EDID. Sending SET_DISPLAY immediately
|
||||
// receives error 5001 while that change is pending. GET_DISPLAY returns
|
||||
// an empty property during the transition; wait for a real mode report.
|
||||
val current = awaitDisplay { (it == 5) == stereo }
|
||||
if (current == mode) return current
|
||||
set(XrealProtocol.SET_DISPLAY, mode)
|
||||
return awaitDisplay { it == mode }
|
||||
}
|
||||
private fun awaitDisplay(matches: (Int) -> Boolean): Int {
|
||||
val deadline = System.nanoTime() + 8_000_000_000L
|
||||
repeat(40) {
|
||||
val reply = exchange(XrealProtocol.GET_DISPLAY)
|
||||
val value = XrealProtocol.numeric(reply)
|
||||
if (value != null && matches(value)) return value
|
||||
if (value == null && !XrealProtocol.acknowledged(reply))
|
||||
throw IOException("Invalid XREAL display transition response")
|
||||
if (System.nanoTime() > deadline) throw IOException("XREAL display transition timed out")
|
||||
Thread.sleep(settleDelayMs)
|
||||
}
|
||||
throw IOException("XREAL display transition did not complete")
|
||||
}
|
||||
fun set(command: Int, value: Int) {
|
||||
val reply = exchange(command, value)
|
||||
if (!XrealProtocol.acknowledged(reply)) throw IOException("XREAL command %04x was not acknowledged: %s".format(
|
||||
command, reply.take(32).joinToString("") { "%02x".format(it) }))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
/** First One-family target. Recovery and Air USB products must not use this transport. */
|
||||
object XrealModel : GlassesModel {
|
||||
const val VENDOR_ID = 0x3318
|
||||
const val PRODUCT_1S = 0x043e
|
||||
const val NATIVE_60HZ = 29 // Captured on 1S: 1920 x 1200 at 60 Hz.
|
||||
override val name = "XREAL 1S"
|
||||
override val controllable = true
|
||||
override val brightnessMax = 9
|
||||
override val volumeMax = 0
|
||||
override val hasSunblock = true
|
||||
override val diagonalFovDeg = 52f
|
||||
override val sbs90 = false
|
||||
|
||||
fun matches(vid: Int, pid: Int) = vid == VENDOR_ID && pid == PRODUCT_1S
|
||||
fun isDisplay(name: String) = name.contains("XREAL", ignoreCase = true)
|
||||
|
||||
// One-family EDID configuration values, not VITURE's display mode IDs.
|
||||
override fun mode(hz: Int, sbs: Boolean) = when {
|
||||
sbs -> 5
|
||||
hz >= 120 -> 4
|
||||
hz >= 90 -> 3
|
||||
else -> NATIVE_60HZ
|
||||
}
|
||||
override fun modeForStep(step: Int): Int? = null
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import kotlin.math.abs
|
||||
|
||||
/** 1S radians/sec and device nanoseconds, mapped using directed hardware captures. */
|
||||
internal class XrealOrientationFilter {
|
||||
private var previous = 0L
|
||||
private var rotation = Quaternion.IDENTITY
|
||||
private var initialized = false
|
||||
private var bias = Vec3(0f, 0f, 0f)
|
||||
|
||||
fun update(sample: XrealProtocol.Sample): FloatArray? {
|
||||
if (sample.timeNs <= previous) return null
|
||||
val dt = if (previous == 0L) 0f else (sample.timeNs - previous) * 1e-9f
|
||||
previous = sample.timeNs
|
||||
// 1S reports right/down/forward. Rotate 180 degrees about X into the
|
||||
// shared right/up/back basis before either gravity alignment or fusion.
|
||||
// Captures: level gravity -Y; looking left -gyroY; up +gyroX; right tilt +gyroZ.
|
||||
val accel = Vec3(sample.accel.x, -sample.accel.y, -sample.accel.z)
|
||||
val gyro = Vec3(sample.gyro.x, -sample.gyro.y, -sample.gyro.z)
|
||||
val norm = accel.length()
|
||||
val gravityValid = abs(norm - 9.81f) < 1.5f
|
||||
if (dt > 0.25f) initialized = false // Never integrate a reconnect/stall as motion.
|
||||
if (!initialized) {
|
||||
if (!gravityValid) return null
|
||||
val up = accel.normalized()
|
||||
val axis = up.cross(Vec3(0f, 1f, 0f))
|
||||
rotation = if (up.y < -0.9999f) Quaternion.fromAxisAngle(Vec3(1f, 0f, 0f), Math.PI.toFloat())
|
||||
else Quaternion(1f + up.y, axis.x, axis.y, axis.z).normalized()
|
||||
initialized = true
|
||||
} else if (dt in 0.0001f..0.05f) {
|
||||
if (gravityValid && gyro.length() < 0.04f) {
|
||||
val alpha = (dt * 0.5f).coerceAtMost(1f)
|
||||
bias = bias * (1f - alpha) + gyro * alpha
|
||||
}
|
||||
var rate = gyro - bias
|
||||
if (gravityValid) {
|
||||
val predictedUp = rotation.conjugate().rotate(Vec3(0f, 1f, 0f))
|
||||
rate += accel.normalized().cross(predictedUp) * 1.5f
|
||||
}
|
||||
rotation = rotation.integrateBodyRate(rate, dt).normalized()
|
||||
}
|
||||
// Same right/up/back -> shared glasses basis used by the RayNeo adapter.
|
||||
return RayNeoOrientationFilter.glassesPose(floatArrayOf(rotation.w, rotation.x, rotation.y, rotation.z))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
package com.droiddeck.launcher.glasses
|
||||
|
||||
import java.io.EOFException
|
||||
import java.io.IOException
|
||||
import java.io.InputStream
|
||||
import java.nio.ByteBuffer
|
||||
import java.nio.ByteOrder
|
||||
|
||||
/** Independent implementation of the One-family wire format; see xreal-investigation.md. */
|
||||
internal object XrealProtocol {
|
||||
const val HOST = "169.254.2.1"
|
||||
const val REPORT_PORT = 52998
|
||||
const val CONTROL_PORT = 52999
|
||||
const val REPORT = 0x2836
|
||||
const val GET_DISPLAY = 0x275e
|
||||
const val SET_DISPLAY = 0x275f
|
||||
const val SET_INPUT = 0x2822
|
||||
const val SET_BRIGHTNESS = 0x271c
|
||||
const val SET_DIMMER = 0x2727
|
||||
const val MAX_BODY = 64 * 1024
|
||||
|
||||
data class Frame(val command: Int, val body: ByteArray)
|
||||
data class Sample(val timeNs: Long, val gyro: Vec3, val accel: Vec3)
|
||||
|
||||
fun read(input: InputStream): Frame {
|
||||
val deadline = System.nanoTime() + 3_000_000_000L
|
||||
val header = ByteArray(6)
|
||||
input.readFully(header, deadline)
|
||||
val size = ByteBuffer.wrap(header, 2, 4).int
|
||||
if (size !in 0..MAX_BODY) throw IOException("Invalid XREAL frame length: $size")
|
||||
return Frame(((header[0].toInt() and 255) shl 8) or (header[1].toInt() and 255),
|
||||
ByteArray(size).also { input.readFully(it, deadline) })
|
||||
}
|
||||
|
||||
private fun InputStream.readFully(bytes: ByteArray, deadline: Long) {
|
||||
var at = 0
|
||||
while (at < bytes.size) {
|
||||
if (System.nanoTime() > deadline) throw IOException("XREAL frame timed out")
|
||||
val count = read(bytes, at, bytes.size - at)
|
||||
if (count < 0) throw EOFException("XREAL stream closed mid-frame")
|
||||
if (count == 0) throw IOException("XREAL stream made no progress")
|
||||
at += count
|
||||
}
|
||||
}
|
||||
|
||||
fun sample(frame: Frame): Sample? {
|
||||
if (frame.command != REPORT || frame.body.size != 128) return null
|
||||
val b = ByteBuffer.wrap(frame.body).order(ByteOrder.LITTLE_ENDIAN)
|
||||
if (b.getInt(24) != 11) return null // Magnetometer reports are a different type.
|
||||
val time = b.getLong(8)
|
||||
val gyro = Vec3(b.getFloat(28), b.getFloat(32), b.getFloat(36))
|
||||
val accel = Vec3(b.getFloat(40), b.getFloat(44), b.getFloat(48))
|
||||
if (time <= 0 || !gyro.valid(100f) || !accel.valid(200f)) return null
|
||||
return Sample(time, gyro, accel)
|
||||
}
|
||||
|
||||
private fun Vec3.valid(limit: Float) = listOf(x, y, z).all { it.isFinite() && kotlin.math.abs(it) <= limit }
|
||||
|
||||
fun request(command: Int, transaction: Int, value: Int? = null): ByteArray {
|
||||
require(transaction > 0)
|
||||
require(value == null || value in 0..127)
|
||||
val payload = if (value == null) byteArrayOf(0x18, 0) else byteArrayOf(0x1a, 2, 8, value.toByte())
|
||||
return ByteBuffer.allocate(10 + payload.size).putShort(command.toShort())
|
||||
.putInt(4 + payload.size).putInt(transaction or Int.MIN_VALUE).put(payload).array()
|
||||
}
|
||||
|
||||
/** No acknowledgement is inferred from a write; match both command and transaction. */
|
||||
fun response(frame: Frame, command: Int, transaction: Int): ByteArray? {
|
||||
if (frame.command != command || frame.body.size < 4 ||
|
||||
ByteBuffer.wrap(frame.body).int != transaction) return null
|
||||
return frame.body.copyOfRange(4, frame.body.size)
|
||||
}
|
||||
|
||||
fun acknowledged(payload: ByteArray) = payload.contentEquals(byteArrayOf(0x22, 0))
|
||||
fun numeric(payload: ByteArray): Int? = if (payload.size == 4 &&
|
||||
payload[0] == 0x22.toByte() && payload[1] == 2.toByte() && payload[2] == 0x10.toByte() &&
|
||||
payload[3].toInt() in 0..127) payload[3].toInt() else null
|
||||
}
|
||||
@@ -12,13 +12,15 @@ data class Cardboard(
|
||||
val screenWidthDm: Int = 40,
|
||||
val screenDistanceDm: Int = 30,
|
||||
val headTracking: Boolean = true,
|
||||
val mono: Boolean = false,
|
||||
) {
|
||||
val lens: Viewer get() = Viewer.byId(viewer)
|
||||
|
||||
fun push(panelWidthMm: Float, panelHeightMm: Float) {
|
||||
fun push(panelWidthMm: Float, panelHeightMm: Float, fovOverrideDeg: Float = 0f, displayHz: Float = 0f) {
|
||||
val l = lens
|
||||
WaylandCompositor.nativeSetCardboard(enabled, source, ipdMm / 1000f, screenDistanceDm / 10f,
|
||||
screenWidthDm / 10f, l.k1, l.k2, fovDeg(l, panelHeightMm), lensInset(panelWidthMm, l.lensSeparationMm), l.chroma)
|
||||
screenWidthDm / 10f, l.k1, l.k2, if (fovOverrideDeg > 0f) fovOverrideDeg else fovDeg(l, panelHeightMm),
|
||||
if (mono) 0f else lensInset(panelWidthMm, l.lensSeparationMm), l.chroma, mono, displayHz)
|
||||
}
|
||||
|
||||
fun encode(): JSONObject = JSONObject()
|
||||
@@ -29,7 +31,8 @@ data class Cardboard(
|
||||
val lensSeparationMm: Float, val chroma: Float, val fallbackFovDeg: Float) {
|
||||
CARDBOARD_V2("cardboard2", 0.34f, 0.55f, 39.3f, 63.9f, 0.006f, 90f),
|
||||
CARDBOARD_V1("cardboard1", 0.441f, 0.156f, 42f, 60f, 0.004f, 80f),
|
||||
NONE("none", 0f, 0f, 0f, 0f, 0f, 60f);
|
||||
NONE("none", 0f, 0f, 0f, 0f, 0f, 60f),
|
||||
VITURE("viture", 0f, 0f, 0f, 0f, 0f, 23.3f);
|
||||
|
||||
companion object {
|
||||
fun byId(id: String): Viewer = entries.firstOrNull { it.id == id } ?: CARDBOARD_V2
|
||||
@@ -123,6 +126,10 @@ object HeadPose {
|
||||
a[r * 3] * b[c] + a[r * 3 + 1] * b[3 + c] + a[r * 3 + 2] * b[6 + c]
|
||||
}
|
||||
|
||||
private val GLASSES_AXES = floatArrayOf(0f, 0f, -1f, -1f, 0f, 0f, 0f, 1f, 0f)
|
||||
|
||||
fun fromGlasses(x: Float, y: Float, z: Float, w: Float): FloatArray = multiply(fromQuaternion(x, y, z, w), GLASSES_AXES)
|
||||
|
||||
fun transpose(m: FloatArray): FloatArray = floatArrayOf(m[0], m[3], m[6], m[1], m[4], m[7], m[2], m[5], m[8])
|
||||
|
||||
fun apply(m: FloatArray, v: FloatArray): FloatArray = floatArrayOf(
|
||||
|
||||
@@ -10,19 +10,42 @@ import android.os.HandlerThread
|
||||
import android.util.Log
|
||||
import com.droiddeck.launcher.wayland.WaylandCompositor
|
||||
|
||||
class HeadTracker(context: Context, private val rotation: () -> Int) : SensorEventListener {
|
||||
class HeadTracker(context: Context, private val glasses: Boolean = false, private val rotation: () -> Int) : SensorEventListener {
|
||||
private val sensors = context.getSystemService(Context.SENSOR_SERVICE) as SensorManager
|
||||
private val sensor: Sensor? = sensors.getDefaultSensor(Sensor.TYPE_GAME_ROTATION_VECTOR)
|
||||
?: sensors.getDefaultSensor(Sensor.TYPE_ROTATION_VECTOR)
|
||||
private var thread: HandlerThread? = null
|
||||
private var started = false
|
||||
private val glassesLock = Any()
|
||||
@Volatile private var reference: HeadPose.Reference? = null
|
||||
@Volatile private var recenterPending = true
|
||||
@Volatile private var latest: FloatArray? = null
|
||||
|
||||
private val glassesSink = com.droiddeck.launcher.glasses.Glasses.PoseSink { x, y, z, w, _ ->
|
||||
val m = HeadPose.fromGlasses(x, y, z, w)
|
||||
synchronized(glassesLock) {
|
||||
if (!started) return@PoseSink
|
||||
latest = m
|
||||
apply(m)
|
||||
}
|
||||
}
|
||||
|
||||
val usesGlasses: Boolean get() = glasses
|
||||
|
||||
val available: Boolean get() = sensor != null
|
||||
val sensorName: String get() = sensor?.let { "${it.name} (${if (it.type == Sensor.TYPE_GAME_ROTATION_VECTOR) "game rotation vector" else "rotation vector"})" } ?: "none"
|
||||
|
||||
fun start() {
|
||||
if (glasses) {
|
||||
synchronized(glassesLock) {
|
||||
if (started) return
|
||||
started = true
|
||||
recenterPending = true
|
||||
}
|
||||
com.droiddeck.launcher.glasses.Glasses.setPoseSink(glassesSink)
|
||||
WaylandCompositor.nativeLog("cardboard", "head tracking on: VITURE glasses IMU")
|
||||
return
|
||||
}
|
||||
val s = sensor ?: run {
|
||||
WaylandCompositor.nativeLog("cardboard", "no rotation sensor: the virtual screen stays head-locked")
|
||||
WaylandCompositor.nativeSetCardboardPose(HeadPose.IDENTITY, false)
|
||||
@@ -37,6 +60,15 @@ class HeadTracker(context: Context, private val rotation: () -> Int) : SensorEve
|
||||
}
|
||||
|
||||
fun stop() {
|
||||
if (glasses) {
|
||||
synchronized(glassesLock) {
|
||||
if (!started) return
|
||||
started = false
|
||||
WaylandCompositor.nativeSetCardboardPose(HeadPose.IDENTITY, false)
|
||||
}
|
||||
com.droiddeck.launcher.glasses.Glasses.clearPoseSink(glassesSink)
|
||||
return
|
||||
}
|
||||
val t = thread ?: return
|
||||
sensors.unregisterListener(this)
|
||||
t.quitSafely()
|
||||
@@ -46,7 +78,8 @@ class HeadTracker(context: Context, private val rotation: () -> Int) : SensorEve
|
||||
|
||||
fun recenter() {
|
||||
recenterPending = true
|
||||
latest?.let { apply(it) }
|
||||
if (glasses) synchronized(glassesLock) { if (started) latest?.let { apply(it) } }
|
||||
else latest?.let { apply(it) }
|
||||
}
|
||||
|
||||
override fun onSensorChanged(event: SensorEvent) {
|
||||
|
||||
@@ -5,6 +5,8 @@ import com.droiddeck.launcher.agent.AgentGuest
|
||||
import org.json.JSONObject
|
||||
|
||||
object SteamVr {
|
||||
const val APP_ID = "250820"
|
||||
|
||||
enum class Status {
|
||||
UNAVAILABLE,
|
||||
NOT_INSTALLED,
|
||||
@@ -13,6 +15,23 @@ object SteamVr {
|
||||
RUNNING,
|
||||
}
|
||||
|
||||
internal fun shouldPoll(status: Status?, cardboardEnabled: Boolean, pictureSwitched: Boolean,
|
||||
glassesControl: Boolean, requested: Boolean): Boolean =
|
||||
requested || cardboardEnabled || pictureSwitched || glassesControl ||
|
||||
status == Status.RUNNING || status == Status.INSTALLING
|
||||
|
||||
internal fun autoAction(
|
||||
status: Status, requested: Boolean, loading: Boolean, steamSession: Boolean,
|
||||
startTried: Boolean, installTried: Boolean,
|
||||
): String? {
|
||||
if (!requested || loading || !steamSession || startTried) return null
|
||||
return when (status) {
|
||||
Status.READY -> "start"
|
||||
Status.NOT_INSTALLED -> if (installTried) null else "install"
|
||||
else -> null
|
||||
}
|
||||
}
|
||||
|
||||
fun status(context: Context): Status = runCatching {
|
||||
parse(AgentGuest.call(context, JSONObject().put("kind", "vr").put("action", "status"), STATUS_TIMEOUT_MS))
|
||||
}.getOrDefault(Status.UNAVAILABLE)
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
package com.droiddeck.launcher.gpu
|
||||
|
||||
import org.json.JSONObject
|
||||
import java.io.File
|
||||
import java.io.RandomAccessFile
|
||||
import java.nio.file.Files
|
||||
import java.nio.file.attribute.BasicFileAttributes
|
||||
|
||||
/** Only new compositor output is evidence of a fault in the current session. */
|
||||
internal class SteamVrDeviceLossLog(private val file: File) {
|
||||
private var identity: Any? = identity()
|
||||
private var position = file.length()
|
||||
private var prefix = runCatching {
|
||||
RandomAccessFile(file, "r").use { input -> ByteArray(minOf(64, input.length()).toInt()).also(input::readFully) }
|
||||
}.getOrDefault(byteArrayOf())
|
||||
private var partial = ""
|
||||
private var handled = false
|
||||
|
||||
private fun identity(): Any? = runCatching {
|
||||
Files.readAttributes(file.toPath(), BasicFileAttributes::class.java).fileKey()
|
||||
}.getOrNull()
|
||||
|
||||
fun poll(): Boolean {
|
||||
if (handled) return true
|
||||
if (!file.isFile) return false
|
||||
return runCatching {
|
||||
val current = identity()
|
||||
RandomAccessFile(file, "r").use { input ->
|
||||
val head = ByteArray(minOf(64, input.length()).toInt()).also(input::readFully)
|
||||
val replaced = prefix.isNotEmpty() && !head.take(prefix.size).toByteArray().contentEquals(prefix)
|
||||
if (current != identity || input.length() < position || replaced) {
|
||||
position = 0
|
||||
partial = ""
|
||||
}
|
||||
identity = current
|
||||
prefix = head
|
||||
// Fault loops can produce megabytes per second. Bound work and keep the newest output.
|
||||
if (input.length() - position > MAX_READ) {
|
||||
position = input.length() - MAX_READ
|
||||
partial = ""
|
||||
}
|
||||
input.seek(position)
|
||||
val bytes = ByteArray(minOf(MAX_READ, input.length() - position).toInt())
|
||||
input.readFully(bytes)
|
||||
position = input.filePointer
|
||||
val text = partial + bytes.toString(Charsets.UTF_8)
|
||||
val end = text.lastIndexOf('\n')
|
||||
partial = text.substring(end + 1).takeLast(2048)
|
||||
if (end >= 0 && isDeviceLoss(text.substring(0, end + 1))) handled = true
|
||||
handled
|
||||
}
|
||||
}.getOrDefault(false)
|
||||
}
|
||||
|
||||
companion object {
|
||||
private const val MAX_READ = 128 * 1024L
|
||||
internal fun isDeviceLoss(text: String): Boolean =
|
||||
Regex("\\bVK_ERROR_DEVICE_LOST\\b|\\bvkerror\\s*=\\s*-4\\b", RegexOption.IGNORE_CASE).containsMatchIn(text)
|
||||
}
|
||||
}
|
||||
|
||||
internal object SteamVrRecoveryPolicy {
|
||||
const val COOLDOWN_MS = 10 * 60 * 1000L
|
||||
const val STOP_MS = 15_000L
|
||||
const val LOGS_MS = 30_000L
|
||||
private const val POLL_MS = 250L
|
||||
|
||||
enum class Shutdown { COMPLETE, LOGS_PENDING, STUCK }
|
||||
|
||||
fun canRestartAutomatically(lastAttempt: Long, now: Long): Boolean =
|
||||
lastAttempt == 0L || (now >= lastAttempt && now - lastAttempt >= COOLDOWN_MS)
|
||||
|
||||
fun awaitShutdown(session: () -> JSONObject?, now: () -> Long, sleep: (Long) -> Unit): Shutdown {
|
||||
fun await(ms: Long, done: (JSONObject) -> Boolean): Boolean {
|
||||
val deadline = now() + ms
|
||||
while (true) {
|
||||
if (session()?.let(done) == true) return true
|
||||
if (now() >= deadline) return false
|
||||
sleep(POLL_MS)
|
||||
}
|
||||
}
|
||||
if (!await(STOP_MS) { it.optString("phase") in setOf("IDLE", "FAILED") }) return Shutdown.STUCK
|
||||
return if (await(LOGS_MS) { !it.optBoolean("artifactsAvailable") || it.optBoolean("artifactsComplete") }) Shutdown.COMPLETE
|
||||
else Shutdown.LOGS_PENDING
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
package com.droiddeck.launcher.gpu
|
||||
|
||||
import android.app.Activity
|
||||
import android.app.AlertDialog
|
||||
import android.content.Context
|
||||
import android.content.Intent
|
||||
import android.os.Handler
|
||||
import android.os.Looper
|
||||
import com.droiddeck.launcher.R
|
||||
import com.droiddeck.launcher.runtime.LinuxRuntime
|
||||
import com.droiddeck.launcher.session.SessionEvents
|
||||
import java.io.File
|
||||
|
||||
/** Process-wide cursor survives Activity recreation; a new session ignores older log errors. */
|
||||
internal object SteamVrHealth {
|
||||
private var session: String? = null
|
||||
private var log: SteamVrDeviceLossLog? = null
|
||||
|
||||
@Synchronized fun begin(context: Context, sessionId: String?) {
|
||||
if (sessionId == session && log != null) return
|
||||
session = sessionId
|
||||
log = SteamVrDeviceLossLog(File(LinuxRuntime.rootDir(context),
|
||||
"root/.local/share/Steam/logs/vrcompositor.txt"))
|
||||
}
|
||||
|
||||
@Synchronized fun deviceLost(): Boolean = log?.poll() == true
|
||||
}
|
||||
|
||||
internal class SteamVrRecovery(private val activity: Activity, private val foreground: () -> Boolean) {
|
||||
private val handler = Handler(Looper.getMainLooper())
|
||||
private var dialog: AlertDialog? = null
|
||||
private var handled = false
|
||||
private var countdown: Runnable? = null
|
||||
|
||||
fun detected() {
|
||||
if (handled || !foreground() || activity.isFinishing || activity.isDestroyed) return
|
||||
handled = true
|
||||
SessionEvents.record("steamvr.device_lost")
|
||||
val prefs = activity.getSharedPreferences(PREFS, Context.MODE_PRIVATE)
|
||||
val automatic = SteamVrRecoveryPolicy.canRestartAutomatically(prefs.getLong(LAST_ATTEMPT, 0), System.currentTimeMillis())
|
||||
var seconds = 8
|
||||
fun message() = if (automatic) activity.getString(R.string.steamvr_recovery_countdown, seconds)
|
||||
else activity.getString(R.string.steamvr_recovery_repeated)
|
||||
val prompt = AlertDialog.Builder(activity)
|
||||
.setTitle(R.string.steamvr_recovery_title)
|
||||
.setMessage(message())
|
||||
.setPositiveButton(R.string.steamvr_recovery_restart) { _, _ -> restart() }
|
||||
.setNegativeButton(R.string.steamvr_recovery_cancel) { _, _ -> }
|
||||
.create()
|
||||
prompt.setOnDismissListener { countdown?.let(handler::removeCallbacks); countdown = null; dialog = null }
|
||||
dialog = prompt
|
||||
// A touch meant for the game must not silently dismiss the recovery controls.
|
||||
prompt.setCanceledOnTouchOutside(false)
|
||||
prompt.show()
|
||||
if (automatic) {
|
||||
countdown = object : Runnable {
|
||||
override fun run() {
|
||||
if (!foreground() || activity.isFinishing || activity.isDestroyed) { handled = false; prompt.dismiss(); return }
|
||||
if (--seconds <= 0) { prompt.dismiss(); restart(); return }
|
||||
prompt.setMessage(message())
|
||||
handler.postDelayed(this, 1000)
|
||||
}
|
||||
}.also { handler.postDelayed(it, 1000) }
|
||||
}
|
||||
}
|
||||
|
||||
private fun restart() {
|
||||
if (!foreground()) return
|
||||
// Commit before crossing processes so a repeated fault cannot create a restart loop.
|
||||
if (!activity.getSharedPreferences(PREFS, Context.MODE_PRIVATE).edit()
|
||||
.putLong(LAST_ATTEMPT, System.currentTimeMillis()).commit()) return
|
||||
SessionEvents.record("steamvr.recovery_requested")
|
||||
activity.startActivity(Intent(activity, SteamVrRestartActivity::class.java)
|
||||
.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK))
|
||||
}
|
||||
|
||||
fun close() { countdown?.let(handler::removeCallbacks); countdown = null; dialog?.dismiss() }
|
||||
|
||||
companion object {
|
||||
internal const val PREFS = "steamvr_recovery"
|
||||
internal const val LAST_ATTEMPT = "lastAttempt"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
package com.droiddeck.launcher.gpu
|
||||
|
||||
import android.util.AtomicFile
|
||||
import org.json.JSONObject
|
||||
import java.io.File
|
||||
|
||||
internal object SteamVrRecoverySettings {
|
||||
/** After a graphics crash SteamVR can quarantine our HMD, leaving a server with no VR output. */
|
||||
fun unblockHeadset(file: File): Boolean {
|
||||
if (!file.isFile) return false
|
||||
val settings = JSONObject(file.readText())
|
||||
val driver = settings.optJSONObject("driver_droiddeck") ?: return false
|
||||
if (!driver.optBoolean("blocked_by_safe_mode")) return false
|
||||
// Preserve every other setting, including safe mode and blocks on other drivers.
|
||||
driver.put("blocked_by_safe_mode", false)
|
||||
val atomic = AtomicFile(file)
|
||||
val output = atomic.startWrite()
|
||||
try {
|
||||
output.write(settings.toString(3).toByteArray(Charsets.UTF_8))
|
||||
atomic.finishWrite(output)
|
||||
} catch (e: Exception) {
|
||||
atomic.failWrite(output)
|
||||
throw e
|
||||
}
|
||||
return true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
package com.droiddeck.launcher.gpu
|
||||
|
||||
import android.app.Activity
|
||||
import android.app.ActivityManager
|
||||
import android.content.Intent
|
||||
import android.net.Uri
|
||||
import android.os.Bundle
|
||||
import android.os.Process
|
||||
import android.os.SystemClock
|
||||
import android.view.Gravity
|
||||
import android.widget.TextView
|
||||
import com.droiddeck.launcher.R
|
||||
import com.droiddeck.launcher.SessionActivity
|
||||
import com.droiddeck.launcher.session.SessionService
|
||||
import org.json.JSONObject
|
||||
|
||||
/** A visible, private process survives destruction of the app's native graphics context. */
|
||||
class SteamVrRestartActivity : Activity() {
|
||||
override fun onCreate(savedInstanceState: Bundle?) {
|
||||
super.onCreate(savedInstanceState)
|
||||
setContentView(TextView(this).apply {
|
||||
setText(R.string.steamvr_recovery_restarting)
|
||||
gravity = Gravity.CENTER
|
||||
textSize = 22f
|
||||
})
|
||||
Thread({ recover() }, "steamvr-process-recovery").start()
|
||||
}
|
||||
|
||||
private fun recover() {
|
||||
try {
|
||||
val manager = getSystemService(ACTIVITY_SERVICE) as ActivityManager
|
||||
val main = manager.runningAppProcesses?.firstOrNull {
|
||||
it.processName == packageName && it.uid == Process.myUid() && it.pid != Process.myPid()
|
||||
} ?: error("DroidDeck process is unavailable")
|
||||
val uri = Uri.parse("content://$packageName.agent")
|
||||
contentResolver.call(uri, "stop", null, null)
|
||||
val shutdown = SteamVrRecoveryPolicy.awaitShutdown({
|
||||
runCatching {
|
||||
contentResolver.call(uri, "state", null, null)?.getString("json")?.let(::JSONObject)?.optJSONObject("session")
|
||||
}.getOrNull()
|
||||
}, SystemClock::elapsedRealtime, SystemClock::sleep)
|
||||
if (shutdown != SteamVrRecoveryPolicy.Shutdown.COMPLETE)
|
||||
android.util.Log.w("SteamVrRecovery", "Restarting before the session finished stopping: $shutdown")
|
||||
// Only the verified main process, never a caller-supplied PID or an external app.
|
||||
Process.killProcess(main.pid)
|
||||
val exited = SystemClock.elapsedRealtime() + 5000
|
||||
while (java.io.File("/proc/${main.pid}").exists() && SystemClock.elapsedRealtime() < exited) SystemClock.sleep(50)
|
||||
check(!java.io.File("/proc/${main.pid}").exists()) { "DroidDeck process did not exit" }
|
||||
val settings = java.io.File(com.droiddeck.launcher.runtime.LinuxRuntime.rootDir(this),
|
||||
"root/.local/share/Steam/config/steamvr.vrsettings")
|
||||
runCatching { SteamVrRecoverySettings.unblockHeadset(settings) }
|
||||
.onSuccess { if (it) android.util.Log.i("SteamVrRecovery", "Cleared the DroidDeck HMD crash block before restarting SteamVR") }
|
||||
.onFailure { android.util.Log.w("SteamVrRecovery", "Could not read the SteamVR settings; restarting with them unchanged", it) }
|
||||
runOnUiThread {
|
||||
startActivity(Intent(this, SessionActivity::class.java)
|
||||
.putExtra(SessionService.EXTRA_MODE, SessionService.MODE_STEAM)
|
||||
.putExtra(SessionActivity.EXTRA_STEAMVR, true)
|
||||
.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK or Intent.FLAG_ACTIVITY_CLEAR_TASK))
|
||||
finishAndRemoveTask()
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
android.util.Log.e("SteamVrRecovery", "Could not restart DroidDeck", e)
|
||||
runOnUiThread {
|
||||
setContentView(TextView(this).apply {
|
||||
setText(R.string.steamvr_recovery_failed)
|
||||
gravity = Gravity.CENTER
|
||||
textSize = 22f
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,255 @@
|
||||
package com.droiddeck.launcher.handtracking
|
||||
|
||||
import com.droiddeck.launcher.glasses.Vec3
|
||||
import kotlin.math.abs
|
||||
import kotlin.math.atan2
|
||||
import kotlin.math.hypot
|
||||
|
||||
internal class Landmark(val x: Float, val y: Float, val visibility: Float)
|
||||
|
||||
internal class WorldPoint(val x: Float, val y: Float, val z: Float)
|
||||
|
||||
internal class Intrinsics(val fx: Float, val fy: Float, val cx: Float, val cy: Float, val width: Int, val height: Int)
|
||||
|
||||
internal class CameraLevel(up: Vec3) {
|
||||
val right: Vec3
|
||||
val up: Vec3
|
||||
val back: Vec3
|
||||
val valid: Boolean
|
||||
|
||||
init {
|
||||
val u = up.normalized()
|
||||
val forward = Vec3(0f, 0f, 1f)
|
||||
val along = u.x * forward.x + u.y * forward.y + u.z * forward.z
|
||||
val horizontal = Vec3(forward.x - u.x * along, forward.y - u.y * along, forward.z - u.z * along)
|
||||
valid = u.length() > 0.5f && horizontal.length() > MIN_HORIZONTAL
|
||||
this.up = u
|
||||
back = (horizontal * -1f).normalized()
|
||||
right = this.up.cross(back)
|
||||
}
|
||||
|
||||
fun level(v: Vec3) = Vec3(dot(v, right), dot(v, up), dot(v, back))
|
||||
|
||||
private fun dot(a: Vec3, b: Vec3) = a.x * b.x + a.y * b.y + a.z * b.z
|
||||
|
||||
companion object {
|
||||
private const val MIN_HORIZONTAL = 0.25f
|
||||
}
|
||||
}
|
||||
|
||||
internal class BodyFrame(
|
||||
val hands: Array<Vec3?>,
|
||||
val faceYaw: Float?,
|
||||
val distance: Float,
|
||||
val shoulderSpan: Float,
|
||||
val seen: Boolean,
|
||||
val estimated: BooleanArray = BooleanArray(2),
|
||||
val edgeRoom: Float = 1f,
|
||||
val joints: Array<FloatArray?> = arrayOfNulls(2),
|
||||
)
|
||||
|
||||
internal class HandFix(val palm: Vec3, val x: Float, val y: Float, val extent: Float, val points: FloatArray, val world: List<WorldPoint> = emptyList())
|
||||
|
||||
internal object Pnp {
|
||||
fun translation(count: Int, k: Intrinsics, x: (Int) -> Float, y: (Int) -> Float, world: (Int) -> WorldPoint): Vec3? {
|
||||
val ata = Array(3) { DoubleArray(3) }
|
||||
val atb = DoubleArray(3)
|
||||
for (i in 0 until count) {
|
||||
val a = ((x(i) - k.cx) / k.fx).toDouble()
|
||||
val b = ((y(i) - k.cy) / k.fy).toDouble()
|
||||
val w = world(i)
|
||||
add(ata, atb, doubleArrayOf(1.0, 0.0, -a), a * w.z - w.x)
|
||||
add(ata, atb, doubleArrayOf(0.0, 1.0, -b), b * w.z - w.y)
|
||||
}
|
||||
val t = solve3(ata, atb) ?: return null
|
||||
return Vec3(t[0].toFloat(), t[1].toFloat(), t[2].toFloat())
|
||||
}
|
||||
|
||||
private fun add(ata: Array<DoubleArray>, atb: DoubleArray, row: DoubleArray, rhs: Double) {
|
||||
for (i in 0..2) {
|
||||
atb[i] += row[i] * rhs
|
||||
for (j in 0..2) ata[i][j] += row[i] * row[j]
|
||||
}
|
||||
}
|
||||
|
||||
private fun solve3(a: Array<DoubleArray>, b: DoubleArray): DoubleArray? {
|
||||
val det = det3(a)
|
||||
if (abs(det) < 1e-12) return null
|
||||
return DoubleArray(3) { col -> det3(Array(3) { r -> DoubleArray(3) { c -> if (c == col) b[r] else a[r][c] } }) / det }
|
||||
}
|
||||
|
||||
private fun det3(m: Array<DoubleArray>) = m[0][0] * (m[1][1] * m[2][2] - m[1][2] * m[2][1]) -
|
||||
m[0][1] * (m[1][0] * m[2][2] - m[1][2] * m[2][0]) + m[0][2] * (m[1][0] * m[2][1] - m[1][1] * m[2][0])
|
||||
}
|
||||
|
||||
internal object BodyPose {
|
||||
const val NOSE = 0
|
||||
const val LEFT_EYE = 2
|
||||
const val RIGHT_EYE = 5
|
||||
const val LEFT_EAR = 7
|
||||
const val RIGHT_EAR = 8
|
||||
const val RIGHT_MOUTH = 10
|
||||
const val LEFT_SHOULDER = 11
|
||||
const val RIGHT_SHOULDER = 12
|
||||
const val LEFT_ELBOW = 13
|
||||
const val RIGHT_ELBOW = 14
|
||||
const val LEFT_WRIST = 15
|
||||
const val RIGHT_WRIST = 16
|
||||
const val LEFT_PINKY = 17
|
||||
const val RIGHT_PINKY = 18
|
||||
const val LEFT_INDEX = 19
|
||||
const val RIGHT_INDEX = 20
|
||||
const val LEFT_HIP = 23
|
||||
const val RIGHT_HIP = 24
|
||||
|
||||
private val ANCHORS = intArrayOf(NOSE, LEFT_EYE, RIGHT_EYE, LEFT_EAR, RIGHT_EAR, LEFT_SHOULDER, RIGHT_SHOULDER, LEFT_HIP, RIGHT_HIP)
|
||||
private val WRISTS = intArrayOf(LEFT_WRIST, RIGHT_WRIST)
|
||||
private val PINKIES = intArrayOf(LEFT_PINKY, RIGHT_PINKY)
|
||||
private val INDEXES = intArrayOf(LEFT_INDEX, RIGHT_INDEX)
|
||||
const val MIN_VISIBILITY = 0.6f
|
||||
|
||||
fun solve(
|
||||
image: List<Landmark>,
|
||||
world: List<WorldPoint>,
|
||||
k: Intrinsics,
|
||||
level: CameraLevel,
|
||||
fixes: Array<HandFix?>? = null,
|
||||
held: FloatArray? = null,
|
||||
): BodyFrame? {
|
||||
if (image.size <= RIGHT_HIP || world.size <= RIGHT_HIP || !level.valid) return null
|
||||
val t = translation(image, world, k) ?: return null
|
||||
fun at(i: Int): Vec3 {
|
||||
val w = world[i]
|
||||
val depth = w.z + t.z
|
||||
val px = image[i].x * k.width
|
||||
val py = image[i].y * k.height
|
||||
return Vec3((px - k.cx) / k.fx * depth, (py - k.cy) / k.fy * depth, depth)
|
||||
}
|
||||
fun visible(i: Int) = image[i].visibility >= MIN_VISIBILITY && image[i].x in 0f..1f && image[i].y in 0f..1f
|
||||
val eyesSeen = visible(LEFT_EYE) && visible(RIGHT_EYE)
|
||||
val shouldersSeen = visible(LEFT_SHOULDER) && visible(RIGHT_SHOULDER)
|
||||
val hands = arrayOfNulls<Vec3>(2)
|
||||
val estimated = BooleanArray(2)
|
||||
val joints = arrayOfNulls<FloatArray>(2)
|
||||
if (fixes == null) {
|
||||
val head = if (eyesSeen) (at(LEFT_EYE) + at(RIGHT_EYE)) * 0.5f else null
|
||||
if (head != null) for (side in 0..1) {
|
||||
if (!visible(WRISTS[side])) continue
|
||||
val grip = (at(WRISTS[side]) + at(PINKIES[side]) + at(INDEXES[side])) * (1f / 3f)
|
||||
hands[side] = level.level(grip - head)
|
||||
}
|
||||
} else {
|
||||
val head = if (eyesSeen) eyes(image, world, k) else null
|
||||
if (head != null) for (side in 0..1) {
|
||||
val fix = fixes.getOrNull(side)
|
||||
if (fix != null) {
|
||||
hands[side] = level.level(fix.palm - head)
|
||||
joints[side] = handJoints(fix.world, level)
|
||||
continue
|
||||
}
|
||||
val depth = held?.getOrNull(side)?.takeIf { it.isFinite() && it > 0f } ?: continue
|
||||
if (!visible(WRISTS[side])) continue
|
||||
val gx = (image[WRISTS[side]].x + image[PINKIES[side]].x + image[INDEXES[side]].x) / 3f * k.width
|
||||
val gy = (image[WRISTS[side]].y + image[PINKIES[side]].y + image[INDEXES[side]].y) / 3f * k.height
|
||||
hands[side] = level.level(Vec3((gx - k.cx) / k.fx * depth, (gy - k.cy) / k.fy * depth, depth) - head)
|
||||
estimated[side] = true
|
||||
}
|
||||
}
|
||||
val faceYaw = if (visible(NOSE) && visible(LEFT_EAR) && visible(RIGHT_EAR)) {
|
||||
val n = world[NOSE]
|
||||
val l = world[LEFT_EAR]
|
||||
val r = world[RIGHT_EAR]
|
||||
val f = level.level(Vec3(n.x - (l.x + r.x) / 2, n.y - (l.y + r.y) / 2, n.z - (l.z + r.z) / 2))
|
||||
if (hypot(f.x, f.z) > 0.02f) atan2(-f.x, -f.z) else null
|
||||
} else null
|
||||
val span = if (shouldersSeen) hypot(image[LEFT_SHOULDER].x - image[RIGHT_SHOULDER].x,
|
||||
(image[LEFT_SHOULDER].y - image[RIGHT_SHOULDER].y) * k.height / k.width) else 0f
|
||||
var room = 1f
|
||||
for (i in EDGE_JOINTS) {
|
||||
if (image[i].visibility < MIN_VISIBILITY) continue
|
||||
room = minOf(room, image[i].x, 1f - image[i].x, image[i].y, 1f - image[i].y)
|
||||
}
|
||||
return BodyFrame(hands, faceYaw, t.z, span, eyesSeen && shouldersSeen, estimated, room, joints)
|
||||
}
|
||||
|
||||
fun handJoints(world: List<WorldPoint>, level: CameraLevel): FloatArray? {
|
||||
if (world.size < HAND_POINTS) return null
|
||||
var mx = 0f
|
||||
var my = 0f
|
||||
var mz = 0f
|
||||
for (i in PALM) {
|
||||
mx += world[i].x
|
||||
my += world[i].y
|
||||
mz += world[i].z
|
||||
}
|
||||
val n = PALM.size.toFloat()
|
||||
val out = FloatArray(HAND_POINTS * 3)
|
||||
for (i in 0 until HAND_POINTS) {
|
||||
val v = level.level(Vec3(world[i].x - mx / n, world[i].y - my / n, world[i].z - mz / n))
|
||||
out[i * 3] = v.x
|
||||
out[i * 3 + 1] = v.y
|
||||
out[i * 3 + 2] = v.z
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
fun eyes(image: List<Landmark>, world: List<WorldPoint>, k: Intrinsics): Vec3? {
|
||||
val used = HEAD.filter { image[it].visibility >= MIN_VISIBILITY }
|
||||
if (used.size < MIN_HEAD_POINTS) return null
|
||||
val t = Pnp.translation(used.size, k, { image[used[it]].x * k.width }, { image[used[it]].y * k.height }, { world[used[it]] })
|
||||
?: return null
|
||||
if (!(t.z > MIN_DISTANCE && t.z < MAX_DISTANCE)) return null
|
||||
val l = world[LEFT_EYE]
|
||||
val r = world[RIGHT_EYE]
|
||||
return Vec3(t.x + (l.x + r.x) / 2, t.y + (l.y + r.y) / 2, t.z + (l.z + r.z) / 2)
|
||||
}
|
||||
|
||||
fun handFix(points: FloatArray, world: List<WorldPoint>, k: Intrinsics): HandFix? {
|
||||
if (world.size < HAND_POINTS || points.size < HAND_POINTS * 2) return null
|
||||
val t = Pnp.translation(HAND_POINTS, k, { points[it * 2] }, { points[it * 2 + 1] }, { world[it] }) ?: return null
|
||||
if (!(t.z > MIN_HAND_DISTANCE && t.z < MAX_DISTANCE)) return null
|
||||
var px = 0f
|
||||
var py = 0f
|
||||
var wx = 0f
|
||||
var wy = 0f
|
||||
var wz = 0f
|
||||
for (i in PALM) {
|
||||
px += points[i * 2]
|
||||
py += points[i * 2 + 1]
|
||||
wx += world[i].x
|
||||
wy += world[i].y
|
||||
wz += world[i].z
|
||||
}
|
||||
val n = PALM.size.toFloat()
|
||||
var minX = Float.MAX_VALUE
|
||||
var maxX = -Float.MAX_VALUE
|
||||
var minY = Float.MAX_VALUE
|
||||
var maxY = -Float.MAX_VALUE
|
||||
for (i in 0 until HAND_POINTS) {
|
||||
minX = minOf(minX, points[i * 2])
|
||||
maxX = maxOf(maxX, points[i * 2])
|
||||
minY = minOf(minY, points[i * 2 + 1])
|
||||
maxY = maxOf(maxY, points[i * 2 + 1])
|
||||
}
|
||||
return HandFix(Vec3(t.x + wx / n, t.y + wy / n, t.z + wz / n), px / n, py / n, maxOf(maxX - minX, maxY - minY), points, world)
|
||||
}
|
||||
|
||||
private fun translation(image: List<Landmark>, world: List<WorldPoint>, k: Intrinsics): Vec3? {
|
||||
val used = ANCHORS.filter { image[it].visibility >= MIN_VISIBILITY }
|
||||
if (used.size < MIN_ANCHORS) return null
|
||||
val t = Pnp.translation(used.size, k, { image[used[it]].x * k.width }, { image[used[it]].y * k.height }, { world[used[it]] })
|
||||
?: return null
|
||||
if (!(t.z > MIN_DISTANCE && t.z < MAX_DISTANCE)) return null
|
||||
return t
|
||||
}
|
||||
|
||||
const val HAND_POINTS = 21
|
||||
private val HEAD = (NOSE..RIGHT_MOUTH).toList()
|
||||
private val PALM = intArrayOf(0, 5, 9, 13, 17)
|
||||
private val EDGE_JOINTS = intArrayOf(LEFT_ELBOW, RIGHT_ELBOW, LEFT_WRIST, RIGHT_WRIST)
|
||||
private const val MIN_ANCHORS = 4
|
||||
private const val MIN_HEAD_POINTS = 4
|
||||
private const val MIN_DISTANCE = 0.2f
|
||||
private const val MIN_HAND_DISTANCE = 0.05f
|
||||
private const val MAX_DISTANCE = 6f
|
||||
}
|
||||
@@ -0,0 +1,323 @@
|
||||
package com.droiddeck.launcher.handtracking
|
||||
|
||||
import android.content.Context
|
||||
import android.graphics.Bitmap
|
||||
import android.graphics.Canvas
|
||||
import android.graphics.Matrix
|
||||
import android.hardware.Sensor
|
||||
import android.hardware.SensorEvent
|
||||
import android.hardware.SensorEventListener
|
||||
import android.hardware.SensorManager
|
||||
import android.hardware.camera2.CaptureRequest
|
||||
import android.os.Build
|
||||
import android.os.PowerManager
|
||||
import android.os.SystemClock
|
||||
import android.util.Log
|
||||
import android.util.Range
|
||||
import android.util.Size
|
||||
import androidx.camera.camera2.interop.Camera2CameraInfo
|
||||
import androidx.camera.camera2.interop.Camera2Interop
|
||||
import androidx.camera.camera2.interop.ExperimentalCamera2Interop
|
||||
import androidx.camera.core.CameraSelector
|
||||
import androidx.camera.core.ImageAnalysis
|
||||
import androidx.camera.core.ImageProxy
|
||||
import androidx.camera.core.resolutionselector.AspectRatioStrategy
|
||||
import androidx.camera.core.resolutionselector.ResolutionSelector
|
||||
import androidx.camera.core.resolutionselector.ResolutionStrategy
|
||||
import androidx.camera.lifecycle.ProcessCameraProvider
|
||||
import androidx.core.content.ContextCompat
|
||||
import androidx.lifecycle.Lifecycle
|
||||
import androidx.lifecycle.LifecycleOwner
|
||||
import androidx.lifecycle.LifecycleRegistry
|
||||
import com.droiddeck.launcher.glasses.Vec3
|
||||
import com.google.mediapipe.framework.image.BitmapImageBuilder
|
||||
import com.google.mediapipe.tasks.core.BaseOptions
|
||||
import com.google.mediapipe.tasks.core.Delegate
|
||||
import com.google.mediapipe.tasks.vision.core.RunningMode
|
||||
import com.google.mediapipe.tasks.vision.poselandmarker.PoseLandmarker
|
||||
import kotlinx.coroutines.flow.MutableStateFlow
|
||||
import kotlinx.coroutines.flow.StateFlow
|
||||
import kotlinx.coroutines.flow.asStateFlow
|
||||
import java.io.File
|
||||
import java.nio.ByteBuffer
|
||||
import java.nio.ByteOrder
|
||||
import java.util.concurrent.Executors
|
||||
import kotlin.math.abs
|
||||
|
||||
@androidx.annotation.OptIn(ExperimentalCamera2Interop::class)
|
||||
internal class HandCamera(
|
||||
context: Context,
|
||||
private val output: File?,
|
||||
private val preview: Boolean,
|
||||
) : LifecycleOwner {
|
||||
class View(
|
||||
val image: Bitmap,
|
||||
val front: Boolean,
|
||||
val landmarks: List<Landmark>?,
|
||||
val placement: Placement,
|
||||
val advice: Advice,
|
||||
)
|
||||
|
||||
private val context = context.applicationContext
|
||||
private val registry = LifecycleRegistry(this)
|
||||
override val lifecycle: Lifecycle get() = registry
|
||||
private val executor = Executors.newSingleThreadExecutor { Thread(it, "hand-camera") }
|
||||
private val sensors = this.context.getSystemService(SensorManager::class.java)
|
||||
private val power = this.context.getSystemService(PowerManager::class.java)
|
||||
private val state = MutableStateFlow<View?>(null)
|
||||
val view: StateFlow<View?> = state.asStateFlow()
|
||||
private val broken = MutableStateFlow(false)
|
||||
val failed: StateFlow<Boolean> = broken.asStateFlow()
|
||||
@Volatile private var up: Vec3? = null
|
||||
@Volatile private var thermal = 0
|
||||
@Volatile private var stopped = false
|
||||
@Volatile var bareHands = false
|
||||
set(value) {
|
||||
if (field != value) Log.i(TAG, if (value) "no controllers: tracking bare hands" else "controllers in use: placing them only")
|
||||
field = value
|
||||
}
|
||||
private var provider: ProcessCameraProvider? = null
|
||||
private var analysis: ImageAnalysis? = null
|
||||
private var landmarker: PoseLandmarker? = null
|
||||
private var hands: HandModel? = null
|
||||
private val heldDepth = FloatArray(2) { Float.NaN }
|
||||
private val heldAt = LongArray(2)
|
||||
private var shm: HandCameraShm? = null
|
||||
private var camera: TrackingCamera? = null
|
||||
private var zoom = 1f
|
||||
private var lastRun = 0L
|
||||
private var lastStampMs = 0L
|
||||
private var skipped = 0
|
||||
private var recorder: HandCameraRecorder? = null
|
||||
private var rawFrame: Bitmap? = null
|
||||
private var turnedFrame: Bitmap? = null
|
||||
|
||||
private val gravity = object : SensorEventListener {
|
||||
override fun onSensorChanged(event: SensorEvent) {
|
||||
val v = Vec3(event.values[0], event.values[1], event.values[2])
|
||||
val last = up
|
||||
up = if (event.sensor.type == Sensor.TYPE_GRAVITY || last == null) v else last + (v - last) * ACCEL_SMOOTHING
|
||||
}
|
||||
|
||||
override fun onAccuracyChanged(sensor: Sensor, accuracy: Int) {}
|
||||
}
|
||||
|
||||
private val heat: Any? = if (Build.VERSION.SDK_INT >= 29) PowerManager.OnThermalStatusChangedListener { thermal = it } else null
|
||||
|
||||
init {
|
||||
registry.currentState = Lifecycle.State.CREATED
|
||||
}
|
||||
|
||||
fun start(side: String?) {
|
||||
val chosen = TrackingCameras.pick(HandTracking.cameras(context), side)
|
||||
if (chosen == null) {
|
||||
broken.value = true
|
||||
return
|
||||
}
|
||||
camera = chosen
|
||||
zoom = chosen.zoom()
|
||||
recorder = HandCameraRecorder.start(context, output, preview, chosen)
|
||||
val sensor = sensors?.getDefaultSensor(Sensor.TYPE_GRAVITY) ?: sensors?.getDefaultSensor(Sensor.TYPE_ACCELEROMETER)
|
||||
if (sensor != null) sensors?.registerListener(gravity, sensor, SensorManager.SENSOR_DELAY_UI)
|
||||
if (Build.VERSION.SDK_INT >= 29) {
|
||||
thermal = power?.currentThermalStatus ?: 0
|
||||
power?.addThermalStatusListener(ContextCompat.getMainExecutor(context), heat as PowerManager.OnThermalStatusChangedListener)
|
||||
}
|
||||
executor.execute { prepare() }
|
||||
val future = ProcessCameraProvider.getInstance(context)
|
||||
future.addListener({
|
||||
if (stopped) return@addListener
|
||||
runCatching { bind(future.get(), chosen) }.onFailure {
|
||||
Log.w(TAG, "camera ${chosen.id} could not start", it)
|
||||
broken.value = true
|
||||
}
|
||||
}, ContextCompat.getMainExecutor(context))
|
||||
}
|
||||
|
||||
fun stop() {
|
||||
if (stopped) return
|
||||
stopped = true
|
||||
sensors?.unregisterListener(gravity)
|
||||
if (Build.VERSION.SDK_INT >= 29) power?.removeThermalStatusListener(heat as PowerManager.OnThermalStatusChangedListener)
|
||||
analysis?.let {
|
||||
it.clearAnalyzer()
|
||||
runCatching { provider?.unbind(it) }
|
||||
}
|
||||
registry.currentState = Lifecycle.State.DESTROYED
|
||||
executor.execute {
|
||||
runCatching { landmarker?.close() }
|
||||
landmarker = null
|
||||
hands?.close()
|
||||
hands = null
|
||||
shm?.close()
|
||||
shm = null
|
||||
recorder?.close()
|
||||
recorder = null
|
||||
}
|
||||
executor.shutdown()
|
||||
}
|
||||
|
||||
private fun bind(cameras: ProcessCameraProvider, chosen: TrackingCamera) {
|
||||
val selector = CameraSelector.Builder()
|
||||
.addCameraFilter { infos -> infos.filter { Camera2CameraInfo.from(it).cameraId == chosen.id } }
|
||||
.build()
|
||||
val builder = ImageAnalysis.Builder()
|
||||
val extender = Camera2Interop.Extender(builder)
|
||||
chosen.fps?.let { extender.setCaptureRequestOption(CaptureRequest.CONTROL_AE_TARGET_FPS_RANGE, Range(it.first, it.second)) }
|
||||
recorder?.let { extender.setSessionCaptureCallback(it.callback) }
|
||||
val use = builder
|
||||
.setResolutionSelector(
|
||||
ResolutionSelector.Builder()
|
||||
.setAspectRatioStrategy(AspectRatioStrategy.RATIO_4_3_FALLBACK_AUTO_STRATEGY)
|
||||
.setResolutionStrategy(ResolutionStrategy(Size(ANALYSIS_WIDTH, ANALYSIS_HEIGHT), ResolutionStrategy.FALLBACK_RULE_CLOSEST_HIGHER_THEN_LOWER))
|
||||
.build(),
|
||||
)
|
||||
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
|
||||
.setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_RGBA_8888)
|
||||
.build()
|
||||
use.setAnalyzer(executor, ::analyze)
|
||||
val bound = cameras.bindToLifecycle(this, selector, use)
|
||||
if (zoom < 1f) bound.cameraControl.setZoomRatio(zoom)
|
||||
provider = cameras
|
||||
analysis = use
|
||||
registry.currentState = Lifecycle.State.RESUMED
|
||||
Log.i(TAG, "camera ${chosen.id} (${if (chosen.front) "front" else "rear"}, ${chosen.fovDegrees.toInt()}°, zoom $zoom, ${chosen.fps} fps) " +
|
||||
if (preview) "checking placement" else "tracking hands")
|
||||
}
|
||||
|
||||
private fun prepare() {
|
||||
if (stopped) return
|
||||
runCatching {
|
||||
val bytes = context.assets.open(MODEL).use { it.readBytes() }
|
||||
val model = ByteBuffer.allocateDirect(bytes.size).order(ByteOrder.nativeOrder()).put(bytes)
|
||||
model.rewind()
|
||||
val options = PoseLandmarker.PoseLandmarkerOptions.builder()
|
||||
.setBaseOptions(BaseOptions.builder().setModelAssetBuffer(model).setDelegate(Delegate.CPU).build())
|
||||
.setRunningMode(RunningMode.VIDEO)
|
||||
.setNumPoses(1)
|
||||
.setMinPoseDetectionConfidence(MIN_CONFIDENCE)
|
||||
.setMinPosePresenceConfidence(MIN_CONFIDENCE)
|
||||
.setOutputSegmentationMasks(false)
|
||||
.build()
|
||||
landmarker = PoseLandmarker.createFromOptions(context, options)
|
||||
if (output != null) shm = HandCameraShm.open(output)
|
||||
}.onFailure {
|
||||
Log.w(TAG, "pose model could not load", it)
|
||||
broken.value = true
|
||||
}
|
||||
if (landmarker != null) hands = runCatching { HandModel.open(context) }.onFailure { Log.w(TAG, "hand model could not load", it) }.getOrNull()
|
||||
}
|
||||
|
||||
private fun analyze(image: ImageProxy) {
|
||||
try {
|
||||
val chosen = camera ?: return
|
||||
val pose = landmarker ?: return
|
||||
val arrival = System.nanoTime()
|
||||
if (stopped || arrival - lastRun < interval()) {
|
||||
skipped++
|
||||
return
|
||||
}
|
||||
lastRun = arrival
|
||||
val exposed = exposure(image.imageInfo.timestamp, arrival, chosen)
|
||||
val stampMs = maxOf(exposed / 1_000_000L, lastStampMs + 1)
|
||||
lastStampMs = stampMs
|
||||
val gravity = up
|
||||
val placement = if (gravity != null) CameraGeometry.placement(chosen.front, gravity) else Placement.UNKNOWN
|
||||
val turns = if (gravity != null) CameraGeometry.uprightTurns(chosen.front, chosen.orientation, gravity)
|
||||
else image.imageInfo.rotationDegrees / 90
|
||||
val frame = upright(raw(image), turns)
|
||||
val result = pose.detectForVideo(BitmapImageBuilder(frame).build(), stampMs)
|
||||
val inferred = System.nanoTime()
|
||||
val found = result.landmarks().firstOrNull()
|
||||
val marks = found?.map { Landmark(it.x(), it.y(), it.visibility().orElse(0f)) }
|
||||
val world = result.worldLandmarks().firstOrNull()?.map { WorldPoint(it.x(), it.y(), it.z()) }
|
||||
val k = chosen.intrinsics(image.width, image.height, zoom, turns)
|
||||
val model = hands
|
||||
val fixes = if (marks != null && model != null) model.find(frame, marks, k, stampMs) else null
|
||||
val handsDone = System.nanoTime()
|
||||
fixes?.forEachIndexed { side, fix ->
|
||||
if (fix != null) {
|
||||
heldDepth[side] = fix.palm.z
|
||||
heldAt[side] = exposed
|
||||
}
|
||||
}
|
||||
val held = FloatArray(2) { if (exposed - heldAt[it] <= HOLD_NS) heldDepth[it] else Float.NaN }
|
||||
val body = if (marks != null && world != null && gravity != null) {
|
||||
BodyPose.solve(marks, world, k, CameraLevel(CameraGeometry.upInCamera(chosen.front, chosen.orientation, turns, gravity)),
|
||||
if (model != null) fixes ?: arrayOfNulls(2) else null, held)
|
||||
} else null
|
||||
shm?.write(exposed, body, bareHands)
|
||||
if (preview) state.value = View(frame, chosen.front, marks, placement, PlacementAdvice.of(placement, body))
|
||||
recorder?.frame(HandCameraRecorder.Frame(exposed, image.imageInfo.timestamp, arrival, inferred, System.nanoTime(), skipped,
|
||||
thermal, turns, placement, gravity, k,
|
||||
found?.map { floatArrayOf(it.x(), it.y(), it.z(), it.visibility().orElse(0f), it.presence().orElse(0f)) },
|
||||
world, body, handsDone, fixes), frame)
|
||||
skipped = 0
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "camera frame skipped", e)
|
||||
} finally {
|
||||
image.close()
|
||||
}
|
||||
}
|
||||
|
||||
private fun raw(image: ImageProxy): Bitmap {
|
||||
val plane = image.planes[0]
|
||||
if (plane.pixelStride != 4 || plane.rowStride != image.width * 4) return image.toBitmap()
|
||||
val bitmap = rawFrame?.takeIf { it.width == image.width && it.height == image.height }
|
||||
?: Bitmap.createBitmap(image.width, image.height, Bitmap.Config.ARGB_8888).also { rawFrame = it }
|
||||
plane.buffer.rewind()
|
||||
bitmap.copyPixelsFromBuffer(plane.buffer)
|
||||
return bitmap
|
||||
}
|
||||
|
||||
private fun upright(raw: Bitmap, turns: Int): Bitmap {
|
||||
val quarter = Math.floorMod(turns, 4)
|
||||
if (quarter == 0) return if (preview) raw.copy(Bitmap.Config.ARGB_8888, false) else raw
|
||||
val width = if (quarter % 2 == 1) raw.height else raw.width
|
||||
val height = if (quarter % 2 == 1) raw.width else raw.height
|
||||
val target = if (preview) Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
|
||||
else turnedFrame?.takeIf { it.width == width && it.height == height }
|
||||
?: Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888).also { turnedFrame = it }
|
||||
val matrix = Matrix()
|
||||
matrix.postRotate(90f * quarter)
|
||||
when (quarter) {
|
||||
1 -> matrix.postTranslate(raw.height.toFloat(), 0f)
|
||||
2 -> matrix.postTranslate(raw.width.toFloat(), raw.height.toFloat())
|
||||
else -> matrix.postTranslate(0f, raw.width.toFloat())
|
||||
}
|
||||
Canvas(target).drawBitmap(raw, matrix, null)
|
||||
return target
|
||||
}
|
||||
|
||||
private fun exposure(stamp: Long, arrival: Long, chosen: TrackingCamera): Long {
|
||||
val monotonic = if (chosen.realtime) stamp - (SystemClock.elapsedRealtimeNanos() - System.nanoTime()) else stamp
|
||||
return if (abs(arrival - monotonic) < MAX_DELAY_NS) monotonic else arrival - TYPICAL_DELAY_NS
|
||||
}
|
||||
|
||||
private fun interval(): Long {
|
||||
val fps = when {
|
||||
thermal >= SEVERE -> HOT_FPS
|
||||
thermal >= MODERATE -> WARM_FPS
|
||||
else -> FPS
|
||||
}
|
||||
return 1_000_000_000L / fps - SLACK_NS
|
||||
}
|
||||
|
||||
companion object {
|
||||
private const val TAG = "HandCamera"
|
||||
private const val MODEL = "handtracking/pose_landmarker_lite.task"
|
||||
private const val ANALYSIS_WIDTH = 960
|
||||
private const val ANALYSIS_HEIGHT = 720
|
||||
private const val MIN_CONFIDENCE = 0.5f
|
||||
private const val FPS = 15
|
||||
private const val WARM_FPS = 10
|
||||
private const val HOT_FPS = 6
|
||||
private const val MODERATE = 2
|
||||
private const val SEVERE = 3
|
||||
private const val SLACK_NS = 8_000_000L
|
||||
private const val MAX_DELAY_NS = 1_000_000_000L
|
||||
private const val TYPICAL_DELAY_NS = 60_000_000L
|
||||
private const val ACCEL_SMOOTHING = 0.1f
|
||||
private const val HOLD_NS = 500_000_000L
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,203 @@
|
||||
package com.droiddeck.launcher.handtracking
|
||||
|
||||
import android.content.Context
|
||||
import android.graphics.Bitmap
|
||||
import android.hardware.camera2.CameraCaptureSession
|
||||
import android.hardware.camera2.CaptureRequest
|
||||
import android.hardware.camera2.CaptureResult
|
||||
import android.hardware.camera2.TotalCaptureResult
|
||||
import android.util.Log
|
||||
import com.droiddeck.launcher.glasses.Vec3
|
||||
import java.io.BufferedWriter
|
||||
import java.io.File
|
||||
import java.io.FileOutputStream
|
||||
import java.text.SimpleDateFormat
|
||||
import java.util.Date
|
||||
import java.util.Locale
|
||||
import java.util.concurrent.Executors
|
||||
import java.util.concurrent.atomic.AtomicBoolean
|
||||
import kotlin.math.max
|
||||
import kotlin.math.roundToInt
|
||||
|
||||
internal class HandCameraRecorder private constructor(private val dir: File, private val traceFlag: File?) {
|
||||
class Capture(val exposureNs: Long, val frameDurationNs: Long, val iso: Int)
|
||||
|
||||
class Frame(
|
||||
val exposureNs: Long,
|
||||
val sensorNs: Long,
|
||||
val arrivalNs: Long,
|
||||
val inferredNs: Long,
|
||||
val doneNs: Long,
|
||||
val skipped: Int,
|
||||
val thermal: Int,
|
||||
val turns: Int,
|
||||
val placement: Placement,
|
||||
val gravity: Vec3?,
|
||||
val intrinsics: Intrinsics?,
|
||||
val marks: List<FloatArray>?,
|
||||
val world: List<WorldPoint>?,
|
||||
val body: BodyFrame?,
|
||||
val handsNs: Long = 0L,
|
||||
val fixes: Array<HandFix?>? = null,
|
||||
)
|
||||
|
||||
private val writer = Executors.newSingleThreadExecutor { Thread(it, "hand-camera-record") }
|
||||
private val lines: BufferedWriter = File(dir, "frames.jsonl").bufferedWriter()
|
||||
private val busy = AtomicBoolean(false)
|
||||
private val captures = object : LinkedHashMap<Long, Capture>() {
|
||||
override fun removeEldestEntry(eldest: MutableMap.MutableEntry<Long, Capture>?) = size > KEEP_CAPTURES
|
||||
}
|
||||
private var count = 0
|
||||
@Volatile private var closed = false
|
||||
|
||||
val callback = object : CameraCaptureSession.CaptureCallback() {
|
||||
override fun onCaptureCompleted(session: CameraCaptureSession, request: CaptureRequest, result: TotalCaptureResult) {
|
||||
val stamp = result.get(CaptureResult.SENSOR_TIMESTAMP) ?: return
|
||||
val capture = Capture(result.get(CaptureResult.SENSOR_EXPOSURE_TIME) ?: 0L,
|
||||
result.get(CaptureResult.SENSOR_FRAME_DURATION) ?: 0L, result.get(CaptureResult.SENSOR_SENSITIVITY) ?: 0)
|
||||
synchronized(captures) { captures[stamp] = capture }
|
||||
}
|
||||
}
|
||||
|
||||
fun frame(frame: Frame, image: Bitmap) {
|
||||
if (closed || count >= MAX_FRAMES) return
|
||||
val index = ++count
|
||||
val capture = synchronized(captures) { captures[frame.sensorNs] }
|
||||
val small = if (busy.compareAndSet(false, true)) runCatching { shrink(image) }.getOrNull().also { if (it == null) busy.set(false) } else null
|
||||
val name = if (small != null) "%05d.jpg".format(Locale.ROOT, index) else null
|
||||
val line = json(index, frame, capture, name)
|
||||
val queued = runCatching { writer.execute {
|
||||
if (small != null) {
|
||||
runCatching { FileOutputStream(File(dir, name!!)).use { small.compress(Bitmap.CompressFormat.JPEG, QUALITY, it) } }
|
||||
small.recycle()
|
||||
busy.set(false)
|
||||
}
|
||||
runCatching {
|
||||
lines.write(line)
|
||||
lines.newLine()
|
||||
if (index % FLUSH_EVERY == 0) lines.flush()
|
||||
}
|
||||
} }
|
||||
if (queued.isFailure) {
|
||||
small?.recycle()
|
||||
busy.set(false)
|
||||
}
|
||||
}
|
||||
|
||||
fun close() {
|
||||
if (closed) return
|
||||
closed = true
|
||||
traceFlag?.delete()
|
||||
writer.execute { runCatching { lines.close() } }
|
||||
writer.shutdown()
|
||||
Log.i(TAG, "recorded $count frames to $dir")
|
||||
}
|
||||
|
||||
private fun shrink(image: Bitmap): Bitmap {
|
||||
val scale = SIDE.toFloat() / max(image.width, image.height)
|
||||
if (scale >= 1f) return image.copy(Bitmap.Config.ARGB_8888, false)
|
||||
return Bitmap.createScaledBitmap(image, (image.width * scale).roundToInt(), (image.height * scale).roundToInt(), true)
|
||||
}
|
||||
|
||||
private fun json(index: Int, f: Frame, capture: Capture?, jpg: String?): String = buildString {
|
||||
append("{\"i\":").append(index)
|
||||
append(",\"exp\":").append(f.exposureNs)
|
||||
append(",\"sensor\":").append(f.sensorNs)
|
||||
append(",\"arr\":").append(f.arrivalNs)
|
||||
append(",\"inf\":").append(f.inferredNs)
|
||||
append(",\"done\":").append(f.doneNs)
|
||||
append(",\"hand\":").append(f.handsNs)
|
||||
append(",\"skip\":").append(f.skipped)
|
||||
append(",\"heat\":").append(f.thermal)
|
||||
append(",\"turns\":").append(f.turns)
|
||||
append(",\"place\":\"").append(f.placement.name).append('"')
|
||||
append(",\"g\":").vec(f.gravity)
|
||||
f.intrinsics?.let { append(",\"k\":[").num(it.fx).append(',').num(it.cx).append(',').num(it.cy).append(',').append(it.width).append(',').append(it.height).append(']') }
|
||||
capture?.let { append(",\"ae\":[").append(it.exposureNs).append(',').append(it.frameDurationNs).append(',').append(it.iso).append(']') }
|
||||
append(",\"lm\":")
|
||||
if (f.marks == null) append("null") else {
|
||||
append('[')
|
||||
f.marks.forEachIndexed { i, m ->
|
||||
if (i > 0) append(',')
|
||||
append('[')
|
||||
m.forEachIndexed { j, v -> if (j > 0) append(','); num(v) }
|
||||
append(']')
|
||||
}
|
||||
append(']')
|
||||
}
|
||||
append(",\"wl\":")
|
||||
if (f.world == null) append("null") else {
|
||||
append('[')
|
||||
f.world.forEachIndexed { i, w ->
|
||||
if (i > 0) append(',')
|
||||
append('[').num(w.x).append(',').num(w.y).append(',').num(w.z).append(']')
|
||||
}
|
||||
append(']')
|
||||
}
|
||||
f.body?.let { b ->
|
||||
append(",\"hands\":[").vec(b.hands[0]).append(',').vec(b.hands[1]).append(']')
|
||||
append(",\"face\":"); if (b.faceYaw == null) append("null") else num(b.faceYaw)
|
||||
append(",\"est\":[").append(b.estimated[0]).append(',').append(b.estimated[1]).append(']')
|
||||
append(",\"room\":").num(b.edgeRoom)
|
||||
append(",\"dist\":").num(b.distance)
|
||||
append(",\"span\":").num(b.shoulderSpan)
|
||||
append(",\"joints\":[")
|
||||
b.joints.forEachIndexed { i, j ->
|
||||
if (i > 0) append(',')
|
||||
if (j == null) append("null") else {
|
||||
append('[')
|
||||
j.forEachIndexed { k, v -> if (k > 0) append(','); num(v) }
|
||||
append(']')
|
||||
}
|
||||
}
|
||||
append(']')
|
||||
}
|
||||
f.fixes?.let { fixes ->
|
||||
append(",\"fix\":[")
|
||||
fixes.forEachIndexed { i, fix ->
|
||||
if (i > 0) append(',')
|
||||
if (fix == null) append("null") else {
|
||||
append("{\"palm\":").vec(fix.palm).append(",\"at\":[").num(fix.x).append(',').num(fix.y).append("],\"size\":").num(fix.extent)
|
||||
append(",\"pts\":[")
|
||||
fix.points.forEachIndexed { j, v -> if (j > 0) append(','); num(v) }
|
||||
append("],\"world\":[")
|
||||
fix.world.forEachIndexed { j, w -> if (j > 0) append(','); num(w.x).append(',').num(w.y).append(',').num(w.z) }
|
||||
append("]}")
|
||||
}
|
||||
}
|
||||
append(']')
|
||||
}
|
||||
append(",\"jpg\":"); if (jpg == null) append("null") else append('"').append(jpg).append('"')
|
||||
append('}')
|
||||
}
|
||||
|
||||
private fun StringBuilder.num(v: Float): StringBuilder = if (v.isFinite()) append(v) else append("null")
|
||||
|
||||
private fun StringBuilder.vec(v: Vec3?): StringBuilder =
|
||||
if (v == null) append("null") else append('[').num(v.x).append(',').num(v.y).append(',').num(v.z).append(']')
|
||||
|
||||
companion object {
|
||||
private const val TAG = "HandCamera"
|
||||
const val FLAG = "handcam-record"
|
||||
const val TRACE_FLAG = "droiddeck-handtrace.on"
|
||||
private const val MAX_FRAMES = 5400
|
||||
private const val KEEP_CAPTURES = 32
|
||||
private const val SIDE = 640
|
||||
private const val QUALITY = 85
|
||||
private const val FLUSH_EVERY = 30
|
||||
|
||||
fun start(context: Context, output: File?, preview: Boolean, camera: TrackingCamera): HandCameraRecorder? {
|
||||
if (!File(context.filesDir, FLAG).exists()) return null
|
||||
return runCatching {
|
||||
val stamp = SimpleDateFormat("yyyyMMdd-HHmmss", Locale.ROOT).format(Date())
|
||||
val dir = File(context.cacheDir, "handcam-rec/$stamp-${if (preview) "check" else "track"}").apply { mkdirs() }
|
||||
File(dir, "camera.json").writeText("{\"id\":\"${camera.id}\",\"front\":${camera.front},\"orientation\":${camera.orientation}," +
|
||||
"\"focal\":${camera.focalPixels},\"active\":[${camera.activeWidth},${camera.activeHeight}]," +
|
||||
"\"fps\":${camera.fps?.let { "[${it.first},${it.second}]" } ?: "null"}}")
|
||||
val flag = output?.parentFile?.let { File(it, TRACE_FLAG) }?.also { it.createNewFile() }
|
||||
Log.i(TAG, "recording hand camera to $dir")
|
||||
HandCameraRecorder(dir, flag)
|
||||
}.onFailure { Log.w(TAG, "hand camera recording unavailable", it) }.getOrNull()
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
package com.droiddeck.launcher.handtracking
|
||||
|
||||
import android.util.Log
|
||||
import com.droiddeck.launcher.glasses.Vec3
|
||||
import com.droiddeck.launcher.input.FakeInputWriter
|
||||
import java.io.File
|
||||
import java.io.RandomAccessFile
|
||||
import java.nio.ByteOrder
|
||||
import java.nio.MappedByteBuffer
|
||||
import java.nio.channels.FileChannel
|
||||
|
||||
internal class HandCameraShm private constructor(
|
||||
private val file: RandomAccessFile,
|
||||
private val map: MappedByteBuffer,
|
||||
private val fence: () -> Unit,
|
||||
) {
|
||||
private var seq = map.getInt(SEQ).let { if (it and 1 != 0) it + 1 else it }
|
||||
private var frame = 0
|
||||
|
||||
init {
|
||||
begin()
|
||||
map.putInt(MAGIC_AT, MAGIC)
|
||||
map.putInt(VERSION_AT, VERSION)
|
||||
for (at in FLAGS until SIZE step 4) map.putInt(at, 0)
|
||||
end()
|
||||
}
|
||||
|
||||
fun write(timeNs: Long, body: BodyFrame?, bareHands: Boolean = false) {
|
||||
begin()
|
||||
val face = body?.faceYaw
|
||||
map.putInt(FLAGS, RUNNING or (if (face != null) FACE else 0) or (if (bareHands) BARE_HANDS else 0))
|
||||
map.putLong(TIME, timeNs)
|
||||
map.putInt(FRAME, ++frame)
|
||||
map.putFloat(FACE_YAW, face ?: 0f)
|
||||
for (side in 0..1) {
|
||||
val at = HANDS + side * HAND_SIZE
|
||||
val hand: Vec3? = body?.hands?.getOrNull(side)
|
||||
val joints = if (hand != null) body?.joints?.getOrNull(side)?.takeIf { it.size == JOINT_FLOATS } else null
|
||||
val seen = if (body?.estimated?.getOrNull(side) == true) SEEN or ESTIMATED else SEEN
|
||||
map.putInt(at, if (hand == null) 0 else if (joints != null) seen or JOINTS else seen)
|
||||
map.putFloat(at + 4, hand?.x ?: 0f)
|
||||
map.putFloat(at + 8, hand?.y ?: 0f)
|
||||
map.putFloat(at + 12, hand?.z ?: 0f)
|
||||
for (i in 0 until JOINT_FLOATS) map.putFloat(at + JOINTS_AT + i * 4, joints?.get(i) ?: 0f)
|
||||
}
|
||||
end()
|
||||
}
|
||||
|
||||
fun close() {
|
||||
begin()
|
||||
for (at in FLAGS until SIZE step 4) map.putInt(at, 0)
|
||||
end()
|
||||
runCatching { file.close() }
|
||||
}
|
||||
|
||||
private fun begin() {
|
||||
map.putInt(SEQ, ++seq)
|
||||
fence()
|
||||
}
|
||||
|
||||
private fun end() {
|
||||
fence()
|
||||
map.putInt(SEQ, ++seq)
|
||||
}
|
||||
|
||||
companion object {
|
||||
const val FILE = "droiddeck-handcam"
|
||||
const val SIZE = 568
|
||||
private const val MAGIC = 0x4D434844
|
||||
private const val VERSION = 2
|
||||
private const val MAGIC_AT = 0
|
||||
private const val VERSION_AT = 4
|
||||
private const val SEQ = 8
|
||||
private const val FLAGS = 12
|
||||
private const val TIME = 16
|
||||
private const val FRAME = 24
|
||||
private const val FACE_YAW = 28
|
||||
private const val HANDS = 32
|
||||
private const val HAND_SIZE = 268
|
||||
private const val JOINTS_AT = 16
|
||||
private const val JOINT_FLOATS = BodyPose.HAND_POINTS * 3
|
||||
private const val RUNNING = 1
|
||||
private const val FACE = 2
|
||||
private const val BARE_HANDS = 4
|
||||
private const val SEEN = 1
|
||||
private const val ESTIMATED = 2
|
||||
private const val JOINTS = 4
|
||||
|
||||
fun open(path: File, fence: () -> Unit = FakeInputWriter::storeFence): HandCameraShm? = runCatching {
|
||||
path.parentFile?.mkdirs()
|
||||
val file = RandomAccessFile(path, "rw")
|
||||
try {
|
||||
file.setLength(SIZE.toLong())
|
||||
val map = file.channel.map(FileChannel.MapMode.READ_WRITE, 0, SIZE.toLong())
|
||||
map.order(ByteOrder.LITTLE_ENDIAN)
|
||||
HandCameraShm(file, map, fence)
|
||||
} catch (e: Exception) {
|
||||
file.close()
|
||||
throw e
|
||||
}
|
||||
}.onFailure { Log.w("HandCamera", "hand camera file unavailable: $path", it) }.getOrNull()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,172 @@
|
||||
package com.droiddeck.launcher.handtracking
|
||||
|
||||
import android.content.Context
|
||||
import android.graphics.Bitmap
|
||||
import android.graphics.Canvas
|
||||
import android.graphics.Color
|
||||
import android.graphics.Matrix
|
||||
import android.graphics.Paint
|
||||
import com.google.mediapipe.framework.image.BitmapImageBuilder
|
||||
import com.google.mediapipe.tasks.components.containers.Landmark as MpLandmark
|
||||
import com.google.mediapipe.tasks.components.containers.NormalizedLandmark
|
||||
import com.google.mediapipe.tasks.core.BaseOptions
|
||||
import com.google.mediapipe.tasks.core.Delegate
|
||||
import com.google.mediapipe.tasks.vision.core.RunningMode
|
||||
import com.google.mediapipe.tasks.vision.handlandmarker.HandLandmarker
|
||||
import java.nio.ByteBuffer
|
||||
import java.nio.ByteOrder
|
||||
import kotlin.math.hypot
|
||||
import kotlin.math.max
|
||||
|
||||
internal class HandModel private constructor(private val landmarkers: List<HandLandmarker>, private val starter: HandLandmarker) {
|
||||
private class Track(val x: Float, val y: Float, val size: Float)
|
||||
|
||||
private val crops = List(2) { Bitmap.createBitmap(CROP, CROP, Bitmap.Config.ARGB_8888) }
|
||||
private val startCrop = Bitmap.createBitmap(START_CROP, START_CROP, Bitmap.Config.ARGB_8888)
|
||||
private val tracks = arrayOfNulls<Track>(2)
|
||||
private val starts = IntArray(2)
|
||||
private val paint = Paint(Paint.FILTER_BITMAP_FLAG)
|
||||
private val matrix = Matrix()
|
||||
|
||||
fun find(frame: Bitmap, marks: List<Landmark>, k: Intrinsics, stampMs: Long): Array<HandFix?> {
|
||||
val fixes = arrayOfNulls<HandFix>(2)
|
||||
if (marks.size <= BodyPose.RIGHT_INDEX) return fixes
|
||||
val shoulders = hypot((marks[BodyPose.LEFT_SHOULDER].x - marks[BodyPose.RIGHT_SHOULDER].x) * k.width,
|
||||
(marks[BodyPose.LEFT_SHOULDER].y - marks[BodyPose.RIGHT_SHOULDER].y) * k.height)
|
||||
for (side in 0..1) {
|
||||
val wrist = marks[WRISTS[side]]
|
||||
val wx = wrist.x * k.width
|
||||
val wy = wrist.y * k.height
|
||||
val fix = tracks[side]?.let { follow(frame, it, side, k, stampMs) }?.takeIf { belongs(it, marks, side, shoulders, k) }
|
||||
?: start(frame, marks, side, wx, wy, shoulders, k)?.takeIf { belongs(it, marks, side, shoulders, k) }
|
||||
if (fix == null) {
|
||||
tracks[side] = null
|
||||
continue
|
||||
}
|
||||
fixes[side] = fix
|
||||
starts[side] = 0
|
||||
tracks[side] = Track(fix.x, fix.y, max(fix.extent, shoulders * MIN_HAND))
|
||||
}
|
||||
val left = fixes[0]
|
||||
val right = fixes[1]
|
||||
if (left != null && right != null && hypot(left.x - right.x, left.y - right.y) < max(left.extent, right.extent) * SAME_HAND) {
|
||||
val drop = if (gap(left, marks, 0, k) <= gap(right, marks, 1, k)) 1 else 0
|
||||
fixes[drop] = null
|
||||
tracks[drop] = null
|
||||
}
|
||||
return fixes
|
||||
}
|
||||
|
||||
fun close() {
|
||||
landmarkers.forEach { runCatching { it.close() } }
|
||||
runCatching { starter.close() }
|
||||
crops.forEach { it.recycle() }
|
||||
startCrop.recycle()
|
||||
}
|
||||
|
||||
private fun follow(frame: Bitmap, track: Track, side: Int, k: Intrinsics, stampMs: Long): HandFix? {
|
||||
val size = max(track.size * CROP_HANDS, MIN_CROP)
|
||||
val x0 = track.x - size / 2
|
||||
val y0 = track.y - size / 2
|
||||
draw(frame, crops[side], x0, y0, size)
|
||||
val result = landmarkers[side].detectForVideo(BitmapImageBuilder(crops[side]).build(), stampMs)
|
||||
val found = result.landmarks().firstOrNull() ?: return null
|
||||
val world = result.worldLandmarks().firstOrNull() ?: return null
|
||||
return fixOf(found, world, x0, y0, size, k)
|
||||
}
|
||||
|
||||
private fun start(frame: Bitmap, marks: List<Landmark>, side: Int, wx: Float, wy: Float, shoulders: Float, k: Intrinsics): HandFix? {
|
||||
val wrist = marks[WRISTS[side]]
|
||||
if (wrist.visibility < MIN_VISIBILITY || shoulders < 1f) return null
|
||||
if (wrist.x < -EDGE || wrist.x > 1f + EDGE || wrist.y < -EDGE || wrist.y > 1f + EDGE) return null
|
||||
val wide = starts[side]++ % 2 == 0
|
||||
val tx = (marks[PINKIES[side]].x + marks[INDEXES[side]].x) / 2 * k.width - wx
|
||||
val ty = (marks[PINKIES[side]].y + marks[INDEXES[side]].y) / 2 * k.height - wy
|
||||
val length = hypot(tx, ty)
|
||||
val reach = if (wide) 0f else shoulders * TOWARD_FINGERS
|
||||
val cx = if (length > 1e-3f) wx + tx / length * reach else wx
|
||||
val cy = if (length > 1e-3f) wy + ty / length * reach else wy
|
||||
val size = max(shoulders * if (wide) WIDE_SHOULDERS else START_SHOULDERS, MIN_CROP)
|
||||
val x0 = cx - size / 2
|
||||
val y0 = cy - size / 2
|
||||
draw(frame, startCrop, x0, y0, size)
|
||||
val result = starter.detect(BitmapImageBuilder(startCrop).build())
|
||||
var best: HandFix? = null
|
||||
var bestGap = Float.MAX_VALUE
|
||||
result.landmarks().forEachIndexed { i, found ->
|
||||
val world = result.worldLandmarks().getOrNull(i) ?: return@forEachIndexed
|
||||
val fix = fixOf(found, world, x0, y0, size, k) ?: return@forEachIndexed
|
||||
val gap = hypot(fix.points[0] - wx, fix.points[1] - wy)
|
||||
if (gap < bestGap) {
|
||||
best = fix
|
||||
bestGap = gap
|
||||
}
|
||||
}
|
||||
return best
|
||||
}
|
||||
|
||||
private fun draw(frame: Bitmap, crop: Bitmap, x0: Float, y0: Float, size: Float) {
|
||||
crop.eraseColor(Color.BLACK)
|
||||
matrix.setTranslate(-x0, -y0)
|
||||
matrix.postScale(crop.width / size, crop.height / size)
|
||||
Canvas(crop).drawBitmap(frame, matrix, paint)
|
||||
}
|
||||
|
||||
private fun fixOf(found: List<NormalizedLandmark>, world: List<MpLandmark>, x0: Float, y0: Float, size: Float, k: Intrinsics): HandFix? {
|
||||
if (found.size < BodyPose.HAND_POINTS || world.size < BodyPose.HAND_POINTS) return null
|
||||
val points = FloatArray(BodyPose.HAND_POINTS * 2) { i ->
|
||||
val p = found[i / 2]
|
||||
if (i % 2 == 0) x0 + p.x() * size else y0 + p.y() * size
|
||||
}
|
||||
return BodyPose.handFix(points, world.map { WorldPoint(it.x(), it.y(), it.z()) }, k)
|
||||
}
|
||||
|
||||
private fun belongs(fix: HandFix, marks: List<Landmark>, side: Int, shoulders: Float, k: Intrinsics): Boolean {
|
||||
val own = marks[WRISTS[side]]
|
||||
if (hypot(fix.x - own.x * k.width, fix.y - own.y * k.height) > max(fix.extent, shoulders) * MAX_WRIST_GAP) return false
|
||||
val other = marks[WRISTS[1 - side]]
|
||||
if (other.visibility < MIN_VISIBILITY || other.x !in 0f..1f || other.y !in 0f..1f) return true
|
||||
return hypot(fix.points[0] - other.x * k.width, fix.points[1] - other.y * k.height) >=
|
||||
hypot(fix.points[0] - own.x * k.width, fix.points[1] - own.y * k.height)
|
||||
}
|
||||
|
||||
private fun gap(fix: HandFix, marks: List<Landmark>, side: Int, k: Intrinsics): Float =
|
||||
hypot(fix.x - marks[WRISTS[side]].x * k.width, fix.y - marks[WRISTS[side]].y * k.height)
|
||||
|
||||
companion object {
|
||||
private const val MODEL = "handtracking/hand_landmarker.task"
|
||||
private val WRISTS = intArrayOf(BodyPose.LEFT_WRIST, BodyPose.RIGHT_WRIST)
|
||||
private val PINKIES = intArrayOf(BodyPose.LEFT_PINKY, BodyPose.RIGHT_PINKY)
|
||||
private val INDEXES = intArrayOf(BodyPose.LEFT_INDEX, BodyPose.RIGHT_INDEX)
|
||||
private const val CROP = 256
|
||||
private const val START_CROP = 384
|
||||
private const val CROP_HANDS = 2.8f
|
||||
private const val MIN_CROP = 48f
|
||||
private const val START_SHOULDERS = 2.5f
|
||||
private const val WIDE_SHOULDERS = 4f
|
||||
private const val MIN_HAND = 0.25f
|
||||
private const val TOWARD_FINGERS = 0.35f
|
||||
private const val MAX_WRIST_GAP = 3f
|
||||
private const val SAME_HAND = 0.5f
|
||||
private const val EDGE = 0.05f
|
||||
private const val MIN_VISIBILITY = 0.5f
|
||||
private const val MIN_CONFIDENCE = 0.4f
|
||||
|
||||
fun open(context: Context): HandModel {
|
||||
val bytes = context.assets.open(MODEL).use { it.readBytes() }
|
||||
fun landmarker(mode: RunningMode, hands: Int): HandLandmarker {
|
||||
val model = ByteBuffer.allocateDirect(bytes.size).order(ByteOrder.nativeOrder()).put(bytes)
|
||||
model.rewind()
|
||||
return HandLandmarker.createFromOptions(context, HandLandmarker.HandLandmarkerOptions.builder()
|
||||
.setBaseOptions(BaseOptions.builder().setModelAssetBuffer(model).setDelegate(Delegate.CPU).build())
|
||||
.setRunningMode(mode)
|
||||
.setNumHands(hands)
|
||||
.setMinHandDetectionConfidence(MIN_CONFIDENCE)
|
||||
.setMinHandPresenceConfidence(MIN_CONFIDENCE)
|
||||
.setMinTrackingConfidence(MIN_CONFIDENCE)
|
||||
.build())
|
||||
}
|
||||
return HandModel(List(2) { landmarker(RunningMode.VIDEO, 1) }, landmarker(RunningMode.IMAGE, 2))
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,110 @@
|
||||
package com.droiddeck.launcher.handtracking
|
||||
|
||||
import android.Manifest
|
||||
import android.content.Context
|
||||
import android.content.pm.PackageManager
|
||||
import android.os.Build
|
||||
import com.droiddeck.launcher.input.ControllerPrefs
|
||||
import kotlinx.coroutines.flow.MutableStateFlow
|
||||
import kotlinx.coroutines.flow.StateFlow
|
||||
import kotlinx.coroutines.flow.asStateFlow
|
||||
|
||||
internal object HandTracking {
|
||||
data class Settings(
|
||||
val enabled: Boolean = false,
|
||||
val asked: Boolean = false,
|
||||
val side: String? = null,
|
||||
val permitted: Boolean = false,
|
||||
val sides: Set<String> = emptySet(),
|
||||
) {
|
||||
val usable: Boolean get() = sides.isNotEmpty()
|
||||
val ready: Boolean get() = enabled && permitted && usable
|
||||
}
|
||||
|
||||
private val state = MutableStateFlow(Settings())
|
||||
val settings: StateFlow<Settings> = state.asStateFlow()
|
||||
private val checks = MutableStateFlow(0)
|
||||
val checkRequests: StateFlow<Int> = checks.asStateFlow()
|
||||
@Volatile private var cameras: List<TrackingCamera>? = null
|
||||
|
||||
val supported: Boolean get() = Build.VERSION.SDK_INT >= 31
|
||||
|
||||
fun load(context: Context): Settings {
|
||||
val prefs = ControllerPrefs.prefs(context)
|
||||
val found = cameras(context)
|
||||
val next = Settings(
|
||||
enabled = prefs.getBoolean(ENABLED, false),
|
||||
asked = prefs.getBoolean(ASKED, false),
|
||||
side = prefs.getString(SIDE, null),
|
||||
permitted = permitted(context),
|
||||
sides = if (supported) TrackingCameras.sides(found) else emptySet(),
|
||||
)
|
||||
state.value = next
|
||||
return next
|
||||
}
|
||||
|
||||
fun cameras(context: Context): List<TrackingCamera> =
|
||||
cameras ?: TrackingCameras.list(context.applicationContext).also { cameras = it }
|
||||
|
||||
fun camera(context: Context): TrackingCamera? = TrackingCameras.pick(cameras(context), state.value.side)
|
||||
|
||||
fun permitted(context: Context): Boolean =
|
||||
context.checkSelfPermission(Manifest.permission.CAMERA) == PackageManager.PERMISSION_GRANTED
|
||||
|
||||
fun refreshPermission(context: Context) {
|
||||
state.value = state.value.copy(permitted = permitted(context))
|
||||
}
|
||||
|
||||
fun allow(context: Context, side: String?) {
|
||||
val edit = ControllerPrefs.prefs(context).edit().putBoolean(ASKED, true).putBoolean(ENABLED, true)
|
||||
if (side != null) edit.putString(SIDE, side)
|
||||
edit.apply()
|
||||
load(context)
|
||||
}
|
||||
|
||||
fun decline(context: Context) {
|
||||
ControllerPrefs.prefs(context).edit().putBoolean(ASKED, true).putBoolean(ENABLED, false).apply()
|
||||
load(context)
|
||||
}
|
||||
|
||||
fun setEnabled(context: Context, on: Boolean) {
|
||||
ControllerPrefs.prefs(context).edit().putBoolean(ASKED, true).putBoolean(ENABLED, on).apply()
|
||||
load(context)
|
||||
}
|
||||
|
||||
fun setSide(context: Context, side: String) {
|
||||
ControllerPrefs.prefs(context).edit().putString(SIDE, side).apply()
|
||||
load(context)
|
||||
}
|
||||
|
||||
fun shouldAsk(settings: Settings, headset: Boolean): Boolean = headset && !settings.asked && settings.usable
|
||||
|
||||
fun wanted(context: Context): Boolean = supported && load(context).ready
|
||||
|
||||
fun requestCheck() {
|
||||
checks.value++
|
||||
}
|
||||
|
||||
private const val ENABLED = "handCamera"
|
||||
private const val ASKED = "handCameraAsked"
|
||||
private const val SIDE = "handCameraSide"
|
||||
}
|
||||
|
||||
internal enum class Advice { STARTING, STAND_UP, UPRIGHT, NOT_SEEN, MOVE_BACK, MOVE_CLOSER, SHOW_HANDS, GOOD }
|
||||
|
||||
internal object PlacementAdvice {
|
||||
const val MIN_SPAN = 0.12f
|
||||
const val MAX_SPAN = 0.26f
|
||||
const val MIN_ROOM = 0.05f
|
||||
|
||||
fun of(placement: Placement, body: BodyFrame?): Advice = when {
|
||||
placement == Placement.FACE_UP || placement == Placement.FACE_DOWN -> Advice.STAND_UP
|
||||
placement == Placement.POINTS_UP || placement == Placement.POINTS_DOWN -> Advice.UPRIGHT
|
||||
body == null || !body.seen -> Advice.NOT_SEEN
|
||||
body.shoulderSpan > MAX_SPAN -> Advice.MOVE_BACK
|
||||
body.shoulderSpan < MIN_SPAN -> Advice.MOVE_CLOSER
|
||||
body.edgeRoom < MIN_ROOM -> Advice.MOVE_BACK
|
||||
body.hands.any { it == null } -> Advice.SHOW_HANDS
|
||||
else -> Advice.GOOD
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,136 @@
|
||||
package com.droiddeck.launcher.handtracking
|
||||
|
||||
import android.content.Context
|
||||
import android.hardware.camera2.CameraCharacteristics
|
||||
import android.hardware.camera2.CameraManager
|
||||
import android.os.Build
|
||||
import com.droiddeck.launcher.glasses.Vec3
|
||||
import kotlin.math.asin
|
||||
import kotlin.math.atan
|
||||
import kotlin.math.max
|
||||
import kotlin.math.min
|
||||
import kotlin.math.tan
|
||||
|
||||
internal class TrackingCamera(
|
||||
val id: String,
|
||||
val front: Boolean,
|
||||
val orientation: Int,
|
||||
val focalPixels: Float,
|
||||
val activeWidth: Int,
|
||||
val activeHeight: Int,
|
||||
val minZoom: Float,
|
||||
val realtime: Boolean,
|
||||
val fps: Pair<Int, Int>? = null,
|
||||
) {
|
||||
val fovDegrees: Float get() = Math.toDegrees(2.0 * atan(activeWidth / 2.0 / focalPixels)).toFloat()
|
||||
|
||||
fun zoom(): Float {
|
||||
if (minZoom >= 1f) return 1f
|
||||
val needed = (tan(Math.toRadians(fovDegrees / 2.0)) / tan(Math.toRadians(MAX_FOV / 2.0))).toFloat()
|
||||
return min(1f, max(minZoom, needed))
|
||||
}
|
||||
|
||||
fun intrinsics(bufferWidth: Int, bufferHeight: Int, zoom: Float, turns: Int): Intrinsics {
|
||||
val aspect = bufferWidth.toFloat() / bufferHeight
|
||||
val region = min(activeWidth.toFloat(), activeHeight * aspect) / zoom
|
||||
val f = focalPixels * bufferWidth / region
|
||||
val quarter = turns and 1 == 1
|
||||
val w = if (quarter) bufferHeight else bufferWidth
|
||||
val h = if (quarter) bufferWidth else bufferHeight
|
||||
return Intrinsics(f, f, w / 2f, h / 2f, w, h)
|
||||
}
|
||||
|
||||
companion object {
|
||||
const val MAX_FOV = 100.0
|
||||
}
|
||||
}
|
||||
|
||||
internal object TrackingCameras {
|
||||
const val REAR = "rear"
|
||||
const val FRONT = "front"
|
||||
private const val MAX_FPS = 30
|
||||
|
||||
fun list(context: Context): List<TrackingCamera> {
|
||||
val manager = context.getSystemService(CameraManager::class.java) ?: return emptyList()
|
||||
val ids = runCatching { manager.cameraIdList.toList() }.getOrDefault(emptyList())
|
||||
return ids.mapNotNull { id -> runCatching { describe(id, manager.getCameraCharacteristics(id)) }.getOrNull() }
|
||||
}
|
||||
|
||||
private fun describe(id: String, c: CameraCharacteristics): TrackingCamera? {
|
||||
val facing = c.get(CameraCharacteristics.LENS_FACING) ?: return null
|
||||
if (facing == CameraCharacteristics.LENS_FACING_EXTERNAL) return null
|
||||
val caps = c.get(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES) ?: return null
|
||||
if (CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_BACKWARD_COMPATIBLE !in caps) return null
|
||||
val active = c.get(CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE) ?: return null
|
||||
val pixels = c.get(CameraCharacteristics.SENSOR_INFO_PIXEL_ARRAY_SIZE) ?: return null
|
||||
val physical = c.get(CameraCharacteristics.SENSOR_INFO_PHYSICAL_SIZE) ?: return null
|
||||
val focal = c.get(CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS)?.firstOrNull() ?: return null
|
||||
val calibrated = c.get(CameraCharacteristics.LENS_INTRINSIC_CALIBRATION)?.firstOrNull()?.takeIf { it > 1f }
|
||||
val focalPixels = calibrated ?: (focal * pixels.width / physical.width)
|
||||
val minZoom = if (Build.VERSION.SDK_INT >= 30) c.get(CameraCharacteristics.CONTROL_ZOOM_RATIO_RANGE)?.lower ?: 1f else 1f
|
||||
val realtime = c.get(CameraCharacteristics.SENSOR_INFO_TIMESTAMP_SOURCE) == CameraCharacteristics.SENSOR_INFO_TIMESTAMP_SOURCE_REALTIME
|
||||
val fps = steadyFps(c.get(CameraCharacteristics.CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES)?.map { it.lower to it.upper }.orEmpty())
|
||||
return TrackingCamera(id, facing == CameraCharacteristics.LENS_FACING_FRONT, c.get(CameraCharacteristics.SENSOR_ORIENTATION) ?: 0,
|
||||
focalPixels, active.width(), active.height(), minZoom, realtime, fps)
|
||||
}
|
||||
|
||||
fun steadyFps(ranges: List<Pair<Int, Int>>): Pair<Int, Int>? =
|
||||
ranges.filter { it.second <= MAX_FPS }.maxWithOrNull(compareBy<Pair<Int, Int>> { it.first }.thenByDescending { it.second })
|
||||
|
||||
fun sides(cameras: List<TrackingCamera>): Set<String> = cameras.mapTo(LinkedHashSet()) { if (it.front) FRONT else REAR }
|
||||
|
||||
fun pick(cameras: List<TrackingCamera>, side: String?): TrackingCamera? {
|
||||
val wanted = cameras.filter { side == null || (it.front == (side == FRONT)) }.ifEmpty { cameras }
|
||||
val preferred = if (side == null) wanted.filter { !it.front }.ifEmpty { wanted } else wanted
|
||||
return preferred.maxWithOrNull(compareBy<TrackingCamera> { min(widest(it), TrackingCamera.MAX_FOV.toFloat()) }
|
||||
.thenBy { it.activeWidth.toLong() * it.activeHeight })
|
||||
}
|
||||
|
||||
private fun widest(c: TrackingCamera): Float =
|
||||
Math.toDegrees(2.0 * atan(tan(Math.toRadians(c.fovDegrees / 2.0)) / c.zoom())).toFloat()
|
||||
}
|
||||
|
||||
internal object CameraGeometry {
|
||||
fun axes(front: Boolean, orientation: Int, turns: Int): Pair<Vec3, Vec3> {
|
||||
val t = Math.toRadians(orientation.toDouble())
|
||||
val c = Math.cos(t).toFloat()
|
||||
val s = Math.sin(t).toFloat()
|
||||
var u = if (front) Vec3(-c, -s, 0f) else Vec3(c, -s, 0f)
|
||||
var v = if (front) Vec3(s, -c, 0f) else Vec3(-s, -c, 0f)
|
||||
repeat(Math.floorMod(turns, 4)) {
|
||||
val nu = v * -1f
|
||||
v = u
|
||||
u = nu
|
||||
}
|
||||
return u to v
|
||||
}
|
||||
|
||||
fun upInCamera(front: Boolean, orientation: Int, turns: Int, up: Vec3): Vec3 {
|
||||
val (u, v) = axes(front, orientation, turns)
|
||||
val n = up.normalized()
|
||||
return Vec3(dot(n, u), dot(n, v), dot(n, u.cross(v)))
|
||||
}
|
||||
|
||||
fun uprightTurns(front: Boolean, orientation: Int, up: Vec3): Int =
|
||||
(0..3).minBy { turns -> dot(up.normalized(), axes(front, orientation, turns).second) }
|
||||
|
||||
fun placement(front: Boolean, up: Vec3): Placement {
|
||||
val n = up.normalized()
|
||||
if (n.length() < 0.5f) return Placement.UNKNOWN
|
||||
if (n.z > FLAT) return Placement.FACE_UP
|
||||
if (n.z < -FLAT) return Placement.FACE_DOWN
|
||||
val elevation = Math.toDegrees(asin((if (front) n.z else -n.z).toDouble().coerceIn(-1.0, 1.0)))
|
||||
return when {
|
||||
elevation > STEEP -> Placement.POINTS_UP
|
||||
elevation < -STEEP -> Placement.POINTS_DOWN
|
||||
else -> Placement.STANDING
|
||||
}
|
||||
}
|
||||
|
||||
private fun dot(a: Vec3, b: Vec3) = a.x * b.x + a.y * b.y + a.z * b.z
|
||||
|
||||
private const val FLAT = 0.9f
|
||||
private const val STEEP = 45.0
|
||||
}
|
||||
|
||||
internal enum class Placement { UNKNOWN, STANDING, FACE_UP, FACE_DOWN, POINTS_UP, POINTS_DOWN }
|
||||
@@ -47,6 +47,7 @@ object ControllerPrefs {
|
||||
val qamButton: Boolean,
|
||||
val keyboardButton: Boolean,
|
||||
val selectSteam: Boolean,
|
||||
val joyConVr: Boolean,
|
||||
val snap: Boolean,
|
||||
val customLayout: Boolean,
|
||||
val mapping: Map<String, String>,
|
||||
@@ -67,6 +68,7 @@ object ControllerPrefs {
|
||||
qamButton = p.getBoolean("qamButton", true),
|
||||
keyboardButton = p.getBoolean("keyboardButton", true),
|
||||
selectSteam = p.getBoolean("selectSteam", false),
|
||||
joyConVr = joyConVr(context),
|
||||
snap = p.getBoolean("snap", true),
|
||||
customLayout = p.all.keys.any { it.startsWith("layout.") },
|
||||
mapping = mappable.associate { (id, _) -> id to target(context, id) },
|
||||
@@ -127,6 +129,12 @@ object ControllerPrefs {
|
||||
prefs(context).edit().putBoolean("selectSteam", on).apply()
|
||||
}
|
||||
|
||||
fun joyConVr(context: Context): Boolean = prefs(context).getBoolean("joyConVr", true)
|
||||
|
||||
fun setJoyConVr(context: Context, on: Boolean) {
|
||||
prefs(context).edit().putBoolean("joyConVr", on).apply()
|
||||
}
|
||||
|
||||
fun setSnap(context: Context, on: Boolean) {
|
||||
prefs(context).edit().putBoolean("snap", on).apply()
|
||||
}
|
||||
|
||||
@@ -119,6 +119,10 @@ public class FakeInputWriter {
|
||||
|
||||
private static native void nativeStoreFence();
|
||||
|
||||
public static void storeFence() {
|
||||
nativeStoreFence();
|
||||
}
|
||||
|
||||
private final File eventFile;
|
||||
private final int slot;
|
||||
private final ByteBuffer buffer = ByteBuffer.allocateDirect(BUFFER_SIZE);
|
||||
|
||||
@@ -0,0 +1,143 @@
|
||||
package com.droiddeck.launcher.input
|
||||
|
||||
import android.view.InputDevice
|
||||
import android.view.KeyEvent
|
||||
import android.view.MotionEvent
|
||||
|
||||
object JoyCon {
|
||||
const val LEFT = 0
|
||||
const val RIGHT = 1
|
||||
|
||||
const val VR_TRIGGER = 1 shl 0
|
||||
const val VR_GRIP = 1 shl 1
|
||||
const val VR_A = 1 shl 2
|
||||
const val VR_B = 1 shl 3
|
||||
const val VR_STICK = 1 shl 4
|
||||
const val VR_SYSTEM = 1 shl 5
|
||||
const val VR_RUMBLE_SLOT = 0x100
|
||||
|
||||
private const val VENDOR = 0x057e
|
||||
private const val PRODUCT_LEFT = 0x2006
|
||||
private const val PRODUCT_RIGHT = 0x2007
|
||||
|
||||
private const val BTN_SOUTH = 0x130
|
||||
private const val BTN_EAST = 0x131
|
||||
private const val BTN_NORTH = 0x133
|
||||
private const val BTN_WEST = 0x134
|
||||
private const val BTN_Z = 0x135
|
||||
private const val BTN_TL = 0x136
|
||||
private const val BTN_TR = 0x137
|
||||
private const val BTN_TL2 = 0x138
|
||||
private const val BTN_TR2 = 0x139
|
||||
private const val BTN_SELECT = 0x13a
|
||||
private const val BTN_START = 0x13b
|
||||
private const val BTN_MODE = 0x13c
|
||||
private const val BTN_THUMBL = 0x13d
|
||||
private const val BTN_THUMBR = 0x13e
|
||||
private const val BTN_DPAD_UP = 0x220
|
||||
private const val BTN_DPAD_DOWN = 0x221
|
||||
private const val BTN_DPAD_LEFT = 0x222
|
||||
private const val BTN_DPAD_RIGHT = 0x223
|
||||
private val NO_VR_DEVICES = intArrayOf(PadBridge.NO_CONTROLLER, PadBridge.NO_CONTROLLER)
|
||||
private var vrOwner: Any? = null
|
||||
@Volatile private var vrDevices = NO_VR_DEVICES
|
||||
|
||||
@JvmStatic
|
||||
fun side(device: InputDevice?): Int = when {
|
||||
device == null || device.isVirtual || device.vendorId != VENDOR -> -1
|
||||
device.productId == PRODUCT_LEFT -> LEFT
|
||||
device.productId == PRODUCT_RIGHT -> RIGHT
|
||||
else -> -1
|
||||
}
|
||||
|
||||
@JvmStatic
|
||||
fun stickAxes(side: Int, device: InputDevice, source: Int): IntArray = when {
|
||||
side == LEFT -> intArrayOf(MotionEvent.AXIS_X, MotionEvent.AXIS_Y)
|
||||
device.getMotionRange(MotionEvent.AXIS_RX, source) != null -> intArrayOf(MotionEvent.AXIS_RX, MotionEvent.AXIS_RY)
|
||||
else -> intArrayOf(MotionEvent.AXIS_Z, MotionEvent.AXIS_RZ)
|
||||
}
|
||||
|
||||
@JvmStatic
|
||||
fun isRail(side: Int, scanCode: Int): Boolean = when (side) {
|
||||
LEFT -> scanCode == BTN_TR || scanCode == BTN_TR2
|
||||
RIGHT -> scanCode == BTN_TL || scanCode == BTN_TL2
|
||||
else -> false
|
||||
}
|
||||
|
||||
internal fun padKey(side: Int, scanCode: Int): Int = when (side) {
|
||||
LEFT -> when (scanCode) {
|
||||
BTN_TL -> KeyEvent.KEYCODE_BUTTON_L1
|
||||
BTN_TL2 -> KeyEvent.KEYCODE_BUTTON_L2
|
||||
BTN_SELECT -> KeyEvent.KEYCODE_BUTTON_SELECT
|
||||
BTN_THUMBL -> KeyEvent.KEYCODE_BUTTON_THUMBL
|
||||
BTN_Z -> KeyEvent.KEYCODE_BUTTON_Z
|
||||
BTN_DPAD_UP -> KeyEvent.KEYCODE_DPAD_UP
|
||||
BTN_DPAD_DOWN -> KeyEvent.KEYCODE_DPAD_DOWN
|
||||
BTN_DPAD_LEFT -> KeyEvent.KEYCODE_DPAD_LEFT
|
||||
BTN_DPAD_RIGHT -> KeyEvent.KEYCODE_DPAD_RIGHT
|
||||
else -> KeyEvent.KEYCODE_UNKNOWN
|
||||
}
|
||||
RIGHT -> when (scanCode) {
|
||||
BTN_SOUTH -> KeyEvent.KEYCODE_BUTTON_A
|
||||
BTN_EAST -> KeyEvent.KEYCODE_BUTTON_B
|
||||
BTN_NORTH -> KeyEvent.KEYCODE_BUTTON_Y
|
||||
BTN_WEST -> KeyEvent.KEYCODE_BUTTON_X
|
||||
BTN_TR -> KeyEvent.KEYCODE_BUTTON_R1
|
||||
BTN_TR2 -> KeyEvent.KEYCODE_BUTTON_R2
|
||||
BTN_START -> KeyEvent.KEYCODE_BUTTON_START
|
||||
BTN_THUMBR -> KeyEvent.KEYCODE_BUTTON_THUMBR
|
||||
BTN_MODE -> KeyEvent.KEYCODE_BUTTON_MODE
|
||||
else -> KeyEvent.KEYCODE_UNKNOWN
|
||||
}
|
||||
else -> KeyEvent.KEYCODE_UNKNOWN
|
||||
}
|
||||
|
||||
@JvmStatic
|
||||
fun normalize(event: KeyEvent): KeyEvent {
|
||||
val side = side(event.device)
|
||||
if (side < 0) return event
|
||||
val code = padKey(side, event.scanCode)
|
||||
if (code == KeyEvent.KEYCODE_UNKNOWN || code == event.keyCode) return event
|
||||
return KeyEvent(event.downTime, event.eventTime, event.action, code, event.repeatCount, event.metaState,
|
||||
event.deviceId, event.scanCode, event.flags, event.source)
|
||||
}
|
||||
|
||||
internal fun vrButton(side: Int, scanCode: Int): Int = when (side) {
|
||||
LEFT -> when (scanCode) {
|
||||
BTN_TL2 -> VR_TRIGGER
|
||||
BTN_TL, BTN_TR, BTN_TR2 -> VR_GRIP
|
||||
BTN_DPAD_DOWN, BTN_DPAD_RIGHT -> VR_A
|
||||
BTN_DPAD_UP, BTN_DPAD_LEFT, BTN_SELECT -> VR_B
|
||||
BTN_THUMBL -> VR_STICK
|
||||
BTN_Z -> VR_SYSTEM
|
||||
else -> 0
|
||||
}
|
||||
RIGHT -> when (scanCode) {
|
||||
BTN_TR2 -> VR_TRIGGER
|
||||
BTN_TR, BTN_TL, BTN_TL2 -> VR_GRIP
|
||||
BTN_SOUTH, BTN_EAST -> VR_A
|
||||
BTN_NORTH, BTN_WEST, BTN_START -> VR_B
|
||||
BTN_THUMBR -> VR_STICK
|
||||
BTN_MODE -> VR_SYSTEM
|
||||
else -> 0
|
||||
}
|
||||
else -> 0
|
||||
}
|
||||
|
||||
@JvmStatic
|
||||
fun vrRumbleDevice(slot: Int): Int? =
|
||||
if (slot == VR_RUMBLE_SLOT + LEFT || slot == VR_RUMBLE_SLOT + RIGHT) vrDevices[slot - VR_RUMBLE_SLOT] else null
|
||||
|
||||
@Synchronized
|
||||
internal fun setVrDevices(owner: Any, devices: IntArray) {
|
||||
vrOwner = owner
|
||||
vrDevices = devices
|
||||
}
|
||||
|
||||
@Synchronized
|
||||
internal fun clearVrDevices(owner: Any) {
|
||||
if (vrOwner !== owner) return
|
||||
vrOwner = null
|
||||
vrDevices = NO_VR_DEVICES
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,213 @@
|
||||
package com.droiddeck.launcher.input
|
||||
|
||||
import android.content.Context
|
||||
import com.droiddeck.launcher.glasses.Quaternion
|
||||
import com.droiddeck.launcher.glasses.Vec3
|
||||
import kotlin.math.abs
|
||||
import kotlin.math.exp
|
||||
import kotlin.math.roundToInt
|
||||
import kotlin.math.sqrt
|
||||
|
||||
internal class JoyConAccelCalibration {
|
||||
var bias = Vec3(0f, 0f, 0f)
|
||||
private set
|
||||
var scale = 1f
|
||||
private set
|
||||
var ready = false
|
||||
private set
|
||||
private val sums = HashMap<Int, DoubleArray>()
|
||||
|
||||
fun restore(saved: FloatArray?) {
|
||||
if (saved == null || saved.size != 4 || saved.any { it.isNaN() || it.isInfinite() }) return
|
||||
val b = Vec3(saved[0], saved[1], saved[2])
|
||||
if (!plausible(b, saved[3])) return
|
||||
bias = b
|
||||
scale = saved[3]
|
||||
ready = true
|
||||
}
|
||||
|
||||
fun saved(): FloatArray? = if (ready) floatArrayOf(bias.x, bias.y, bias.z, scale) else null
|
||||
|
||||
fun correct(raw: Vec3): Vec3 = if (ready) (raw - bias) * scale else raw
|
||||
|
||||
fun addResting(raw: Vec3): Boolean {
|
||||
val n = raw.length()
|
||||
if (n < 1e-3f) return false
|
||||
val key = bin(raw.x / n) * 9 + bin(raw.y / n) * 3 + bin(raw.z / n)
|
||||
val s = sums.getOrPut(key) { DoubleArray(4) }
|
||||
s[0] += raw.x.toDouble()
|
||||
s[1] += raw.y.toDouble()
|
||||
s[2] += raw.z.toDouble()
|
||||
s[3] += 1.0
|
||||
return s[3] % SOLVE_EVERY == 0.0 && solve()
|
||||
}
|
||||
|
||||
private fun solve(): Boolean {
|
||||
val points = sums.values.filter { it[3] >= MIN_SAMPLES }.map { doubleArrayOf(it[0] / it[3], it[1] / it[3], it[2] / it[3]) }
|
||||
if (points.size < MIN_BINS || smallestSpread(points) < MIN_SPREAD) return false
|
||||
val ata = Array(4) { DoubleArray(4) }
|
||||
val atb = DoubleArray(4)
|
||||
for (p in points) {
|
||||
val row = doubleArrayOf(2 * p[0], 2 * p[1], 2 * p[2], 1.0)
|
||||
val rhs = p[0] * p[0] + p[1] * p[1] + p[2] * p[2]
|
||||
for (i in 0..3) {
|
||||
atb[i] += row[i] * rhs
|
||||
for (j in 0..3) ata[i][j] += row[i] * row[j]
|
||||
}
|
||||
}
|
||||
val x = solve4(ata, atb) ?: return false
|
||||
val radius2 = x[3] + x[0] * x[0] + x[1] * x[1] + x[2] * x[2]
|
||||
if (radius2 <= 0) return false
|
||||
val b = Vec3(x[0].toFloat(), x[1].toFloat(), x[2].toFloat())
|
||||
val s = (GRAVITY / sqrt(radius2)).toFloat()
|
||||
if (!plausible(b, s)) return false
|
||||
bias = b
|
||||
scale = s
|
||||
ready = true
|
||||
return true
|
||||
}
|
||||
|
||||
private fun bin(v: Float) = (v * 1.5f).roundToInt().coerceIn(-1, 1) + 1
|
||||
|
||||
companion object {
|
||||
private const val GRAVITY = 9.80665
|
||||
private const val MIN_SAMPLES = 20.0
|
||||
private const val MIN_BINS = 5
|
||||
private const val MIN_SPREAD = 2.0
|
||||
private const val SOLVE_EVERY = 25.0
|
||||
private const val MAX_BIAS = 1.5f
|
||||
private const val MAX_SCALE_ERROR = 0.1f
|
||||
|
||||
private fun plausible(b: Vec3, s: Float) = b.length() <= MAX_BIAS && abs(s - 1f) <= MAX_SCALE_ERROR
|
||||
|
||||
private fun smallestSpread(points: List<DoubleArray>): Double {
|
||||
val mean = DoubleArray(3) { i -> points.sumOf { it[i] } / points.size }
|
||||
val c = Array(3) { DoubleArray(3) }
|
||||
for (p in points) for (i in 0..2) for (j in 0..2) c[i][j] += (p[i] - mean[i]) * (p[j] - mean[j])
|
||||
return sqrt(maxOf(smallestEigenvalue(c), 0.0))
|
||||
}
|
||||
|
||||
private fun smallestEigenvalue(m: Array<DoubleArray>): Double {
|
||||
val a = Array(3) { m[it].copyOf() }
|
||||
repeat(50) {
|
||||
var p = 0
|
||||
var q = 1
|
||||
for ((i, j) in listOf(0 to 2, 1 to 2)) if (abs(a[i][j]) > abs(a[p][q])) { p = i; q = j }
|
||||
if (abs(a[p][q]) < 1e-12) return minOf(a[0][0], a[1][1], a[2][2])
|
||||
val theta = (a[q][q] - a[p][p]) / (2 * a[p][q])
|
||||
val t = (if (theta >= 0) 1.0 else -1.0) / (abs(theta) + sqrt(theta * theta + 1))
|
||||
val c = 1 / sqrt(t * t + 1)
|
||||
val s = t * c
|
||||
for (k in 0..2) {
|
||||
val kp = a[k][p]
|
||||
val kq = a[k][q]
|
||||
a[k][p] = c * kp - s * kq
|
||||
a[k][q] = s * kp + c * kq
|
||||
}
|
||||
for (k in 0..2) {
|
||||
val pk = a[p][k]
|
||||
val qk = a[q][k]
|
||||
a[p][k] = c * pk - s * qk
|
||||
a[q][k] = s * pk + c * qk
|
||||
}
|
||||
}
|
||||
return minOf(a[0][0], a[1][1], a[2][2])
|
||||
}
|
||||
|
||||
private fun solve4(a: Array<DoubleArray>, b: DoubleArray): DoubleArray? {
|
||||
val m = Array(4) { i -> DoubleArray(5) { j -> if (j < 4) a[i][j] else b[i] } }
|
||||
for (col in 0..3) {
|
||||
val pivot = (col..3).maxBy { abs(m[it][col]) }
|
||||
if (abs(m[pivot][col]) < 1e-9) return null
|
||||
val swap = m[col]
|
||||
m[col] = m[pivot]
|
||||
m[pivot] = swap
|
||||
for (r in 0..3) {
|
||||
if (r == col) continue
|
||||
val f = m[r][col] / m[col][col]
|
||||
for (k in col..4) m[r][k] -= f * m[col][k]
|
||||
}
|
||||
}
|
||||
return DoubleArray(4) { m[it][4] / m[it][it] }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
internal class JoyConMotion {
|
||||
var velocity = Vec3(0f, 0f, 0f)
|
||||
private set
|
||||
var displacement = Vec3(0f, 0f, 0f)
|
||||
private set
|
||||
var inertial = Vec3(0f, 0f, 0f)
|
||||
private set
|
||||
var resting = false
|
||||
private set
|
||||
private var gravity = Float.NaN
|
||||
private var average = Vec3(0f, 0f, 0f)
|
||||
private var calmFor = 0f
|
||||
private var stillFor = 0f
|
||||
private var movingFor = 0f
|
||||
|
||||
fun update(rotation: Quaternion, accel: Vec3, rate: Vec3, calibrated: Boolean, dt: Float) {
|
||||
if (dt <= 0f) return
|
||||
if (gravity.isNaN()) {
|
||||
gravity = accel.length()
|
||||
average = accel
|
||||
}
|
||||
if (calibrated) gravity = GRAVITY
|
||||
average += (accel - average) * (dt / AVERAGE_SECONDS).coerceAtMost(1f)
|
||||
val world = rotation.rotate(accel)
|
||||
val linear = Vec3(world.x, world.y - gravity, world.z)
|
||||
inertial = inertial * exp(-dt / INERTIAL_SECONDS) + linear * dt
|
||||
val calm = rate.length() < CALM_RATE && (accel - average).length() < CALM_ACCEL
|
||||
calmFor = if (calm) calmFor + dt else 0f
|
||||
val still = rate.length() < CALM_RATE &&
|
||||
(linear.length() < STILL_ACCEL || calmFor >= if (calibrated) CALIBRATED_CALM_SECONDS else UNCALIBRATED_CALM_SECONDS)
|
||||
stillFor = if (still) stillFor + dt else 0f
|
||||
resting = calmFor >= RESTING_SECONDS && rate.length() < RESTING_RATE
|
||||
if (stillFor >= STILL_SECONDS) {
|
||||
movingFor = 0f
|
||||
velocity *= exp(-dt / SETTLE_SECONDS)
|
||||
if (!calibrated) gravity += (accel.length() - gravity) * (dt / GRAVITY_SECONDS).coerceAtMost(1f)
|
||||
} else {
|
||||
movingFor += dt
|
||||
val leak = if (movingFor < FREE_SECONDS) LEAK_SECONDS else LONG_MOVE_LEAK_SECONDS
|
||||
velocity = velocity * exp(-dt / leak) + linear * dt
|
||||
}
|
||||
displacement += velocity * dt
|
||||
}
|
||||
|
||||
companion object {
|
||||
private const val GRAVITY = 9.80665f
|
||||
private const val AVERAGE_SECONDS = 0.1f
|
||||
private const val CALM_RATE = 0.2f
|
||||
private const val CALM_ACCEL = 0.25f
|
||||
private const val STILL_ACCEL = 0.12f
|
||||
private const val STILL_SECONDS = 0.2f
|
||||
private const val UNCALIBRATED_CALM_SECONDS = 0.5f
|
||||
private const val CALIBRATED_CALM_SECONDS = 2f
|
||||
private const val SETTLE_SECONDS = 0.05f
|
||||
private const val LEAK_SECONDS = 20f
|
||||
private const val FREE_SECONDS = 1f
|
||||
private const val LONG_MOVE_LEAK_SECONDS = 0.2f
|
||||
private const val GRAVITY_SECONDS = 0.5f
|
||||
private const val INERTIAL_SECONDS = 30f
|
||||
private const val RESTING_SECONDS = 0.3f
|
||||
private const val RESTING_RATE = 0.05f
|
||||
}
|
||||
}
|
||||
|
||||
internal object JoyConCalibrations {
|
||||
fun load(context: Context, descriptor: String?): FloatArray? {
|
||||
if (descriptor.isNullOrEmpty()) return null
|
||||
val text = ControllerPrefs.prefs(context).getString(KEY + descriptor, null) ?: return null
|
||||
return text.split(',').mapNotNull { it.toFloatOrNull() }.toFloatArray().takeIf { it.size == 4 }
|
||||
}
|
||||
|
||||
fun save(context: Context, descriptor: String?, values: FloatArray?) {
|
||||
if (descriptor.isNullOrEmpty() || values == null) return
|
||||
ControllerPrefs.prefs(context).edit().putString(KEY + descriptor, values.joinToString(",")).apply()
|
||||
}
|
||||
|
||||
private const val KEY = "joyConAccel."
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
package com.droiddeck.launcher.input
|
||||
|
||||
import com.droiddeck.launcher.glasses.Quaternion
|
||||
import com.droiddeck.launcher.glasses.Vec3
|
||||
import kotlin.math.abs
|
||||
|
||||
internal class JoyConOrientation {
|
||||
var rotation = Quaternion.IDENTITY
|
||||
private set
|
||||
var rate = Vec3(0f, 0f, 0f)
|
||||
private set
|
||||
var ready = false
|
||||
private set
|
||||
private var bias = Vec3(0f, 0f, 0f)
|
||||
private var stillFor = 0f
|
||||
|
||||
fun update(accelKernel: FloatArray, gyroKernel: FloatArray, dt: Float) = update(axes(accelKernel), axes(gyroKernel), dt)
|
||||
|
||||
fun update(accel: Vec3, gyro: Vec3, dt: Float) {
|
||||
val norm = accel.length()
|
||||
val gravityValid = abs(norm - GRAVITY) < GRAVITY_TOLERANCE
|
||||
if (!ready) {
|
||||
if (!gravityValid) return
|
||||
val up = accel.normalized()
|
||||
val axis = up.cross(WORLD_UP)
|
||||
rotation = if (up.y < -0.9999f) Quaternion.fromAxisAngle(Vec3(1f, 0f, 0f), Math.PI.toFloat())
|
||||
else Quaternion(1f + up.y, axis.x, axis.y, axis.z).normalized()
|
||||
rate = Vec3(0f, 0f, 0f)
|
||||
ready = true
|
||||
return
|
||||
}
|
||||
if (dt <= 0f) return
|
||||
val still = gravityValid && abs(norm - GRAVITY) < STILL_ACCEL && (gyro - bias).length() < STILL_RATE
|
||||
stillFor = if (still) stillFor + dt else 0f
|
||||
if (stillFor >= STILL_SECONDS) {
|
||||
val alpha = (dt / BIAS_SECONDS).coerceAtMost(1f)
|
||||
bias = bias * (1f - alpha) + gyro * alpha
|
||||
}
|
||||
rate = gyro - bias
|
||||
var corrected = rate
|
||||
if (gravityValid && rate.length() < MOVING_RATE) {
|
||||
val predictedUp = rotation.conjugate().rotate(WORLD_UP)
|
||||
corrected += accel.normalized().cross(predictedUp) * (GAIN * (1f - abs(norm - GRAVITY) / GRAVITY_TOLERANCE))
|
||||
}
|
||||
rotation = rotation.integrateBodyRate(corrected, dt).normalized()
|
||||
}
|
||||
|
||||
companion object {
|
||||
private val WORLD_UP = Vec3(0f, 1f, 0f)
|
||||
private const val GRAVITY = 9.80665f
|
||||
private const val GRAVITY_TOLERANCE = 1.0f
|
||||
private const val STILL_ACCEL = 0.5f
|
||||
private const val STILL_RATE = 0.05f
|
||||
private const val STILL_SECONDS = 0.3f
|
||||
private const val BIAS_SECONDS = 2f
|
||||
private const val GAIN = 1.0f
|
||||
private const val MOVING_RATE = 1.0f
|
||||
|
||||
fun axes(v: FloatArray) = Vec3(-v[1], v[2], -v[0])
|
||||
}
|
||||
}
|
||||
|
||||
internal class JoyConReports {
|
||||
class Sample(val accel: FloatArray, val gyro: FloatArray, val dt: Float)
|
||||
|
||||
private val pending = ArrayList<FloatArray>(4)
|
||||
private var pendingAt = 0L
|
||||
private var previousFirst: FloatArray? = null
|
||||
private var previousLast: FloatArray? = null
|
||||
private var previousSample: FloatArray? = null
|
||||
private var previousAt = 0L
|
||||
private var order = NEWEST_FIRST_START
|
||||
|
||||
val size: Int get() = pending.size
|
||||
|
||||
fun add(accel: FloatArray, gyro: FloatArray, arrivalNs: Long) {
|
||||
if (pending.isEmpty()) pendingAt = arrivalNs
|
||||
pending += floatArrayOf(accel[0], accel[1], accel[2], gyro[0], gyro[1], gyro[2])
|
||||
}
|
||||
|
||||
fun flush(): List<Sample> {
|
||||
if (pending.isEmpty()) return emptyList()
|
||||
val first = pending.first()
|
||||
val last = pending.last()
|
||||
if (pending.size > 1) {
|
||||
if (previousFirst?.contentEquals(last) == true) order = (order + 1).coerceAtMost(ORDER_LIMIT)
|
||||
else if (previousLast?.contentEquals(first) == true) order = (order - 1).coerceAtLeast(-ORDER_LIMIT)
|
||||
}
|
||||
val chronological = if (order >= 0) pending.asReversed() else pending
|
||||
val fresh = ArrayList<FloatArray>(chronological.size)
|
||||
var before = previousSample
|
||||
for (sample in chronological) {
|
||||
if (before?.contentEquals(sample) != true) fresh += sample
|
||||
before = sample
|
||||
}
|
||||
val used = fresh.ifEmpty { listOf(chronological.last()) }
|
||||
val dt = if (previousAt == 0L) 0f else ((pendingAt - previousAt).coerceAtLeast(0L) * 1e-9f / used.size).coerceAtMost(MAX_STEP_S)
|
||||
val out = used.map { Sample(it.copyOfRange(0, 3), it.copyOfRange(3, 6), dt) }
|
||||
previousFirst = first
|
||||
previousLast = last
|
||||
previousSample = chronological.last()
|
||||
previousAt = pendingAt
|
||||
pending.clear()
|
||||
return out
|
||||
}
|
||||
|
||||
companion object {
|
||||
const val SAMPLES_PER_REPORT = 3
|
||||
private const val NEWEST_FIRST_START = 2
|
||||
private const val ORDER_LIMIT = 20
|
||||
private const val MAX_STEP_S = 0.05f
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
package com.droiddeck.launcher.input
|
||||
|
||||
import android.util.Log
|
||||
import com.droiddeck.launcher.glasses.Quaternion
|
||||
import com.droiddeck.launcher.glasses.Vec3
|
||||
import java.io.File
|
||||
import java.io.RandomAccessFile
|
||||
import java.nio.ByteOrder
|
||||
import java.nio.MappedByteBuffer
|
||||
import java.nio.channels.FileChannel
|
||||
|
||||
internal class JoyConShm private constructor(
|
||||
private val file: RandomAccessFile,
|
||||
private val map: MappedByteBuffer,
|
||||
private val fence: () -> Unit,
|
||||
) {
|
||||
class Hand(
|
||||
val connected: Boolean,
|
||||
val charging: Boolean = false,
|
||||
val buttons: Int = 0,
|
||||
val rotation: Quaternion = Quaternion.IDENTITY,
|
||||
val rate: Vec3 = Vec3(0f, 0f, 0f),
|
||||
val displacement: Vec3 = Vec3(0f, 0f, 0f),
|
||||
val velocity: Vec3 = Vec3(0f, 0f, 0f),
|
||||
val inertial: Vec3 = Vec3(0f, 0f, 0f),
|
||||
val stickX: Float = 0f,
|
||||
val stickY: Float = 0f,
|
||||
val battery: Float = -1f,
|
||||
val recenter: Int = 0,
|
||||
val timeNs: Long = 0L,
|
||||
)
|
||||
|
||||
private var seq = map.getInt(SEQ).let { if (it and 1 != 0) it + 1 else it }
|
||||
|
||||
init {
|
||||
begin()
|
||||
map.putInt(MAGIC_AT, MAGIC)
|
||||
map.putInt(VERSION_AT, VERSION)
|
||||
for (at in HANDS until SIZE step 4) map.putInt(at, 0)
|
||||
end()
|
||||
}
|
||||
|
||||
fun write(side: Int, hand: Hand) {
|
||||
val at = HANDS + side * HAND_SIZE
|
||||
begin()
|
||||
map.putInt(at, (if (hand.connected) CONNECTED else 0) or (if (hand.charging) CHARGING else 0))
|
||||
map.putInt(at + 4, hand.buttons)
|
||||
map.putFloat(at + 8, hand.rotation.w)
|
||||
map.putFloat(at + 12, hand.rotation.x)
|
||||
map.putFloat(at + 16, hand.rotation.y)
|
||||
map.putFloat(at + 20, hand.rotation.z)
|
||||
map.putFloat(at + 24, hand.rate.x)
|
||||
map.putFloat(at + 28, hand.rate.y)
|
||||
map.putFloat(at + 32, hand.rate.z)
|
||||
map.putFloat(at + 36, hand.stickX)
|
||||
map.putFloat(at + 40, hand.stickY)
|
||||
map.putFloat(at + 44, hand.battery)
|
||||
map.putInt(at + 48, hand.recenter)
|
||||
map.putLong(at + 56, hand.timeNs)
|
||||
map.putFloat(at + 64, hand.displacement.x)
|
||||
map.putFloat(at + 68, hand.displacement.y)
|
||||
map.putFloat(at + 72, hand.displacement.z)
|
||||
map.putFloat(at + 76, hand.velocity.x)
|
||||
map.putFloat(at + 80, hand.velocity.y)
|
||||
map.putFloat(at + 84, hand.velocity.z)
|
||||
map.putFloat(at + 88, hand.inertial.x)
|
||||
map.putFloat(at + 92, hand.inertial.y)
|
||||
map.putFloat(at + 96, hand.inertial.z)
|
||||
map.putInt(at + 100, 0)
|
||||
end()
|
||||
}
|
||||
|
||||
fun close() {
|
||||
write(JoyCon.LEFT, Hand(connected = false))
|
||||
write(JoyCon.RIGHT, Hand(connected = false))
|
||||
runCatching { file.close() }
|
||||
}
|
||||
|
||||
private fun begin() {
|
||||
map.putInt(SEQ, ++seq)
|
||||
fence()
|
||||
}
|
||||
|
||||
private fun end() {
|
||||
fence()
|
||||
map.putInt(SEQ, ++seq)
|
||||
}
|
||||
|
||||
companion object {
|
||||
const val FILE = "droiddeck-joycon"
|
||||
const val SIZE = 224
|
||||
private const val MAGIC = 0x4E434A44
|
||||
private const val VERSION = 3
|
||||
private const val MAGIC_AT = 0
|
||||
private const val VERSION_AT = 4
|
||||
private const val SEQ = 8
|
||||
private const val HANDS = 16
|
||||
private const val HAND_SIZE = 104
|
||||
private const val CONNECTED = 1
|
||||
private const val CHARGING = 2
|
||||
|
||||
fun open(path: File, fence: () -> Unit = FakeInputWriter::storeFence): JoyConShm? = runCatching {
|
||||
path.parentFile?.mkdirs()
|
||||
val file = RandomAccessFile(path, "rw")
|
||||
try {
|
||||
file.setLength(SIZE.toLong())
|
||||
val map = file.channel.map(FileChannel.MapMode.READ_WRITE, 0, SIZE.toLong())
|
||||
map.order(ByteOrder.LITTLE_ENDIAN)
|
||||
JoyConShm(file, map, fence)
|
||||
} catch (e: Exception) {
|
||||
file.close()
|
||||
throw e
|
||||
}
|
||||
}.onFailure { Log.w("JoyConVr", "Joy-Con pose file unavailable: $path", it) }.getOrNull()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,363 @@
|
||||
package com.droiddeck.launcher.input
|
||||
|
||||
import android.content.Context
|
||||
import android.hardware.BatteryState
|
||||
import android.hardware.Sensor
|
||||
import android.hardware.SensorEvent
|
||||
import android.hardware.SensorEventListener
|
||||
import android.hardware.SensorManager
|
||||
import android.os.CombinedVibration
|
||||
import android.os.Handler
|
||||
import android.os.HandlerThread
|
||||
import android.os.Looper
|
||||
import android.os.Process
|
||||
import android.os.VibrationEffect
|
||||
import android.util.Log
|
||||
import android.view.InputDevice
|
||||
import android.view.KeyEvent
|
||||
import android.view.MotionEvent
|
||||
import androidx.annotation.RequiresApi
|
||||
import java.io.File
|
||||
import kotlin.math.min
|
||||
import kotlin.math.sqrt
|
||||
|
||||
@RequiresApi(31)
|
||||
class JoyConVr(
|
||||
private val context: Context,
|
||||
private val path: File,
|
||||
private val onDevicesChanged: (taken: List<Int>, released: List<Int>) -> Unit,
|
||||
) {
|
||||
private val thread = HandlerThread("joycon-vr", Process.THREAD_PRIORITY_DISPLAY).apply { start() }
|
||||
private val worker = Handler(thread.looper)
|
||||
private val main = Handler(Looper.getMainLooper())
|
||||
@Volatile private var owned = IntArray(0)
|
||||
private var wanted = false
|
||||
private var closed = false
|
||||
private var retries = 0
|
||||
private var shm: JoyConShm? = null
|
||||
private val hands = arrayOfNulls<Hand>(2)
|
||||
private val retry = Runnable { apply() }
|
||||
|
||||
fun setActive(on: Boolean) {
|
||||
worker.post {
|
||||
if (closed || wanted == on) return@post
|
||||
wanted = on
|
||||
retries = 0
|
||||
apply()
|
||||
}
|
||||
}
|
||||
|
||||
fun refresh() {
|
||||
worker.post {
|
||||
if (closed) return@post
|
||||
retries = 0
|
||||
apply()
|
||||
}
|
||||
}
|
||||
|
||||
fun owns(deviceId: Int): Boolean = owned.contains(deviceId)
|
||||
|
||||
val holding: Boolean get() = owned.isNotEmpty()
|
||||
|
||||
fun onKeyEvent(event: KeyEvent): Boolean {
|
||||
if (!owns(event.deviceId)) return false
|
||||
if (event.repeatCount > 0) return true
|
||||
val down = when (event.action) {
|
||||
KeyEvent.ACTION_DOWN -> true
|
||||
KeyEvent.ACTION_UP -> false
|
||||
else -> return true
|
||||
}
|
||||
val id = event.deviceId
|
||||
val scanCode = event.scanCode
|
||||
worker.post { hand(id)?.key(scanCode, down) }
|
||||
return true
|
||||
}
|
||||
|
||||
fun onMotionEvent(event: MotionEvent): Boolean {
|
||||
if (!owns(event.deviceId)) return false
|
||||
if (!event.isFromSource(InputDevice.SOURCE_JOYSTICK) || event.actionMasked != MotionEvent.ACTION_MOVE) return true
|
||||
val device = event.device ?: return true
|
||||
val side = JoyCon.side(device)
|
||||
if (side < 0) return true
|
||||
val axes = JoyCon.stickAxes(side, device, event.source)
|
||||
val x = event.getAxisValue(axes[0])
|
||||
val y = event.getAxisValue(axes[1])
|
||||
val id = event.deviceId
|
||||
worker.post { hand(id)?.stick(x, -y) }
|
||||
return true
|
||||
}
|
||||
|
||||
fun releaseButtons() {
|
||||
worker.post { hands.forEach { it?.release() } }
|
||||
}
|
||||
|
||||
fun close() {
|
||||
worker.post {
|
||||
closed = true
|
||||
wanted = false
|
||||
apply()
|
||||
shm?.close()
|
||||
shm = null
|
||||
}
|
||||
thread.quitSafely()
|
||||
}
|
||||
|
||||
private fun hand(id: Int): Hand? = hands.firstOrNull { it?.device?.id == id }
|
||||
|
||||
private fun apply() {
|
||||
worker.removeCallbacks(retry)
|
||||
val found = arrayOfNulls<InputDevice>(2)
|
||||
var waiting = false
|
||||
if (wanted && !closed) {
|
||||
for (id in InputDevice.getDeviceIds()) {
|
||||
val device = InputDevice.getDevice(id) ?: continue
|
||||
val side = JoyCon.side(device)
|
||||
if (side < 0 || found[side] != null) continue
|
||||
if (hasMotion(device)) found[side] = device else waiting = true
|
||||
}
|
||||
}
|
||||
val before = owned
|
||||
for (side in 0..1) {
|
||||
val current = hands[side]
|
||||
val device = found[side]
|
||||
if (current != null && current.device.id == device?.id) continue
|
||||
current?.stop()
|
||||
hands[side] = null
|
||||
if (device != null) {
|
||||
hands[side] = start(side, device)
|
||||
if (hands[side] == null) waiting = true
|
||||
}
|
||||
}
|
||||
val now = hands.mapNotNull { it?.device?.id }.toIntArray()
|
||||
owned = now
|
||||
if (closed) JoyCon.clearVrDevices(this)
|
||||
else JoyCon.setVrDevices(this, IntArray(2) { hands[it]?.device?.id ?: PadBridge.NO_CONTROLLER })
|
||||
if (!closed) {
|
||||
val taken = now.filter { it !in before }
|
||||
val released = before.filter { it !in now }
|
||||
if (taken.isNotEmpty() || released.isNotEmpty()) main.post { onDevicesChanged(taken, released) }
|
||||
}
|
||||
if (wanted && waiting && retries < MAX_RETRIES) {
|
||||
retries++
|
||||
worker.postDelayed(retry, RETRY_MS)
|
||||
}
|
||||
}
|
||||
|
||||
private fun hasMotion(device: InputDevice): Boolean =
|
||||
runCatching {
|
||||
val sensors = device.sensorManager
|
||||
sensors.getDefaultSensor(Sensor.TYPE_GYROSCOPE) != null && sensors.getDefaultSensor(Sensor.TYPE_ACCELEROMETER) != null
|
||||
}.getOrDefault(false)
|
||||
|
||||
private fun start(side: Int, device: InputDevice): Hand? {
|
||||
val out = shm ?: JoyConShm.open(path)?.also { shm = it } ?: return null
|
||||
val sensors = runCatching { device.sensorManager }.getOrNull() ?: return null
|
||||
val gyro = runCatching { sensors.getDefaultSensor(Sensor.TYPE_GYROSCOPE) }.getOrNull() ?: return null
|
||||
val accel = runCatching { sensors.getDefaultSensor(Sensor.TYPE_ACCELEROMETER) }.getOrNull() ?: return null
|
||||
val hand = Hand(side, device, sensors, out)
|
||||
val listening = runCatching {
|
||||
sensors.registerListener(hand, accel, SensorManager.SENSOR_DELAY_FASTEST, worker) &&
|
||||
sensors.registerListener(hand, gyro, SensorManager.SENSOR_DELAY_FASTEST, worker)
|
||||
}.onFailure { Log.w(TAG, "Joy-Con motion unavailable for \"${device.name}\"", it) }.getOrDefault(false)
|
||||
if (!listening) {
|
||||
runCatching { sensors.unregisterListener(hand) }
|
||||
return null
|
||||
}
|
||||
Log.i(TAG, "${SIDES[side]} Joy-Con \"${device.name}\" (id ${device.id}) is a VR controller")
|
||||
return hand
|
||||
}
|
||||
|
||||
private inner class Hand(
|
||||
val side: Int,
|
||||
val device: InputDevice,
|
||||
private val sensors: SensorManager,
|
||||
private val out: JoyConShm,
|
||||
) : SensorEventListener {
|
||||
private val filter = JoyConOrientation()
|
||||
private val motion = JoyConMotion()
|
||||
private val calibration = JoyConAccelCalibration().apply { restore(JoyConCalibrations.load(context, device.descriptor)) }
|
||||
private val reports = JoyConReports()
|
||||
private val accel = FloatArray(3)
|
||||
private var accelStamp = 0L
|
||||
private var haveAccel = false
|
||||
private val pressed = HashSet<Int>()
|
||||
private var stickX = 0f
|
||||
private var stickY = 0f
|
||||
private var systemDown = false
|
||||
private var systemTap = false
|
||||
private var held = false
|
||||
private var recenter = 0
|
||||
private var battery = -1f
|
||||
private var charging = false
|
||||
private var batteryAt = 0L
|
||||
private var sampleAt = 0L
|
||||
private var live = true
|
||||
private val hold = Runnable {
|
||||
if (!systemDown || held) return@Runnable
|
||||
held = true
|
||||
recenter++
|
||||
confirm()
|
||||
publish()
|
||||
}
|
||||
private val tapEnd = Runnable {
|
||||
systemTap = false
|
||||
publish()
|
||||
}
|
||||
private val flush = Runnable { flushReport() }
|
||||
|
||||
override fun onSensorChanged(event: SensorEvent) {
|
||||
if (!live) return
|
||||
when (event.sensor.type) {
|
||||
Sensor.TYPE_ACCELEROMETER -> {
|
||||
if (event.timestamp == accelStamp) return
|
||||
accelStamp = event.timestamp
|
||||
event.values.copyInto(accel, 0, 0, 3)
|
||||
haveAccel = true
|
||||
}
|
||||
Sensor.TYPE_GYROSCOPE -> {
|
||||
if (!haveAccel) return
|
||||
reports.add(accel, event.values, System.nanoTime())
|
||||
worker.removeCallbacks(flush)
|
||||
if (reports.size >= JoyConReports.SAMPLES_PER_REPORT) flushReport() else worker.postDelayed(flush, FLUSH_MS)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private fun flushReport() {
|
||||
val samples = reports.flush()
|
||||
if (samples.isEmpty() || !live) return
|
||||
for (sample in samples) {
|
||||
val raw = JoyConOrientation.axes(sample.accel)
|
||||
val corrected = calibration.correct(raw)
|
||||
filter.update(corrected, JoyConOrientation.axes(sample.gyro), sample.dt)
|
||||
if (!filter.ready) continue
|
||||
motion.update(filter.rotation, corrected, filter.rate, calibration.ready, sample.dt)
|
||||
if (motion.resting && calibration.addResting(raw)) JoyConCalibrations.save(context, device.descriptor, calibration.saved())
|
||||
}
|
||||
val now = System.nanoTime()
|
||||
sampleAt = now
|
||||
if (now - batteryAt > BATTERY_NS) refreshBattery(now)
|
||||
publish()
|
||||
}
|
||||
|
||||
override fun onAccuracyChanged(sensor: Sensor?, accuracy: Int) = Unit
|
||||
|
||||
fun key(scanCode: Int, down: Boolean) {
|
||||
if (!live) return
|
||||
val bit = JoyCon.vrButton(side, scanCode)
|
||||
if (bit == 0) return
|
||||
if (bit == JoyCon.VR_SYSTEM) {
|
||||
system(down)
|
||||
return
|
||||
}
|
||||
if (down) pressed += scanCode else pressed -= scanCode
|
||||
publish()
|
||||
}
|
||||
|
||||
fun stick(x: Float, y: Float) {
|
||||
if (!live) return
|
||||
val length = sqrt(x * x + y * y)
|
||||
if (length < STICK_DEADZONE) {
|
||||
stickX = 0f
|
||||
stickY = 0f
|
||||
} else {
|
||||
val scale = min((length - STICK_DEADZONE) / (1f - STICK_DEADZONE), 1f) / length
|
||||
stickX = x * scale
|
||||
stickY = y * scale
|
||||
}
|
||||
publish()
|
||||
}
|
||||
|
||||
fun release() {
|
||||
if (!live) return
|
||||
pressed.clear()
|
||||
stickX = 0f
|
||||
stickY = 0f
|
||||
systemDown = false
|
||||
systemTap = false
|
||||
held = false
|
||||
worker.removeCallbacks(hold)
|
||||
worker.removeCallbacks(tapEnd)
|
||||
publish()
|
||||
}
|
||||
|
||||
fun stop() {
|
||||
live = false
|
||||
runCatching { sensors.unregisterListener(this) }
|
||||
worker.removeCallbacks(hold)
|
||||
worker.removeCallbacks(tapEnd)
|
||||
worker.removeCallbacks(flush)
|
||||
out.write(side, JoyConShm.Hand(connected = false))
|
||||
Log.i(TAG, "${SIDES[side]} Joy-Con \"${device.name}\" (id ${device.id}) is a game pad again")
|
||||
}
|
||||
|
||||
private fun system(down: Boolean) {
|
||||
if (down == systemDown) return
|
||||
systemDown = down
|
||||
if (down) {
|
||||
held = false
|
||||
worker.postDelayed(hold, HOLD_MS)
|
||||
return
|
||||
}
|
||||
worker.removeCallbacks(hold)
|
||||
if (held) return
|
||||
systemTap = true
|
||||
publish()
|
||||
worker.removeCallbacks(tapEnd)
|
||||
worker.postDelayed(tapEnd, TAP_MS)
|
||||
}
|
||||
|
||||
private fun buttons(): Int {
|
||||
var bits = if (systemTap) JoyCon.VR_SYSTEM else 0
|
||||
for (scanCode in pressed) bits = bits or JoyCon.vrButton(side, scanCode)
|
||||
return bits
|
||||
}
|
||||
|
||||
private fun publish() {
|
||||
out.write(side, JoyConShm.Hand(
|
||||
connected = filter.ready,
|
||||
charging = charging,
|
||||
buttons = buttons(),
|
||||
rotation = filter.rotation,
|
||||
rate = filter.rate,
|
||||
displacement = motion.displacement,
|
||||
velocity = motion.velocity,
|
||||
inertial = motion.inertial,
|
||||
stickX = stickX,
|
||||
stickY = stickY,
|
||||
battery = battery,
|
||||
recenter = recenter,
|
||||
timeNs = sampleAt,
|
||||
))
|
||||
}
|
||||
|
||||
private fun refreshBattery(now: Long) {
|
||||
batteryAt = now
|
||||
val state = runCatching { device.batteryState }.getOrNull() ?: return
|
||||
battery = if (state.isPresent && !state.capacity.isNaN()) state.capacity else -1f
|
||||
charging = state.isPresent && state.status == BatteryState.STATUS_CHARGING
|
||||
}
|
||||
|
||||
private fun confirm() {
|
||||
if (!ControllerPrefs.rumbleEnabled(context)) return
|
||||
runCatching {
|
||||
device.vibratorManager.vibrate(CombinedVibration.createParallel(
|
||||
VibrationEffect.createOneShot(CONFIRM_MS, VibrationEffect.DEFAULT_AMPLITUDE)))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
companion object {
|
||||
const val FILE = JoyConShm.FILE
|
||||
private const val TAG = "JoyConVr"
|
||||
private val SIDES = arrayOf("left", "right")
|
||||
private const val MAX_RETRIES = 6
|
||||
private const val RETRY_MS = 500L
|
||||
private const val HOLD_MS = 800L
|
||||
private const val TAP_MS = 120L
|
||||
private const val CONFIRM_MS = 80L
|
||||
private const val BATTERY_NS = 30_000_000_000L
|
||||
private const val STICK_DEADZONE = 0.12f
|
||||
private const val FLUSH_MS = 3L
|
||||
}
|
||||
}
|
||||
@@ -92,6 +92,7 @@ class OnScreenControls(
|
||||
private val groups = controls.map { it.group }.distinct()
|
||||
|
||||
private var buttonsOnly = false
|
||||
val isButtonsOnly: Boolean get() = buttonsOnly
|
||||
private var quickHidden = false
|
||||
private var quickPressedBy = -1
|
||||
private var keyboardPressedBy = -1
|
||||
|
||||
@@ -46,6 +46,8 @@ public final class PadBridge {
|
||||
private final FakeInputWriter writer;
|
||||
private final PadState state = new PadState();
|
||||
private final int[] owners = new int[SLOTS];
|
||||
private final int[] partners = new int[SLOTS];
|
||||
private volatile java.util.function.IntPredicate excluded = id -> false;
|
||||
private final FakeInputWriter[] playerWriters = new FakeInputWriter[SLOTS];
|
||||
private final PadState[] playerStates = new PadState[SLOTS];
|
||||
private final PadState effectiveState = new PadState();
|
||||
@@ -72,8 +74,10 @@ public final class PadBridge {
|
||||
public static final int NO_CONTROLLER = -1;
|
||||
private static volatile int activeControllerId = NO_CONTROLLER;
|
||||
private static final AtomicIntegerArray slotControllers = new AtomicIntegerArray(SLOTS);
|
||||
private static final AtomicIntegerArray slotJoyCons = new AtomicIntegerArray(SLOTS * 2);
|
||||
static {
|
||||
for (int slot = 0; slot < SLOTS; slot++) slotControllers.set(slot, NO_CONTROLLER);
|
||||
for (int i = 0; i < SLOTS * 2; i++) slotJoyCons.set(i, NO_CONTROLLER);
|
||||
}
|
||||
|
||||
public static int activeControllerId() { return activeControllerId; }
|
||||
@@ -82,10 +86,16 @@ public final class PadBridge {
|
||||
return slot >= 0 && slot < SLOTS ? slotControllers.get(slot) : activeControllerId;
|
||||
}
|
||||
|
||||
public static int joyConForSlot(int slot, int side) {
|
||||
return slot >= 0 && slot < SLOTS && (side == JoyCon.LEFT || side == JoyCon.RIGHT)
|
||||
? slotJoyCons.get(slot * 2 + side) : NO_CONTROLLER;
|
||||
}
|
||||
|
||||
public PadBridge(File fakeInputDir) {
|
||||
this.fakeInputDir = fakeInputDir;
|
||||
writer = new FakeInputWriter(fakeInputDir.getAbsolutePath(), 0);
|
||||
Arrays.fill(owners, NO_CONTROLLER);
|
||||
Arrays.fill(partners, NO_CONTROLLER);
|
||||
for (int slot = 1; slot < SLOTS; slot++) playerStates[slot] = new PadState();
|
||||
}
|
||||
|
||||
@@ -93,6 +103,10 @@ public final class PadBridge {
|
||||
selectSteam = on;
|
||||
}
|
||||
|
||||
public void setExcluded(java.util.function.IntPredicate devices) {
|
||||
excluded = devices;
|
||||
}
|
||||
|
||||
public synchronized boolean start() {
|
||||
if (!open) {
|
||||
open = writer.open();
|
||||
@@ -106,7 +120,9 @@ public final class PadBridge {
|
||||
activeControllerId = NO_CONTROLLER;
|
||||
for (int slot = 0; slot < SLOTS; slot++) {
|
||||
owners[slot] = NO_CONTROLLER;
|
||||
partners[slot] = NO_CONTROLLER;
|
||||
slotControllers.set(slot, NO_CONTROLLER);
|
||||
clearJoyCons(slot);
|
||||
if (playerWriters[slot] != null) {
|
||||
playerWriters[slot].destroy();
|
||||
playerWriters[slot] = null;
|
||||
@@ -206,7 +222,7 @@ public final class PadBridge {
|
||||
public synchronized void syncConnectedControllers() {
|
||||
for (int deviceId : InputDevice.getDeviceIds()) {
|
||||
InputDevice device = InputDevice.getDevice(deviceId);
|
||||
if (isFromController(device)) publish(slotFor(device));
|
||||
if (isFromController(device) && !excluded.test(deviceId)) publish(slotFor(device));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -224,8 +240,10 @@ public final class PadBridge {
|
||||
}
|
||||
|
||||
public synchronized boolean onKeyEvent(KeyEvent event) {
|
||||
if (!isControllerKeyEvent(event)) return false;
|
||||
if (!isControllerKeyEvent(event) || excluded.test(event.getDeviceId())) return false;
|
||||
int slot = slotFor(event.getDevice());
|
||||
int side = JoyCon.side(event.getDevice());
|
||||
if (side >= 0 && partners[slot] != NO_CONTROLLER && JoyCon.isRail(side, event.getScanCode())) return true;
|
||||
PadState pad = stateFor(slot);
|
||||
boolean pressed = event.getAction() == KeyEvent.ACTION_DOWN;
|
||||
switch (event.getKeyCode()) {
|
||||
@@ -269,16 +287,32 @@ public final class PadBridge {
|
||||
}
|
||||
|
||||
public synchronized boolean onMotionEvent(MotionEvent event) {
|
||||
if (!isControllerMotionEvent(event)) return false;
|
||||
if (!isControllerMotionEvent(event) || excluded.test(event.getDeviceId())) return false;
|
||||
if (event.getAction() != MotionEvent.ACTION_MOVE) return false;
|
||||
int slot = slotFor(event.getDevice());
|
||||
PadState pad = stateFor(slot);
|
||||
noteDevice(event.getDevice(), slot);
|
||||
statAxes++;
|
||||
scheduleStats();
|
||||
InputDevice device = event.getDevice();
|
||||
int side = JoyCon.side(device);
|
||||
if (side >= 0) {
|
||||
int[] stick = JoyCon.stickAxes(side, device, event.getSource());
|
||||
float x = axis(event, stick[0]);
|
||||
float y = axis(event, stick[1]);
|
||||
if (side == JoyCon.LEFT) {
|
||||
pad.leftX = x;
|
||||
pad.leftY = y;
|
||||
} else {
|
||||
pad.rightX = x;
|
||||
pad.rightY = y;
|
||||
}
|
||||
if (Math.max(Math.abs(x), Math.abs(y)) > 0.5f) notePlayerInput();
|
||||
publish(slot);
|
||||
return true;
|
||||
}
|
||||
pad.leftX = axis(event, MotionEvent.AXIS_X);
|
||||
pad.leftY = axis(event, MotionEvent.AXIS_Y);
|
||||
InputDevice device = event.getDevice();
|
||||
boolean wiredXboxLayout = device != null
|
||||
&& ((device.getVendorId() == 0x20d6 && device.getProductId() == 0x2063)
|
||||
|| (device.getVendorId() == 0x0e6f && device.getProductId() == 0x02f0))
|
||||
@@ -358,7 +392,7 @@ public final class PadBridge {
|
||||
|
||||
public synchronized void onDeviceRemoved(int deviceId) {
|
||||
for (int slot = 0; slot < SLOTS; slot++) {
|
||||
if (owners[slot] == deviceId) free(slot);
|
||||
if (owners[slot] == deviceId || partners[slot] == deviceId) leave(slot, deviceId);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -425,12 +459,26 @@ public final class PadBridge {
|
||||
|
||||
private int slotFor(InputDevice device) {
|
||||
int id = device.getId();
|
||||
if (owners[0] == id) return 0;
|
||||
if (owners[0] == id || partners[0] == id) return 0;
|
||||
for (int slot = 1; slot < SLOTS; slot++) {
|
||||
if (owners[slot] == id) return slot;
|
||||
if (owners[slot] == id || partners[slot] == id) return slot;
|
||||
}
|
||||
for (int slot = 0; slot < SLOTS; slot++) {
|
||||
if (owners[slot] != NO_CONTROLLER && InputDevice.getDevice(owners[slot]) == null) free(slot);
|
||||
if (partners[slot] != NO_CONTROLLER && InputDevice.getDevice(partners[slot]) == null) leave(slot, partners[slot]);
|
||||
if (owners[slot] != NO_CONTROLLER && InputDevice.getDevice(owners[slot]) == null) leave(slot, owners[slot]);
|
||||
}
|
||||
int side = JoyCon.side(device);
|
||||
for (int slot = 0; side >= 0 && slot < SLOTS; slot++) {
|
||||
if (owners[slot] == NO_CONTROLLER || partners[slot] != NO_CONTROLLER) continue;
|
||||
InputDevice owner = InputDevice.getDevice(owners[slot]);
|
||||
if (JoyCon.side(owner) != 1 - side) continue;
|
||||
partners[slot] = id;
|
||||
slotJoyCons.set(slot * 2 + side, id);
|
||||
slotJoyCons.set(slot * 2 + 1 - side, owner.getId());
|
||||
if (slot == 0) releaseBackPaddles();
|
||||
stateFor(slot).clear();
|
||||
Log.i(TAG, "\"" + device.getName() + "\" (id " + id + ") joins \"" + owner.getName() + "\" as player " + (slot + 1));
|
||||
return slot;
|
||||
}
|
||||
for (int slot = 0; slot < SLOTS; slot++) {
|
||||
if (owners[slot] != NO_CONTROLLER) continue;
|
||||
@@ -446,7 +494,8 @@ public final class PadBridge {
|
||||
}
|
||||
|
||||
private boolean isPrimary(KeyEvent event) {
|
||||
return owners[0] == event.getDevice().getId();
|
||||
int id = event.getDevice().getId();
|
||||
return owners[0] == id || partners[0] == id;
|
||||
}
|
||||
|
||||
private void releaseBackPaddles() {
|
||||
@@ -454,9 +503,32 @@ public final class PadBridge {
|
||||
DeckControls.INSTANCE.setGrip(DeckControls.R4, false);
|
||||
}
|
||||
|
||||
private void leave(int slot, int id) {
|
||||
if (partners[slot] == NO_CONTROLLER) {
|
||||
free(slot);
|
||||
return;
|
||||
}
|
||||
Log.i(TAG, "player " + (slot + 1) + " Joy-Con (id " + id + ") left");
|
||||
if (owners[slot] == id) owners[slot] = partners[slot];
|
||||
partners[slot] = NO_CONTROLLER;
|
||||
clearJoyCons(slot);
|
||||
if (slotControllers.get(slot) == id) slotControllers.set(slot, owners[slot]);
|
||||
if (activeControllerId == id) activeControllerId = owners[slot];
|
||||
if (slot == 0) releaseBackPaddles();
|
||||
stateFor(slot).clear();
|
||||
publish(slot);
|
||||
}
|
||||
|
||||
private static void clearJoyCons(int slot) {
|
||||
slotJoyCons.set(slot * 2, NO_CONTROLLER);
|
||||
slotJoyCons.set(slot * 2 + 1, NO_CONTROLLER);
|
||||
}
|
||||
|
||||
private void free(int slot) {
|
||||
Log.i(TAG, "player " + (slot + 1) + " (id " + owners[slot] + ") left");
|
||||
owners[slot] = NO_CONTROLLER;
|
||||
partners[slot] = NO_CONTROLLER;
|
||||
clearJoyCons(slot);
|
||||
slotControllers.set(slot, NO_CONTROLLER);
|
||||
if (slot == 0) {
|
||||
releaseBackPaddles();
|
||||
|
||||
@@ -7,26 +7,21 @@ import android.content.Context
|
||||
import android.os.Build
|
||||
import android.provider.Settings
|
||||
import android.util.Log
|
||||
import android.view.Display
|
||||
|
||||
object PerfMode {
|
||||
private const val TAG = "PerfMode"
|
||||
|
||||
fun apply(a: Activity, fpsLimit: Int = 0): String {
|
||||
fun apply(a: Activity, fpsLimit: Int = 0, refreshLimit: Int = 0): String {
|
||||
val parts = ArrayList<String>(2)
|
||||
|
||||
parts += try {
|
||||
val display = if (Build.VERSION.SDK_INT >= 30) a.display else @Suppress("DEPRECATION") a.windowManager.defaultDisplay
|
||||
if (display == null || Build.VERSION.SDK_INT < 23) "display mode unchanged" else {
|
||||
val cur = display.mode
|
||||
val cap = peakRefreshCap(a)
|
||||
val sameSize = display.supportedModes
|
||||
.filter { it.physicalWidth == cur.physicalWidth && it.physicalHeight == cur.physicalHeight }
|
||||
.filter { it.refreshRate <= cap + 0.5f }
|
||||
.ifEmpty { display.supportedModes.filter { it.modeId == cur.modeId } }
|
||||
val best = sameSize.filter { cadenceFits(it.refreshRate, fpsLimit) }.maxByOrNull { it.refreshRate }
|
||||
?: sameSize.maxByOrNull { it.refreshRate }
|
||||
if (best != null && best.modeId != cur.modeId &&
|
||||
(best.refreshRate > cur.refreshRate + 0.5f || !cadenceFits(cur.refreshRate, fpsLimit))) {
|
||||
val cap = minOf(peakRefreshCap(a), refreshLimit.takeIf { it > 0 }?.toFloat() ?: Float.POSITIVE_INFINITY)
|
||||
val best = selectMode(display.supportedModes, cur, fpsLimit, cap)
|
||||
if (best != null && a.window.attributes.preferredDisplayModeId != best.modeId) {
|
||||
a.window.attributes = a.window.attributes.apply { preferredDisplayModeId = best.modeId }
|
||||
"display ${cur.refreshRate.toInt()} → ${best.refreshRate.toInt()} Hz (mode ${best.modeId})"
|
||||
} else "display ${cur.refreshRate.toInt()} Hz"
|
||||
@@ -59,6 +54,16 @@ object PerfMode {
|
||||
return if (peak >= 1f) peak else Float.POSITIVE_INFINITY
|
||||
}
|
||||
|
||||
internal fun selectMode(modes: Array<Display.Mode>, current: Display.Mode, fpsLimit: Int, cap: Float): Display.Mode? {
|
||||
val sameSize = modes.filter {
|
||||
it.physicalWidth == current.physicalWidth && it.physicalHeight == current.physicalHeight
|
||||
}
|
||||
val allowed = sameSize.filter { it.refreshRate <= cap + 0.5f }
|
||||
return allowed.filter { cadenceFits(it.refreshRate, fpsLimit) }.maxByOrNull { it.refreshRate }
|
||||
?: allowed.maxByOrNull { it.refreshRate }
|
||||
?: sameSize.minByOrNull { it.refreshRate }
|
||||
}
|
||||
|
||||
fun cadenceFits(hz: Float, fpsLimit: Int): Boolean {
|
||||
if (fpsLimit <= 0) return true
|
||||
val ratio = hz / fpsLimit
|
||||
|
||||
@@ -16,6 +16,7 @@ import android.view.InputDevice
|
||||
import androidx.annotation.RequiresApi
|
||||
import com.droiddeck.launcher.core.SessionPart
|
||||
import com.droiddeck.launcher.input.ControllerPrefs
|
||||
import com.droiddeck.launcher.input.JoyCon
|
||||
import com.droiddeck.launcher.input.PadBridge
|
||||
import java.io.DataInputStream
|
||||
import kotlin.math.max
|
||||
@@ -26,6 +27,7 @@ class RumbleComponent : SessionPart() {
|
||||
private val playing = HashMap<Int, Motors>()
|
||||
private val effects = HashMap<Int, Effect>()
|
||||
private val controllers = HashMap<Int, Motors?>()
|
||||
private val joyConPairs = HashMap<Long, Motors>()
|
||||
private var preferences: SharedPreferences? = null
|
||||
private var enabled = false
|
||||
private var motorThread: HandlerThread? = null
|
||||
@@ -78,6 +80,7 @@ class RumbleComponent : SessionPart() {
|
||||
motorThread = null
|
||||
phone = null
|
||||
controllers.clear()
|
||||
joyConPairs.clear()
|
||||
val s = server
|
||||
server = null
|
||||
try { s?.close() } catch (e: Exception) { }
|
||||
@@ -125,10 +128,24 @@ class RumbleComponent : SessionPart() {
|
||||
}
|
||||
|
||||
private fun target(slot: Int): Motors? {
|
||||
JoyCon.vrRumbleDevice(slot)?.let { hand -> return if (hand == PadBridge.NO_CONTROLLER) null else motors(hand) }
|
||||
val left = PadBridge.joyConForSlot(slot, JoyCon.LEFT)
|
||||
val right = PadBridge.joyConForSlot(slot, JoyCon.RIGHT)
|
||||
if (left != PadBridge.NO_CONTROLLER && right != PadBridge.NO_CONTROLLER) {
|
||||
val key = (left.toLong() shl 32) or (right.toLong() and 0xFFFFFFFFL)
|
||||
joyConPairs[key]?.let { return it }
|
||||
val l = motors(left)
|
||||
val r = motors(right)
|
||||
if (l != null && r != null) return JoyConPairMotors(l, r).also { joyConPairs[key] = it }
|
||||
}
|
||||
val id = PadBridge.controllerForSlot(slot)
|
||||
if (id == PadBridge.NO_CONTROLLER) return phone
|
||||
return motors(id) ?: phone
|
||||
}
|
||||
|
||||
private fun motors(id: Int): Motors? {
|
||||
if (!controllers.containsKey(id)) controllers[id] = try { controllerMotors(id) } catch (e: Exception) { null }
|
||||
return controllers[id] ?: phone
|
||||
return controllers[id]
|
||||
}
|
||||
|
||||
private fun cancelAll() {
|
||||
@@ -201,6 +218,25 @@ class RumbleComponent : SessionPart() {
|
||||
}
|
||||
}
|
||||
|
||||
@RequiresApi(31)
|
||||
private class JoyConMotors(override val name: String, private val manager: VibratorManager) : Motors {
|
||||
override fun play(strong: Int, weak: Int, ms: Long) =
|
||||
manager.vibrate(CombinedVibration.createParallel(oneShot(manager.defaultVibrator, max(strong, weak), ms)))
|
||||
override fun cancel() = manager.cancel()
|
||||
}
|
||||
|
||||
private class JoyConPairMotors(private val left: Motors, private val right: Motors) : Motors {
|
||||
override val name = "${left.name} and ${right.name}"
|
||||
override fun play(strong: Int, weak: Int, ms: Long) {
|
||||
if (strong == 0) left.cancel() else left.play(strong, 0, ms)
|
||||
if (weak == 0) right.cancel() else right.play(0, weak, ms)
|
||||
}
|
||||
override fun cancel() {
|
||||
left.cancel()
|
||||
right.cancel()
|
||||
}
|
||||
}
|
||||
|
||||
private class LegacyPadMotors(override val name: String, private val vibrator: Vibrator) : Motors {
|
||||
override fun play(strong: Int, weak: Int, ms: Long) = vibrator.vibrate(oneShot(vibrator, max(strong, weak), ms))
|
||||
override fun cancel() {
|
||||
@@ -230,7 +266,8 @@ class RumbleComponent : SessionPart() {
|
||||
if (Build.VERSION.SDK_INT >= 31) {
|
||||
val manager = device.vibratorManager
|
||||
val ids = manager.vibratorIds.sortedArray()
|
||||
return if (ids.isEmpty()) null else PadMotors(name, manager, ids)
|
||||
if (ids.isEmpty()) return null
|
||||
return if (JoyCon.side(device) >= 0) JoyConMotors(name, manager) else PadMotors(name, manager, ids)
|
||||
}
|
||||
@Suppress("DEPRECATION")
|
||||
val vibrator = device.vibrator
|
||||
|
||||
@@ -31,6 +31,7 @@ object SessionEvents {
|
||||
SessionState.programArgs = emptyList()
|
||||
SessionState.steamUi = null
|
||||
SessionState.steamUrl = null
|
||||
SessionState.steamVr = false
|
||||
SessionState.firstFrameSeen = false
|
||||
SessionState.suspended = false
|
||||
}
|
||||
|
||||
@@ -104,14 +104,19 @@ object SessionFiles {
|
||||
val msitools = listOf("msiinfo", "cabextract", "7z", "7z.so", "libmsi-1.0.so.0", "libgsf-1.so.114", "libgcab-1.0.so.0")
|
||||
.map { "usr/local/lib/droiddeck-msitools/$it" } +
|
||||
listOf("NOTICE", "GPL-2", "GPL-3", "LGPL-2.1", "7zip-License", "7zip-unRarLicense").map { "usr/local/share/licenses/droiddeck-msitools/$it" }
|
||||
val vr = listOf(
|
||||
val handModels = listOf("left", "right").flatMap { side ->
|
||||
listOf("droiddeck_hand_$side.json", "hand.mtl", "hand.png", "hand_open.obj", "hand_fist.obj", "thumb_index_open.obj",
|
||||
"thumb_index_pinch.obj")
|
||||
.map { "droiddeck/resources/rendermodels/droiddeck_hand_$side/$it" }
|
||||
}
|
||||
val vr = (listOf(
|
||||
"droiddeck/bin/linuxarm64/driver_droiddeck.so",
|
||||
"droiddeck/bin/linuxarm64/xr-display-mode",
|
||||
"droiddeck/driver.vrdrivermanifest",
|
||||
"droiddeck/resources/settings/default.vrsettings",
|
||||
"layer/libVkLayer_droiddeck_vr.so",
|
||||
"layer/VK_LAYER_DROIDDECK_vr.json",
|
||||
).map { "usr/local/lib/droiddeck-vr/$it" to "usr/local/lib/droiddeck-vr/$it" } +
|
||||
) + handModels).map { "usr/local/lib/droiddeck-vr/$it" to "usr/local/lib/droiddeck-vr/$it" } +
|
||||
listOf("usr/local/lib/droiddeck-vr/layer/VK_LAYER_DROIDDECK_vr_implicit.json" to
|
||||
"usr/share/vulkan/implicit_layer.d/droiddeck_vr.json")
|
||||
val fexPreloads = listOf("x86_64", "i386").flatMap { arch ->
|
||||
|
||||
@@ -1299,6 +1299,7 @@ class SessionService : Service() {
|
||||
val failureStatus = SessionState.failureStatus ?: status
|
||||
SessionEvents.fail(code, message, failureStatus)
|
||||
}
|
||||
com.droiddeck.launcher.glasses.Glasses.set3dForVr(false)
|
||||
SessionState.notifyEnded(status)
|
||||
stopForeground(STOP_FOREGROUND_REMOVE)
|
||||
stopSelf()
|
||||
|
||||
@@ -53,6 +53,9 @@ object SessionState {
|
||||
var steamUi: String? = null
|
||||
@Volatile
|
||||
var steamUrl: String? = null
|
||||
// A Play SteamVR request can arrive while an existing session is visible.
|
||||
// Wake the status/launch effect even if no glasses surface has attached yet.
|
||||
var steamVr by mutableStateOf(false)
|
||||
var relaunch: android.content.Intent? = null
|
||||
@JvmStatic var hdr = false
|
||||
|
||||
|
||||
@@ -29,6 +29,7 @@ class ControllerActions(
|
||||
val onQamButton: (Boolean) -> Unit,
|
||||
val onKeyboardButton: (Boolean) -> Unit,
|
||||
val onSelectSteam: (Boolean) -> Unit,
|
||||
val onJoyConVr: (Boolean) -> Unit,
|
||||
val onEditLayout: () -> Unit,
|
||||
val onResetLayout: () -> Unit,
|
||||
val onMapping: () -> Unit,
|
||||
@@ -72,6 +73,7 @@ fun ColumnScope.ControllerRows(host: MenuHost, oscMode: String, c: ControllerPre
|
||||
ToggleRow(host, "controller-qam", stringResource(R.string.ctrl_qam_button), null, c.qamButton, onChange = a.onQamButton)
|
||||
ToggleRow(host, "controller-keyboard", stringResource(R.string.ctrl_keyboard_button), null, c.keyboardButton, onChange = a.onKeyboardButton)
|
||||
ToggleRow(host, "controller-select-steam", stringResource(R.string.ctrl_select_steam), stringResource(R.string.ctrl_select_steam_hint), c.selectSteam, onChange = a.onSelectSteam)
|
||||
ToggleRow(host, "controller-joycon-vr", stringResource(R.string.ctrl_joycon_vr), stringResource(R.string.ctrl_joycon_vr_hint), c.joyConVr, onChange = a.onJoyConVr)
|
||||
SettingsRow(stringResource(R.string.ctrl_layout), if (c.customLayout) stringResource(R.string.ctrl_layout_custom) else stringResource(R.string.ctrl_layout_default)) {
|
||||
Row(horizontalArrangement = Arrangement.spacedBy(6.dp)) {
|
||||
SecondaryButton(stringResource(R.string.ctrl_edit)) { a.onEditLayout() }
|
||||
|
||||
@@ -0,0 +1,484 @@
|
||||
package com.droiddeck.launcher.ui
|
||||
|
||||
import android.view.KeyEvent
|
||||
import androidx.activity.compose.BackHandler
|
||||
import androidx.compose.animation.AnimatedContent
|
||||
import androidx.compose.animation.core.Animatable
|
||||
import androidx.compose.animation.fadeIn
|
||||
import androidx.compose.animation.fadeOut
|
||||
import androidx.compose.animation.slideInHorizontally
|
||||
import androidx.compose.animation.slideOutHorizontally
|
||||
import androidx.compose.animation.togetherWith
|
||||
import androidx.compose.foundation.Canvas
|
||||
import androidx.compose.foundation.LocalIndication
|
||||
import androidx.compose.foundation.background
|
||||
import androidx.compose.foundation.clickable
|
||||
import androidx.compose.foundation.focusGroup
|
||||
import androidx.compose.foundation.gestures.detectTapGestures
|
||||
import androidx.compose.foundation.hoverable
|
||||
import androidx.compose.foundation.interaction.MutableInteractionSource
|
||||
import androidx.compose.foundation.layout.Box
|
||||
import androidx.compose.foundation.layout.BoxWithConstraintsScope
|
||||
import androidx.compose.foundation.layout.Column
|
||||
import androidx.compose.foundation.layout.fillMaxSize
|
||||
import androidx.compose.foundation.layout.fillMaxWidth
|
||||
import androidx.compose.foundation.layout.height
|
||||
import androidx.compose.foundation.layout.heightIn
|
||||
import androidx.compose.foundation.layout.offset
|
||||
import androidx.compose.foundation.layout.padding
|
||||
import androidx.compose.foundation.layout.size
|
||||
import androidx.compose.foundation.layout.width
|
||||
import androidx.compose.foundation.rememberScrollState
|
||||
import androidx.compose.foundation.shape.CircleShape
|
||||
import androidx.compose.foundation.verticalScroll
|
||||
import androidx.compose.material.icons.Icons
|
||||
import androidx.compose.material.icons.filled.Close
|
||||
import androidx.compose.material3.Icon
|
||||
import androidx.compose.material3.MaterialTheme
|
||||
import androidx.compose.runtime.Composable
|
||||
import androidx.compose.runtime.CompositionLocalProvider
|
||||
import androidx.compose.runtime.DisposableEffect
|
||||
import androidx.compose.runtime.LaunchedEffect
|
||||
import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
import androidx.compose.runtime.remember
|
||||
import androidx.compose.runtime.rememberCoroutineScope
|
||||
import androidx.compose.runtime.rememberUpdatedState
|
||||
import androidx.compose.runtime.setValue
|
||||
import androidx.compose.runtime.withFrameNanos
|
||||
import androidx.compose.ui.Alignment
|
||||
import androidx.compose.ui.ExperimentalComposeUiApi
|
||||
import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.draw.clip
|
||||
import androidx.compose.ui.focus.FocusDirection
|
||||
import androidx.compose.ui.focus.FocusRequester
|
||||
import androidx.compose.ui.focus.focusProperties
|
||||
import androidx.compose.ui.focus.focusRequester
|
||||
import androidx.compose.ui.focus.onFocusChanged
|
||||
import androidx.compose.ui.geometry.Offset
|
||||
import androidx.compose.ui.geometry.Rect
|
||||
import androidx.compose.ui.graphics.Color
|
||||
import androidx.compose.ui.graphics.Path
|
||||
import androidx.compose.ui.graphics.drawscope.Stroke
|
||||
import androidx.compose.ui.graphics.graphicsLayer
|
||||
import androidx.compose.ui.graphics.lerp
|
||||
import androidx.compose.ui.input.InputMode
|
||||
import androidx.compose.ui.input.key.onPreviewKeyEvent
|
||||
import androidx.compose.ui.input.nestedscroll.NestedScrollConnection
|
||||
import androidx.compose.ui.input.nestedscroll.NestedScrollSource
|
||||
import androidx.compose.ui.input.nestedscroll.nestedScroll
|
||||
import androidx.compose.ui.input.pointer.pointerInput
|
||||
import androidx.compose.ui.layout.boundsInParent
|
||||
import androidx.compose.ui.layout.onGloballyPositioned
|
||||
import androidx.compose.ui.layout.onSizeChanged
|
||||
import androidx.compose.ui.platform.LocalDensity
|
||||
import androidx.compose.ui.platform.LocalInputModeManager
|
||||
import androidx.compose.ui.platform.testTag
|
||||
import androidx.compose.ui.res.stringResource
|
||||
import androidx.compose.ui.semantics.Role
|
||||
import androidx.compose.ui.semantics.contentDescription
|
||||
import androidx.compose.ui.semantics.semantics
|
||||
import androidx.compose.ui.unit.IntOffset
|
||||
import androidx.compose.ui.unit.Velocity
|
||||
import androidx.compose.ui.unit.dp
|
||||
import androidx.compose.ui.util.lerp
|
||||
import androidx.compose.ui.zIndex
|
||||
import com.droiddeck.launcher.R
|
||||
import kotlinx.coroutines.CoroutineScope
|
||||
import kotlinx.coroutines.Job
|
||||
import kotlinx.coroutines.coroutineScope
|
||||
import kotlinx.coroutines.delay
|
||||
import kotlinx.coroutines.launch
|
||||
import kotlin.math.roundToInt
|
||||
|
||||
private const val DROP_PIECES = 12
|
||||
|
||||
internal class DropSheetState(private val scope: CoroutineScope) {
|
||||
var open by mutableStateOf(false)
|
||||
var shown by mutableStateOf(false)
|
||||
var lipBack by mutableStateOf(false)
|
||||
var tab by mutableStateOf(0)
|
||||
var travel by mutableStateOf(1)
|
||||
var settledByDrag = false
|
||||
var engaged = false
|
||||
private set
|
||||
val pull = Animatable(0f)
|
||||
val lift = Animatable(0f)
|
||||
val ride = Animatable(0f)
|
||||
val sink = Animatable(0f)
|
||||
val hot = Animatable(1f)
|
||||
val lipShow = Animatable(0f)
|
||||
val lipGone = Animatable(0f)
|
||||
val pieces = List(DROP_PIECES) { Animatable(0f) }
|
||||
var travelPx = 1f
|
||||
private var from = 0f
|
||||
var progress = 0f
|
||||
private set
|
||||
private var dragJob: Job? = null
|
||||
|
||||
fun cancelDrag() {
|
||||
dragJob?.cancel()
|
||||
dragJob = null
|
||||
}
|
||||
|
||||
fun reset() {
|
||||
cancelDrag()
|
||||
engaged = false
|
||||
settledByDrag = false
|
||||
open = false
|
||||
shown = false
|
||||
lipBack = false
|
||||
scope.launch {
|
||||
listOf(pull, lift, ride, sink, lipGone).forEach { it.snapTo(0f) }
|
||||
pieces.forEach { it.snapTo(0f) }
|
||||
hot.snapTo(1f)
|
||||
}
|
||||
}
|
||||
|
||||
fun beginDrag() {
|
||||
cancelDrag()
|
||||
settledByDrag = false
|
||||
engaged = true
|
||||
from = if (open) 1f else 0f
|
||||
progress = from
|
||||
if (!open) shown = true
|
||||
}
|
||||
|
||||
fun drag(dy: Float) = follow((progress + dy / travelPx).coerceIn(0f, 1.04f))
|
||||
|
||||
fun follow(p: Float) {
|
||||
progress = p
|
||||
scope.launch {
|
||||
lift.snapTo(p); ride.snapTo(p); sink.snapTo(p.coerceIn(0f, 1f)); hot.snapTo(0f)
|
||||
lipGone.snapTo(1f)
|
||||
pieces.forEach { it.snapTo(p.coerceIn(0f, 1f)) }
|
||||
}
|
||||
}
|
||||
|
||||
fun settle(v: Float, flick: Float) {
|
||||
engaged = false
|
||||
val opens = when {
|
||||
v > flick -> true
|
||||
v < -flick -> false
|
||||
from > 0.5f -> progress > 0.7f
|
||||
else -> progress > 0.3f
|
||||
}
|
||||
settledByDrag = true
|
||||
cancelDrag()
|
||||
dragJob = scope.launch {
|
||||
val to = if (opens) 1f else 0f
|
||||
coroutineScope {
|
||||
launch { lift.animateTo(to, Motion.sp(if (opens) 0.7f else 1f, 380f)) }
|
||||
launch { ride.animateTo(to, Motion.sp(if (opens) 0.7f else 0.55f, if (opens) 380f else 300f)) }
|
||||
launch { sink.animateTo(to, Motion.tw(220)) }
|
||||
pieces.forEachIndexed { i, piece -> launch { if (opens) delay(Motion.ms(i * 35).toLong()); piece.animateTo(to, Motion.tw(if (opens) 200 else 90)) } }
|
||||
}
|
||||
if (!opens) { shown = false; lipGone.snapTo(0f); hot.snapTo(1f); lipBack = false }
|
||||
}
|
||||
if (open != opens) open = opens else settledByDrag = false
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
internal fun rememberDropSheetState(): DropSheetState {
|
||||
val scope = rememberCoroutineScope()
|
||||
return remember { DropSheetState(scope) }
|
||||
}
|
||||
|
||||
@OptIn(ExperimentalComposeUiApi::class, androidx.compose.animation.ExperimentalAnimationApi::class)
|
||||
@Composable
|
||||
internal fun BoxWithConstraintsScope.DropSheet(
|
||||
state: DropSheetState,
|
||||
tabs: List<Pair<String, String>>,
|
||||
covered: () -> Float = { 0f },
|
||||
button: @Composable (enabled: Boolean) -> Unit,
|
||||
rows: @Composable (tab: String) -> Unit,
|
||||
) {
|
||||
val density = LocalDensity.current
|
||||
val colors = MaterialTheme.colorScheme
|
||||
val pal = LocalPalette.current
|
||||
val inputMode = LocalInputModeManager.current
|
||||
val narrow = LocalNarrowPane.current
|
||||
val chipFocus = remember(tabs.size) { List(tabs.size) { FocusRequester() } }
|
||||
val buttonFocus = remember { FocusRequester() }
|
||||
val leftForButton = remember { booleanArrayOf(false) }
|
||||
var rest by remember { mutableStateOf<Rect?>(null) }
|
||||
val openState = rememberUpdatedState(state.open)
|
||||
if (state.tab > tabs.lastIndex) state.tab = 0
|
||||
fun focusButton() { if (inputMode.inputMode == InputMode.Keyboard) runCatching { buttonFocus.requestFocus() } }
|
||||
fun close() {
|
||||
if (!state.open) return
|
||||
state.open = false
|
||||
focusButton()
|
||||
}
|
||||
val lifecycle = androidx.compose.ui.platform.LocalLifecycleOwner.current.lifecycle
|
||||
DisposableEffect(lifecycle) {
|
||||
val obs = androidx.lifecycle.LifecycleEventObserver { _, e ->
|
||||
if (e == androidx.lifecycle.Lifecycle.Event.ON_STOP && (state.open || state.shown || state.engaged)) state.reset()
|
||||
}
|
||||
lifecycle.addObserver(obs)
|
||||
onDispose { lifecycle.removeObserver(obs) }
|
||||
}
|
||||
LaunchedEffect(state.open) {
|
||||
if (state.settledByDrag) {
|
||||
state.settledByDrag = false
|
||||
if (state.open) {
|
||||
if (inputMode.inputMode == InputMode.Keyboard) { withFrameNanos { }; runCatching { chipFocus[state.tab].requestFocus() } }
|
||||
} else focusButton()
|
||||
return@LaunchedEffect
|
||||
}
|
||||
if (state.engaged) return@LaunchedEffect
|
||||
state.cancelDrag()
|
||||
if (state.open) {
|
||||
state.lipBack = false
|
||||
state.shown = true
|
||||
coroutineScope {
|
||||
launch { state.pull.animateTo(24f, Motion.sp(0.7f, 380f)) }
|
||||
launch { state.sink.animateTo(1f, Motion.tw(260)) }
|
||||
launch {
|
||||
delay(Motion.ms(120).toLong())
|
||||
launch { state.lipGone.animateTo(1f, Motion.tw(140)) }
|
||||
launch { state.pull.animateTo(0f, Motion.sp(1f, 500f)) }
|
||||
launch { state.lift.animateTo(1f, Motion.sp(0.7f, 380f)) }
|
||||
launch { state.ride.animateTo(1f, Motion.sp(0.7f, 380f)) }
|
||||
launch { state.hot.snapTo(1f); delay(Motion.ms(100).toLong()); state.hot.animateTo(0f, Motion.tw(200)) }
|
||||
}
|
||||
state.pieces.forEachIndexed { i, p -> launch { delay(Motion.ms(120 + 260 + i * 55).toLong()); p.animateTo(1f, Motion.tw(220)) } }
|
||||
if (inputMode.inputMode == InputMode.Keyboard) launch {
|
||||
delay(Motion.ms(300).toLong())
|
||||
withFrameNanos { }
|
||||
runCatching { chipFocus[state.tab].requestFocus() }
|
||||
}
|
||||
}
|
||||
} else if (state.shown) {
|
||||
coroutineScope {
|
||||
state.pieces.forEach { launch { it.animateTo(0f, Motion.tw(90)) } }
|
||||
launch { delay(Motion.ms(60).toLong()); state.lift.animateTo(0f, Motion.sp(1f, 380f)) }
|
||||
launch { delay(Motion.ms(100).toLong()); state.ride.animateTo(0f, Motion.sp(0.55f, 300f)) }
|
||||
launch { state.sink.animateTo(0f, Motion.tw(300)) }
|
||||
launch { delay(Motion.ms(420).toLong()); state.lipGone.snapTo(0f); state.lipBack = true }
|
||||
}
|
||||
state.shown = false
|
||||
state.lipBack = false
|
||||
state.lipGone.snapTo(0f)
|
||||
state.hot.snapTo(1f)
|
||||
} else {
|
||||
listOf(state.pull, state.sink, state.lift, state.ride, state.lipGone).forEach { it.snapTo(0f) }
|
||||
state.pieces.forEach { it.snapTo(0f) }
|
||||
state.hot.snapTo(1f)
|
||||
}
|
||||
}
|
||||
BackHandler(enabled = state.open) { close() }
|
||||
if (state.ride.isRunning || (state.shown && !state.open)) VeilRing()
|
||||
|
||||
val lipWanted = !state.open && (!state.shown || state.lipBack) && covered() < 0.5f
|
||||
LaunchedEffect(lipWanted) {
|
||||
if (lipWanted) {
|
||||
if (!state.lipBack) delay(Motion.ms(160).toLong())
|
||||
state.lipShow.animateTo(1f, Motion.tw(220))
|
||||
} else {
|
||||
delay(Motion.ms(140).toLong())
|
||||
state.lipShow.animateTo(0f, Motion.tw(220))
|
||||
}
|
||||
}
|
||||
|
||||
val flick = with(density) { 600.dp.toPx() }
|
||||
val rowsPush = remember(state) {
|
||||
object : NestedScrollConnection {
|
||||
override fun onPreScroll(available: Offset, source: NestedScrollSource): Offset {
|
||||
if (!state.engaged || available.y <= 0f) return Offset.Zero
|
||||
val before = state.progress
|
||||
state.follow((before + available.y / state.travelPx).coerceAtMost(1f))
|
||||
return Offset(0f, (state.progress - before) * state.travelPx)
|
||||
}
|
||||
override fun onPostScroll(consumed: Offset, available: Offset, source: NestedScrollSource): Offset {
|
||||
if (available.y >= 0f || source != NestedScrollSource.Drag || !openState.value) return Offset.Zero
|
||||
if (!state.engaged) state.beginDrag()
|
||||
state.follow((state.progress + available.y / state.travelPx).coerceIn(0f, 1f))
|
||||
return available
|
||||
}
|
||||
override suspend fun onPreFling(available: Velocity): Velocity {
|
||||
if (!state.engaged) return Velocity.Zero
|
||||
state.settle(available.y, flick)
|
||||
return available
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
val w = constraints.maxWidth.toFloat()
|
||||
val h = constraints.maxHeight.toFloat()
|
||||
val inset = with(density) { 12.dp.toPx() }
|
||||
val edge = with(density) { 3.dp.toPx() }
|
||||
val buttonH = rest?.height ?: with(density) { 56.dp.toPx() }
|
||||
val openBottom = h - sheetTop(h, buttonH, with(density) { 16.dp.toPx() }, edge)
|
||||
val lower = rest?.let { (openBottom - edge - buttonH / 2f) - it.top } ?: 0f
|
||||
state.travelPx = openBottom.coerceAtLeast(1f)
|
||||
|
||||
if (state.shown || state.open) Box(
|
||||
Modifier.fillMaxSize()
|
||||
.graphicsLayer { alpha = state.sink.value }.background(Color.Black.copy(alpha = 0.42f))
|
||||
.focusProperties { canFocus = false }
|
||||
.clickable(interactionSource = remember { MutableInteractionSource() }, indication = null, enabled = state.open) { close() },
|
||||
)
|
||||
|
||||
if (state.shown) {
|
||||
val bottom = lerp(0f, openBottom, state.lift.value)
|
||||
val sheetLeft = inset
|
||||
val sheetRight = w - inset
|
||||
val corner = with(density) { 18.dp.toPx() }
|
||||
val rim = with(density) { 1.4.dp.toPx() }
|
||||
val path = remember { Path() }
|
||||
val rimPath = remember { Path() }
|
||||
Canvas(Modifier.fillMaxSize()) {
|
||||
val t = -corner
|
||||
rimPath.reset()
|
||||
rimPath.moveTo(sheetLeft, t)
|
||||
rimPath.lineTo(sheetLeft, bottom - corner)
|
||||
rimPath.arcTo(Rect(sheetLeft, bottom - 2 * corner, sheetLeft + 2 * corner, bottom), 180f, -90f, false)
|
||||
rimPath.lineTo(sheetRight - corner, bottom)
|
||||
rimPath.arcTo(Rect(sheetRight - 2 * corner, bottom - 2 * corner, sheetRight, bottom), 90f, -90f, false)
|
||||
rimPath.lineTo(sheetRight, t)
|
||||
path.reset()
|
||||
path.addPath(rimPath)
|
||||
path.close()
|
||||
drawPath(path, lerp(colors.surface, pal.signal, 0.22f * state.hot.value))
|
||||
drawPath(rimPath, pal.signal.copy(alpha = lerp(0.55f, 0.9f, state.hot.value)), style = Stroke(rim))
|
||||
}
|
||||
val pill = rest?.translate(0f, lower)
|
||||
val sheetW = sheetRight - sheetLeft
|
||||
val chipsH = with(density) { 44.dp.toPx() }
|
||||
val closeW = with(density) { 64.dp.toPx() }
|
||||
val chipsBottom = with(density) {
|
||||
if (narrow && pill != null) pill.top - 10.dp.toPx() else openBottom - 14.dp.toPx()
|
||||
}
|
||||
val chipsTop = chipsBottom - chipsH
|
||||
val chipsLeft = with(density) {
|
||||
if (narrow || pill == null) 16.dp.toPx() else pill.right - sheetLeft + 20.dp.toPx()
|
||||
}
|
||||
val chipsRight = sheetW - closeW
|
||||
val rowsBottom = chipsTop - with(density) { 8.dp.toPx() }
|
||||
Box(
|
||||
Modifier
|
||||
.offset { IntOffset(sheetLeft.roundToInt(), (bottom - openBottom).roundToInt()) }
|
||||
.size(with(density) { sheetW.toDp() }, with(density) { openBottom.toDp() })
|
||||
.pointerInput(Unit) { detectTapGestures { } }
|
||||
.controllerBack { close() }
|
||||
.bumpers(
|
||||
onPrevious = { if (state.tab > 0) { state.travel = -1; state.tab--; runCatching { chipFocus[state.tab].requestFocus() } } },
|
||||
onNext = { if (state.tab < tabs.lastIndex) { state.travel = 1; state.tab++; runCatching { chipFocus[state.tab].requestFocus() } } },
|
||||
)
|
||||
.focusProperties {
|
||||
exit = { dir ->
|
||||
if (!openState.value) FocusRequester.Default
|
||||
else if (dir == FocusDirection.Left) { leftForButton[0] = true; buttonFocus }
|
||||
else FocusRequester.Cancel
|
||||
}
|
||||
}
|
||||
.focusGroup(),
|
||||
) {
|
||||
Box(
|
||||
Modifier
|
||||
.nestedScroll(rowsPush)
|
||||
.fillMaxWidth()
|
||||
.height(with(density) { rowsBottom.coerceAtLeast(0f).toDp() }),
|
||||
) {
|
||||
AnimatedContent(
|
||||
targetState = state.tab.coerceIn(0, tabs.lastIndex.coerceAtLeast(0)),
|
||||
transitionSpec = {
|
||||
val d = state.travel
|
||||
val shift = with(density) { 24.dp.roundToPx() }
|
||||
(slideInHorizontally(Motion.tw(240)) { shift * d } + fadeIn(Motion.tw(240)))
|
||||
.togetherWith(slideOutHorizontally(Motion.tw(240)) { -shift * d } + fadeOut(Motion.tw(240)))
|
||||
},
|
||||
label = "dropTab",
|
||||
) { t ->
|
||||
val leaving = transition.targetState == androidx.compose.animation.EnterExitState.PostExit
|
||||
val leavingNow = rememberUpdatedState(leaving)
|
||||
CompositionLocalProvider(LocalSheetBeat provides remember { SheetBeat(state.pieces) }) {
|
||||
Column(
|
||||
Modifier.fillMaxSize()
|
||||
.focusProperties { enter = { if (leavingNow.value) FocusRequester.Cancel else FocusRequester.Default } }
|
||||
.focusGroup()
|
||||
.verticalScroll(rememberScrollState())
|
||||
.padding(start = 16.dp, end = 16.dp, top = 8.dp, bottom = 8.dp),
|
||||
) { tabs.getOrNull(t)?.let { rows(it.first) } }
|
||||
}
|
||||
}
|
||||
}
|
||||
Box(
|
||||
Modifier
|
||||
.offset { IntOffset(chipsLeft.roundToInt(), chipsTop.roundToInt()) }
|
||||
.zIndex(1f)
|
||||
.width(with(density) { (chipsRight - chipsLeft).coerceAtLeast(0f).toDp() })
|
||||
.graphicsLayer { val v = state.pieces[0].value; alpha = v; translationY = (1f - v) * -6.dp.toPx() }
|
||||
.onPreviewKeyEvent { e ->
|
||||
val k = e.nativeKeyEvent
|
||||
if (k.keyCode == KeyEvent.KEYCODE_DPAD_DOWN) { if (k.action == KeyEvent.ACTION_DOWN) close(); true } else false
|
||||
},
|
||||
) {
|
||||
SheetChips(tabs.map { it.second }, state.tab, chipFocus, width = LocalSheetChipWidth.current) { i -> state.travel = if (i > state.tab) 1 else -1; state.tab = i }
|
||||
}
|
||||
val closeSource = remember { MutableInteractionSource() }
|
||||
val closeHot = rememberHot(closeSource)
|
||||
Box(
|
||||
contentAlignment = Alignment.Center,
|
||||
modifier = Modifier
|
||||
.offset { IntOffset((sheetW - 52.dp.toPx()).roundToInt(), chipsTop.roundToInt()) }
|
||||
.zIndex(1f).size(44.dp).testTag("drop-close")
|
||||
.clip(CircleShape).glideBorder(closeHot, CircleShape, pal.signal)
|
||||
.hoverable(closeSource)
|
||||
.clickable(interactionSource = closeSource, indication = LocalIndication.current, enabled = state.open, role = Role.Button) { close() }
|
||||
.controllerConfirm(enabled = state.open) { close() },
|
||||
) { Icon(Icons.Filled.Close, stringResource(R.string.common_close), tint = colors.onSurfaceVariant, modifier = Modifier.size(20.dp)) }
|
||||
}
|
||||
}
|
||||
|
||||
val enabled = covered() < 0.5f
|
||||
val sideInset = if (narrow) 20.dp else 44.dp
|
||||
val play = LocalPlayRect.current
|
||||
Box(
|
||||
Modifier
|
||||
.offset { IntOffset(play?.left?.roundToInt() ?: sideInset.roundToPx(), sideInset.roundToPx()) }
|
||||
.then(if (play != null) Modifier.width(with(density) { play.width.toDp() }) else Modifier)
|
||||
.onGloballyPositioned { rest = it.boundsInParent() }
|
||||
.graphicsLayer { translationY = lower * state.ride.value; alpha = 1f - covered() }
|
||||
.zIndex(2f)
|
||||
.focusRequester(buttonFocus)
|
||||
.focusProperties { canFocus = enabled }
|
||||
.onFocusChanged {
|
||||
if (it.hasFocus && leftForButton[0]) { leftForButton[0] = false; if (openState.value) close() }
|
||||
}
|
||||
.onPreviewKeyEvent { e ->
|
||||
val k = e.nativeKeyEvent
|
||||
if (k.keyCode != KeyEvent.KEYCODE_DPAD_UP && !(state.open && k.keyCode == KeyEvent.KEYCODE_DPAD_RIGHT)) false
|
||||
else {
|
||||
if (k.action == KeyEvent.ACTION_DOWN && k.repeatCount == 0) {
|
||||
if (state.open) runCatching { chipFocus[state.tab].requestFocus() } else if (enabled) state.open = true
|
||||
}
|
||||
true
|
||||
}
|
||||
},
|
||||
) { button(enabled) }
|
||||
|
||||
val r0 = rest
|
||||
if (r0 != null && (state.lipShow.value > 0.01f || state.pull.value > 0.5f)) {
|
||||
var handleSize by remember { mutableStateOf(androidx.compose.ui.unit.IntSize.Zero) }
|
||||
val lipH = 32.dp + state.pull.value.dp
|
||||
val drop = { if (!state.open && enabled) state.open = true }
|
||||
val settingsLabel = stringResource(R.string.drawer_page_settings)
|
||||
Box(
|
||||
contentAlignment = Alignment.BottomCenter,
|
||||
modifier = Modifier
|
||||
.offset { IntOffset((r0.center.x - handleSize.width / 2f).roundToInt(), (r0.top + lower * state.ride.value - handleSize.height + (1f - state.lipShow.value) * 8.dp.toPx()).roundToInt()) }
|
||||
.onSizeChanged { handleSize = it }
|
||||
.graphicsLayer { alpha = state.lipShow.value * (1f - state.lipGone.value) * (1f - covered()) }
|
||||
.zIndex(2f)
|
||||
.heightIn(min = 44.dp).testTag("drop-open")
|
||||
.semantics { contentDescription = settingsLabel }
|
||||
.focusProperties { canFocus = false }
|
||||
.clickable(interactionSource = remember { MutableInteractionSource() }, indication = null, role = Role.Button, onClick = drop),
|
||||
) {
|
||||
Handle(lipH, above = true)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -151,6 +151,7 @@ class FrontEndState(
|
||||
|
||||
class FrontEndActions(
|
||||
val onPlay: () -> Unit,
|
||||
val onPlaySteamVr: () -> Unit = {},
|
||||
val onPlayDesktopUi: () -> Unit,
|
||||
val onSteamUrl: (String) -> Unit = {},
|
||||
val onSteamGame: (Library.SteamGame) -> Unit,
|
||||
|
||||
@@ -206,7 +206,7 @@ internal fun SetupPanel(
|
||||
),
|
||||
) {
|
||||
PageHeader(stringResource(R.string.setup_title)) {
|
||||
Box(Modifier.weight(1f)) { SheetChips(tabs, tab, chipFocus, pick) }
|
||||
Box(Modifier.weight(1f)) { SheetChips(tabs, tab, chipFocus, onPick = pick) }
|
||||
}
|
||||
Column(modifier = Modifier.weight(1f).fillMaxWidth().verticalScroll(rememberScrollState()).padding(bottom = 24.dp)) {
|
||||
when (tab) {
|
||||
|
||||
@@ -22,6 +22,7 @@ import androidx.compose.foundation.interaction.collectIsHoveredAsState
|
||||
import androidx.compose.foundation.interaction.collectIsPressedAsState
|
||||
import androidx.compose.foundation.layout.Arrangement
|
||||
import androidx.compose.foundation.layout.Box
|
||||
import androidx.compose.foundation.layout.BoxWithConstraints
|
||||
import androidx.compose.foundation.layout.ExperimentalLayoutApi
|
||||
import androidx.compose.foundation.layout.FlowRow
|
||||
import androidx.compose.foundation.layout.Row
|
||||
@@ -38,6 +39,7 @@ import androidx.compose.material.icons.filled.ChevronLeft
|
||||
import androidx.compose.ui.graphics.vector.ImageVector
|
||||
import androidx.compose.material3.CircularProgressIndicator
|
||||
import androidx.compose.material3.Icon
|
||||
import androidx.compose.material3.LocalTextStyle
|
||||
import androidx.compose.material3.MaterialTheme
|
||||
import androidx.compose.material3.Text
|
||||
import androidx.compose.runtime.Composable
|
||||
@@ -57,6 +59,8 @@ import androidx.compose.ui.graphics.Color
|
||||
import androidx.compose.ui.graphics.graphicsLayer
|
||||
import androidx.compose.ui.semantics.Role
|
||||
import androidx.compose.ui.semantics.contentDescription
|
||||
import androidx.compose.ui.text.TextStyle
|
||||
import androidx.compose.ui.text.rememberTextMeasurer
|
||||
import androidx.compose.ui.text.font.FontWeight
|
||||
import androidx.compose.ui.text.style.TextOverflow
|
||||
import androidx.compose.ui.unit.dp
|
||||
@@ -148,6 +152,7 @@ internal fun PrimaryButton(
|
||||
modifier: Modifier = Modifier,
|
||||
large: Boolean = false,
|
||||
icon: ImageVector? = null,
|
||||
fill: Boolean = false,
|
||||
onClick: () -> Unit,
|
||||
) {
|
||||
val colors = MaterialTheme.colorScheme
|
||||
@@ -165,7 +170,8 @@ internal fun PrimaryButton(
|
||||
val lift by animateFloatAsState(if (hot) 14f else 6f, Motion.tw(300), label = "btnLift")
|
||||
val placed = remember { arrayOfNulls<androidx.compose.ui.layout.LayoutCoordinates>(1) }
|
||||
Row(
|
||||
verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(8.dp),
|
||||
verticalAlignment = Alignment.CenterVertically,
|
||||
horizontalArrangement = if (fill) Arrangement.spacedBy(8.dp, Alignment.CenterHorizontally) else Arrangement.spacedBy(8.dp),
|
||||
modifier = modifier.then(track)
|
||||
.onGloballyPositioned { placed[0] = it }
|
||||
.heightIn(min = if (compact) 44.dp else 48.dp)
|
||||
@@ -181,18 +187,34 @@ internal fun PrimaryButton(
|
||||
onClick()
|
||||
}
|
||||
.padding(
|
||||
start = if (large) 20.dp else if (compact) 10.dp else if (icon != null) 14.dp else 18.dp,
|
||||
end = if (large) 26.dp else if (compact) 10.dp else 18.dp,
|
||||
start = if (fill) 12.dp else if (large) 20.dp else if (compact) 10.dp else if (icon != null) 14.dp else 18.dp,
|
||||
end = if (fill) 12.dp else if (large) 26.dp else if (compact) 10.dp else 18.dp,
|
||||
top = if (large) 16.dp else if (compact) 7.dp else 11.dp,
|
||||
bottom = if (large) 16.dp else if (compact) 7.dp else 11.dp,
|
||||
),
|
||||
) {
|
||||
val fg = if (enabled) colors.onPrimary else colors.onSurfaceVariant
|
||||
if (icon != null) Icon(icon, contentDescription = null, tint = fg, modifier = Modifier.size(if (large) 20.dp else 17.dp))
|
||||
Text(
|
||||
text, fontSize = if (large) 17.sp else if (compact) 12.sp else 15.sp,
|
||||
fontWeight = if (large) FontWeight.Bold else FontWeight.SemiBold, letterSpacing = 0.5.sp, color = fg, maxLines = 1,
|
||||
)
|
||||
val style = LocalTextStyle.current.merge(TextStyle(
|
||||
fontSize = if (large) 17.sp else if (compact) 12.sp else 15.sp,
|
||||
fontWeight = if (large) FontWeight.Bold else FontWeight.SemiBold, letterSpacing = 0.5.sp, color = fg,
|
||||
))
|
||||
if (fill) FittedLabel(text, style, Modifier.weight(1f, fill = false))
|
||||
else Text(text, style = style, maxLines = 1)
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun FittedLabel(text: String, style: TextStyle, modifier: Modifier = Modifier) {
|
||||
BoxWithConstraints(modifier) {
|
||||
val measurer = rememberTextMeasurer()
|
||||
val max = constraints.maxWidth
|
||||
val size = remember(text, style, max) {
|
||||
var size = style.fontSize
|
||||
while (size.value > 10f && measurer.measure(text, style.copy(fontSize = size), maxLines = 1, softWrap = false).size.width > max) size = (size.value - 0.5f).sp
|
||||
size
|
||||
}
|
||||
Text(text, style = style.copy(fontSize = size), maxLines = 1, softWrap = false)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
package com.droiddeck.launcher.ui
|
||||
|
||||
import androidx.compose.foundation.layout.Column
|
||||
import androidx.compose.material3.AlertDialog
|
||||
import androidx.compose.material3.LinearProgressIndicator
|
||||
import androidx.compose.material3.Text
|
||||
import androidx.compose.material3.TextButton
|
||||
import androidx.compose.runtime.*
|
||||
import androidx.compose.ui.res.stringResource
|
||||
import com.droiddeck.launcher.R
|
||||
import com.droiddeck.launcher.glasses.CalibrationPhase
|
||||
import com.droiddeck.launcher.glasses.GlassesCalibrationState
|
||||
|
||||
@Composable
|
||||
internal fun GlassesCalibrationControls(
|
||||
state: GlassesCalibrationState,
|
||||
onStart: () -> Unit,
|
||||
onCancel: () -> Unit,
|
||||
onReset: () -> Unit,
|
||||
onDone: () -> Unit,
|
||||
) {
|
||||
var shown by remember { mutableStateOf(false) }
|
||||
var reset by remember { mutableStateOf(false) }
|
||||
val cancel by rememberUpdatedState(onCancel)
|
||||
DisposableEffect(Unit) { onDispose { cancel() } }
|
||||
ActionRow(stringResource(R.string.glasses_calibration),
|
||||
stringResource(if (state.saved) R.string.glasses_calibration_saved_hint else R.string.glasses_calibration_hint),
|
||||
stringResource(R.string.glasses_calibration_open), { onCancel(); shown = true })
|
||||
if (state.saved && !state.running) ActionRow(stringResource(R.string.glasses_calibration_reset), null,
|
||||
stringResource(R.string.glasses_calibration_reset_button), { reset = true })
|
||||
if (reset) AlertDialog(
|
||||
onDismissRequest = { reset = false },
|
||||
title = { Text(stringResource(R.string.glasses_calibration_reset)) },
|
||||
text = { Text(stringResource(R.string.glasses_calibration_reset_hint)) },
|
||||
confirmButton = { TextButton(onClick = { onReset(); reset = false }) { Text(stringResource(R.string.glasses_calibration_reset_button)) } },
|
||||
dismissButton = { TextButton(onClick = { reset = false }) { Text(stringResource(R.string.common_cancel)) } },
|
||||
)
|
||||
if (shown) AlertDialog(
|
||||
onDismissRequest = { onCancel(); shown = false },
|
||||
title = { Text(stringResource(R.string.glasses_calibration)) },
|
||||
text = { Column {
|
||||
Text(stringResource(when (state.phase) {
|
||||
CalibrationPhase.IDLE -> R.string.glasses_calibration_instructions
|
||||
CalibrationPhase.SETTLING -> R.string.glasses_calibration_settling
|
||||
CalibrationPhase.COLLECTING -> R.string.glasses_calibration_collecting
|
||||
CalibrationPhase.MOVING -> R.string.glasses_calibration_moving
|
||||
CalibrationPhase.TURN_OVER -> R.string.glasses_calibration_turn_over
|
||||
CalibrationPhase.SAVED -> R.string.glasses_calibration_complete
|
||||
CalibrationPhase.FAILED -> R.string.glasses_calibration_failed
|
||||
}))
|
||||
if (state.running) {
|
||||
LinearProgressIndicator(progress = { state.progress / 100f })
|
||||
Text("${state.progress}%")
|
||||
}
|
||||
} },
|
||||
confirmButton = {
|
||||
if (!state.running) TextButton(onClick = {
|
||||
if (state.phase == CalibrationPhase.SAVED) { onDone(); onCancel(); shown = false }
|
||||
else onStart()
|
||||
}) { Text(stringResource(if (state.phase == CalibrationPhase.SAVED) R.string.glasses_calibration_done else R.string.glasses_calibration_start)) }
|
||||
},
|
||||
dismissButton = { TextButton(onClick = { onCancel(); shown = false }) { Text(stringResource(R.string.common_cancel)) } },
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,268 @@
|
||||
package com.droiddeck.launcher.ui
|
||||
|
||||
import androidx.compose.foundation.background
|
||||
import androidx.compose.foundation.border
|
||||
import androidx.compose.foundation.clickable
|
||||
import androidx.compose.foundation.interaction.MutableInteractionSource
|
||||
import androidx.compose.foundation.interaction.collectIsPressedAsState
|
||||
import androidx.compose.foundation.layout.Box
|
||||
import androidx.compose.foundation.layout.Column
|
||||
import androidx.compose.foundation.layout.Spacer
|
||||
import androidx.compose.foundation.layout.WindowInsets
|
||||
import androidx.compose.foundation.layout.displayCutout
|
||||
import androidx.compose.foundation.layout.fillMaxSize
|
||||
import androidx.compose.foundation.layout.height
|
||||
import androidx.compose.foundation.layout.offset
|
||||
import androidx.compose.foundation.layout.padding
|
||||
import androidx.compose.foundation.layout.size
|
||||
import androidx.compose.foundation.layout.windowInsetsPadding
|
||||
import androidx.compose.foundation.shape.CircleShape
|
||||
import androidx.compose.foundation.shape.RoundedCornerShape
|
||||
import androidx.compose.material3.Icon
|
||||
import androidx.compose.material3.Text
|
||||
import androidx.compose.runtime.Composable
|
||||
import androidx.compose.runtime.remember
|
||||
import androidx.compose.ui.Alignment
|
||||
import androidx.compose.ui.draw.clip
|
||||
import androidx.compose.ui.platform.testTag
|
||||
import androidx.compose.ui.semantics.Role
|
||||
import androidx.compose.ui.unit.IntOffset
|
||||
import androidx.compose.ui.unit.sp
|
||||
import androidx.compose.runtime.getValue
|
||||
import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.graphics.Color
|
||||
import androidx.compose.ui.graphics.SolidColor
|
||||
import androidx.compose.ui.graphics.StrokeCap
|
||||
import androidx.compose.ui.graphics.StrokeJoin
|
||||
import androidx.compose.ui.graphics.vector.ImageVector
|
||||
import androidx.compose.ui.graphics.vector.path
|
||||
import androidx.compose.ui.res.stringResource
|
||||
import androidx.compose.ui.unit.dp
|
||||
import androidx.compose.runtime.collectAsState
|
||||
import com.droiddeck.launcher.R
|
||||
import com.droiddeck.launcher.glasses.Glasses
|
||||
import com.droiddeck.launcher.glasses.GlassesRendering
|
||||
import com.droiddeck.launcher.glasses.GlassesSettings
|
||||
import com.droiddeck.launcher.glasses.GlassesState
|
||||
import com.droiddeck.launcher.handtracking.HandTracking
|
||||
import java.util.Locale
|
||||
|
||||
internal val GlassesIcon: ImageVector by lazy {
|
||||
ImageVector.Builder("Glasses", 24.dp, 24.dp, 24f, 24f).apply {
|
||||
path(fill = null, stroke = SolidColor(Color.Black), strokeLineWidth = 1.8f, strokeLineCap = StrokeCap.Round, strokeLineJoin = StrokeJoin.Round) {
|
||||
moveTo(2.5f, 10f); lineTo(21.5f, 10f)
|
||||
moveTo(3.2f, 10f); lineTo(3.8f, 15.2f); quadTo(4f, 17f, 6f, 17f); lineTo(8.6f, 17f); quadTo(10.2f, 17f, 10.6f, 15.4f); lineTo(11.2f, 12.6f)
|
||||
quadTo(12f, 11.6f, 12.8f, 12.6f); lineTo(13.4f, 15.4f); quadTo(13.8f, 17f, 15.4f, 17f); lineTo(18f, 17f); quadTo(20f, 17f, 20.2f, 15.2f); lineTo(20.8f, 10f)
|
||||
moveTo(2.5f, 10f); lineTo(1.8f, 7.6f)
|
||||
moveTo(21.5f, 10f); lineTo(22.2f, 7.6f)
|
||||
}
|
||||
}.build()
|
||||
}
|
||||
|
||||
@Composable
|
||||
internal fun rememberGlasses(): Pair<GlassesState, GlassesSettings> {
|
||||
val g by Glasses.state.collectAsState()
|
||||
val s by Glasses.settings.collectAsState()
|
||||
return g to s
|
||||
}
|
||||
|
||||
internal object GlassesTab {
|
||||
const val DISPLAY = "display"
|
||||
const val HARDWARE = "hardware"
|
||||
const val TRACKING = "tracking"
|
||||
}
|
||||
|
||||
@Composable
|
||||
internal fun glassesTabs(g: GlassesState): List<Pair<String, String>> = buildList {
|
||||
add(GlassesTab.DISPLAY to stringResource(R.string.glasses_display))
|
||||
if (g.canControl) {
|
||||
if (g.hardwareControls) add(GlassesTab.HARDWARE to stringResource(R.string.glasses_hardware))
|
||||
add(GlassesTab.TRACKING to stringResource(R.string.glasses_tracking))
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
internal fun GlassesGroups(
|
||||
host: MenuHost,
|
||||
g: GlassesState,
|
||||
s: GlassesSettings,
|
||||
track: (String) -> Modifier = { Modifier },
|
||||
onRecenter: (() -> Unit)? = null,
|
||||
inSession: Boolean = true,
|
||||
outputActive: Boolean = inSession,
|
||||
steamVrRunning: Boolean = false,
|
||||
tab: String? = null,
|
||||
steamVr: @Composable () -> Unit = {},
|
||||
) {
|
||||
val display = tab == null || tab == GlassesTab.DISPLAY
|
||||
val picture = GlassesRendering.picture(s, g.sbs, steamVrRunning)
|
||||
val usesGlasses = GlassesRendering.usesGlasses(g, outputActive, steamVrRunning)
|
||||
val virtualScreen = usesGlasses && picture == GlassesRendering.Picture.VIRTUAL_SCREEN
|
||||
val directView = usesGlasses && picture == GlassesRendering.Picture.STEREO
|
||||
if (display) SettingsGroup(g.model) {
|
||||
if (g.experimental) SettingsRow(stringResource(R.string.glasses_control), stringResource(R.string.glasses_xreal_experimental)) {}
|
||||
val status = if (g.hasDisplay) stringResource(R.string.glasses_status, g.width, g.height, kotlin.math.round(g.hz).toInt(),
|
||||
stringResource(if (g.sbs) R.string.glasses_mode_3d else R.string.glasses_mode_2d)) else null
|
||||
when {
|
||||
g.noSignal -> SettingsRow(stringResource(R.string.glasses_output), stringResource(R.string.glasses_no_signal)) {}
|
||||
!g.controllable -> {
|
||||
if (status != null) SettingsRow(stringResource(R.string.glasses_output), status) {}
|
||||
SettingsRow(stringResource(R.string.glasses_control), g.controlError ?: stringResource(R.string.glasses_no_control)) {}
|
||||
}
|
||||
g.permission == Glasses.Permission.ASKING ->
|
||||
SettingsRow(stringResource(R.string.glasses_control), stringResource(R.string.glasses_asking)) {}
|
||||
!g.connected -> ActionRow(stringResource(R.string.glasses_control),
|
||||
g.controlError ?: stringResource(if (g.permission == Glasses.Permission.DENIED) R.string.glasses_denied else R.string.glasses_allow_hint),
|
||||
stringResource(R.string.glasses_allow), { Glasses.requestControl() })
|
||||
status != null -> SettingsRow(stringResource(R.string.glasses_output), status) {}
|
||||
}
|
||||
ToggleRow(host, "glasses-game", stringResource(R.string.glasses_game_on), stringResource(R.string.glasses_game_on_hint),
|
||||
s.gameOnGlasses, chipModifier = track("glasses-game")) { Glasses.setGameOnGlasses(it) }
|
||||
}
|
||||
if (display && (g.canControl || g.hasDisplay)) SettingsGroup(stringResource(R.string.glasses_display)) {
|
||||
if (g.canControl) {
|
||||
if (!g.sbs) ChoiceRow(host, "glasses-hz", stringResource(R.string.glasses_refresh), null,
|
||||
listOf(60, 90, 120).map { it to "$it Hz" }, s.refreshHz, chipModifier = track("glasses-hz")) { Glasses.setRefreshHz(it) }
|
||||
if (g.sbsSupport == GlassesState.SBS_FAILS && !g.sbs) {
|
||||
ActionRow(stringResource(R.string.glasses_3d), stringResource(R.string.glasses_3d_failed),
|
||||
stringResource(R.string.stores_dl_retry), { Glasses.set3d(true) })
|
||||
} else {
|
||||
if (inSession || g.sbs) ToggleRow(host, "glasses-3d", stringResource(R.string.glasses_3d), stringResource(R.string.glasses_3d_hint),
|
||||
g.sbs, chipModifier = track("glasses-3d")) { Glasses.set3d(it) }
|
||||
if (g.sbs) {
|
||||
if (g.sbs90) ChoiceRow(host, "glasses-3d-hz", stringResource(R.string.glasses_3d_refresh), null,
|
||||
listOf(60, 90).map { it to "$it Hz" }, s.sbsHz, chipModifier = track("glasses-3d-hz")) { Glasses.setSbsHz(it) }
|
||||
else SettingsRow(stringResource(R.string.glasses_3d_refresh), "60 Hz") {}
|
||||
}
|
||||
}
|
||||
}
|
||||
ToggleRow(host, "glasses-auto3d", stringResource(R.string.glasses_auto3d), stringResource(R.string.glasses_auto3d_hint),
|
||||
s.auto3d, chipModifier = track("glasses-auto3d")) { Glasses.setAuto3d(it) }
|
||||
ChoiceRow(host, "glasses-picture", stringResource(R.string.glasses_picture), stringResource(R.string.glasses_picture_hint),
|
||||
listOf(
|
||||
GlassesSettings.PICTURE_AUTO to stringResource(R.string.glasses_picture_auto),
|
||||
GlassesSettings.PICTURE_SCREEN to stringResource(R.string.glasses_picture_screen),
|
||||
GlassesSettings.PICTURE_SBS to stringResource(R.string.glasses_picture_sbs),
|
||||
), s.picture3d, chipModifier = track("glasses-picture"), hintLines = 2) { Glasses.setPicture3d(it) }
|
||||
when {
|
||||
!inSession -> SettingsRow(stringResource(R.string.glasses_picture_screen), stringResource(R.string.glasses_screen_next_session)) {}
|
||||
!usesGlasses -> SettingsRow(stringResource(R.string.glasses_picture_screen), stringResource(R.string.glasses_screen_no_output)) {}
|
||||
picture == GlassesRendering.Picture.FLAT -> ActionRow(
|
||||
stringResource(R.string.glasses_picture_screen), stringResource(R.string.glasses_screen_flat_hint),
|
||||
stringResource(R.string.glasses_screen_use), { Glasses.setPicture3d(GlassesSettings.PICTURE_SCREEN) })
|
||||
directView -> SettingsRow(stringResource(R.string.glasses_picture_screen),
|
||||
stringResource(if (steamVrRunning) R.string.glasses_screen_direct_hint else R.string.glasses_screen_sbs_hint)) {}
|
||||
}
|
||||
if (virtualScreen || (!inSession && s.picture3d != GlassesSettings.PICTURE_SBS)) {
|
||||
SliderRow(stringResource(R.string.glasses_screen_size), null, s.screenWidthDm, 5..60, step = 1,
|
||||
format = { String.format(Locale.US, "%.1f m", it / 10f) }, modifier = track("glasses-width")) { Glasses.setScreenWidthDm(it) }
|
||||
SliderRow(stringResource(R.string.glasses_screen_distance), null, s.screenDistanceDm, 5..100, step = 1,
|
||||
format = { String.format(Locale.US, "%.1f m", it / 10f) }, modifier = track("glasses-distance")) { Glasses.setScreenDistanceDm(it) }
|
||||
}
|
||||
steamVr()
|
||||
if ((directView && steamVrRunning) || !inSession) SliderRow(stringResource(R.string.glasses_vr_fov), stringResource(R.string.glasses_vr_fov_hint), s.vrFovDeg,
|
||||
GlassesSettings.VR_FOV_MIN..GlassesSettings.VR_FOV_MAX, step = 1,
|
||||
format = { "$it°" }, modifier = track("glasses-vr-fov")) { Glasses.setVrFovDeg(it) }
|
||||
}
|
||||
if ((tab == null || tab == GlassesTab.HARDWARE) && g.canControl && g.hardwareControls) SettingsGroup(stringResource(R.string.glasses_hardware)) {
|
||||
if (g.brightnessMax > 0) SliderRow(stringResource(R.string.glasses_brightness), null, g.brightness.coerceAtLeast(0), 0..g.brightnessMax, step = 1,
|
||||
modifier = track("glasses-brightness")) { Glasses.setBrightness(it) }
|
||||
if (g.volumeMax > 0) SliderRow(stringResource(R.string.glasses_volume), null, g.volume.coerceAtLeast(0), 0..g.volumeMax, step = 1,
|
||||
modifier = track("glasses-volume")) { Glasses.setVolume(it) }
|
||||
if (g.hasSunblock) ToggleRow(host, "glasses-sunblock", stringResource(R.string.glasses_sunblock), stringResource(R.string.glasses_sunblock_hint),
|
||||
g.sunblock, chipModifier = track("glasses-sunblock")) { Glasses.setSunblock(it) }
|
||||
}
|
||||
if ((tab == null || tab == GlassesTab.TRACKING) && g.canControl) SettingsGroup(stringResource(R.string.glasses_tracking)) {
|
||||
val trackingHint = when {
|
||||
inSession && !usesGlasses -> R.string.glasses_screen_no_output
|
||||
inSession && picture == GlassesRendering.Picture.FLAT -> R.string.glasses_tracking_flat_hint
|
||||
inSession && directView && !steamVrRunning -> R.string.glasses_tracking_sbs_hint
|
||||
else -> R.string.glasses_head_tracking_hint
|
||||
}
|
||||
ToggleRow(host, "glasses-tracking", stringResource(R.string.glasses_head_tracking), stringResource(trackingHint),
|
||||
s.headTracking, chipModifier = track("glasses-tracking")) { Glasses.setHeadTracking(it) }
|
||||
if (onRecenter != null && s.headTracking && (virtualScreen || (directView && steamVrRunning))) ActionRow(stringResource(R.string.glasses_recenter), null,
|
||||
stringResource(R.string.cardboard_recenter_button), onRecenter)
|
||||
val calibration by Glasses.calibration.collectAsState()
|
||||
if (calibration.supported) GlassesCalibrationControls(calibration,
|
||||
onStart = Glasses::startCalibration, onCancel = Glasses::cancelCalibration,
|
||||
onReset = Glasses::resetCalibration, onDone = { onRecenter?.invoke() })
|
||||
}
|
||||
val context = androidx.compose.ui.platform.LocalContext.current
|
||||
val hands by HandTracking.settings.collectAsState()
|
||||
androidx.compose.runtime.LaunchedEffect(Unit) { HandTracking.load(context) }
|
||||
if ((tab == null || tab == GlassesTab.TRACKING) && g.canControl && hands.usable) SettingsGroup(stringResource(R.string.hand_camera_group)) {
|
||||
HandCameraRows(host, track, inSession)
|
||||
}
|
||||
}
|
||||
|
||||
private fun padButtonColors(tint: Int, opacity: Int, pressed: Boolean): Pair<Color, Color> {
|
||||
val base = Color(tint)
|
||||
val alpha = opacity.coerceIn(0, 100) / 100f
|
||||
fun light(a: Int, f: Float) = Color(base.red + (1 - base.red) * f, base.green + (1 - base.green) * f, base.blue + (1 - base.blue) * f, a * alpha / 255f)
|
||||
return Color(base.red * 0.12f, base.green * 0.12f, base.blue * 0.12f, 190 / 255f) to (if (pressed) light(230, 0.7f) else light(150, 0.25f))
|
||||
}
|
||||
|
||||
@Composable
|
||||
internal fun GlassesCornerButton(slot: Int, tint: Int, opacity: Int, onClick: () -> Unit) {
|
||||
val density = androidx.compose.ui.platform.LocalDensity.current
|
||||
androidx.compose.foundation.layout.BoxWithConstraints(
|
||||
Modifier.fillMaxSize().windowInsetsPadding(WindowInsets.displayCutout),
|
||||
) {
|
||||
val x = if (slot < 0) maxWidth / 2 + 56.dp else 34.dp + 56.dp * slot
|
||||
val bottom = if (slot < 0) 44.dp else 34.dp
|
||||
val source = remember { MutableInteractionSource() }
|
||||
val pressed by source.collectIsPressedAsState()
|
||||
val (fill, line) = padButtonColors(tint, opacity, pressed)
|
||||
Box(
|
||||
contentAlignment = Alignment.Center,
|
||||
modifier = Modifier
|
||||
.offset { with(density) { IntOffset((x - 18.dp).roundToPx(), (maxHeight - bottom - 18.dp).roundToPx()) } }
|
||||
.size(36.dp)
|
||||
.clip(CircleShape)
|
||||
.background(fill)
|
||||
.border(1.5.dp, line, CircleShape)
|
||||
.clickable(interactionSource = source, indication = null, role = Role.Button, onClick = onClick)
|
||||
.testTag("glasses-button"),
|
||||
) {
|
||||
Icon(GlassesIcon, stringResource(R.string.glasses_open), tint = line, modifier = Modifier.size(22.dp))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
internal fun GlassesRecenterButton(tint: Int, opacity: Int, onClick: () -> Unit) {
|
||||
val source = remember { MutableInteractionSource() }
|
||||
val pressed by source.collectIsPressedAsState()
|
||||
val (fill, line) = padButtonColors(tint, opacity, pressed)
|
||||
val shape = RoundedCornerShape(50)
|
||||
Box(
|
||||
Modifier.fillMaxSize().windowInsetsPadding(WindowInsets.displayCutout).padding(top = 12.dp),
|
||||
contentAlignment = Alignment.TopCenter,
|
||||
) {
|
||||
Box(
|
||||
contentAlignment = Alignment.Center,
|
||||
modifier = Modifier
|
||||
.height(36.dp)
|
||||
.clip(shape)
|
||||
.background(fill)
|
||||
.border(1.5.dp, line, shape)
|
||||
.clickable(interactionSource = source, indication = null, role = Role.Button, onClick = onClick)
|
||||
.padding(horizontal = 18.dp)
|
||||
.testTag("glasses-recenter"),
|
||||
) {
|
||||
Text(stringResource(R.string.glasses_recenter_button), color = line, fontSize = 14.sp)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
internal fun GlassesPlayingHint(model: String) {
|
||||
Box(Modifier.fillMaxSize(), contentAlignment = Alignment.Center) {
|
||||
Column(horizontalAlignment = Alignment.CenterHorizontally) {
|
||||
Icon(GlassesIcon, null, tint = Color.White.copy(alpha = 0.35f), modifier = Modifier.size(56.dp))
|
||||
Spacer(Modifier.height(10.dp))
|
||||
Text(stringResource(R.string.glasses_playing, model), color = Color.White.copy(alpha = 0.45f), fontSize = 15.sp)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,345 @@
|
||||
package com.droiddeck.launcher.ui
|
||||
|
||||
import android.Manifest
|
||||
import android.content.Intent
|
||||
import android.net.Uri
|
||||
import android.provider.Settings
|
||||
import android.speech.tts.TextToSpeech
|
||||
import androidx.activity.compose.rememberLauncherForActivityResult
|
||||
import androidx.activity.result.contract.ActivityResultContracts
|
||||
import androidx.compose.foundation.Canvas
|
||||
import androidx.compose.foundation.Image
|
||||
import androidx.compose.foundation.background
|
||||
import androidx.compose.foundation.layout.Arrangement
|
||||
import androidx.compose.foundation.layout.Box
|
||||
import androidx.compose.foundation.layout.Column
|
||||
import androidx.compose.foundation.layout.Row
|
||||
import androidx.compose.foundation.layout.Spacer
|
||||
import androidx.compose.foundation.layout.aspectRatio
|
||||
import androidx.compose.foundation.layout.fillMaxHeight
|
||||
import androidx.compose.foundation.layout.fillMaxSize
|
||||
import androidx.compose.foundation.layout.fillMaxWidth
|
||||
import androidx.compose.foundation.layout.heightIn
|
||||
import androidx.compose.foundation.layout.padding
|
||||
import androidx.compose.foundation.layout.size
|
||||
import androidx.compose.foundation.rememberScrollState
|
||||
import androidx.compose.foundation.selection.selectable
|
||||
import androidx.compose.foundation.shape.RoundedCornerShape
|
||||
import androidx.compose.foundation.verticalScroll
|
||||
import androidx.compose.material3.AlertDialog
|
||||
import androidx.compose.material3.CircularProgressIndicator
|
||||
import androidx.compose.material3.MaterialTheme
|
||||
import androidx.compose.material3.RadioButton
|
||||
import androidx.compose.material3.RadioButtonDefaults
|
||||
import androidx.compose.material3.Text
|
||||
import androidx.compose.material3.TextButton
|
||||
import androidx.compose.runtime.Composable
|
||||
import androidx.compose.runtime.DisposableEffect
|
||||
import androidx.compose.runtime.LaunchedEffect
|
||||
import androidx.compose.runtime.collectAsState
|
||||
import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
import androidx.compose.runtime.remember
|
||||
import androidx.compose.runtime.rememberUpdatedState
|
||||
import androidx.compose.runtime.saveable.rememberSaveable
|
||||
import androidx.compose.runtime.setValue
|
||||
import androidx.compose.ui.Alignment
|
||||
import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.draw.clip
|
||||
import androidx.compose.ui.geometry.Offset
|
||||
import androidx.compose.ui.graphics.Color
|
||||
import androidx.compose.ui.graphics.StrokeCap
|
||||
import androidx.compose.ui.graphics.asImageBitmap
|
||||
import androidx.compose.ui.graphics.drawscope.DrawScope
|
||||
import androidx.compose.ui.graphics.graphicsLayer
|
||||
import androidx.compose.ui.layout.ContentScale
|
||||
import androidx.compose.ui.platform.LocalContext
|
||||
import androidx.compose.ui.platform.LocalLifecycleOwner
|
||||
import androidx.compose.ui.platform.testTag
|
||||
import androidx.compose.ui.res.stringResource
|
||||
import androidx.compose.ui.semantics.Role
|
||||
import androidx.compose.ui.text.font.FontWeight
|
||||
import androidx.compose.ui.unit.dp
|
||||
import androidx.compose.ui.unit.sp
|
||||
import androidx.compose.ui.window.Dialog
|
||||
import androidx.compose.ui.window.DialogProperties
|
||||
import androidx.lifecycle.Lifecycle
|
||||
import androidx.lifecycle.LifecycleEventObserver
|
||||
import com.droiddeck.launcher.R
|
||||
import com.droiddeck.launcher.handtracking.Advice
|
||||
import com.droiddeck.launcher.handtracking.BodyPose
|
||||
import com.droiddeck.launcher.handtracking.HandCamera
|
||||
import com.droiddeck.launcher.handtracking.HandTracking
|
||||
import com.droiddeck.launcher.handtracking.Landmark
|
||||
import com.droiddeck.launcher.handtracking.TrackingCameras
|
||||
import kotlinx.coroutines.delay
|
||||
import java.util.Locale
|
||||
|
||||
@Composable
|
||||
internal fun HandCameraHost(check: Boolean?, onAskPermission: () -> Unit, onPlay: () -> Unit, onClose: () -> Unit) {
|
||||
if (!HandTracking.supported) return
|
||||
val context = LocalContext.current
|
||||
val (glasses, _) = rememberGlasses()
|
||||
val settings by HandTracking.settings.collectAsState()
|
||||
LaunchedEffect(Unit) { HandTracking.load(context) }
|
||||
var prompted by rememberSaveable { mutableStateOf(false) }
|
||||
if (!prompted && HandTracking.shouldAsk(settings, glasses.present && glasses.canControl)) {
|
||||
HandCameraPrompt(settings.sides,
|
||||
onAllow = { side ->
|
||||
prompted = true
|
||||
HandTracking.allow(context, side)
|
||||
if (!HandTracking.permitted(context)) onAskPermission()
|
||||
},
|
||||
onNotNow = {
|
||||
prompted = true
|
||||
HandTracking.decline(context)
|
||||
})
|
||||
}
|
||||
val requests by HandTracking.checkRequests.collectAsState()
|
||||
var handled by remember { mutableStateOf(requests) }
|
||||
val fromSettings = requests != handled
|
||||
if (check != null || fromSettings) {
|
||||
PlacementCheck(settings, play = check == true && !fromSettings,
|
||||
onPlay = { handled = requests; onPlay() },
|
||||
onClose = { handled = requests; onClose() })
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
internal fun HandCameraPrompt(sides: Set<String>, onAllow: (String?) -> Unit, onNotNow: () -> Unit) {
|
||||
val colors = MaterialTheme.colorScheme
|
||||
val pal = LocalPalette.current
|
||||
val choose = sides.size > 1
|
||||
var side by remember { mutableStateOf(TrackingCameras.REAR) }
|
||||
AlertDialog(
|
||||
onDismissRequest = onNotNow,
|
||||
modifier = Modifier.testTag("hand-camera-prompt"),
|
||||
title = { Text(stringResource(R.string.hand_camera_prompt_title)) },
|
||||
text = {
|
||||
Column(Modifier.verticalScroll(rememberScrollState()), verticalArrangement = Arrangement.spacedBy(10.dp)) {
|
||||
Text(stringResource(R.string.hand_camera_prompt_body), fontSize = 14.sp)
|
||||
if (choose) listOf(
|
||||
Triple(TrackingCameras.REAR, R.string.hand_camera_rear, R.string.hand_camera_rear_hint),
|
||||
Triple(TrackingCameras.FRONT, R.string.hand_camera_front, R.string.hand_camera_front_hint),
|
||||
).forEach { (id, label, hint) ->
|
||||
Row(
|
||||
verticalAlignment = Alignment.CenterVertically,
|
||||
modifier = Modifier.fillMaxWidth().clip(RoundedCornerShape(8.dp))
|
||||
.selectable(side == id, role = Role.RadioButton) { side = id }
|
||||
.testTag("hand-camera-$id")
|
||||
.padding(vertical = 4.dp),
|
||||
) {
|
||||
RadioButton(side == id, onClick = null,
|
||||
colors = RadioButtonDefaults.colors(selectedColor = pal.signal, unselectedColor = colors.onSurfaceVariant))
|
||||
Column(Modifier.padding(start = 10.dp)) {
|
||||
Text(stringResource(label), fontSize = 15.sp, color = colors.onBackground)
|
||||
Text(stringResource(hint), fontSize = 13.sp, color = colors.onSurfaceVariant)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
confirmButton = {
|
||||
TextButton(onClick = { onAllow(if (choose) side else null) }, modifier = Modifier.testTag("hand-camera-allow")) {
|
||||
Text(stringResource(R.string.hand_camera_allow))
|
||||
}
|
||||
},
|
||||
dismissButton = {
|
||||
TextButton(onClick = onNotNow, modifier = Modifier.testTag("hand-camera-not-now")) { Text(stringResource(R.string.hand_camera_not_now)) }
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
@Composable
|
||||
internal fun HandCameraRows(host: MenuHost, track: (String) -> Modifier, inSession: Boolean) {
|
||||
if (!HandTracking.supported) return
|
||||
val context = LocalContext.current
|
||||
val lifecycle = LocalLifecycleOwner.current.lifecycle
|
||||
DisposableEffect(lifecycle) {
|
||||
HandTracking.load(context)
|
||||
val watch = LifecycleEventObserver { _, event -> if (event == Lifecycle.Event.ON_RESUME) HandTracking.refreshPermission(context) }
|
||||
lifecycle.addObserver(watch)
|
||||
onDispose { lifecycle.removeObserver(watch) }
|
||||
}
|
||||
val s by HandTracking.settings.collectAsState()
|
||||
if (!s.usable) return
|
||||
var denied by remember { mutableStateOf(false) }
|
||||
val ask = rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
|
||||
HandTracking.refreshPermission(context)
|
||||
denied = !granted
|
||||
}
|
||||
ToggleRow(host, "hand-camera", stringResource(R.string.hand_camera_setting), stringResource(R.string.hand_camera_setting_hint),
|
||||
s.enabled, chipModifier = track("hand-camera")) { on ->
|
||||
HandTracking.setEnabled(context, on)
|
||||
if (on && !s.permitted) ask.launch(Manifest.permission.CAMERA)
|
||||
}
|
||||
if (!s.enabled) return
|
||||
if (s.sides.size > 1) ChoiceRow(host, "hand-camera-side", stringResource(R.string.hand_camera_choice), null,
|
||||
listOf(TrackingCameras.REAR to stringResource(R.string.hand_camera_rear), TrackingCameras.FRONT to stringResource(R.string.hand_camera_front)),
|
||||
s.side ?: TrackingCameras.REAR, chipModifier = track("hand-camera-side")) { HandTracking.setSide(context, it) }
|
||||
when {
|
||||
!s.permitted && denied -> ActionRow(stringResource(R.string.hand_camera_access), stringResource(R.string.hand_camera_access_denied),
|
||||
stringResource(R.string.hand_camera_open_settings), {
|
||||
context.startActivity(Intent(Settings.ACTION_APPLICATION_DETAILS_SETTINGS, Uri.fromParts("package", context.packageName, null))
|
||||
.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK))
|
||||
})
|
||||
!s.permitted -> ActionRow(stringResource(R.string.hand_camera_access), stringResource(R.string.hand_camera_access_hint),
|
||||
stringResource(R.string.hand_camera_allow_button), { ask.launch(Manifest.permission.CAMERA) })
|
||||
!inSession -> ActionRow(stringResource(R.string.hand_camera_placement), stringResource(R.string.hand_camera_placement_hint),
|
||||
stringResource(R.string.hand_camera_check), { HandTracking.requestCheck() })
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun PlacementCheck(settings: HandTracking.Settings, play: Boolean, onPlay: () -> Unit, onClose: () -> Unit) {
|
||||
val context = LocalContext.current
|
||||
val side = settings.side
|
||||
val camera = remember(side) { HandCamera(context, null, preview = true) }
|
||||
DisposableEffect(camera) {
|
||||
camera.start(side)
|
||||
onDispose { camera.stop() }
|
||||
}
|
||||
val view by camera.view.collectAsState()
|
||||
val failed by camera.failed.collectAsState()
|
||||
val front = view?.front ?: (TrackingCameras.pick(HandTracking.cameras(context), side)?.front == true)
|
||||
val advice = when {
|
||||
failed -> null
|
||||
else -> view?.advice ?: Advice.STARTING
|
||||
}
|
||||
val adviceText = stringResource(advice?.let(::adviceLabel) ?: R.string.hand_camera_failed)
|
||||
val start by rememberUpdatedState(onPlay)
|
||||
if (play) LaunchedEffect(advice == Advice.GOOD) {
|
||||
if (advice == Advice.GOOD) {
|
||||
delay(GOOD_HOLD_MS)
|
||||
start()
|
||||
}
|
||||
}
|
||||
if (!front) SpokenAdvice(advice, adviceText)
|
||||
Dialog(onDismissRequest = onClose, properties = DialogProperties(usePlatformDefaultWidth = false, decorFitsSystemWindows = false)) {
|
||||
PlacementCheckContent(view, front, advice, adviceText, play, switchable = settings.sides.size > 1,
|
||||
onSwitch = { HandTracking.setSide(context, if (front) TrackingCameras.REAR else TrackingCameras.FRONT) },
|
||||
onPlay = onPlay, onClose = onClose)
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
internal fun PlacementCheckContent(
|
||||
view: HandCamera.View?,
|
||||
front: Boolean,
|
||||
advice: Advice?,
|
||||
adviceText: String,
|
||||
play: Boolean,
|
||||
switchable: Boolean,
|
||||
onSwitch: () -> Unit,
|
||||
onPlay: () -> Unit,
|
||||
onClose: () -> Unit,
|
||||
) {
|
||||
val colors = MaterialTheme.colorScheme
|
||||
val pal = LocalPalette.current
|
||||
Row(
|
||||
horizontalArrangement = Arrangement.spacedBy(24.dp),
|
||||
modifier = Modifier.fillMaxSize().background(pal.background).padding(24.dp).testTag("hand-camera-check"),
|
||||
) {
|
||||
Box(Modifier.weight(1.3f).fillMaxHeight(), contentAlignment = Alignment.Center) { CameraPreview(view) }
|
||||
Column(Modifier.weight(1f).fillMaxHeight().verticalScroll(rememberScrollState()), verticalArrangement = Arrangement.spacedBy(14.dp)) {
|
||||
Text(stringResource(R.string.hand_camera_check_title), fontSize = 22.sp, fontWeight = FontWeight.Bold, color = colors.onBackground)
|
||||
Text(stringResource(if (front) R.string.hand_camera_how_front else R.string.hand_camera_how_rear), fontSize = 15.sp, color = colors.onSurfaceVariant)
|
||||
Text(adviceText, fontSize = 19.sp, fontWeight = FontWeight.SemiBold,
|
||||
color = if (advice == Advice.GOOD) pal.good else colors.onBackground, modifier = Modifier.testTag("hand-camera-advice"))
|
||||
Spacer(Modifier.heightIn(min = 8.dp))
|
||||
Row(horizontalArrangement = Arrangement.spacedBy(10.dp)) {
|
||||
if (switchable) SecondaryButton(stringResource(R.string.hand_camera_switch), compact = true, onClick = onSwitch)
|
||||
if (play) {
|
||||
SecondaryButton(stringResource(R.string.hand_camera_skip), compact = true, modifier = Modifier.testTag("hand-camera-skip"), onClick = onPlay)
|
||||
SecondaryButton(stringResource(R.string.common_cancel), compact = true, onClick = onClose)
|
||||
} else {
|
||||
SecondaryButton(stringResource(R.string.hand_camera_done), compact = true, modifier = Modifier.testTag("hand-camera-done"), onClick = onClose)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun CameraPreview(view: HandCamera.View?) {
|
||||
val pal = LocalPalette.current
|
||||
val shape = RoundedCornerShape(16.dp)
|
||||
if (view == null) {
|
||||
Box(Modifier.fillMaxWidth().aspectRatio(4f / 3f).clip(shape).background(pal.surface), contentAlignment = Alignment.Center) {
|
||||
CircularProgressIndicator(color = pal.signal, modifier = Modifier.size(36.dp))
|
||||
}
|
||||
return
|
||||
}
|
||||
val image = remember(view) { view.image.asImageBitmap() }
|
||||
Box(Modifier.aspectRatio(view.image.width.toFloat() / view.image.height).clip(shape)) {
|
||||
Image(image, null, contentScale = ContentScale.FillBounds,
|
||||
modifier = Modifier.fillMaxSize().graphicsLayer { if (view.front) scaleX = -1f })
|
||||
Canvas(Modifier.fillMaxSize()) { drawSkeleton(view.landmarks, view.front, pal.signal, pal.good) }
|
||||
}
|
||||
}
|
||||
|
||||
private fun DrawScope.drawSkeleton(marks: List<Landmark>?, mirrored: Boolean, line: Color, hand: Color) {
|
||||
if (marks == null || marks.size <= BodyPose.RIGHT_HIP) return
|
||||
fun at(i: Int) = Offset((if (mirrored) 1f - marks[i].x else marks[i].x) * size.width, marks[i].y * size.height)
|
||||
fun seen(i: Int) = marks[i].visibility >= BodyPose.MIN_VISIBILITY
|
||||
val width = size.minDimension / 120f
|
||||
for ((a, b) in BONES) if (seen(a) && seen(b)) drawLine(line, at(a), at(b), strokeWidth = width, cap = StrokeCap.Round)
|
||||
for (i in JOINTS) if (seen(i)) drawCircle(line, width * 1.2f, at(i))
|
||||
for (wrist in listOf(BodyPose.LEFT_WRIST, BodyPose.RIGHT_WRIST)) {
|
||||
if (!seen(wrist)) continue
|
||||
val grip = (at(wrist) + at(wrist + 2) + at(wrist + 4)) / 3f
|
||||
drawCircle(hand.copy(alpha = 0.35f), width * 5f, grip)
|
||||
drawCircle(hand, width * 2.2f, grip)
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun SpokenAdvice(advice: Advice?, text: String) {
|
||||
val context = LocalContext.current
|
||||
var ready by remember { mutableStateOf(false) }
|
||||
val speech = remember {
|
||||
var engine: TextToSpeech? = null
|
||||
engine = TextToSpeech(context.applicationContext) { status ->
|
||||
if (status == TextToSpeech.SUCCESS) {
|
||||
engine?.language = Locale.getDefault()
|
||||
ready = true
|
||||
}
|
||||
}
|
||||
engine
|
||||
}
|
||||
DisposableEffect(speech) { onDispose { speech.shutdown() } }
|
||||
LaunchedEffect(advice, ready) {
|
||||
if (!ready || advice == null || advice == Advice.STARTING) return@LaunchedEffect
|
||||
delay(SPEAK_AFTER_MS)
|
||||
while (true) {
|
||||
speech.speak(text, TextToSpeech.QUEUE_FLUSH, null, "hand-camera")
|
||||
if (advice == Advice.GOOD) break
|
||||
delay(SPEAK_AGAIN_MS)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private fun adviceLabel(advice: Advice): Int = when (advice) {
|
||||
Advice.STARTING -> R.string.hand_camera_starting
|
||||
Advice.STAND_UP -> R.string.hand_camera_stand_up
|
||||
Advice.UPRIGHT -> R.string.hand_camera_upright
|
||||
Advice.NOT_SEEN -> R.string.hand_camera_not_seen
|
||||
Advice.MOVE_BACK -> R.string.hand_camera_move_back
|
||||
Advice.MOVE_CLOSER -> R.string.hand_camera_move_closer
|
||||
Advice.SHOW_HANDS -> R.string.hand_camera_show_hands
|
||||
Advice.GOOD -> R.string.hand_camera_good
|
||||
}
|
||||
|
||||
private val BONES = listOf(
|
||||
BodyPose.LEFT_SHOULDER to BodyPose.RIGHT_SHOULDER,
|
||||
BodyPose.LEFT_SHOULDER to BodyPose.LEFT_ELBOW, BodyPose.LEFT_ELBOW to BodyPose.LEFT_WRIST,
|
||||
BodyPose.RIGHT_SHOULDER to BodyPose.RIGHT_ELBOW, BodyPose.RIGHT_ELBOW to BodyPose.RIGHT_WRIST,
|
||||
BodyPose.LEFT_SHOULDER to BodyPose.LEFT_HIP, BodyPose.RIGHT_SHOULDER to BodyPose.RIGHT_HIP, BodyPose.LEFT_HIP to BodyPose.RIGHT_HIP,
|
||||
BodyPose.NOSE to BodyPose.LEFT_EYE, BodyPose.NOSE to BodyPose.RIGHT_EYE,
|
||||
BodyPose.LEFT_EYE to BodyPose.LEFT_EAR, BodyPose.RIGHT_EYE to BodyPose.RIGHT_EAR,
|
||||
)
|
||||
private val JOINTS = listOf(BodyPose.NOSE, BodyPose.LEFT_SHOULDER, BodyPose.RIGHT_SHOULDER, BodyPose.LEFT_ELBOW, BodyPose.RIGHT_ELBOW,
|
||||
BodyPose.LEFT_HIP, BodyPose.RIGHT_HIP)
|
||||
private const val GOOD_HOLD_MS = 1200L
|
||||
private const val SPEAK_AFTER_MS = 1200L
|
||||
private const val SPEAK_AGAIN_MS = 6000L
|
||||
@@ -23,6 +23,7 @@ import androidx.compose.foundation.interaction.collectIsHoveredAsState
|
||||
import androidx.compose.foundation.layout.Arrangement
|
||||
import androidx.compose.foundation.layout.Box
|
||||
import androidx.compose.foundation.layout.BoxWithConstraints
|
||||
import androidx.compose.foundation.layout.BoxWithConstraintsScope
|
||||
import androidx.compose.foundation.layout.Column
|
||||
import androidx.compose.foundation.layout.Row
|
||||
import androidx.compose.foundation.layout.Spacer
|
||||
@@ -42,6 +43,7 @@ import androidx.compose.foundation.shape.RoundedCornerShape
|
||||
import androidx.compose.foundation.verticalScroll
|
||||
import androidx.compose.material.icons.Icons
|
||||
import androidx.compose.material.icons.filled.Close
|
||||
import androidx.compose.material.icons.filled.KeyboardArrowDown
|
||||
import androidx.compose.material.icons.filled.KeyboardArrowUp
|
||||
import androidx.compose.ui.platform.testTag
|
||||
import kotlinx.coroutines.Job
|
||||
@@ -52,6 +54,7 @@ import androidx.compose.runtime.Composable
|
||||
import androidx.compose.runtime.CompositionLocalProvider
|
||||
import androidx.compose.runtime.DisposableEffect
|
||||
import androidx.compose.runtime.LaunchedEffect
|
||||
import androidx.compose.runtime.compositionLocalOf
|
||||
import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.mutableStateMapOf
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
@@ -81,6 +84,7 @@ import androidx.compose.ui.geometry.RoundRect
|
||||
import androidx.compose.ui.graphics.Color
|
||||
import androidx.compose.ui.graphics.Path
|
||||
import androidx.compose.ui.graphics.drawscope.Stroke
|
||||
import androidx.compose.ui.graphics.drawscope.scale
|
||||
import androidx.compose.ui.graphics.graphicsLayer
|
||||
import androidx.compose.ui.graphics.lerp
|
||||
import androidx.compose.ui.input.InputMode
|
||||
@@ -102,6 +106,7 @@ import androidx.compose.ui.text.style.TextOverflow
|
||||
import androidx.compose.ui.unit.Dp
|
||||
import androidx.compose.ui.unit.IntOffset
|
||||
import androidx.compose.ui.unit.dp
|
||||
import androidx.compose.ui.unit.isSpecified
|
||||
import androidx.compose.ui.unit.sp
|
||||
import androidx.compose.ui.util.lerp
|
||||
import com.droiddeck.launcher.R
|
||||
@@ -138,6 +143,15 @@ internal class SheetBeat(private val pieces: List<Animatable<Float, AnimationVec
|
||||
|
||||
internal val LocalSheetBeat = staticCompositionLocalOf<SheetBeat?> { null }
|
||||
|
||||
internal val LocalSheetChipWidth = compositionLocalOf { Dp.Unspecified }
|
||||
|
||||
internal val LocalPlayRect = compositionLocalOf<Rect?> { null }
|
||||
|
||||
private val ChipGap = 6.dp
|
||||
|
||||
internal fun sheetChipWidth(row: Dp, count: Int): Dp =
|
||||
if (count > 0 && row / count >= 96.dp) (row - ChipGap * (count - 1)) / count else 0.dp
|
||||
|
||||
@Composable
|
||||
internal fun Modifier.sheetPiece(): Modifier {
|
||||
val beat = LocalSheetBeat.current ?: return this
|
||||
@@ -173,6 +187,8 @@ internal fun LiftHero(
|
||||
onOpenChange: (Boolean) -> Unit = {},
|
||||
start: Dp = 44.dp,
|
||||
bottom: Dp = 44.dp,
|
||||
drop: DropSheetState? = null,
|
||||
overlay: @Composable BoxWithConstraintsScope.(lifted: () -> Float) -> Unit = {},
|
||||
) {
|
||||
val density = LocalDensity.current
|
||||
val inputMode = LocalInputModeManager.current
|
||||
@@ -306,7 +322,7 @@ internal fun LiftHero(
|
||||
val lipFocus = remember { FocusRequester() }
|
||||
val settingsLabel = stringResource(R.string.drawer_page_settings)
|
||||
var lipFocused by remember { mutableStateOf(false) }
|
||||
val lipWanted = !open && (!shown || lipBack) && (playFocused || playHovered || touchOnly || lipFocused)
|
||||
val lipWanted = !open && (!shown || lipBack) && (playFocused || playHovered || touchOnly || lipFocused) && drop?.let { it.open || it.shown } != true
|
||||
LaunchedEffect(lipWanted) {
|
||||
if (lipWanted) {
|
||||
if (!lipBack) delay(Motion.ms(160).toLong())
|
||||
@@ -385,18 +401,35 @@ internal fun LiftHero(
|
||||
}
|
||||
BoxWithConstraints(
|
||||
modifier.fillMaxSize().onGloballyPositioned { heroAt = it.boundsInRoot().topLeft }
|
||||
.pointerInput(Unit) {
|
||||
.pointerInput(drop) {
|
||||
val tracker = androidx.compose.ui.input.pointer.util.VelocityTracker()
|
||||
var decided = false
|
||||
val dropFlick = 600.dp.toPx()
|
||||
detectVerticalDragGestures(
|
||||
onDragStart = { tracker.resetTracking(); decided = false; drag.engaged = false },
|
||||
onDragEnd = { if (drag.engaged) settleDrag(tracker.calculateVelocity().y) },
|
||||
onDragCancel = { if (drag.engaged) settleDrag(0f) },
|
||||
onDragEnd = {
|
||||
if (drag.engaged) settleDrag(tracker.calculateVelocity().y)
|
||||
else if (drop?.engaged == true) drop.settle(tracker.calculateVelocity().y, dropFlick)
|
||||
},
|
||||
onDragCancel = {
|
||||
if (drag.engaged) settleDrag(0f)
|
||||
else if (drop?.engaged == true) drop.settle(0f, dropFlick)
|
||||
},
|
||||
) { change, amount ->
|
||||
if (launching) return@detectVerticalDragGestures
|
||||
if (!decided) {
|
||||
decided = true
|
||||
if (if (open) amount > 0f else amount < 0f) beginDrag()
|
||||
when {
|
||||
drop != null && (drop.open || drop.shown) -> if (drop.open && amount < 0f) drop.beginDrag()
|
||||
open || shown || amount < 0f -> if (if (open) amount > 0f else amount < 0f) beginDrag()
|
||||
drop != null && amount > 0f -> drop.beginDrag()
|
||||
}
|
||||
}
|
||||
if (drop?.engaged == true) {
|
||||
change.consume()
|
||||
tracker.addPosition(change.uptimeMillis, change.position)
|
||||
drop.drag(amount)
|
||||
return@detectVerticalDragGestures
|
||||
}
|
||||
if (!drag.engaged) return@detectVerticalDragGestures
|
||||
change.consume()
|
||||
@@ -413,9 +446,15 @@ internal fun LiftHero(
|
||||
val openTop = sheetTop(h, buttonH, with(density) { 16.dp.toPx() }, with(density) { 3.dp.toPx() })
|
||||
val raise = restRect?.let { (openTop + with(density) { 3.dp.toPx() } - buttonH / 2f) - it.top } ?: 0f
|
||||
travelPx[0] = (h - openTop).coerceAtLeast(1f)
|
||||
val pill = restRect?.translate(0f, raise)
|
||||
val chipsX = with(density) {
|
||||
if (narrow || pill == null) 16.dp.toPx() else pill.right + 20.dp.toPx() - inset
|
||||
}
|
||||
val chipWidth = with(density) { sheetChipWidth((w - 2 * inset - chipsX - 64.dp.toPx()).coerceAtLeast(0f).toDp(), tabs.size) }
|
||||
|
||||
val foot = h
|
||||
Box(Modifier.fillMaxSize().clipToBounds().graphicsLayer { val k = 1f - 0.05f * sink.value; scaleX = k; scaleY = k }) { background() }
|
||||
val wallSink = { maxOf(sink.value, drop?.sink?.value ?: 0f) }
|
||||
Box(Modifier.fillMaxSize().clipToBounds().graphicsLayer { val k = 1f - 0.05f * wallSink(); scaleX = k; scaleY = k }) { background() }
|
||||
scrim { sink.value }
|
||||
if (shown || open) Box(
|
||||
Modifier.fillMaxSize()
|
||||
@@ -432,7 +471,6 @@ internal fun LiftHero(
|
||||
val rimPath = remember { Path() }
|
||||
val corner = with(density) { SheetCorner.toPx() }
|
||||
val rim = with(density) { 1.4.dp.toPx() }
|
||||
val pill = restRect?.translate(0f, raise)
|
||||
Canvas(Modifier.fillMaxSize()) {
|
||||
val f = fold.value
|
||||
val ft = foldTop.value
|
||||
@@ -459,9 +497,6 @@ internal fun LiftHero(
|
||||
drawPath(rimPath, rimColor, style = Stroke(rim))
|
||||
}
|
||||
}
|
||||
val chipsX = with(density) {
|
||||
if (narrow || pill == null) 16.dp.toPx() else pill.right + 20.dp.toPx() - sheetLeft
|
||||
}
|
||||
val chipsY = with(density) { if (narrow && pill != null) (pill.bottom - openTop) + 10.dp.toPx() else ChipsTop.toPx() }
|
||||
val rowsY = with(density) { maxOf(RowsTop.toPx(), chipsY + 52.dp.toPx()) }
|
||||
val sheetW = sheetRight - sheetLeft
|
||||
@@ -537,13 +572,13 @@ internal fun LiftHero(
|
||||
}
|
||||
|
||||
Column(Modifier.align(Alignment.BottomStart).padding(start = start, bottom = bottom, end = 20.dp)) {
|
||||
Box(Modifier.graphicsLayer { alpha = 1f - sink.value }) { words() }
|
||||
Box(Modifier.graphicsLayer { alpha = 1f - wallSink() }) { words() }
|
||||
Spacer(Modifier.height(18.dp))
|
||||
Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(10.dp)) {
|
||||
Box(
|
||||
Modifier
|
||||
.onGloballyPositioned { rest = it.boundsInRoot() }
|
||||
.graphicsLayer { translationY = raise * ride.value }
|
||||
.graphicsLayer { translationY = raise * ride.value; alpha = 1f - (drop?.sink?.value ?: 0f) }
|
||||
.onFocusChanged {
|
||||
if (it.hasFocus && leftForPlay[0]) { leftForPlay[0] = false; if (openState.value && !launchingState.value) close() }
|
||||
playFocused = it.hasFocus
|
||||
@@ -570,7 +605,7 @@ internal fun LiftHero(
|
||||
val besideShown = !(open || shown)
|
||||
Row(
|
||||
verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(10.dp),
|
||||
modifier = Modifier.graphicsLayer { alpha = 1f - sink.value }
|
||||
modifier = Modifier.graphicsLayer { alpha = 1f - wallSink() }
|
||||
.focusProperties { canFocus = besideShown },
|
||||
) { if (besideShown || sink.value < 0.99f) beside() }
|
||||
}
|
||||
@@ -585,7 +620,7 @@ internal fun LiftHero(
|
||||
Modifier
|
||||
.offset { IntOffset((r0.center.x - handleW / 2f).roundToInt(), (r0.bottom + raise * ride.value + (lipShow.value - 1f) * 8.dp.toPx()).roundToInt()) }
|
||||
.onSizeChanged { handleW = it.width.toFloat() }
|
||||
.graphicsLayer { alpha = lipShow.value * (1f - lipGone.value) }
|
||||
.graphicsLayer { alpha = lipShow.value * (1f - lipGone.value) * (1f - (drop?.sink?.value ?: 0f)) }
|
||||
.heightIn(min = 44.dp).testTag("sheet-open")
|
||||
.semantics { contentDescription = settingsLabel }
|
||||
.then(
|
||||
@@ -599,14 +634,15 @@ internal fun LiftHero(
|
||||
}
|
||||
}
|
||||
}
|
||||
CompositionLocalProvider(LocalSheetChipWidth provides chipWidth, LocalPlayRect provides restRect) { overlay { sink.value } }
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun Handle(height: Dp) {
|
||||
internal fun Handle(height: Dp, above: Boolean = false) {
|
||||
val colors = MaterialTheme.colorScheme
|
||||
val pal = LocalPalette.current
|
||||
val shape = RoundedCornerShape(bottomStart = 12.dp, bottomEnd = 12.dp)
|
||||
val shape = if (above) RoundedCornerShape(topStart = 12.dp, topEnd = 12.dp) else RoundedCornerShape(bottomStart = 12.dp, bottomEnd = 12.dp)
|
||||
val still = Motion.scale == 0f
|
||||
val bob = androidx.compose.animation.core.rememberInfiniteTransition(label = "handle")
|
||||
val lift by bob.animateFloat(
|
||||
@@ -625,18 +661,18 @@ private fun Handle(height: Dp) {
|
||||
Row(
|
||||
verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(4.dp),
|
||||
modifier = Modifier.height(height).clip(shape).background(colors.surface)
|
||||
.drawBorderSidesAndFoot(pal.line, 12.dp)
|
||||
.drawBorderSidesAndFoot(pal.line, 12.dp, flip = above)
|
||||
.padding(start = 14.dp, end = 10.dp),
|
||||
) {
|
||||
Text(stringResource(R.string.drawer_page_settings), fontSize = 14.sp, fontWeight = FontWeight.SemiBold, color = colors.onSurfaceVariant, maxLines = 1)
|
||||
Icon(
|
||||
Icons.Filled.KeyboardArrowUp, null, tint = colors.onSurfaceVariant,
|
||||
modifier = Modifier.size(16.dp).graphicsLayer { translationY = if (still) 0f else -3.dp.toPx() * lift },
|
||||
if (above) Icons.Filled.KeyboardArrowDown else Icons.Filled.KeyboardArrowUp, null, tint = colors.onSurfaceVariant,
|
||||
modifier = Modifier.size(16.dp).graphicsLayer { translationY = if (still) 0f else (if (above) 3.dp else -3.dp).toPx() * lift },
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
private fun Modifier.drawBorderSidesAndFoot(color: Color, corner: Dp): Modifier =
|
||||
private fun Modifier.drawBorderSidesAndFoot(color: Color, corner: Dp, flip: Boolean = false): Modifier =
|
||||
this.drawWithContent {
|
||||
drawContent()
|
||||
val c = corner.toPx()
|
||||
@@ -649,30 +685,40 @@ private fun Modifier.drawBorderSidesAndFoot(color: Color, corner: Dp): Modifier
|
||||
arcTo(Rect(size.width - 2 * c, size.height - 2 * c + sw / 2, size.width - sw / 2, size.height - sw / 2), 90f, -90f, false)
|
||||
lineTo(size.width - sw / 2, 0f)
|
||||
}
|
||||
drawPath(p, color, style = Stroke(sw))
|
||||
if (flip) scale(1f, -1f) { drawPath(p, color, style = Stroke(sw)) } else drawPath(p, color, style = Stroke(sw))
|
||||
}
|
||||
|
||||
@OptIn(ExperimentalComposeUiApi::class)
|
||||
@Composable
|
||||
internal fun SheetChips(labels: List<String>, selected: Int, focus: List<FocusRequester>? = null, onPick: (Int) -> Unit) {
|
||||
internal fun SheetChips(
|
||||
labels: List<String>,
|
||||
selected: Int,
|
||||
focus: List<FocusRequester>? = null,
|
||||
width: Dp = Dp.Unspecified,
|
||||
onPick: (Int) -> Unit,
|
||||
) {
|
||||
val pal = LocalPalette.current
|
||||
val bounds = remember { mutableStateMapOf<Int, Rect>() }
|
||||
val tint = rememberUpdatedState(pal.signal)
|
||||
val chips = focus ?: remember(labels.size) { List(labels.size) { FocusRequester() } }
|
||||
var within by remember { mutableStateOf(false) }
|
||||
LaunchedEffect(selected) { if (within) runCatching { chips[selected].requestFocus() } }
|
||||
BoxWithConstraints(Modifier.fillMaxWidth().onFocusChanged { within = it.hasFocus }) {
|
||||
val equal = maxWidth / labels.size.coerceAtLeast(1) >= 96.dp
|
||||
BoxWithConstraints(
|
||||
Modifier.fillMaxWidth().onFocusChanged { within = it.hasFocus },
|
||||
contentAlignment = Alignment.CenterStart,
|
||||
) {
|
||||
val equal = !width.isSpecified && maxWidth / labels.size.coerceAtLeast(1) >= 96.dp
|
||||
val least = if (width.isSpecified) maxOf(width, 64.dp) else 64.dp
|
||||
val scroll = rememberScrollState()
|
||||
Box(if (equal) Modifier.fillMaxWidth() else Modifier.edgeFade(scroll).horizontalScroll(scroll).padding(3.dp)) {
|
||||
Row(
|
||||
horizontalArrangement = Arrangement.spacedBy(6.dp),
|
||||
horizontalArrangement = Arrangement.spacedBy(ChipGap),
|
||||
modifier = (if (equal) Modifier.fillMaxWidth() else Modifier)
|
||||
.focusProperties { enter = { chips.getOrNull(selected) ?: FocusRequester.Default } }
|
||||
.focusGroup(),
|
||||
) {
|
||||
labels.forEachIndexed { i, label ->
|
||||
val m = (if (equal) Modifier.weight(1f) else Modifier.widthIn(min = 64.dp))
|
||||
val m = (if (equal) Modifier.weight(1f) else Modifier.widthIn(min = least))
|
||||
.onGloballyPositioned { bounds[i] = it.boundsInParent() }
|
||||
.focusRequester(chips[i])
|
||||
SheetChip(label, i == selected, m.onFocusChanged { if (it.isFocused && i != selected) onPick(i) }) { onPick(i) }
|
||||
|
||||
@@ -2,6 +2,7 @@ package com.droiddeck.launcher.ui
|
||||
|
||||
internal enum class SessionDrawerPage(val firstControl: String) {
|
||||
CONTROLLER("touch"),
|
||||
GLASSES("glasses-game"),
|
||||
SECOND_SCREEN("second-screen:none"),
|
||||
DISPLAY("hud"),
|
||||
EFFECTS("look"),
|
||||
@@ -10,8 +11,10 @@ internal enum class SessionDrawerPage(val firstControl: String) {
|
||||
SESSION("suspend"),
|
||||
}
|
||||
|
||||
internal fun sessionDrawerPages(hasSecondScreen: Boolean): List<SessionDrawerPage> =
|
||||
SessionDrawerPage.entries.filter { hasSecondScreen || it != SessionDrawerPage.SECOND_SCREEN }
|
||||
internal fun sessionDrawerPages(hasSecondScreen: Boolean, hasGlasses: Boolean = false): List<SessionDrawerPage> =
|
||||
SessionDrawerPage.entries.filter {
|
||||
(hasSecondScreen || it != SessionDrawerPage.SECOND_SCREEN) && (hasGlasses || it != SessionDrawerPage.GLASSES)
|
||||
}
|
||||
|
||||
internal fun List<SessionDrawerPage>.step(current: SessionDrawerPage, direction: Int): SessionDrawerPage {
|
||||
val index = indexOf(current).coerceAtLeast(0)
|
||||
|
||||
@@ -149,6 +149,7 @@ import java.util.Locale
|
||||
|
||||
private val SessionDrawerPage.title: Int get() = when (this) {
|
||||
SessionDrawerPage.CONTROLLER -> R.string.drawer_page_controller
|
||||
SessionDrawerPage.GLASSES -> R.string.drawer_page_glasses
|
||||
SessionDrawerPage.SECOND_SCREEN -> R.string.drawer_second_screen
|
||||
SessionDrawerPage.DISPLAY -> R.string.drawer_page_display
|
||||
SessionDrawerPage.VR -> R.string.vr_title
|
||||
@@ -159,6 +160,7 @@ private val SessionDrawerPage.title: Int get() = when (this) {
|
||||
|
||||
private val SessionDrawerPage.icon get() = when (this) {
|
||||
SessionDrawerPage.CONTROLLER -> Icons.Outlined.SportsEsports
|
||||
SessionDrawerPage.GLASSES -> GlassesIcon
|
||||
SessionDrawerPage.SECOND_SCREEN -> Icons.Outlined.ScreenShare
|
||||
SessionDrawerPage.DISPLAY -> Icons.Outlined.DesktopWindows
|
||||
SessionDrawerPage.VR -> VrHeadsetIcon
|
||||
@@ -211,6 +213,8 @@ class DrawerActions(
|
||||
val cardboard: Cardboard = Cardboard.OFF,
|
||||
val cardboardSensor: Boolean = false,
|
||||
val steamVr: SteamVr.Status? = null,
|
||||
val glassesPresent: Boolean = false,
|
||||
val glassesOutput: Boolean = false,
|
||||
val immersiveSteamVr: Boolean = false,
|
||||
val textureAnisotropy: Int? = 0,
|
||||
val textureLodBias: String? = TextureFiltering.LOD_BIAS_OFF,
|
||||
@@ -247,6 +251,7 @@ class DrawerActions(
|
||||
val onEffects: (ScreenEffects) -> Unit = {},
|
||||
val onCardboard: (Cardboard) -> Unit = {},
|
||||
val onCardboardRecenter: () -> Unit = {},
|
||||
val onGlassesRecenter: () -> Unit = {},
|
||||
val onSteamVr: (String) -> Unit = {},
|
||||
val onTextureAnisotropy: (Int?) -> Unit = {},
|
||||
val onTextureLodBias: (String?) -> Unit = {},
|
||||
@@ -295,7 +300,7 @@ class DrawerActions(
|
||||
@OptIn(ExperimentalComposeUiApi::class)
|
||||
@Composable
|
||||
internal fun SessionDrawer(open: Boolean, requestedPage: SessionDrawerPage, controllerActive: Boolean, onPageChange: (SessionDrawerPage) -> Unit, a: DrawerActions) {
|
||||
val pages = sessionDrawerPages(a.steam && a.secondScreenDisplays.isNotEmpty())
|
||||
val pages = sessionDrawerPages(a.steam && a.secondScreenDisplays.isNotEmpty(), a.glassesPresent)
|
||||
val page = requestedPage.takeIf { it in pages } ?: SessionDrawerPage.CONTROLLER
|
||||
LaunchedEffect(requestedPage, pages) { if (page != requestedPage) onPageChange(page) }
|
||||
val colors = MaterialTheme.colorScheme
|
||||
@@ -465,7 +470,7 @@ internal fun SessionDrawer(open: Boolean, requestedPage: SessionDrawerPage, cont
|
||||
}
|
||||
SessionDrawerPage.VR -> {
|
||||
SteamVrGroup(a) { key -> focus.track(page, key) }
|
||||
CardboardGroup(host, a) { key -> focus.track(page, key) }
|
||||
if (!a.glassesOutput) CardboardGroup(host, a) { key -> focus.track(page, key) }
|
||||
}
|
||||
SessionDrawerPage.CONTROLLER -> {
|
||||
SettingsGroup(stringResource(R.string.drawer_page_controller)) {
|
||||
@@ -532,6 +537,12 @@ internal fun SessionDrawer(open: Boolean, requestedPage: SessionDrawerPage, cont
|
||||
SessionDrawerPage.SECOND_SCREEN -> {
|
||||
SecondScreenGroup(host, a) { key -> focus.track(page, key) }
|
||||
}
|
||||
SessionDrawerPage.GLASSES -> {
|
||||
val (g, s) = rememberGlasses()
|
||||
GlassesGroups(host, g, s, { key -> focus.track(page, key) }, onRecenter = a.onGlassesRecenter,
|
||||
outputActive = a.glassesOutput, steamVrRunning = a.steamVr == SteamVr.Status.RUNNING,
|
||||
steamVr = { SteamVrRow(a) })
|
||||
}
|
||||
SessionDrawerPage.GAMES -> {
|
||||
if (a.steam) SelectedGameProfileGroup(host, a) { key -> focus.track(page, key) }
|
||||
Text(stringResource(R.string.fex_next_launch), fontSize = 12.sp,
|
||||
@@ -954,6 +965,19 @@ private fun SteamVrGroup(a: DrawerActions, track: (String) -> Modifier) {
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun SteamVrRow(a: DrawerActions) {
|
||||
val vrLabel = stringResource(R.string.cardboard_steamvr)
|
||||
when (a.steamVr) {
|
||||
SteamVr.Status.NOT_INSTALLED -> ActionRow(vrLabel, null, stringResource(R.string.cardboard_steamvr_install), { a.onSteamVr("install") })
|
||||
SteamVr.Status.INSTALLING -> SettingsRow(vrLabel, stringResource(R.string.cardboard_steamvr_installing)) {}
|
||||
SteamVr.Status.READY -> ActionRow(vrLabel, null, stringResource(R.string.cardboard_steamvr_start), { a.onSteamVr("start") })
|
||||
SteamVr.Status.RUNNING -> ActionRow(vrLabel, null, stringResource(R.string.cardboard_steamvr_stop), { a.onSteamVr("stop") })
|
||||
SteamVr.Status.UNAVAILABLE -> SettingsRow(vrLabel, stringResource(R.string.cardboard_steamvr_unavailable)) {}
|
||||
null -> {}
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun SecondScreenGroup(host: MenuHost, a: DrawerActions, track: (String) -> Modifier) {
|
||||
val colors = MaterialTheme.colorScheme
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package com.droiddeck.launcher.ui
|
||||
|
||||
import androidx.compose.ui.platform.testTag
|
||||
import androidx.compose.foundation.layout.fillMaxWidth
|
||||
import com.droiddeck.launcher.R
|
||||
import androidx.compose.ui.res.stringResource
|
||||
import androidx.compose.animation.core.FastOutSlowInEasing
|
||||
@@ -104,8 +105,24 @@ internal fun SteamHome(s: FrontEndState, a: FrontEndActions, modifier: Modifier)
|
||||
}
|
||||
},
|
||||
)
|
||||
val (glasses, glassesSettings) = rememberGlasses()
|
||||
val headset = glasses.present && glasses.canControl
|
||||
val drop = rememberDropSheetState()
|
||||
LaunchedEffect(headset) { if (!headset && (drop.open || drop.shown)) drop.reset() }
|
||||
LiftHero(
|
||||
key = "steam", tabs = tabs,
|
||||
drop = drop.takeIf { headset },
|
||||
overlay = { lifted ->
|
||||
if (headset) DropSheet(
|
||||
drop, glassesTabs(glasses), covered = lifted,
|
||||
button = { enabled ->
|
||||
PrimaryButton(stringResource(R.string.games_play_steamvr), enabled = !s.busy && glasses.usable, large = true, icon = Icons.Filled.PlayArrow, fill = true,
|
||||
modifier = Modifier.fillMaxWidth().testTag("play-steamvr")) { if (enabled) { drop.open = false; a.onPlaySteamVr() } }
|
||||
},
|
||||
) { tab ->
|
||||
GlassesGroups(rememberMenuHost(), glasses, glassesSettings, inSession = false, tab = tab)
|
||||
}
|
||||
},
|
||||
onOpen = a.onSteamSettings, onPrimary = a.onPlay,
|
||||
modifier = modifier.clipToBounds(),
|
||||
start = if (narrow) 20.dp else 44.dp, bottom = if (narrow) 20.dp else 44.dp,
|
||||
|
||||
@@ -41,7 +41,6 @@ object CompositorHost {
|
||||
resumeVsync()
|
||||
return false
|
||||
}
|
||||
WaylandCompositor.nativeSetOutputRefreshRate(refreshHz)
|
||||
com.droiddeck.launcher.gpu.TpUbwcHint.applyToProcess()
|
||||
WaylandCompositor.nativeStartWithSurface(
|
||||
surface, xdgRuntimeDir, driverPath, libraryName, nativeLibDir,
|
||||
@@ -55,6 +54,13 @@ object CompositorHost {
|
||||
private fun pace(refreshHz: Float, fpsLimit: Int) {
|
||||
WaylandCompositor.nativeSetFpsLimit(fpsLimit)
|
||||
WaylandCompositor.nativeSetLayerFrameRate(if (fpsLimit > 0) fpsLimit.toFloat() else refreshHz)
|
||||
WaylandCompositor.nativeSetOutputRefreshRate(refreshHz)
|
||||
SessionState.refreshHz = refreshHz
|
||||
}
|
||||
|
||||
@Synchronized
|
||||
fun updatePacing(refreshHz: Float, fpsLimit: Int) {
|
||||
if (started) pace(refreshHz, fpsLimit)
|
||||
}
|
||||
|
||||
@Synchronized
|
||||
|
||||
@@ -20,8 +20,8 @@ class HdrProbe(
|
||||
)
|
||||
|
||||
object HdrSupport {
|
||||
fun probe(context: Context): HdrProbe {
|
||||
val display: Display? = if (Build.VERSION.SDK_INT >= 30) {
|
||||
fun probe(context: Context, output: Display? = null): HdrProbe {
|
||||
val display: Display? = output ?: if (Build.VERSION.SDK_INT >= 30) {
|
||||
try { context.display } catch (e: Exception) { null }
|
||||
} else {
|
||||
@Suppress("DEPRECATION")
|
||||
|
||||
@@ -273,7 +273,7 @@ public final class WaylandCompositor {
|
||||
|
||||
public static native void nativeSetCardboard(boolean enabled, int source, float ipdM, float screenDistM,
|
||||
float screenWidthM, float k1, float k2, float fovDeg,
|
||||
float lensInset, float chroma);
|
||||
float lensInset, float chroma, boolean mono, float displayHz);
|
||||
|
||||
public static native void nativeSetCardboardPose(float[] rotation, boolean live);
|
||||
|
||||
|
||||
Reference in new issue
Block a user