diff --git a/docs/development/unified-input-proposal.md b/docs/development/unified-input-proposal.md index 76ee629f0..b64056a1e 100644 --- a/docs/development/unified-input-proposal.md +++ b/docs/development/unified-input-proposal.md @@ -1,322 +1,213 @@ -# Unified input configuration and routing proposal +# Shared input profiles and configurable control surfaces -Status: implemented locally for device evaluation; not published or approved by maintainers. -Prepared on 2026-10-23 against `ac0909ea`. -Related source snapshots: PR #1202 at `42db788f`, PR #1305 at `76fc9b31`. +This change implements a shared routing model for physical controllers and local +screen controls. It is intended for review of the architectural direction, with +a functional configuration UI and hardware validation on AYN Thor. The related +maintainer discussions motivate this work; they are not approval of this +implementation. -## Problem and scope +The configuration UI is intentionally basic. If maintainers accept the +architecture, I am happy to refine the UX/UI in the same PR before merge, or in a +follow-up PR, whichever they prefer. Navigation, binding discovery, layout +editing, accessibility and localization need further attention. The current UI +makes the architecture usable and testable rather than establishing its final +interaction design. -DroidDeck has useful input paths whose configuration and lifecycle rules are -spread across several components. Desktop users need configurable physical -controller bindings. Dual-screen users also need to arrange and configure lower -screen controls without giving up Steam Deck trackpads or Steam Input layouts. +## Why a shared system -Provide one configuration model and reusable action handling while preserving -the existing low-latency transports and Steam's ownership of Steam-managed -bindings. A common model does not require all input to cross a new queue, UI -thread, generic event bus, or virtual device. +Desktop sessions need mouse, keyboard and application actions from a controller. +A dual-screen device may already have a comfortable lower-screen mouse, leaving +its physical sticks available for other actions. The same logical button or pad +can appear on the main screen, lower screen or a physical device; its location +should not dictate which actions it can perform. -The implementation has two editable built-in profiles, Game and -Desktop. Named profile creation, per-application selection, and profile -import/export are future work; the stored model must allow them without changing -the routing contract. +The model separates three concerns: -## Existing paths to preserve +- **Source role:** physical, primary screen or secondary screen. +- **Profile:** the actions assigned to controls in each role. Game and Desktop are + the two editable built-in profiles. Auto selects by session mode; explicit + selection supports a game running inside a desktop session. +- **Surface layout:** the placement and size of local screen controls. The custom + second-screen layout is stored independently for each built-in profile. -| Source | Current path | Meaning | -| --- | --- | --- | -| Android physical pad | `PadBridge` -> `FakeInputWriter` | Gamepad buttons, axes, player slots | -| Built-in screen pad, including the lower-screen Gamepad tab | `PadBridge.applyTouch` -> `FakeInputWriter` | Virtual gamepad input | -| Lower-screen Deck trackpads and back grips | `DeckControlsPanel` -> `DeckControls` -> `FakeInputWriter.writeDeckControls` | Touch coordinates, contact, click/pressure, L4/L5/R4/R5 | -| Steam touch overlay | `SteamTouchControls` -> `SteamTouchDevice` -> Steam | Steam Mobile Touch reports, with Steam-owned bindings and action sets | -| Desktop trackpad and keyboard | Existing gesture/keyboard handlers -> compositor | Direct guest mouse and keyboard input | -| Steam Controller BLE and proposed native DualSense | Driver-specific report paths | Input can bypass Android `KeyEvent` dispatch | +A binding selects native controller output, a DroidDeck action, or disabled. +Native output continues through the existing game/Steam path, optionally with a +compatible gamepad remap. A local action consumes that control so, for example, +R3 assigned to Ctrl+F does not also press the guest's R3 button. -Deck trackpads must not be reduced to mouse deltas or a pair of ordinary joystick -axes. Their contact, coordinates, click state and pressure are meaningful to -Steam. Likewise, Steam Mobile Touch is a distinct protocol, not another spelling -of an Xbox pad. +```mermaid +flowchart TD + Physical[Android controller events] --> PadBridge + Primary[Primary on-screen controls] --> PadBridge + Secondary[Secondary custom controls] --> PadBridge + Deck[Deck pads and grips] --> DeckControls + BLE[Steam Controller BLE reports] --> TritonInputRouter + Profiles[Game / Desktop profile by source role] --> Router[InputRouter] + PadBridge --> Router + DeckControls --> Router + TritonInputRouter --> Router + Router --> Native[Existing pad / Deck report publication] + Router --> Held[Shared key and mouse ownership] + Router --> Commands[Existing menu and keyboard hosts] + Router --> Motion[InputMotion: active stick pointer / scroll only] + Motion --> Held + Held --> Guest[Existing guest keyboard and mouse output] + Native --> Steam[Steam Input or guest controller consumer] + Layout[Independent surface geometry] --> Secondary +``` -See [Steam touch controls](steam-touch-controls.md) for the current integration, -including Steam-owned layouts, action sets and cloud configuration. +These are routing relationships, not additional asynchronous queues. Native +publication is performed by the existing source adapters and writers. -## Separate bindings, screen layout, and device calibration - -The proposed model has three independent parts: - -- **Binding profile:** logical input sources and their actions, with Game and - Desktop defaults and user overrides. -- **Surface layout:** which controls appear on the primary or secondary control - surface, their positions, sizes and presentation. A layout references bindings - or Steam-managed controls; it does not copy Steam's action-set definitions. -- **Device configuration:** hardware identification, axis normalization, - calibration and capabilities. This is separate from a user's desktop layout. - -Use logical surface roles rather than persistent Android display IDs. Controls -need stable IDs independent of their location. Moving a control to the lower -screen must not change its action or create a new guest gamepad. The existing -player-slot and device identities remain authoritative. - -A Game profile can use the lower screen for two Deck trackpads and back grips, -plus selected DroidDeck actions. A Desktop profile can put a mouse trackpad and -keyboard there while mapping the physical sticks to arrow keys, scrolling or -other actions. Connecting a second display must not silently rewrite physical -controller bindings. - -The model should support reusable lower-screen layouts later. Initially expose -existing layouts and a bounded set of configurable controls; a complete freeform -multi-display editor is a separate deliverable. - -## Steam Input ownership - -Each binding explicitly selects an execution path: - -1. **Native/Steam-managed:** send the existing controller or touch report. - Steam remains responsible for game bindings, action sets, layers and its - touch/radial menus. DroidDeck shows an appropriate entry into Steam's editor. -2. **DroidDeck-managed:** execute a guest key combination, mouse action or - application action through the existing output functions. The consumed - control is excluded from the normal controller output. -3. **Disabled:** intentionally produce no action. - -Steam-managed and DroidDeck-managed controls can coexist on one surface. One -activation must have one semantic owner, so a local keyboard shortcut does not -also produce the original controller button. Gamepad-to-gamepad remaps remain -explicit preprocessing before the normal game/Steam path, rather than a second -independent action sent alongside it. - -Steam Input already supports extensive binding and touch-menu behavior. This -proposal concentrates DroidDeck's work on Android surfaces, app shortcuts, -desktop/non-Steam bindings and consistent lifecycle handling. It does not propose -reimplementing Steam's configuration engine or importing its profiles into a -second authoritative store. - -Preserve raw-report fields and output/feedback channels not affected by a local -binding, including motion and haptics. Driver-specific adapters must participate -before publishing consumed controls; an Activity-only interceptor cannot cover -raw HID devices. - -## Configuration and runtime model - -Responsibilities: +## Implementation | Component | Responsibility | | --- | --- | -| `InputProfile` | Version, stable ID, default role, bindings and surface-layout references | -| `SurfaceLayout` | Stable control IDs, display role, geometry and appearance | -| `InputBinding` | Source selector, activation rule, action and execution owner | -| Source adapters | Preserve device identity, normalize inputs and expose capabilities | -| Compiled binding plan | Pre-resolved routes for one active profile and its devices | -| Action executor/output ownership | Execute existing outputs and track held contributions | +| `InputProfile` | Logical controls, source roles and supported actions | +| `InputProfiles` | Versioned persistence, selection, validation and migration | +| `InputSurfaceLayout` | Bounded custom lower-screen geometry, separate from actions | +| `InputRouter` | Resolve channels on profile changes, route events, track held outputs | +| `PadBridge` | Existing player slots plus independently owned local touch sources | +| `DeckControls` | Source ownership for Deck grips, contacts, coordinates and clicks | +| `TritonInputRouter` | Apply routing before Steam Controller BLE report publication | +| `PhysicalInputBindings` | Capture additional Android keys with device identity | +| `HeldInput` | Coordinate router outputs with existing keyboard and gesture outputs | +| `InputMotion` | Frame callback for active synthesized stick pointer/scroll movement | +| `InputTiming` | Optional bounded dispatch-to-publication diagnostics | -Buttons and axis directions can produce keys, key combinations, mouse buttons, -gamepad buttons or app actions. Sticks can retain their gamepad role or produce -pointer velocity, scrolling or keyboard directions. Analog triggers need distinct -press/release thresholds when used as buttons. Capability-aware controls should -explain why an unavailable action cannot run, rather than silently substituting -another behavior. +Supported actions are native output, compatible native remapping, disabled, +keyboard keys/combinations, mouse buttons, pointer movement, scrolling, four +keyboard directions, and existing menu/keyboard/Steam/quick-access commands. +Analog controls used as buttons have separate activation and release thresholds. +Key combinations press modifiers first and release in reverse order. -The same action picker is reused for physical and local screen controls. Screen -geometry stays outside binding execution. Steam-managed widgets retain their -native configuration and protocol adapters. +Standard controller assignments are shared by the physical role. Additional +captured keys match a device identity plus Android keycode/scancode, rather than +persisting a transient Android device ID. System-reserved keys remain with +Android. This is not a per-device calibration system. -Configuration is loaded, validated and compiled when it changes. Dispatch uses -an immutable plan snapshot and preallocated per-source state. JSON decoding, -preference reads, disk writes and layout traversal do not belong in event dispatch. -Use stable symbolic control/action identifiers in storage; transient Android -device IDs are runtime keys, not portable profile identifiers. +### Reuse and migration -Profile selection defaults to the session's mode, with an explicit session -override. Desktop mode can contain a game, so it is not sufficient evidence for -automatic keyboard/mouse interception. An optional profile-switch action uses -the same app-action mechanism as menus and keyboard display. +Existing `FakeInputWriter` transports, player assignment, guest keyboard/mouse +output, `KeyboardHost`, second-screen presentation and `DeckControlsPanel` are +reused. The custom lower-screen editor arranges the existing buttons, sticks, +trackpads and grips; it does not create another emulated controller. -## Ownership and lifecycle +The old touch-only mapping and Select-as-Steam settings migrate once into Game's +primary-screen and physical bindings respectively. Their redundant UI is removed. +Invalid stored entries fall back to defaults. The global settings reset clears +both profile and surface settings without reimporting the old mappings. -- Track held keys and mouse buttons by source. Releasing one source must not - release a key or button another source still owns. -- Cancel only a disappearing surface's or disconnected device's contributions; - avoid a global release that also clears another screen or controller. -- Define arbitration for multiple sources targeting the same analog control or - Deck touch surface. Retain ownership through a contact; do not let concurrent - views overwrite each other's reports or add a second virtual pad implicitly. -- Changing a binding/profile releases its old outputs before installing the new - route. Held controls must return to neutral before triggering a new binding. -- Menus, keyboard capture, pause, focus loss and device removal have explicit - routing rules. Menu navigation retains working confirmation, cancellation and - page controls even when those buttons have gameplay bindings. -- Keep system-reserved Android inputs with the system and retain touchscreen - access to reset a problematic controller assignment. +The shared PC-key definitions are adapted from kingokksa's +[#1305](https://github.com/Droid-Deck/DroidDeck/pull/1305) at `76fc9b31`. +Device capture and identity handling build on the approach in jmarti326's +[#1202](https://github.com/Droid-Deck/DroidDeck/pull/1202) at `42db788f`. +Neither PR is merged wholesale; their broader UI and device behavior should be +coordinated with their authors. -Button chords require a deliberate policy. A dedicated modifier can select a -chord without delaying unrelated controls. A button that also has a single-press -action may require deferred dispatch to disambiguate a chord, hold or double tap. -That delay must be opt-in, visible in configuration and limited to the affected -binding. Do not buffer all controller presses to support optional chords. +### Ownership and lifecycle -## Latency requirements and verification +Each source retains its own contribution. Releasing a physical mouse button does +not release an identical button still held on the lower screen. Closing a surface +releases that surface, including the final release on the existing Epic launcher +mouse route. Existing gesture and keyboard outputs participate in shared key and +mouse-button ownership. -The goal is no additional scheduling stage for normal input, not an unmeasured -claim of zero added latency. +Native pad merging uses OR for buttons, the maximum trigger value, and the +strongest complete stick vector. A native Deck trackpad retains one source's +coordinates until that contact ends, then accepts a waiting contact. Concurrent +surfaces do not overwrite its coordinates during the same contact. -- Identity routes retain the existing direct publication path and report format. -- Ordinary buttons and touch reports must not wait for drawing, Compose state, - a new coroutine, a timer, an event-bus subscriber or another frame. -- Add no hot-path allocations for ordinary binding lookup. Reuse buffers and - keep critical sections short. The current router serializes routing and profile - changes under one monitor to keep shared output ownership consistent; hardware - traces must establish whether this causes contention before publication. -- Continuous stick-to-pointer movement needs a clock even when no new stick - event arrives. Keep that scheduler limited to the active synthesized movement - route; stop it at neutral. It must not schedule ordinary gamepad forwarding. -- Keep existing synchronization and low-latency flags intact until measurements - justify a separate change. Do not weaken shared-memory publication ordering. -- Send UI updates separately from input publication. Marshaling app-menu work to - Android's UI thread must not marshal ordinary controller output with it. +Profile changes, suspension and menu entry release old local outputs. Held +controls must return to neutral before activating their new bindings. Device +removal and surface disposal remove their owned state. The existing menu +navigation path remains available independently of gameplay assignments. -Before publishing the change, record a baseline on Thor with the same build -configuration, controller mode, display refresh rates and low-latency settings. -Compare identity forwarding, a local key binding, stick-to-pointer, Deck trackpad -input and simultaneous upper/lower-screen input under both idle and rendering -load. Measure p50/p95/p99 dispatch-to-publication time, allocations and contention. -Use debug-only sampling or buffered tracing; avoid per-event log writes. +The menu and keyboard commands currently open their respective UI; they are not +toggles. Back/B closes the menu. This can be reconsidered with the UX work. -Android-to-publication measurements do not measure complete button-to-photon -latency. Guest receipt and rendered response need separate measurement, with -clock-domain alignment for cross-process traces or an external high-speed-camera -test. Record benchmark conditions and uncertainty; establish numerical acceptance -budgets from the baseline before presenting latency claims in a PR. +### Hot path and scheduling -Hard acceptance conditions include no new frame/timer wait for identity routes, -no duplicate or lost press/release transitions, and no degradation of current -Deck trackpad behavior. Microbenchmarks alone cannot establish those conditions. +Profile JSON, preferences and layout geometry are processed outside event +routing. Channels resolve their actions when configuration changes. The current +router uses mutable per-source state under one synchronized monitor; it is not an +immutable compiled plan. This keeps shared ownership consistent, but contention +with simultaneous BLE input has not yet been measured on hardware. -## Reuse, migration, and removal +For an unchanged Game binding, `PadSource` returns the original `PadState`. +The BLE identity route returns its original report array. Modified reports use +reusable buffers and preserve unrelated timestamp, motion, pressure and extension +fields. Normal controller events do not wait for a frame, coroutine or timer. -- [#1202](https://github.com/Droid-Deck/DroidDeck/pull/1202): reuse physical capture, - device identification and menu-action UI where suitable. Generalize the - execution contract beyond Activity `KeyEvent` interception. -- [#1305](https://github.com/Droid-Deck/DroidDeck/pull/1305): reuse PC-key definitions, - key selection and screen-control presentation. Adapt keyboard output to shared - ownership; its screen-layout format is not a complete physical-input profile. -- [#581](https://github.com/Droid-Deck/DroidDeck/pull/581): coordinate device - calibration/profile matching, independently of user binding/surface profiles. -- [#1226](https://github.com/Droid-Deck/DroidDeck/pull/1226): use a shared keyboard - action with the appropriate existing host/display, rather than hard-coding a - second keyboard implementation. -- Reuse current guest keyboard/mouse output, `PadBridge`, Deck/Steam touch - transports and the existing second-screen presentation lifecycle. +Only synthesized stick-to-pointer/scroll motion uses `Choreographer`, because a +held stick must keep moving without new device events. Trackpad deltas and button +changes publish synchronously on the normal UI input path. Pointer output arriving +from an off-main-thread adapter is marshalled to the existing UI cursor path; +application commands also use the main handler. Neither schedules native forwarding. -Remove duplicated settings, capture UI and action switches only after their -replacement preserves saved settings and passes behavior tests. Do not remove -working device adapters, Steam touch support or native transport code merely to -make the architecture diagram more uniform. Stage each migration so it can be -reviewed and reverted independently. +The mapped cursor reuses the existing compositor coordinates and Android cursor +layer. Trackpad contact starts from the current pointer position, preventing +jumps when switching sources. Publishing pending Compose snapshot notifications +before drawing avoids the observed every-other-frame cursor update without +changing the stick response curve or adding smoothing. -## Implemented local evaluation build +See [validation](unified-input-validation.md) for measurements and their limits. +The evidence supports no clear typical latency regression in the tested native +path, not zero overhead or a complete input-to-display latency guarantee. -No PR is created yet. Test the complete integrated change on Thor before deciding -whether to publish it. The intended framing is **Shared input profiles and -configurable control surfaces**, with links to the desktop requests -[#860](https://github.com/Droid-Deck/DroidDeck/issues/860) and -[#1356](https://github.com/Droid-Deck/DroidDeck/issues/1356), the centralization -discussion in #1202, and the related work above. Those discussions are context, -not maintainer approval of this implementation. +## Steam ownership and boundaries -### Runtime and configuration +Steam remains responsible for game mappings, action sets, layers, radial menus +and its cloud profiles. Native/Steam-managed here means forwarding the controller +through the existing path; it is not a new integration with Steam's profile API. +Steam Mobile Touch retains its own Steam-managed layout and protocol. Its widgets +do not pass through this local remapper. The VR-specific input path also remains +separate. Existing gestures retain their recognizers and share output ownership. -- `InputProfile` separates actions and source roles from `InputSurfaceLayout`. - `InputProfiles` stores a versioned document and migrates legacy touch mappings - and Select-as-Steam into Game once. Editing one role does not edit another. -- `InputRouter` resolves actions when a profile changes. `PadSource` returns the - original `PadState` on the identity path. Other routes reuse their output - buffers and consume only the selected control. A key combination presses - modifiers first and releases in reverse order. Trigger/direction hysteresis - uses separate press and release thresholds. -- `PadBridge` preserves physical player slots and separately owns each touch - surface. Merging uses OR for buttons, maximum trigger values and the strongest - complete stick vector. Surface removal cannot release another source's hold. -- `DeckControls` keeps per-source contacts, coordinates, clicks and pressure. - Default native actions retain the existing Deck report. Local trackpad actions - use contact-relative deltas, avoiding a jump at first contact. -- `TritonInputRouter` routes Steam Controller BLE reports before publication. Its - identity path returns the original byte array. Other paths preserve packet, - timestamp, inertial data, grip-contact bits, report extensions and pressure. - This adapter is covered by synthetic report tests; physical BLE testing is - still required. -- `PhysicalInputBindings` captures device identity, keycode and scancode, including - controller buttons exposed as keyboard keys. It consumes a configured local - action before normal dispatch and keeps releases paired across profile edits. - Android volume and system keys remain reserved. -- `HeldInput` coordinates local mappings with existing guest keyboard, gesture, - mouse and second-screen outputs. `InputMotion` schedules only synthesized - stick movement/scrolling. Buttons and trackpad deltas publish synchronously; - UI commands use the existing main-thread handler. -- Profile changes, suspension and menu entry release local outputs and require - held inputs to return to neutral before activating their new actions. +The two built-in profiles and the bounded custom lower-screen editor are the +implemented scope. Named profile libraries, import/export, per-app selection, +macros, tap/hold behavior, multi-button activation chords and a complete freeform +multi-display editor are deferred. One button emitting Ctrl+F is supported; +pressing multiple buttons to activate a new binding is not implemented. -### User interface +Native remaps stay within compatible report families. Additional hardware +adapters must join before publishing consumed controls; an Activity-only +interceptor cannot cover every raw device. Future adapters should use this +contract without replacing device-specific calibration or protocol handling. -Input profiles are available from Controller setup and the session's Controller -panel. Auto selects Desktop for a desktop session and Game for other sessions; -manual selection handles games launched from a desktop without guessing from -window focus. The editor exposes physical, primary-screen and secondary-screen -roles, native/remapped gamepad output, local actions and disabled bindings. +## Related upstream work -Custom controls is an additional second-screen mode. Its editor arranges a -bounded set of existing `DeckControlsPanel` buttons, sticks, trackpads and grips. -Layout geometry is independent of bindings and saved per built-in profile. -Android keyboard input reuses `KeyboardHost`; the existing keyboard/trackpad -and Deck panel modes remain available. Shared PC-key definitions are taken from -PR #1305 at `76fc9b31`; capture/identity handling follows PR #1202 at `42db788f`. -Neither PR is merged wholesale. The superseded touch-only mapping page and -Select-as-Steam UI toggle are removed after migration. +Reviewed on 2026-10-24. Several related proposals remain open; this is not a claim +that no other controller work exists or that this implementation supersedes it. -### Boundaries and deferred work +- [#1202](https://github.com/Droid-Deck/DroidDeck/pull/1202): physical shortcuts and + capture. The maintainer's + [centralization feedback](https://github.com/Droid-Deck/DroidDeck/pull/1202#issuecomment-6299647180) + motivates one shared assignment model. Questions there about overlapping + actions inform consumption/ownership; multi-button activation remains deferred. +- [#1305](https://github.com/Droid-Deck/DroidDeck/pull/1305): screen key mappings, + combinations and layout management. This change shares the action model across + physical and screen sources, but does not implement that PR's full layout UX. +- [#1414](https://github.com/Droid-Deck/DroidDeck/pull/1414): external-controller + button and touchpad-press capture, including devices exposing a touchpad press + as pointer input. Our key capture does not yet implement that pointer-contact + adapter. It is a concrete area for coordination. +- [#860](https://github.com/Droid-Deck/DroidDeck/issues/860) and + [#1356](https://github.com/Droid-Deck/DroidDeck/issues/1356): desktop controller + input. This implements controller-driven pointer/keyboard actions, not every + gesture or automatic keyboard behavior requested in #1356. +- [#581](https://github.com/Droid-Deck/DroidDeck/pull/581): hardware calibration and + device profiles remain a separate responsibility from action profiles. +- [#1226](https://github.com/Droid-Deck/DroidDeck/pull/1226): keyboard host selection; + the local action reuses the existing host rather than implementing another IME. +- [#1150](https://github.com/Droid-Deck/DroidDeck/pull/1150): raw DualSense input. + That proposed adapter will need integration before publication; it is not + covered merely by the current Android key capture. -Steam Mobile Touch retains its Steam-owned action sets, layouts and cloud data; -its controls are edited in Steam. It does not pass through DroidDeck's local -remapper. Existing gesture recognition and VR controller handling are retained; -gestures share output ownership, while the VR-specific input path remains owned -by VR. Native remaps are limited to compatible controls within the same report -family. An Android physical key can retain its original behavior or run a local -action; capture does not synthesize an unrelated hardware device. +## Review focus -Named profile libraries, importing/exporting profiles, per-app switching, -reusable layout presets, macros, tap/hold/chord activation and a freeform -multi-display drag editor remain deferred. Key combinations emitted by one -binding are supported; multi-button activation chords are not implemented. -Hardware calibration stays in the existing adapters. No native transport, -Steam haptics path or device driver is replaced. - -### Validation and Thor procedure - -Before changes, the existing controller/PadMouse/on-screen/second-screen tests -passed. The integrated suite exercises routing, shared holds, key order, -hysteresis, neutral gating, profile migration/persistence, raw report preservation, -existing input paths and the Compose binding/layout editors. See the local -Gradle test report for exact results. - -1. Install the local test APK alongside the signed public app; keep runtime data - and baseline configuration available. Do not replace the official signing key. -2. In a Desktop session, test pointer/scroll, mouse buttons, arrow keys, arbitrary - key combinations and captured hardware back buttons. Open and close the drawer - and keyboard with controls held; verify every guest release. -3. Select Game and test Factorio plus Steam UI, first with the default native - profile, then with individual bindings changed to local or disabled. Repeat - with Xbox 360 and Steam Deck virtual-controller modes. -4. Exercise keyboard/trackpad, Deck panels and Custom controls on the lower screen. - Hold the same output from two sources, remove a surface, change profiles while - holding sticks/buttons and reconnect the controller/display. -5. Enable Input diagnostics, close the editor, generate input under idle and - rendering load, then reopen and read p50/p95/p99. Recording is bounded to the - most recent 4096 samples and disabled by default. `InputTiming` measures dispatch - through existing transport publication, including routing. It does not measure - full button-to-photon latency. Use the same instrumentation on a baseline build - before making comparative claims. -6. Record refresh rates, controller/transport mode, build type and low-latency - flags. Compare raw identity, local key, pointer and native Deck routes. Use - external high-speed video or aligned guest traces for end-to-end measurement. - -A passing host test suite does not establish latency on Thor or physical BLE -compatibility. Those remain explicit release gates before a public PR. +The main questions are whether source roles and profile boundaries match the +project's direction, whether adapters join at the right point, and whether the +shared ownership/lifecycle contract is suitable for future input sources. The +validation document distinguishes device-tested behavior, host regressions and +remaining coverage. UX/UI refinement can follow agreement on those decisions. diff --git a/docs/development/unified-input-validation.md b/docs/development/unified-input-validation.md new file mode 100644 index 000000000..0398cdaa4 --- /dev/null +++ b/docs/development/unified-input-validation.md @@ -0,0 +1,189 @@ +# Unified input validation + +Recorded on 2026-10-24. These results distinguish the device-tested candidate from +the later upstream rebase and host regression fixes. They do not establish zero +latency or complete coverage of every controller and session mode. + +## Revisions and environment + +The Thor candidate was `5021e7ac` on base `ac0909ea`, with the local temporary-file, +cursor visibility/handoff/frame cadence, scroll direction and button label fixes +subsequently committed on this branch. Its final APK SHA-256 was +`bb211095d6443e9f85ed1f5d8c5674f527df2937ac6b0efe7d6fc4adcb9dcf62`. + +The branch was then rebased onto upstream `906e0d1f`. Review added regressions and +fixes for final touch-source release, concurrent native trackpad contact ownership, +and resetting the new preferences. Those final changes and the rebase have host +test coverage; the installed Thor APK was not silently replaced and those exact +changes have not yet been revalidated on the device. + +Local Android builds use Java 17 and NDK `28.2.13676358`. The Thor test APK used a +local Gradle init script to install alongside the official app; no application ID +or app-name changes are part of this PR. Local builds use `-PskipRust=true`, so +Steam/Desktop checks do not establish the behavior of the Epic/GOG/Amazon store +engines in a complete release build. + +## Automated checks + +On the rebased implementation, the complete `:app:testDebugUnitTest` run passed: +**2,230 tests across 333 suites, zero failures/errors/skips**. + +The supported `tools/build_local.sh` debug build also passed, including session +asset matching, runtime-input attestation, native export/dependency validation, +ZIP alignment and signature verification. The APK SHA-256 was +`a7d440b4029f9ad67ca6f44c92ccdd6b9eba9e9a758b28ab0e432256697d49b9`. +The only local environment repair was moving a Finder `.DS_Store` file out of the +NDK's host-toolchain directory; the upstream script counted it as another host. +No build checks were disabled to get past the stale-runtime check after rebasing. + +Commands, with Java/SDK paths configured for the local machine: + +```sh +./gradlew :app:testDebugUnitTest -PskipRust=true -PndkVersion=28.2.13676358 +DROIDDECK_BUILD_VARIANT=debug DROIDDECK_SKIP_RUST=1 \ + DROIDDECK_NDK_VERSION=28.2.13676358 bash tools/build_local.sh +# Linux with gcc and g++ available: +python3 -m unittest tools.tests.test_fakeinput_open -v +``` + +Coverage includes profile persistence and migration, invalid bindings, shared +holds, key order, trigger/direction hysteresis, neutral gating on profile changes, +source disposal, raw BLE report preservation, physical key capture, global reset, +existing controller/gesture/keyboard paths, pointer handoff, and Compose settings +and layout editor behavior. + +Two additional ownership regressions were demonstrated failing before their +fixes, then passed in the full suite: + +- Removing the last local touch surface must send the final release to the legacy + Epic mouse route, while removing only one of two holds must preserve the other. +- Concurrent native Deck pad surfaces must retain the first contact's coordinates + until it ends, then transfer ownership and release correctly. + +The native preload regression probe compiles the actual library and compares +`open`, `openat`, `open64` and `openat64` with libc. It covers ordinary file +creation, `O_TMPFILE` permissions, and opening a directory without a mode argument +(12 combinations). The temporary-file fix addresses the KDE cache permissions +problem encountered during the Desktop test; it does not change controller +transport or publication ordering. + +All three test methods passed on Linux x86-64 after rebasing. The same 12 +combinations also passed in the earlier i386 probe. + +Earlier native testing also exposed three i386 ring-test failures, reproduced on +the unmodified pre-feature baseline. These are not presented as passing or fixed +by this PR. Physical BLE coverage is likewise not implied by synthetic tests. + +## Manual AYN Thor results + +The user performed the following checks in the installed candidate. The desktop +application was KWrite; native gamepad input was checked in Steam's controller +input test. A loaded game's input-to-display response was not measured. + +| Area | Observed result | +| --- | --- | +| Native Game profile | Physical buttons, D-pad, both sticks/clicks and triggers work without observed missed input or stuck controls | +| Desktop defaults | Left stick moves pointer; A/B click; X/Y provide Enter/Escape; D-pad supplies arrows; right stick scrolls and stops at neutral | +| Scroll direction | Stick down advances to higher line numbers; stick up returns to earlier lines | +| Cursor visibility | Controller buttons and stick movement no longer hide the mapped pointer | +| Pointer handoff | Lower trackpad and stick can alternate without a jump; click position matches cursor | +| Cursor cadence | User confirmed smooth movement after the frame update fix; measurements below | +| Android keyboard | Text entry works on the lower screen, including while moving the pointer; controls continue after hiding the keyboard | +| Configured shortcut | R3 assigned to Ctrl+F through the settings UI repeatedly opens search; Y/Escape closes it; subsequent typing has no stuck Ctrl | +| Combined keyboard input | Physical modifier with Android keyboard supports Ctrl+A, Ctrl+X, Ctrl+V and Ctrl+S | +| Custom lower screen | Right pad moves pointer, left pad scrolls, A/B click, and View/Select opens the keyboard | +| Shared mouse hold | Physical A and screen A can hold together; releasing the physical button preserves selection until the screen button is released | +| Menu/held input | User reported successful menu/Ctrl release and ordinary typing afterward | +| Android Home/resume | Physical stick, lower trackpad, clicks and keyboard shortcuts work after returning | +| Layout persistence | Removing/readding a button through the editor persists across APK update and restart; defaults restored through the editor | +| Label rendering | Long View/Select and Menu/Start labels fit within their buttons | +| Native Deck lower screen | Both trackpads, clicks, L4/L5/R4/R5 and physical controls work in Steam's input test | + +The editor's individual compatible-native and disabled options are host-tested, +not an exhaustive manual hardware matrix. Menu is currently an open command; +Back/B closes it. Multi-button activation chords are not implemented. + +## Relative latency measurement + +### What was measured + +On the same Thor, timestamps from the built-in controller's evdev stream using +`CLOCK_MONOTONIC` were matched to a read-only guest observer of the published +controller ring. The observer polled with a requested 250 microsecond sleep, +roughly 0.31 ms in practice. Each observation bounds receipt between polls; the +table reports the **upper bound**, not an exact physical latency. + +This path includes Android dispatch and publication. It excludes controller +firmware, Steam Input/game processing, rendering and display scanout. USB was +used for control and collection, not as the timed input transport. This is not a +button-to-photon test. + +The public 0.6.0 build (`22eac083`) was sampled first. A locally instrumented +baseline at `ac0909ea` was then compared with the feature candidate; their relevant +baseline Java/native input implementation matched. Both used Steam Deck emulation, +1280×720 at 60 Hz, Steam's main screen, and the same disabled low-latency options. + +The manual samples used physical B presses. Automatic samples injected B edges +into the built-in controller's evdev path, continuing through normal Android +input dispatch. Settled paired runs used a 150-press warmup before 150 measured +presses (300 edges) at approximately 33°C battery temperature. Earlier preliminary +runs were affected by background preparation and are kept separate. + +### Results + +All values below are milliseconds, kernel event to observer upper bound. + +| Run | Matched edges | p50 | p95 | p99 | Max | +| --- | ---: | ---: | ---: | ---: | ---: | +| Public 0.6.0, manual | 84 | 4.379 | 8.155 | 9.334 | 9.729 | +| Local baseline, manual | 82 | 3.876 | 8.322 | 10.667 | 12.447 | +| Baseline automatic, preliminary | 300 | 3.213 | 6.774 | 7.960 | 8.688 | +| Candidate automatic, preliminary | 300 | 2.694 | 5.197 | 6.725 | 8.164 | +| Candidate automatic, settled 1 | 300 | 3.403 | 6.503 | 9.114 | 10.167 | +| Baseline automatic, settled | 300 | 3.061 | 6.269 | 7.160 | 7.850 | +| Candidate automatic, settled 2 | 300 | 2.999 | 5.620 | 8.171 | 13.653 | +| Final installed candidate, extra check | 300 | 3.039 | 5.905 | 7.470 | 11.684 | + +The extra final-APK check matched all 300 edges without an observed overrun or +extra/missing edge. Its p50 interval was 2.692–3.039 ms. It ran at 32°C with internal +timing disabled, without another paired baseline or the explicit 150-press warmup; +it is supporting regression evidence, not a controlled speed comparison. + +A separate internal sample of 600 dispatch-to-publication observations reported: + +| Build | p50 (µs) | p95 (µs) | p99 (µs) | +| --- | ---: | ---: | ---: | +| Baseline | 112.6 | 274.2 | 442.2 | +| Candidate | 116.9 | 315.2 | 561.7 | + +The 4.3 µs median difference is the difference between those runs, **not an +isolated measurement of router cost**. The tested typical native path remained +within baseline variation. Candidate tail values were higher in some runs and +were not isolated to a cause. No claim of a speedup, zero cost, or unchanged worst +case is supported by these samples. + +### Cursor frame cadence + +The initial stick-driven cursor updated at a median 33.333 ms interval, while +the lower trackpad updated at 16.667 ms. Publishing pending Compose snapshot +notifications before the frame's draw traversal changed the stick median to +16.667 ms. Of 119 intervals, 115 were 14–20 ms and none were 30–37 ms. The remaining +four included longer start/stop gaps, so these are not a continuous-load frame-time +distribution. The user confirmed the visible improvement. Speed, sensitivity and +the stick response curve were unchanged. + +This is a rendering cadence check, separate from event propagation latency. + +## Remaining validation + +- Repeat the short Thor regression pass on the rebased final branch. +- Measure simultaneous BLE and touch input, contention and allocations on hardware. +- Compare Desktop key/pointer and native Deck touch latency quantitatively. +- Repeat under a loaded game/rendering workload and, if needed, measure display + response with high-speed video. +- Exercise physical Bluetooth devices and other device-specific adapters. +- Let upstream CI build and test the complete release configuration, including + store engines omitted from the local Steam/Desktop APK. + +These limits should remain visible in the PR while maintainers review the +architecture. Host tests and a successful APK build do not replace those checks.