mirror of
https://github.com/Droid-Deck/DroidDeck.git
synced 2026-11-20 05:16:01 +00:00
docs: describe input architecture and Thor validation limits
This commit is contained in:
1 parent
6530faaf78
commit
d202233beb
2 files changed
+363
-283
No files matched your search
@@ -1,322 +1,213 @@
|
||||
# Unified input configuration and routing proposal
|
||||
# Shared input profiles and configurable control surfaces
|
||||
|
||||
Status: implemented locally for device evaluation; not published or approved by maintainers.
|
||||
Prepared on 2026-10-23 against `ac0909ea`.
|
||||
Related source snapshots: PR #1202 at `42db788f`, PR #1305 at `76fc9b31`.
|
||||
This change implements a shared routing model for physical controllers and local
|
||||
screen controls. It is intended for review of the architectural direction, with
|
||||
a functional configuration UI and hardware validation on AYN Thor. The related
|
||||
maintainer discussions motivate this work; they are not approval of this
|
||||
implementation.
|
||||
|
||||
## Problem and scope
|
||||
The configuration UI is intentionally basic. If maintainers accept the
|
||||
architecture, I am happy to refine the UX/UI in the same PR before merge, or in a
|
||||
follow-up PR, whichever they prefer. Navigation, binding discovery, layout
|
||||
editing, accessibility and localization need further attention. The current UI
|
||||
makes the architecture usable and testable rather than establishing its final
|
||||
interaction design.
|
||||
|
||||
DroidDeck has useful input paths whose configuration and lifecycle rules are
|
||||
spread across several components. Desktop users need configurable physical
|
||||
controller bindings. Dual-screen users also need to arrange and configure lower
|
||||
screen controls without giving up Steam Deck trackpads or Steam Input layouts.
|
||||
## Why a shared system
|
||||
|
||||
Provide one configuration model and reusable action handling while preserving
|
||||
the existing low-latency transports and Steam's ownership of Steam-managed
|
||||
bindings. A common model does not require all input to cross a new queue, UI
|
||||
thread, generic event bus, or virtual device.
|
||||
Desktop sessions need mouse, keyboard and application actions from a controller.
|
||||
A dual-screen device may already have a comfortable lower-screen mouse, leaving
|
||||
its physical sticks available for other actions. The same logical button or pad
|
||||
can appear on the main screen, lower screen or a physical device; its location
|
||||
should not dictate which actions it can perform.
|
||||
|
||||
The implementation has two editable built-in profiles, Game and
|
||||
Desktop. Named profile creation, per-application selection, and profile
|
||||
import/export are future work; the stored model must allow them without changing
|
||||
the routing contract.
|
||||
The model separates three concerns:
|
||||
|
||||
## Existing paths to preserve
|
||||
- **Source role:** physical, primary screen or secondary screen.
|
||||
- **Profile:** the actions assigned to controls in each role. Game and Desktop are
|
||||
the two editable built-in profiles. Auto selects by session mode; explicit
|
||||
selection supports a game running inside a desktop session.
|
||||
- **Surface layout:** the placement and size of local screen controls. The custom
|
||||
second-screen layout is stored independently for each built-in profile.
|
||||
|
||||
| Source | Current path | Meaning |
|
||||
| --- | --- | --- |
|
||||
| Android physical pad | `PadBridge` -> `FakeInputWriter` | Gamepad buttons, axes, player slots |
|
||||
| Built-in screen pad, including the lower-screen Gamepad tab | `PadBridge.applyTouch` -> `FakeInputWriter` | Virtual gamepad input |
|
||||
| Lower-screen Deck trackpads and back grips | `DeckControlsPanel` -> `DeckControls` -> `FakeInputWriter.writeDeckControls` | Touch coordinates, contact, click/pressure, L4/L5/R4/R5 |
|
||||
| Steam touch overlay | `SteamTouchControls` -> `SteamTouchDevice` -> Steam | Steam Mobile Touch reports, with Steam-owned bindings and action sets |
|
||||
| Desktop trackpad and keyboard | Existing gesture/keyboard handlers -> compositor | Direct guest mouse and keyboard input |
|
||||
| Steam Controller BLE and proposed native DualSense | Driver-specific report paths | Input can bypass Android `KeyEvent` dispatch |
|
||||
A binding selects native controller output, a DroidDeck action, or disabled.
|
||||
Native output continues through the existing game/Steam path, optionally with a
|
||||
compatible gamepad remap. A local action consumes that control so, for example,
|
||||
R3 assigned to Ctrl+F does not also press the guest's R3 button.
|
||||
|
||||
Deck trackpads must not be reduced to mouse deltas or a pair of ordinary joystick
|
||||
axes. Their contact, coordinates, click state and pressure are meaningful to
|
||||
Steam. Likewise, Steam Mobile Touch is a distinct protocol, not another spelling
|
||||
of an Xbox pad.
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Physical[Android controller events] --> PadBridge
|
||||
Primary[Primary on-screen controls] --> PadBridge
|
||||
Secondary[Secondary custom controls] --> PadBridge
|
||||
Deck[Deck pads and grips] --> DeckControls
|
||||
BLE[Steam Controller BLE reports] --> TritonInputRouter
|
||||
Profiles[Game / Desktop profile by source role] --> Router[InputRouter]
|
||||
PadBridge --> Router
|
||||
DeckControls --> Router
|
||||
TritonInputRouter --> Router
|
||||
Router --> Native[Existing pad / Deck report publication]
|
||||
Router --> Held[Shared key and mouse ownership]
|
||||
Router --> Commands[Existing menu and keyboard hosts]
|
||||
Router --> Motion[InputMotion: active stick pointer / scroll only]
|
||||
Motion --> Held
|
||||
Held --> Guest[Existing guest keyboard and mouse output]
|
||||
Native --> Steam[Steam Input or guest controller consumer]
|
||||
Layout[Independent surface geometry] --> Secondary
|
||||
```
|
||||
|
||||
See [Steam touch controls](steam-touch-controls.md) for the current integration,
|
||||
including Steam-owned layouts, action sets and cloud configuration.
|
||||
These are routing relationships, not additional asynchronous queues. Native
|
||||
publication is performed by the existing source adapters and writers.
|
||||
|
||||
## Separate bindings, screen layout, and device calibration
|
||||
|
||||
The proposed model has three independent parts:
|
||||
|
||||
- **Binding profile:** logical input sources and their actions, with Game and
|
||||
Desktop defaults and user overrides.
|
||||
- **Surface layout:** which controls appear on the primary or secondary control
|
||||
surface, their positions, sizes and presentation. A layout references bindings
|
||||
or Steam-managed controls; it does not copy Steam's action-set definitions.
|
||||
- **Device configuration:** hardware identification, axis normalization,
|
||||
calibration and capabilities. This is separate from a user's desktop layout.
|
||||
|
||||
Use logical surface roles rather than persistent Android display IDs. Controls
|
||||
need stable IDs independent of their location. Moving a control to the lower
|
||||
screen must not change its action or create a new guest gamepad. The existing
|
||||
player-slot and device identities remain authoritative.
|
||||
|
||||
A Game profile can use the lower screen for two Deck trackpads and back grips,
|
||||
plus selected DroidDeck actions. A Desktop profile can put a mouse trackpad and
|
||||
keyboard there while mapping the physical sticks to arrow keys, scrolling or
|
||||
other actions. Connecting a second display must not silently rewrite physical
|
||||
controller bindings.
|
||||
|
||||
The model should support reusable lower-screen layouts later. Initially expose
|
||||
existing layouts and a bounded set of configurable controls; a complete freeform
|
||||
multi-display editor is a separate deliverable.
|
||||
|
||||
## Steam Input ownership
|
||||
|
||||
Each binding explicitly selects an execution path:
|
||||
|
||||
1. **Native/Steam-managed:** send the existing controller or touch report.
|
||||
Steam remains responsible for game bindings, action sets, layers and its
|
||||
touch/radial menus. DroidDeck shows an appropriate entry into Steam's editor.
|
||||
2. **DroidDeck-managed:** execute a guest key combination, mouse action or
|
||||
application action through the existing output functions. The consumed
|
||||
control is excluded from the normal controller output.
|
||||
3. **Disabled:** intentionally produce no action.
|
||||
|
||||
Steam-managed and DroidDeck-managed controls can coexist on one surface. One
|
||||
activation must have one semantic owner, so a local keyboard shortcut does not
|
||||
also produce the original controller button. Gamepad-to-gamepad remaps remain
|
||||
explicit preprocessing before the normal game/Steam path, rather than a second
|
||||
independent action sent alongside it.
|
||||
|
||||
Steam Input already supports extensive binding and touch-menu behavior. This
|
||||
proposal concentrates DroidDeck's work on Android surfaces, app shortcuts,
|
||||
desktop/non-Steam bindings and consistent lifecycle handling. It does not propose
|
||||
reimplementing Steam's configuration engine or importing its profiles into a
|
||||
second authoritative store.
|
||||
|
||||
Preserve raw-report fields and output/feedback channels not affected by a local
|
||||
binding, including motion and haptics. Driver-specific adapters must participate
|
||||
before publishing consumed controls; an Activity-only interceptor cannot cover
|
||||
raw HID devices.
|
||||
|
||||
## Configuration and runtime model
|
||||
|
||||
Responsibilities:
|
||||
## Implementation
|
||||
|
||||
| Component | Responsibility |
|
||||
| --- | --- |
|
||||
| `InputProfile` | Version, stable ID, default role, bindings and surface-layout references |
|
||||
| `SurfaceLayout` | Stable control IDs, display role, geometry and appearance |
|
||||
| `InputBinding` | Source selector, activation rule, action and execution owner |
|
||||
| Source adapters | Preserve device identity, normalize inputs and expose capabilities |
|
||||
| Compiled binding plan | Pre-resolved routes for one active profile and its devices |
|
||||
| Action executor/output ownership | Execute existing outputs and track held contributions |
|
||||
| `InputProfile` | Logical controls, source roles and supported actions |
|
||||
| `InputProfiles` | Versioned persistence, selection, validation and migration |
|
||||
| `InputSurfaceLayout` | Bounded custom lower-screen geometry, separate from actions |
|
||||
| `InputRouter` | Resolve channels on profile changes, route events, track held outputs |
|
||||
| `PadBridge` | Existing player slots plus independently owned local touch sources |
|
||||
| `DeckControls` | Source ownership for Deck grips, contacts, coordinates and clicks |
|
||||
| `TritonInputRouter` | Apply routing before Steam Controller BLE report publication |
|
||||
| `PhysicalInputBindings` | Capture additional Android keys with device identity |
|
||||
| `HeldInput` | Coordinate router outputs with existing keyboard and gesture outputs |
|
||||
| `InputMotion` | Frame callback for active synthesized stick pointer/scroll movement |
|
||||
| `InputTiming` | Optional bounded dispatch-to-publication diagnostics |
|
||||
|
||||
Buttons and axis directions can produce keys, key combinations, mouse buttons,
|
||||
gamepad buttons or app actions. Sticks can retain their gamepad role or produce
|
||||
pointer velocity, scrolling or keyboard directions. Analog triggers need distinct
|
||||
press/release thresholds when used as buttons. Capability-aware controls should
|
||||
explain why an unavailable action cannot run, rather than silently substituting
|
||||
another behavior.
|
||||
Supported actions are native output, compatible native remapping, disabled,
|
||||
keyboard keys/combinations, mouse buttons, pointer movement, scrolling, four
|
||||
keyboard directions, and existing menu/keyboard/Steam/quick-access commands.
|
||||
Analog controls used as buttons have separate activation and release thresholds.
|
||||
Key combinations press modifiers first and release in reverse order.
|
||||
|
||||
The same action picker is reused for physical and local screen controls. Screen
|
||||
geometry stays outside binding execution. Steam-managed widgets retain their
|
||||
native configuration and protocol adapters.
|
||||
Standard controller assignments are shared by the physical role. Additional
|
||||
captured keys match a device identity plus Android keycode/scancode, rather than
|
||||
persisting a transient Android device ID. System-reserved keys remain with
|
||||
Android. This is not a per-device calibration system.
|
||||
|
||||
Configuration is loaded, validated and compiled when it changes. Dispatch uses
|
||||
an immutable plan snapshot and preallocated per-source state. JSON decoding,
|
||||
preference reads, disk writes and layout traversal do not belong in event dispatch.
|
||||
Use stable symbolic control/action identifiers in storage; transient Android
|
||||
device IDs are runtime keys, not portable profile identifiers.
|
||||
### Reuse and migration
|
||||
|
||||
Profile selection defaults to the session's mode, with an explicit session
|
||||
override. Desktop mode can contain a game, so it is not sufficient evidence for
|
||||
automatic keyboard/mouse interception. An optional profile-switch action uses
|
||||
the same app-action mechanism as menus and keyboard display.
|
||||
Existing `FakeInputWriter` transports, player assignment, guest keyboard/mouse
|
||||
output, `KeyboardHost`, second-screen presentation and `DeckControlsPanel` are
|
||||
reused. The custom lower-screen editor arranges the existing buttons, sticks,
|
||||
trackpads and grips; it does not create another emulated controller.
|
||||
|
||||
## Ownership and lifecycle
|
||||
The old touch-only mapping and Select-as-Steam settings migrate once into Game's
|
||||
primary-screen and physical bindings respectively. Their redundant UI is removed.
|
||||
Invalid stored entries fall back to defaults. The global settings reset clears
|
||||
both profile and surface settings without reimporting the old mappings.
|
||||
|
||||
- Track held keys and mouse buttons by source. Releasing one source must not
|
||||
release a key or button another source still owns.
|
||||
- Cancel only a disappearing surface's or disconnected device's contributions;
|
||||
avoid a global release that also clears another screen or controller.
|
||||
- Define arbitration for multiple sources targeting the same analog control or
|
||||
Deck touch surface. Retain ownership through a contact; do not let concurrent
|
||||
views overwrite each other's reports or add a second virtual pad implicitly.
|
||||
- Changing a binding/profile releases its old outputs before installing the new
|
||||
route. Held controls must return to neutral before triggering a new binding.
|
||||
- Menus, keyboard capture, pause, focus loss and device removal have explicit
|
||||
routing rules. Menu navigation retains working confirmation, cancellation and
|
||||
page controls even when those buttons have gameplay bindings.
|
||||
- Keep system-reserved Android inputs with the system and retain touchscreen
|
||||
access to reset a problematic controller assignment.
|
||||
The shared PC-key definitions are adapted from kingokksa's
|
||||
[#1305](https://github.com/Droid-Deck/DroidDeck/pull/1305) at `76fc9b31`.
|
||||
Device capture and identity handling build on the approach in jmarti326's
|
||||
[#1202](https://github.com/Droid-Deck/DroidDeck/pull/1202) at `42db788f`.
|
||||
Neither PR is merged wholesale; their broader UI and device behavior should be
|
||||
coordinated with their authors.
|
||||
|
||||
Button chords require a deliberate policy. A dedicated modifier can select a
|
||||
chord without delaying unrelated controls. A button that also has a single-press
|
||||
action may require deferred dispatch to disambiguate a chord, hold or double tap.
|
||||
That delay must be opt-in, visible in configuration and limited to the affected
|
||||
binding. Do not buffer all controller presses to support optional chords.
|
||||
### Ownership and lifecycle
|
||||
|
||||
## Latency requirements and verification
|
||||
Each source retains its own contribution. Releasing a physical mouse button does
|
||||
not release an identical button still held on the lower screen. Closing a surface
|
||||
releases that surface, including the final release on the existing Epic launcher
|
||||
mouse route. Existing gesture and keyboard outputs participate in shared key and
|
||||
mouse-button ownership.
|
||||
|
||||
The goal is no additional scheduling stage for normal input, not an unmeasured
|
||||
claim of zero added latency.
|
||||
Native pad merging uses OR for buttons, the maximum trigger value, and the
|
||||
strongest complete stick vector. A native Deck trackpad retains one source's
|
||||
coordinates until that contact ends, then accepts a waiting contact. Concurrent
|
||||
surfaces do not overwrite its coordinates during the same contact.
|
||||
|
||||
- Identity routes retain the existing direct publication path and report format.
|
||||
- Ordinary buttons and touch reports must not wait for drawing, Compose state,
|
||||
a new coroutine, a timer, an event-bus subscriber or another frame.
|
||||
- Add no hot-path allocations for ordinary binding lookup. Reuse buffers and
|
||||
keep critical sections short. The current router serializes routing and profile
|
||||
changes under one monitor to keep shared output ownership consistent; hardware
|
||||
traces must establish whether this causes contention before publication.
|
||||
- Continuous stick-to-pointer movement needs a clock even when no new stick
|
||||
event arrives. Keep that scheduler limited to the active synthesized movement
|
||||
route; stop it at neutral. It must not schedule ordinary gamepad forwarding.
|
||||
- Keep existing synchronization and low-latency flags intact until measurements
|
||||
justify a separate change. Do not weaken shared-memory publication ordering.
|
||||
- Send UI updates separately from input publication. Marshaling app-menu work to
|
||||
Android's UI thread must not marshal ordinary controller output with it.
|
||||
Profile changes, suspension and menu entry release old local outputs. Held
|
||||
controls must return to neutral before activating their new bindings. Device
|
||||
removal and surface disposal remove their owned state. The existing menu
|
||||
navigation path remains available independently of gameplay assignments.
|
||||
|
||||
Before publishing the change, record a baseline on Thor with the same build
|
||||
configuration, controller mode, display refresh rates and low-latency settings.
|
||||
Compare identity forwarding, a local key binding, stick-to-pointer, Deck trackpad
|
||||
input and simultaneous upper/lower-screen input under both idle and rendering
|
||||
load. Measure p50/p95/p99 dispatch-to-publication time, allocations and contention.
|
||||
Use debug-only sampling or buffered tracing; avoid per-event log writes.
|
||||
The menu and keyboard commands currently open their respective UI; they are not
|
||||
toggles. Back/B closes the menu. This can be reconsidered with the UX work.
|
||||
|
||||
Android-to-publication measurements do not measure complete button-to-photon
|
||||
latency. Guest receipt and rendered response need separate measurement, with
|
||||
clock-domain alignment for cross-process traces or an external high-speed-camera
|
||||
test. Record benchmark conditions and uncertainty; establish numerical acceptance
|
||||
budgets from the baseline before presenting latency claims in a PR.
|
||||
### Hot path and scheduling
|
||||
|
||||
Hard acceptance conditions include no new frame/timer wait for identity routes,
|
||||
no duplicate or lost press/release transitions, and no degradation of current
|
||||
Deck trackpad behavior. Microbenchmarks alone cannot establish those conditions.
|
||||
Profile JSON, preferences and layout geometry are processed outside event
|
||||
routing. Channels resolve their actions when configuration changes. The current
|
||||
router uses mutable per-source state under one synchronized monitor; it is not an
|
||||
immutable compiled plan. This keeps shared ownership consistent, but contention
|
||||
with simultaneous BLE input has not yet been measured on hardware.
|
||||
|
||||
## Reuse, migration, and removal
|
||||
For an unchanged Game binding, `PadSource` returns the original `PadState`.
|
||||
The BLE identity route returns its original report array. Modified reports use
|
||||
reusable buffers and preserve unrelated timestamp, motion, pressure and extension
|
||||
fields. Normal controller events do not wait for a frame, coroutine or timer.
|
||||
|
||||
- [#1202](https://github.com/Droid-Deck/DroidDeck/pull/1202): reuse physical capture,
|
||||
device identification and menu-action UI where suitable. Generalize the
|
||||
execution contract beyond Activity `KeyEvent` interception.
|
||||
- [#1305](https://github.com/Droid-Deck/DroidDeck/pull/1305): reuse PC-key definitions,
|
||||
key selection and screen-control presentation. Adapt keyboard output to shared
|
||||
ownership; its screen-layout format is not a complete physical-input profile.
|
||||
- [#581](https://github.com/Droid-Deck/DroidDeck/pull/581): coordinate device
|
||||
calibration/profile matching, independently of user binding/surface profiles.
|
||||
- [#1226](https://github.com/Droid-Deck/DroidDeck/pull/1226): use a shared keyboard
|
||||
action with the appropriate existing host/display, rather than hard-coding a
|
||||
second keyboard implementation.
|
||||
- Reuse current guest keyboard/mouse output, `PadBridge`, Deck/Steam touch
|
||||
transports and the existing second-screen presentation lifecycle.
|
||||
Only synthesized stick-to-pointer/scroll motion uses `Choreographer`, because a
|
||||
held stick must keep moving without new device events. Trackpad deltas and button
|
||||
changes publish synchronously on the normal UI input path. Pointer output arriving
|
||||
from an off-main-thread adapter is marshalled to the existing UI cursor path;
|
||||
application commands also use the main handler. Neither schedules native forwarding.
|
||||
|
||||
Remove duplicated settings, capture UI and action switches only after their
|
||||
replacement preserves saved settings and passes behavior tests. Do not remove
|
||||
working device adapters, Steam touch support or native transport code merely to
|
||||
make the architecture diagram more uniform. Stage each migration so it can be
|
||||
reviewed and reverted independently.
|
||||
The mapped cursor reuses the existing compositor coordinates and Android cursor
|
||||
layer. Trackpad contact starts from the current pointer position, preventing
|
||||
jumps when switching sources. Publishing pending Compose snapshot notifications
|
||||
before drawing avoids the observed every-other-frame cursor update without
|
||||
changing the stick response curve or adding smoothing.
|
||||
|
||||
## Implemented local evaluation build
|
||||
See [validation](unified-input-validation.md) for measurements and their limits.
|
||||
The evidence supports no clear typical latency regression in the tested native
|
||||
path, not zero overhead or a complete input-to-display latency guarantee.
|
||||
|
||||
No PR is created yet. Test the complete integrated change on Thor before deciding
|
||||
whether to publish it. The intended framing is **Shared input profiles and
|
||||
configurable control surfaces**, with links to the desktop requests
|
||||
[#860](https://github.com/Droid-Deck/DroidDeck/issues/860) and
|
||||
[#1356](https://github.com/Droid-Deck/DroidDeck/issues/1356), the centralization
|
||||
discussion in #1202, and the related work above. Those discussions are context,
|
||||
not maintainer approval of this implementation.
|
||||
## Steam ownership and boundaries
|
||||
|
||||
### Runtime and configuration
|
||||
Steam remains responsible for game mappings, action sets, layers, radial menus
|
||||
and its cloud profiles. Native/Steam-managed here means forwarding the controller
|
||||
through the existing path; it is not a new integration with Steam's profile API.
|
||||
Steam Mobile Touch retains its own Steam-managed layout and protocol. Its widgets
|
||||
do not pass through this local remapper. The VR-specific input path also remains
|
||||
separate. Existing gestures retain their recognizers and share output ownership.
|
||||
|
||||
- `InputProfile` separates actions and source roles from `InputSurfaceLayout`.
|
||||
`InputProfiles` stores a versioned document and migrates legacy touch mappings
|
||||
and Select-as-Steam into Game once. Editing one role does not edit another.
|
||||
- `InputRouter` resolves actions when a profile changes. `PadSource` returns the
|
||||
original `PadState` on the identity path. Other routes reuse their output
|
||||
buffers and consume only the selected control. A key combination presses
|
||||
modifiers first and releases in reverse order. Trigger/direction hysteresis
|
||||
uses separate press and release thresholds.
|
||||
- `PadBridge` preserves physical player slots and separately owns each touch
|
||||
surface. Merging uses OR for buttons, maximum trigger values and the strongest
|
||||
complete stick vector. Surface removal cannot release another source's hold.
|
||||
- `DeckControls` keeps per-source contacts, coordinates, clicks and pressure.
|
||||
Default native actions retain the existing Deck report. Local trackpad actions
|
||||
use contact-relative deltas, avoiding a jump at first contact.
|
||||
- `TritonInputRouter` routes Steam Controller BLE reports before publication. Its
|
||||
identity path returns the original byte array. Other paths preserve packet,
|
||||
timestamp, inertial data, grip-contact bits, report extensions and pressure.
|
||||
This adapter is covered by synthetic report tests; physical BLE testing is
|
||||
still required.
|
||||
- `PhysicalInputBindings` captures device identity, keycode and scancode, including
|
||||
controller buttons exposed as keyboard keys. It consumes a configured local
|
||||
action before normal dispatch and keeps releases paired across profile edits.
|
||||
Android volume and system keys remain reserved.
|
||||
- `HeldInput` coordinates local mappings with existing guest keyboard, gesture,
|
||||
mouse and second-screen outputs. `InputMotion` schedules only synthesized
|
||||
stick movement/scrolling. Buttons and trackpad deltas publish synchronously;
|
||||
UI commands use the existing main-thread handler.
|
||||
- Profile changes, suspension and menu entry release local outputs and require
|
||||
held inputs to return to neutral before activating their new actions.
|
||||
The two built-in profiles and the bounded custom lower-screen editor are the
|
||||
implemented scope. Named profile libraries, import/export, per-app selection,
|
||||
macros, tap/hold behavior, multi-button activation chords and a complete freeform
|
||||
multi-display editor are deferred. One button emitting Ctrl+F is supported;
|
||||
pressing multiple buttons to activate a new binding is not implemented.
|
||||
|
||||
### User interface
|
||||
Native remaps stay within compatible report families. Additional hardware
|
||||
adapters must join before publishing consumed controls; an Activity-only
|
||||
interceptor cannot cover every raw device. Future adapters should use this
|
||||
contract without replacing device-specific calibration or protocol handling.
|
||||
|
||||
Input profiles are available from Controller setup and the session's Controller
|
||||
panel. Auto selects Desktop for a desktop session and Game for other sessions;
|
||||
manual selection handles games launched from a desktop without guessing from
|
||||
window focus. The editor exposes physical, primary-screen and secondary-screen
|
||||
roles, native/remapped gamepad output, local actions and disabled bindings.
|
||||
## Related upstream work
|
||||
|
||||
Custom controls is an additional second-screen mode. Its editor arranges a
|
||||
bounded set of existing `DeckControlsPanel` buttons, sticks, trackpads and grips.
|
||||
Layout geometry is independent of bindings and saved per built-in profile.
|
||||
Android keyboard input reuses `KeyboardHost`; the existing keyboard/trackpad
|
||||
and Deck panel modes remain available. Shared PC-key definitions are taken from
|
||||
PR #1305 at `76fc9b31`; capture/identity handling follows PR #1202 at `42db788f`.
|
||||
Neither PR is merged wholesale. The superseded touch-only mapping page and
|
||||
Select-as-Steam UI toggle are removed after migration.
|
||||
Reviewed on 2026-10-24. Several related proposals remain open; this is not a claim
|
||||
that no other controller work exists or that this implementation supersedes it.
|
||||
|
||||
### Boundaries and deferred work
|
||||
- [#1202](https://github.com/Droid-Deck/DroidDeck/pull/1202): physical shortcuts and
|
||||
capture. The maintainer's
|
||||
[centralization feedback](https://github.com/Droid-Deck/DroidDeck/pull/1202#issuecomment-6299647180)
|
||||
motivates one shared assignment model. Questions there about overlapping
|
||||
actions inform consumption/ownership; multi-button activation remains deferred.
|
||||
- [#1305](https://github.com/Droid-Deck/DroidDeck/pull/1305): screen key mappings,
|
||||
combinations and layout management. This change shares the action model across
|
||||
physical and screen sources, but does not implement that PR's full layout UX.
|
||||
- [#1414](https://github.com/Droid-Deck/DroidDeck/pull/1414): external-controller
|
||||
button and touchpad-press capture, including devices exposing a touchpad press
|
||||
as pointer input. Our key capture does not yet implement that pointer-contact
|
||||
adapter. It is a concrete area for coordination.
|
||||
- [#860](https://github.com/Droid-Deck/DroidDeck/issues/860) and
|
||||
[#1356](https://github.com/Droid-Deck/DroidDeck/issues/1356): desktop controller
|
||||
input. This implements controller-driven pointer/keyboard actions, not every
|
||||
gesture or automatic keyboard behavior requested in #1356.
|
||||
- [#581](https://github.com/Droid-Deck/DroidDeck/pull/581): hardware calibration and
|
||||
device profiles remain a separate responsibility from action profiles.
|
||||
- [#1226](https://github.com/Droid-Deck/DroidDeck/pull/1226): keyboard host selection;
|
||||
the local action reuses the existing host rather than implementing another IME.
|
||||
- [#1150](https://github.com/Droid-Deck/DroidDeck/pull/1150): raw DualSense input.
|
||||
That proposed adapter will need integration before publication; it is not
|
||||
covered merely by the current Android key capture.
|
||||
|
||||
Steam Mobile Touch retains its Steam-owned action sets, layouts and cloud data;
|
||||
its controls are edited in Steam. It does not pass through DroidDeck's local
|
||||
remapper. Existing gesture recognition and VR controller handling are retained;
|
||||
gestures share output ownership, while the VR-specific input path remains owned
|
||||
by VR. Native remaps are limited to compatible controls within the same report
|
||||
family. An Android physical key can retain its original behavior or run a local
|
||||
action; capture does not synthesize an unrelated hardware device.
|
||||
## Review focus
|
||||
|
||||
Named profile libraries, importing/exporting profiles, per-app switching,
|
||||
reusable layout presets, macros, tap/hold/chord activation and a freeform
|
||||
multi-display drag editor remain deferred. Key combinations emitted by one
|
||||
binding are supported; multi-button activation chords are not implemented.
|
||||
Hardware calibration stays in the existing adapters. No native transport,
|
||||
Steam haptics path or device driver is replaced.
|
||||
|
||||
### Validation and Thor procedure
|
||||
|
||||
Before changes, the existing controller/PadMouse/on-screen/second-screen tests
|
||||
passed. The integrated suite exercises routing, shared holds, key order,
|
||||
hysteresis, neutral gating, profile migration/persistence, raw report preservation,
|
||||
existing input paths and the Compose binding/layout editors. See the local
|
||||
Gradle test report for exact results.
|
||||
|
||||
1. Install the local test APK alongside the signed public app; keep runtime data
|
||||
and baseline configuration available. Do not replace the official signing key.
|
||||
2. In a Desktop session, test pointer/scroll, mouse buttons, arrow keys, arbitrary
|
||||
key combinations and captured hardware back buttons. Open and close the drawer
|
||||
and keyboard with controls held; verify every guest release.
|
||||
3. Select Game and test Factorio plus Steam UI, first with the default native
|
||||
profile, then with individual bindings changed to local or disabled. Repeat
|
||||
with Xbox 360 and Steam Deck virtual-controller modes.
|
||||
4. Exercise keyboard/trackpad, Deck panels and Custom controls on the lower screen.
|
||||
Hold the same output from two sources, remove a surface, change profiles while
|
||||
holding sticks/buttons and reconnect the controller/display.
|
||||
5. Enable Input diagnostics, close the editor, generate input under idle and
|
||||
rendering load, then reopen and read p50/p95/p99. Recording is bounded to the
|
||||
most recent 4096 samples and disabled by default. `InputTiming` measures dispatch
|
||||
through existing transport publication, including routing. It does not measure
|
||||
full button-to-photon latency. Use the same instrumentation on a baseline build
|
||||
before making comparative claims.
|
||||
6. Record refresh rates, controller/transport mode, build type and low-latency
|
||||
flags. Compare raw identity, local key, pointer and native Deck routes. Use
|
||||
external high-speed video or aligned guest traces for end-to-end measurement.
|
||||
|
||||
A passing host test suite does not establish latency on Thor or physical BLE
|
||||
compatibility. Those remain explicit release gates before a public PR.
|
||||
The main questions are whether source roles and profile boundaries match the
|
||||
project's direction, whether adapters join at the right point, and whether the
|
||||
shared ownership/lifecycle contract is suitable for future input sources. The
|
||||
validation document distinguishes device-tested behavior, host regressions and
|
||||
remaining coverage. UX/UI refinement can follow agreement on those decisions.
|
||||
@@ -0,0 +1,189 @@
|
||||
# Unified input validation
|
||||
|
||||
Recorded on 2026-10-24. These results distinguish the device-tested candidate from
|
||||
the later upstream rebase and host regression fixes. They do not establish zero
|
||||
latency or complete coverage of every controller and session mode.
|
||||
|
||||
## Revisions and environment
|
||||
|
||||
The Thor candidate was `5021e7ac` on base `ac0909ea`, with the local temporary-file,
|
||||
cursor visibility/handoff/frame cadence, scroll direction and button label fixes
|
||||
subsequently committed on this branch. Its final APK SHA-256 was
|
||||
`bb211095d6443e9f85ed1f5d8c5674f527df2937ac6b0efe7d6fc4adcb9dcf62`.
|
||||
|
||||
The branch was then rebased onto upstream `906e0d1f`. Review added regressions and
|
||||
fixes for final touch-source release, concurrent native trackpad contact ownership,
|
||||
and resetting the new preferences. Those final changes and the rebase have host
|
||||
test coverage; the installed Thor APK was not silently replaced and those exact
|
||||
changes have not yet been revalidated on the device.
|
||||
|
||||
Local Android builds use Java 17 and NDK `28.2.13676358`. The Thor test APK used a
|
||||
local Gradle init script to install alongside the official app; no application ID
|
||||
or app-name changes are part of this PR. Local builds use `-PskipRust=true`, so
|
||||
Steam/Desktop checks do not establish the behavior of the Epic/GOG/Amazon store
|
||||
engines in a complete release build.
|
||||
|
||||
## Automated checks
|
||||
|
||||
On the rebased implementation, the complete `:app:testDebugUnitTest` run passed:
|
||||
**2,230 tests across 333 suites, zero failures/errors/skips**.
|
||||
|
||||
The supported `tools/build_local.sh` debug build also passed, including session
|
||||
asset matching, runtime-input attestation, native export/dependency validation,
|
||||
ZIP alignment and signature verification. The APK SHA-256 was
|
||||
`a7d440b4029f9ad67ca6f44c92ccdd6b9eba9e9a758b28ab0e432256697d49b9`.
|
||||
The only local environment repair was moving a Finder `.DS_Store` file out of the
|
||||
NDK's host-toolchain directory; the upstream script counted it as another host.
|
||||
No build checks were disabled to get past the stale-runtime check after rebasing.
|
||||
|
||||
Commands, with Java/SDK paths configured for the local machine:
|
||||
|
||||
```sh
|
||||
./gradlew :app:testDebugUnitTest -PskipRust=true -PndkVersion=28.2.13676358
|
||||
DROIDDECK_BUILD_VARIANT=debug DROIDDECK_SKIP_RUST=1 \
|
||||
DROIDDECK_NDK_VERSION=28.2.13676358 bash tools/build_local.sh
|
||||
# Linux with gcc and g++ available:
|
||||
python3 -m unittest tools.tests.test_fakeinput_open -v
|
||||
```
|
||||
|
||||
Coverage includes profile persistence and migration, invalid bindings, shared
|
||||
holds, key order, trigger/direction hysteresis, neutral gating on profile changes,
|
||||
source disposal, raw BLE report preservation, physical key capture, global reset,
|
||||
existing controller/gesture/keyboard paths, pointer handoff, and Compose settings
|
||||
and layout editor behavior.
|
||||
|
||||
Two additional ownership regressions were demonstrated failing before their
|
||||
fixes, then passed in the full suite:
|
||||
|
||||
- Removing the last local touch surface must send the final release to the legacy
|
||||
Epic mouse route, while removing only one of two holds must preserve the other.
|
||||
- Concurrent native Deck pad surfaces must retain the first contact's coordinates
|
||||
until it ends, then transfer ownership and release correctly.
|
||||
|
||||
The native preload regression probe compiles the actual library and compares
|
||||
`open`, `openat`, `open64` and `openat64` with libc. It covers ordinary file
|
||||
creation, `O_TMPFILE` permissions, and opening a directory without a mode argument
|
||||
(12 combinations). The temporary-file fix addresses the KDE cache permissions
|
||||
problem encountered during the Desktop test; it does not change controller
|
||||
transport or publication ordering.
|
||||
|
||||
All three test methods passed on Linux x86-64 after rebasing. The same 12
|
||||
combinations also passed in the earlier i386 probe.
|
||||
|
||||
Earlier native testing also exposed three i386 ring-test failures, reproduced on
|
||||
the unmodified pre-feature baseline. These are not presented as passing or fixed
|
||||
by this PR. Physical BLE coverage is likewise not implied by synthetic tests.
|
||||
|
||||
## Manual AYN Thor results
|
||||
|
||||
The user performed the following checks in the installed candidate. The desktop
|
||||
application was KWrite; native gamepad input was checked in Steam's controller
|
||||
input test. A loaded game's input-to-display response was not measured.
|
||||
|
||||
| Area | Observed result |
|
||||
| --- | --- |
|
||||
| Native Game profile | Physical buttons, D-pad, both sticks/clicks and triggers work without observed missed input or stuck controls |
|
||||
| Desktop defaults | Left stick moves pointer; A/B click; X/Y provide Enter/Escape; D-pad supplies arrows; right stick scrolls and stops at neutral |
|
||||
| Scroll direction | Stick down advances to higher line numbers; stick up returns to earlier lines |
|
||||
| Cursor visibility | Controller buttons and stick movement no longer hide the mapped pointer |
|
||||
| Pointer handoff | Lower trackpad and stick can alternate without a jump; click position matches cursor |
|
||||
| Cursor cadence | User confirmed smooth movement after the frame update fix; measurements below |
|
||||
| Android keyboard | Text entry works on the lower screen, including while moving the pointer; controls continue after hiding the keyboard |
|
||||
| Configured shortcut | R3 assigned to Ctrl+F through the settings UI repeatedly opens search; Y/Escape closes it; subsequent typing has no stuck Ctrl |
|
||||
| Combined keyboard input | Physical modifier with Android keyboard supports Ctrl+A, Ctrl+X, Ctrl+V and Ctrl+S |
|
||||
| Custom lower screen | Right pad moves pointer, left pad scrolls, A/B click, and View/Select opens the keyboard |
|
||||
| Shared mouse hold | Physical A and screen A can hold together; releasing the physical button preserves selection until the screen button is released |
|
||||
| Menu/held input | User reported successful menu/Ctrl release and ordinary typing afterward |
|
||||
| Android Home/resume | Physical stick, lower trackpad, clicks and keyboard shortcuts work after returning |
|
||||
| Layout persistence | Removing/readding a button through the editor persists across APK update and restart; defaults restored through the editor |
|
||||
| Label rendering | Long View/Select and Menu/Start labels fit within their buttons |
|
||||
| Native Deck lower screen | Both trackpads, clicks, L4/L5/R4/R5 and physical controls work in Steam's input test |
|
||||
|
||||
The editor's individual compatible-native and disabled options are host-tested,
|
||||
not an exhaustive manual hardware matrix. Menu is currently an open command;
|
||||
Back/B closes it. Multi-button activation chords are not implemented.
|
||||
|
||||
## Relative latency measurement
|
||||
|
||||
### What was measured
|
||||
|
||||
On the same Thor, timestamps from the built-in controller's evdev stream using
|
||||
`CLOCK_MONOTONIC` were matched to a read-only guest observer of the published
|
||||
controller ring. The observer polled with a requested 250 microsecond sleep,
|
||||
roughly 0.31 ms in practice. Each observation bounds receipt between polls; the
|
||||
table reports the **upper bound**, not an exact physical latency.
|
||||
|
||||
This path includes Android dispatch and publication. It excludes controller
|
||||
firmware, Steam Input/game processing, rendering and display scanout. USB was
|
||||
used for control and collection, not as the timed input transport. This is not a
|
||||
button-to-photon test.
|
||||
|
||||
The public 0.6.0 build (`22eac083`) was sampled first. A locally instrumented
|
||||
baseline at `ac0909ea` was then compared with the feature candidate; their relevant
|
||||
baseline Java/native input implementation matched. Both used Steam Deck emulation,
|
||||
1280×720 at 60 Hz, Steam's main screen, and the same disabled low-latency options.
|
||||
|
||||
The manual samples used physical B presses. Automatic samples injected B edges
|
||||
into the built-in controller's evdev path, continuing through normal Android
|
||||
input dispatch. Settled paired runs used a 150-press warmup before 150 measured
|
||||
presses (300 edges) at approximately 33°C battery temperature. Earlier preliminary
|
||||
runs were affected by background preparation and are kept separate.
|
||||
|
||||
### Results
|
||||
|
||||
All values below are milliseconds, kernel event to observer upper bound.
|
||||
|
||||
| Run | Matched edges | p50 | p95 | p99 | Max |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
||||
| Public 0.6.0, manual | 84 | 4.379 | 8.155 | 9.334 | 9.729 |
|
||||
| Local baseline, manual | 82 | 3.876 | 8.322 | 10.667 | 12.447 |
|
||||
| Baseline automatic, preliminary | 300 | 3.213 | 6.774 | 7.960 | 8.688 |
|
||||
| Candidate automatic, preliminary | 300 | 2.694 | 5.197 | 6.725 | 8.164 |
|
||||
| Candidate automatic, settled 1 | 300 | 3.403 | 6.503 | 9.114 | 10.167 |
|
||||
| Baseline automatic, settled | 300 | 3.061 | 6.269 | 7.160 | 7.850 |
|
||||
| Candidate automatic, settled 2 | 300 | 2.999 | 5.620 | 8.171 | 13.653 |
|
||||
| Final installed candidate, extra check | 300 | 3.039 | 5.905 | 7.470 | 11.684 |
|
||||
|
||||
The extra final-APK check matched all 300 edges without an observed overrun or
|
||||
extra/missing edge. Its p50 interval was 2.692–3.039 ms. It ran at 32°C with internal
|
||||
timing disabled, without another paired baseline or the explicit 150-press warmup;
|
||||
it is supporting regression evidence, not a controlled speed comparison.
|
||||
|
||||
A separate internal sample of 600 dispatch-to-publication observations reported:
|
||||
|
||||
| Build | p50 (µs) | p95 (µs) | p99 (µs) |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| Baseline | 112.6 | 274.2 | 442.2 |
|
||||
| Candidate | 116.9 | 315.2 | 561.7 |
|
||||
|
||||
The 4.3 µs median difference is the difference between those runs, **not an
|
||||
isolated measurement of router cost**. The tested typical native path remained
|
||||
within baseline variation. Candidate tail values were higher in some runs and
|
||||
were not isolated to a cause. No claim of a speedup, zero cost, or unchanged worst
|
||||
case is supported by these samples.
|
||||
|
||||
### Cursor frame cadence
|
||||
|
||||
The initial stick-driven cursor updated at a median 33.333 ms interval, while
|
||||
the lower trackpad updated at 16.667 ms. Publishing pending Compose snapshot
|
||||
notifications before the frame's draw traversal changed the stick median to
|
||||
16.667 ms. Of 119 intervals, 115 were 14–20 ms and none were 30–37 ms. The remaining
|
||||
four included longer start/stop gaps, so these are not a continuous-load frame-time
|
||||
distribution. The user confirmed the visible improvement. Speed, sensitivity and
|
||||
the stick response curve were unchanged.
|
||||
|
||||
This is a rendering cadence check, separate from event propagation latency.
|
||||
|
||||
## Remaining validation
|
||||
|
||||
- Repeat the short Thor regression pass on the rebased final branch.
|
||||
- Measure simultaneous BLE and touch input, contention and allocations on hardware.
|
||||
- Compare Desktop key/pointer and native Deck touch latency quantitatively.
|
||||
- Repeat under a loaded game/rendering workload and, if needed, measure display
|
||||
response with high-speed video.
|
||||
- Exercise physical Bluetooth devices and other device-specific adapters.
|
||||
- Let upstream CI build and test the complete release configuration, including
|
||||
store engines omitted from the local Steam/Desktop APK.
|
||||
|
||||
These limits should remain visible in the PR while maintainers review the
|
||||
architecture. Host tests and a successful APK build do not replace those checks.
|
||||
Reference in new issue
Block a user