Unify controller and screen input profiles for game and desktop sessions (#1439)

## Problem and intent

Desktop sessions need configurable mouse, keyboard and application
actions from a controller. On dual-screen devices, a lower-screen
trackpad may already handle the pointer, leaving the physical controls
available for other assignments. This PR gives physical controllers and
local touch controls a shared profile/action model instead of separate
configuration paths.

This follows the centralization direction discussed in
[#1202](https://github.com/Droid-Deck/DroidDeck/pull/1202#issuecomment-6299647180)
and addresses the controller-input portion of #860 and #1356. Those
discussions are context, **not prior maintainer approval** of this
implementation. I would appreciate architectural feedback and guidance
on splitting or coordinating this work if that would make review easier.

## Profiles and configuration

- Controller stays within the existing Setup tabs. Game and Desktop each
have a default-profile selector, followed by one profile list and device
settings.
- Named profiles reference three independently reusable parts: Physical
controller, Main screen and Second screen. Layout geometry and action
bindings are stored separately. Profiles can share physical controls
while assigning different touch actions.
- The session menu selects a complete profile, or expands to mix the
three parts. A mixed selection can be saved as a new profile; Game and
Desktop retain separate selections.
- Both screens use the same full-size editor, with add, move, resize,
snapping, undo and shared properties. The main-screen editor has
movable, collapsible tools and is transparent above a running session.
When editing the secondary display, its entire area is the canvas and
the main display hosts tools/properties.
- The old fixed gamepad, fixed Deck controls and separate Screen keys
editor are replaced with editable presets and the shared renderer.
Existing positions, bindings, shapes, symbols, imported images and
toggle behavior migrate; legacy Screen keys become additional profiles
without replacing the user's selected profile. Migration retains a
backup before removing old preferences.
- Saved layouts have thumbnails and screen targeting. Actions include
compatible native output/remapping, keys/combinations, mouse buttons,
pointer movement, vertical/horizontal scrolling, directional keys,
existing menu/keyboard commands, and disabled. Buttons can use symbols,
shapes and toggle behavior.

## Architecture and latency


```mermaid
flowchart LR
    Library[Profiles + reusable layouts and bindings] --> Router[InputRouter]
    Physical[Physical controller adapters] --> Router
    Touch[Main + secondary InputSurfaceView] --> Sources[Owned pad / Deck sources]
    Sources --> Router
    BLE[Steam Controller BLE adapter] --> Router
    Router --> Native[Existing controller publication]
    Native --> Steam[Steam Input / guest controller]
    Router --> Held[Shared key and mouse ownership]
    Held --> Guest[Existing guest input outputs]
    Router --> UI[Existing menu / keyboard hosts]
```

Bindings resolve when the profile changes, outside event dispatch.
Native forwarding has no additional frame/timer queue; identity routes
reuse existing state/reports. Only synthesized continuous stick
pointer/scroll motion uses a frame callback. One source releasing a
key/button does not release another source's hold. Menu, profile,
suspension and removal transitions release old outputs and gate held
controls until neutral.

Steam continues to own game-specific mappings and action sets. Steam
Mobile Touch and VR retain their specialized paths. Raw device adapters
remain responsible for transport/calibration and must join routing
before publishing consumed controls; this does not claim that every
adapter has been unified.

Historical Thor measurements on the native Game path found
kernel-event-to-guest-observer upper-bound medians of **3.061 ms
baseline versus 2.999–3.403 ms candidate**, with a subsequent
installed-APK check at **3.039 ms / 300 matched edges**. These are
relative software-path measurements, not button-to-photon latency. Some
tails increased, simultaneous BLE contention remains unmeasured, and
**this UI revision has not had a new latency measurement**. The previous
cursor fix changed visual update intervals from roughly 33.3 ms to 16.7
ms without changing sensitivity or adding smoothing.

[Full design, migration and
boundaries](https://github.com/Delete228/DroidDeck/blob/codex/unified-controller-input/docs/development/unified-input-proposal.md).

## Validation

Updated through upstream 542bbf89c; current PR head 08526f828. The
latest CodeRabbit review generated no actionable comments.

- **1,017 tests passed in 133 suites after the latest upstream merge**,
with no failures, errors or skips: all input/UI tests, global reset,
component updates, PiP policy, runtime auto-update and Proton menu
auto-selection. The pre-merge review-fix run passed **967
input/UI/global-reset tests**. The subsequent second-screen refresh fix
passed **35 focused surface/presentation tests**, including three new
state-preservation regressions.
- Full local run before the final review follow-ups: **3,598 tests,
3,573 passed, 25 failed**. The 25 are unchanged upstream macOS
limitations: log-export tests requiring SecureDirectoryStream (24) and a
case-sensitive WoW add-on test (1). The global-reset regression also
passes. Earlier disk-exhausted attempts were discarded; the completed
run bounded temporary resources with one worker and JVM recycling,
without excluding tests or changing repository test configuration.
- APK build, runtime/session assets, native dependencies, alignment and
v1/v2/v3 signatures passed. Installed over the separate Input Test app
on **AYN Thor** with its environment retained. Local packaging omits
Rust store engines; the [full Build APK workflow for 08526f828
passed](https://github.com/Droid-Deck/DroidDeck/actions/runs/42295403480),
including Android/JVM, Python/native, Rust store-engine and Xbox checks.
- Review follow-ups keep migration IDs stable, preserve unreadable
supported-version libraries before recovery, retain forward-version
protection, release keyboard owners on focus loss, handle unavailable
presentation displays, and allow controller confirmation/cancellation
after key capture. Deck router cleanup checks ownership; BLE teardown
synchronously blocks late routing/publication, and shared Triton slot
ownership protects a replacement controller from stale instances. Legacy
Select-to-Steam migrates to Desktop without replacing saved bindings;
mouse backend transitions are serialized with publication. Profile
refresh now updates custom controls in place, preserving
terminal/dashboard state, the field keyboard and WoW chat. No physical
BLE-controller test or new latency measurement was performed.
- The latest 08526f828 APK is installed on Thor. Profile preferences
stayed byte-for-byte identical and the retained Steam environment
reached READY at 1280×720 / 60 Hz. The earlier capture-dialog fix was
also exercised on Thor. APK SHA-256:
`ed6ef888648468b4ebeb94e8f1ae74a195bb1f0a3f27e4fb3646d5462a4beb6b`.
- Upstream integration preserves the new WoW chat panel,
platform-pointer and PiP fixes. The fast-double-tap stick-click rule and
its six regressions were adapted to the shared renderer; WoW field
selection also preserves Ctrl held by a profile.
- Latest Thor UI pass: existing Setup navigation, profile/layout
previews, action persistence, centered element creation, main-screen
editing over Steam, secondary canvas with primary tools/properties, and
X returning from the docked menu to controls.
- Across the documented device sessions, the user tested native physical
buttons/sticks/triggers, Deck pads/grips, desktop
pointer/clicks/scrolling/arrows, Android keyboard with physical input,
Ctrl+F and Ctrl+A/X/V/S, simultaneous physical/touch holds, menu release
and Home/resume. These historical checks are distinguished from the
latest UI-only pass.

[Revision-specific results, APK hashes and
limitations](https://github.com/Delete228/DroidDeck/blob/codex/unified-controller-input/docs/development/unified-input-validation.md).

## Coordination and remaining work

- #1202 (jmarti326): shortcut capture/device identity and the
shared-assignment discussion informed this work.
- #1305 (kingokksa): shared PC-key definitions were adapted from that
PR; this does not absorb its entire feature set.
- #1414, #581, #1226 and #1150 remain relevant for external-device
input, calibration, keyboard routing and DualSense integration. The
DualSense coordination discussion in this PR describes adapter
boundaries and duplicate-stream suppression.

Import/export, automatic per-app selection, macros, multi-button
activation chords and automatic text-field keyboard detection remain out
of scope. A single binding emitting a key combination is implemented.

The configuration UI is now implemented and exercised on Thor. I am
happy to continue UX/accessibility/localization refinements here before
merge, or in follow-up PRs, according to maintainer feedback. The main
review questions remain the shared model, source ownership/lifecycle
contract, migration policy and integration with other input work.


## UI walkthrough on AYN Thor

Actual captures from the installed Input Test build. In paired images,
the main display is above the second display. The test application name
and local build badge are not production branding changes.

<details>
<summary>Controller settings: separate defaults for Desktop and Game,
followed by the profile library.</summary>

<img width="960" alt="Controller settings: separate defaults for Desktop
and Game, followed by the profile library."
src="https://github.com/user-attachments/assets/a4c156d6-9ded-43df-b0f8-e5f34028806d"
/>

</details>

<details>
<summary>Physical controller: bindings grouped by control type, with
each current action visible.</summary>

<img width="960" alt="Physical controller: bindings grouped by control
type, with each current action visible."
src="https://github.com/user-attachments/assets/a8a840e3-19ac-473f-9f55-b06ae9c637b5"
/>

</details>

<details>
<summary>Reusable layouts: bounded previews and separate screen
targets.</summary>

<img width="960" alt="Reusable layouts: bounded previews and separate
screen targets."
src="https://github.com/user-attachments/assets/cf0cf7d1-42f3-46cd-b50b-f01e1fa8ed41"
/>

</details>

<details>
<summary>Main-screen editor over the live Steam UI, with the movable
tools expanded. The background is Steam artwork, not a running
game.</summary>

<img width="960" alt="Main-screen editor over the live Steam UI, with
the movable tools expanded. The background is Steam artwork, not a
running game."
src="https://github.com/user-attachments/assets/0781f7c8-fd10-47e5-8a23-817f945a6289"
/>

</details>

<details>
<summary>Dual-screen editor: tools and selection on the main display,
full-size placement canvas on the second display.</summary>

<img width="720" alt="Dual-screen editor: tools and selection on the
main display, full-size placement canvas on the second display."
src="https://github.com/user-attachments/assets/f13d4ef6-be6d-4d95-9d42-e047e0a63b8a"
/>

</details>

<details>
<summary>The selected lower-screen trackpad stays visible while its
properties open on the main display.</summary>

<img width="720" alt="The selected lower-screen trackpad stays visible
while its properties open on the main display."
src="https://github.com/user-attachments/assets/26baf597-f037-419e-9601-e48f086c0d4f"
/>

</details>

<details>
<summary>Session menu: expand Profile to select the physical,
main-screen and second-screen parts independently.</summary>

<img width="720" alt="Session menu: expand Profile to select the
physical, main-screen and second-screen parts independently."
src="https://github.com/user-attachments/assets/27ac4d9c-68b3-4257-8353-23104e2343de"
/>

</details>
This commit is contained in:
Kurt Himebauch authored and GitHub committed 2026-11-07 08:32:50 -05:00
commit 22a675a664
98 files changed
+10181 -4879

No files matched your search

+393
View File
@@ -0,0 +1,393 @@
# Shared input profiles and configurable control surfaces
This change implements a shared routing model for physical controllers and local
screen controls. It is intended for review of the architectural direction, with
a functional configuration UI and hardware validation on AYN Thor. The related
maintainer discussions motivate this work; they are not approval of this
implementation.
The configuration UI keeps the existing Setup tabs. Controller contains mode
defaults, one profile list and device settings. Reusable layouts and actions are
stored separately but are managed within each profile source. Direct editing on
the target display has been exercised on AYN Thor, including editing above a live
Steam session and keeping the secondary canvas separate from the main-screen
tools. Revision-specific checks and remaining limits are recorded in the
validation report.
## Why a shared system
Desktop sessions need mouse, keyboard and application actions from a controller.
A dual-screen device may already have a comfortable lower-screen mouse, leaving
its physical sticks available for other actions. The same logical button or pad
can appear on the main screen, lower screen or a physical device; its location
should not dictate which actions it can perform.
The model separates three concerns:
- **Source role:** physical controller, game overlay (primary) or auxiliary panel
(secondary). Swapping the game and auxiliary panel between displays preserves
the panel's role and bindings.
- **Profile:** references to a physical binding preset and a layout/binding pair
for each touch role. Game and Desktop are editable bases; named profiles can
share parts. Auto has separate Steam and Desktop assignments. Opening a profile
for editing does not activate it.
- **Surface layout:** stable element IDs, types, names, positions and sizes.
Named custom layouts can be used on either touch role. Actions belong to a
separate binding preset associated with that layout. Two desktop profiles can
share their physical bindings and layout while using different touch actions.
A binding selects native controller output, a DroidDeck action, or disabled.
Native output continues through the existing game/Steam path, optionally with a
compatible gamepad remap. A local action consumes that control so, for example,
R3 assigned to Ctrl+F does not also press the guest's R3 button.
```mermaid
flowchart TD
Physical[Android controller events] --> PadBridge
Primary[Primary on-screen controls] --> PadBridge
Secondary[Secondary custom controls] --> PadBridge
Deck[Deck pads and grips] --> DeckControls
BLE[Steam Controller BLE reports] --> TritonInputRouter
Library[Saved profiles, layouts and binding presets] --> Profiles[Resolved active profile by source role]
Profiles --> Router[InputRouter]
PadBridge --> Router
DeckControls --> Router
TritonInputRouter --> Router
Router --> Native[Existing pad / Deck report publication]
Router --> Held[Shared key and mouse ownership]
Router --> Commands[Existing menu and keyboard hosts]
Router --> Motion[InputMotion: active stick pointer / scroll only]
Motion --> Held
Held --> Guest[Existing guest keyboard and mouse output]
Native --> Steam[Steam Input or guest controller consumer]
Layout[Independent surface geometry and element IDs] --> Secondary
Layout --> Primary
```
These are routing relationships, not additional asynchronous queues. Native
publication is performed by the existing source adapters and writers.
## Implementation
| Component | Responsibility |
| --- | --- |
| `InputProfile` | Logical controls, source roles and supported actions |
| `InputLibraries` / `InputLibrary` | Versioned reusable parts, validated references, selection and migration |
| `InputProfiles` | Compatibility API for the compiled active profile and legacy migration |
| `InputSurfaceLayout` | Bounded normalized touch geometry and legacy layout migration |
| `InputSurfaceSource` | Independently owned per-element routing through existing pad/Deck publishers |
| `InputRouter` | Resolve channels on profile changes, route events, track held outputs |
| `PadBridge` | Existing player slots plus independently owned local touch sources |
| `DeckControls` | Source ownership for Deck grips, contacts, coordinates and clicks |
| `TritonInputRouter` | Apply routing before Steam Controller BLE report publication |
| `PhysicalInputBindings` | Capture additional Android keys with device identity |
| `HeldInput` | Coordinate router outputs with existing keyboard and gesture outputs |
| `InputMotion` | Frame callback for active synthesized stick pointer/scroll movement |
| `InputTiming` | Optional bounded dispatch-to-publication diagnostics |
Supported actions are native output, compatible native remapping, disabled,
keyboard keys/combinations, mouse buttons, pointer movement, scrolling, four
keyboard directions, and existing menu/keyboard/Steam/quick-access commands.
Analog controls used as buttons have separate activation and release thresholds.
Key combinations press modifiers first and release in reverse order.
Standard controller assignments are shared by the physical role. Additional
captured keys match a device identity plus Android keycode/scancode, rather than
persisting a transient Android device ID. System-reserved keys remain with
Android. This is not a per-device calibration system.
### Controller navigation
Setup keeps its existing top-level tabs. Controller starts with Desktop and Game
mode defaults, followed by one profile list and inline device settings. Opening
or editing a profile does not activate it; the session menu owns the active
profile override. Quick-menu overrides are called out beside the default selectors.
Each profile opens Physical controller, Main screen or Second screen. Control
cards and touch element rows open the same large properties editor. Layout
editing uses the full-size canvas and one draggable, expandable Tools palette. Back
returns to the source and profile being edited, rather than the library root.
Saved layouts and reuse commands live inside the relevant source editor; there
are no parallel top-level layout, binding and profile catalogs.
New profiles copy actions and custom geometry independently by default. Users
can explicitly keep physical controls shared at creation, or copy/link a source
from another profile later. Reusing a saved layout can link geometry without
linking actions. Make independent copies both geometry and actions for that
source. Reset actions affects only the selected profile, preserving its layouts
and detaching shared actions before resetting them.
Main-screen visibility remains a global device setting. Color and opacity belong
to each saved touch layout. The standard gamepad is an ordinary editable layout:
its legacy positions, appearance and stick options are migrated into elements.
Other saved resolutions remain selectable layouts. The previous fixed gamepad
renderer and its separate editor are removed; only its placement calculation is
retained for migration. Vibration, radial stick dead zone, Joy-Con VR, global
shortcuts and timing diagnostics are inline below the profile list. This adds no
settings reads or queues to input dispatch.
### Session profile composition
Both the session drawer and launcher controls sheet expose Profile, Physical
controller, Main screen and Second screen. Choosing a profile loads its three
saved parts; choosing one part creates a mode-specific override without editing
the source profile. Game and Desktop remember their choices separately. Auto
resolves the corresponding default profile. Selecting a complete profile clears
part overrides. Save as new profile creates independent copies of all three
parts, including geometry, appearance and actions.
Fixed lower-screen tabs, side selectors and click bars are removed. Deck pads
and grips, one or two Deck pads, grips alone, the mouse trackpad with keyboard,
and a compact gamepad are ordinary editable layout/binding presets. Their menu,
keyboard and click buttons are ordinary elements too. The compact gamepad reuses
the previous panel geometry calculation. Existing fixed-panel selections migrate
to profile controls on first use. Schema 4 also preserves version-3 custom
layouts and converts legacy fixed layout references without changing physical
mappings. Active quick-menu parts cannot be deleted.
Mouse touchpad gestures reuse `TouchpadGestures` for relative movement, tap,
hold-drag, two-finger scrolling and secondary clicks. A tap uses the element's
configured click action; it does not reposition the pointer. Gesture outputs use
owned router channels and are suppressed while input is suspended or until a
held contact returns to neutral after a profile change. Native Deck pad mode
retains contact/position reporting and click behavior for Steam Input.
### Reuse and migration
Existing `FakeInputWriter` transports, player assignment, guest keyboard/mouse
output, `KeyboardHost` and second-screen presentation are reused. The existing
button, stick, d-pad and trackpad views now live in `InputSurfaceView`; there is
no parallel fixed controller panel. The custom editor arranges the existing button, stick, d-pad and trackpad
views; it does not create another emulated controller. Trigger elements provide
press/release input. A new element is unassigned until an action is explicitly
saved. Native sticks and pads retain their existing click gestures.
Movement/contact and click are separate assignments for sticks and trackpads.
Both use the same properties editor and router ownership rules; migrated click
bindings remain editable, and a custom mouse pad can assign its click separately.
The editor uses the target display at actual size. On a dual-screen device,
Add, Undo, Snap, Save and element properties stay on the activity display; the
auxiliary display is a full-size canvas with no toolbar inset. On one display,
a draggable Tools bubble expands into those same tools, including element
properties, and collapses to leave the canvas accessible. Properties replace the
canvas and return to the same draft. Snapping aligns centers and edges with
other elements and the display, including during resize. Done persists geometry and actions together; Cancel discards the
draft after confirmation. The preview has no input sources and cannot send
input to the running session.
Primary custom layouts follow the existing on-screen-controls visibility setting;
Steam's touch-controller mode retains priority. Secondary layouts use Profile controls mode. Terminal, desktop and dashboard
remain separate panel modes. Steam-managed touch controls keep their own editor; a new editable layout
can start empty or copy a saved custom layout.
Legacy Game/Desktop mappings, key captures, active selection and lower-screen
geometry migrate to the library together. Legacy element IDs preserve their
existing logical bindings. New elements use stable independent IDs, so two
buttons can have different actions or hold the same output without releasing
one another. Referenced parts cannot be deleted, and editing an unsupported
library version cannot overwrite it. Shared edits show their scope; copying a
binding preset from a profile relinks only that profile's selected role.
Library decoding, validation and preset resolution happen during configuration
or profile activation. Dispatch uses preconfigured router channels, with no
preference reads, profile lookups or new asynchronous queues. The physical native
identity path and frame-based stick motion remain in place. These properties do
not substitute for a fresh latency comparison on hardware.
The old touch-only mapping and Select-as-Steam settings migrate once into Game's
primary-screen and physical bindings respectively. Their redundant UI is removed.
`PadBridge` retains raw SELECT state; only the profile maps it to GUIDE. The old
flag is read for migration, not applied again during event handling, so explicit
native/disabled/local bindings and profile resets remain authoritative. Attaching
or removing a router cannot change which raw button a SELECT release clears.
Invalid stored entries fall back to defaults. The global settings reset clears
both profile and surface settings without reimporting the old mappings.
The shared PC-key definitions are adapted from kingokksa's
[#1305](https://github.com/Droid-Deck/DroidDeck/pull/1305) at `76fc9b31`.
Device capture and identity handling build on the approach in jmarti326's
[#1202](https://github.com/Droid-Deck/DroidDeck/pull/1202) at `42db788f`.
The initial implementation adapted these parts rather than merging either PR
wholesale. Upstream subsequently merged #1202; the October 30 integration retains
its global physical shortcuts, saved assignments and configuration UI alongside
the profile bindings.
The November 4 integration reuses upstream's common bottom-screen bar, extracted
trackpad/terminal panels and screen-swap presentation. Custom controls coexist
with the dashboard and KDE desktop panels. Routed on-screen stick updates retain
upstream's throttled visual redraws without deferring pad publication. Upstream
hardware-key translation and mouse capture use the existing output ownership and
release capture while the input-profile editor is open.
Explicit captured bindings in the active profile take precedence over a global
physical shortcut for the same key. Otherwise the global shortcut precedes the
default Back/QAM handling and logical controller routing. A consumed shortcut
does not also reach the guest, and its release remains consumed across menu
changes. Shortcut capture remains available from Input profiles, the session
menu and mode settings. Profile editing gates execution of these shortcuts.
### Ownership and lifecycle
Each source retains its own contribution. Releasing a physical mouse button does
not release an identical button still held on the lower screen. Closing a surface
releases that surface, including the final release on the existing Epic launcher
mouse route. Existing gesture and keyboard outputs participate in shared key and
mouse-button ownership.
Native pad merging uses OR for buttons, the maximum trigger value, and the
strongest complete stick vector. A native Deck trackpad retains one source's
coordinates until that contact ends, then accepts a waiting contact. Concurrent
surfaces do not overwrite its coordinates during the same contact.
Profile changes, suspension and menu entry release old local outputs. Held
controls must return to neutral before activating their new bindings. Device
removal and surface disposal remove their owned state. The existing menu
navigation path remains available independently of gameplay assignments.
When an adapter attaches retained input state to a replacement router, it seeds
the new channels without emitting presses and requires held controls to return
to neutral. Already-open pad rings receive the released state immediately;
attaching a router, changing a profile or enabling input does not open an unused
ring. Steam-menu and VR stick-switch requests use separate system-owned button
pulses, so primary-surface bindings cannot suppress or remap those commands.
The menu and keyboard commands currently open their respective UI; they are not
toggles. Back/B closes the menu. This can be reconsidered with the UX work.
### Hot path and scheduling
Profile JSON, preferences and layout geometry are processed outside event
routing. Channels resolve their actions when configuration changes. The current
router uses mutable per-source state under one synchronized monitor; it is not an
immutable compiled plan. This keeps shared ownership consistent, but contention
with simultaneous BLE input has not yet been measured on hardware.
For an unchanged Game binding, `PadSource` returns the original `PadState`.
The BLE identity route returns its original report array. Modified reports use
reusable buffers and preserve unrelated timestamp, motion, pressure and extension
fields. Normal controller events do not wait for a frame, coroutine or timer.
Only synthesized stick-to-pointer/scroll motion uses `Choreographer`, because a
held stick must keep moving without new device events. Trackpad deltas and button
changes publish synchronously on the normal UI input path. Pointer output arriving
from an off-main-thread adapter is marshalled to the existing UI cursor path;
application commands also use the main handler. Neither schedules native forwarding.
The mapped cursor reuses the existing compositor coordinates and Android cursor
layer. Trackpad contact starts from the current pointer position, preventing
jumps when switching sources. Publishing pending Compose snapshot notifications
before drawing avoids the observed every-other-frame cursor update without
changing the stick response curve or adding smoothing.
See [validation](unified-input-validation.md) for measurements and their limits.
The evidence supports no clear typical latency regression in the tested native
path, not zero overhead or a complete input-to-display latency guarantee.
## Steam ownership and boundaries
Steam remains responsible for game mappings, action sets, layers, radial menus
and its cloud profiles. Native/Steam-managed here means forwarding the controller
through the existing path; it is not a new integration with Steam's profile API.
Steam Mobile Touch retains its own Steam-managed layout and protocol. Its widgets
do not pass through this local remapper. The VR-specific input path also remains
separate. Existing gestures retain their recognizers and share output ownership.
The local UI revision includes named profiles, reusable binding presets and
custom layouts edited on either display. Import/export, automatic per-app
selection, macros, configurable tap/hold behavior and multi-button activation
chords remain deferred. One button emitting Ctrl+F is supported; pressing
multiple buttons to activate a new binding is not implemented. Built-in Steam
touch layouts retain their separate Steam-managed editor.
Physical native remaps stay within compatible report families. Generic custom
buttons choose a supported button output explicitly. Additional hardware
adapters must join before publishing consumed controls; an Activity-only
interceptor cannot cover every raw device. Future adapters should use this
contract without replacing device-specific calibration or protocol handling.
## Related upstream work
Reviewed on 2026-10-24. Several related proposals remain open; this is not a claim
that no other controller work exists or that this implementation supersedes it.
- [#1202](https://github.com/Droid-Deck/DroidDeck/pull/1202): physical shortcuts and
capture. The maintainer's
[centralization feedback](https://github.com/Droid-Deck/DroidDeck/pull/1202#issuecomment-6299647180)
motivates one shared assignment model. Questions there about overlapping
actions inform consumption/ownership; multi-button activation remains deferred.
- [#1305](https://github.com/Droid-Deck/DroidDeck/pull/1305): screen key mappings,
combinations and layout management. This change shares the action model across
physical and screen sources, but does not implement that PR's full layout UX.
- [#1414](https://github.com/Droid-Deck/DroidDeck/pull/1414): external-controller
button and touchpad-press capture, including devices exposing a touchpad press
as pointer input. Our key capture does not yet implement that pointer-contact
adapter. It is a concrete area for coordination.
- [#860](https://github.com/Droid-Deck/DroidDeck/issues/860) and
[#1356](https://github.com/Droid-Deck/DroidDeck/issues/1356): desktop controller
input. This implements controller-driven pointer/keyboard actions, not every
gesture or automatic keyboard behavior requested in #1356.
- [#581](https://github.com/Droid-Deck/DroidDeck/pull/581): hardware calibration and
device profiles remain a separate responsibility from action profiles.
- [#1226](https://github.com/Droid-Deck/DroidDeck/pull/1226): keyboard host selection;
the local action reuses the existing host rather than implementing another IME.
- [#1150](https://github.com/Droid-Deck/DroidDeck/pull/1150): raw DualSense input.
That proposed adapter will need integration before publication; it is not
covered merely by the current Android key capture.
## Review focus
The main questions are whether source roles and profile boundaries match the
project's direction, whether adapters join at the right point, and whether the
shared ownership/lifecycle contract is suitable for future input sources. The
validation document distinguishes device-tested behavior, host regressions and
remaining coverage. UX/UI refinement can follow agreement on those decisions.
### Settings polish and display targeting
The session Profile row expands to show the three saved parts and the editor.
Choosing the profile remains a separate selector. Session editing opens its
settings on the spare display, when available; element properties and editor
commands stay on the activity display. Main-screen placement over a running
session uses a transparent canvas so controls can be arranged around game UI.
Existing visible controls are suspended while editing.
Schema 5 adds an explicit main/secondary/both target to saved layouts. Migration
infers the target from existing profile and quick-menu references, preserving
layouts already used on both screens. Layout rows render a miniature with the
same `InputSurfaceView` renderer; unused legacy copies are retained under Other
saved layouts. Newly created layouts belong to the source being edited.
Action editing retains a draft for each route and action type. Sticks assigned
to a digital action press above 55% on either axis and release below 35%; a
trackpad holds a digital action during contact. These semantics are explained
next to the action. Horizontal scroll uses the same router and existing evdev
REL_HWHEEL / Wayland axis transport, with independent fractional accumulation.
No additional event thread or queue is introduced.
### Screen-key consolidation
Schema 6 imports the retired screen-key overlay into normal saved profiles,
layouts and element bindings. The active old set and each named saved set become
separate imported profiles. Current Game/Desktop defaults and quick selections
remain unchanged; the settings page explains where to find the imported profiles.
The old editor, overlay View, guest-key dispatch path and session-menu entries
are removed. Only a one-time reader of the old preferences remains.
Migration converts normalized centers and density-based sizes into the shared
layout geometry. Key chords, four-way directions, separate eight-way stick
bindings, toggle buttons, built-in symbols, custom image references, shape and
opacity are retained. Blank stick diagonals keep their previous behavior of
pressing neighboring cardinal directions. Imported images are decoded once per
view, sampled to at most 512 pixels, and read only from the app's existing image
directory. New buttons can choose text or an existing built-in key symbol.
A converted library and the original raw preferences are committed together
before old preference entries are removed. Invalid data or capacity failures do
not erase the old library or keys, and settings show a pending-import notice.
The backup is recovery data, not a second runtime input configuration.
Toggle buttons use `InputSurfaceSource` and the same router ownership as every
other element. Releasing one owner cannot release another owner's held key.
Menu suspension, profile changes, cancellation and detach release latched output.
Physical native input continues through its existing fast path; no queue, timer
or additional event thread is introduced for these features.
@@ -0,0 +1,703 @@
# Unified input validation
Recorded on 2026-10-24. These results distinguish the device-tested candidate from
the later upstream rebase and host regression fixes. They do not establish zero
latency or complete coverage of every controller and session mode.
## UI publication candidate, 2026-11-06
The implementation was updated through upstream `a1bc8467f` (merge `aa3757eff`).
The generic action selector now says **Scroll**; binding summaries still state
the configured direction. Closing the docked session drawer now invokes its
ordinary close action instead of disabling the second-screen presentation.
The global-reset regression compares against a pristine compiled library, which
includes the editable gamepad's keyboard element.
- All **944 input/UI tests in 125 suites passed**. The final full JVM run completed
**3,598 tests in 493 suites: 3,573 passed, 25 failed, no errors or skips**.
The failures are the same unchanged upstream log-export tests (24) and
case-sensitive WoW add-on import test (1) described below. Input/UI and global
settings reset passed in this full run.
- Initial full runs exhausted this Mac's free disk and produced secondary native
runtime/idle failures. Obsolete local test APKs were removed. The completed run
used a local Gradle init script with one worker and a fresh test JVM every 25
classes to bound temporary-file use. No repository test configuration was
weakened and no tests were excluded from the completed run.
- The APK passed runtime-input, session-asset, native dependency, alignment and
v1/v2/v3 signature checks. SHA-256:
`611814f11cbac4bab7f97dac5c0b0657d85d8dfbd6b40b741bda1fc13efa6f83`.
It was installed over the separate Input Test app with its environment intact;
as with earlier local candidates, Rust store engines were omitted.
- On AYN Thor, closing the lower-screen menu with its X restored the existing
controls instead of revealing Android. The live Steam editor, separate lower
canvas and upper tools/properties, profile previews, saved layouts, and action
pages were exercised. A single Trigger selection added a centered, selected
element; the reported occasional extra tap was not reproduced. The temporary
editor change was discarded.
- The user also verified native controls, profile switching, trackpad/pointer
and scrolling behavior on the immediately preceding UI candidate. This is
distinct from automated UI navigation and the historical latency measurements.
Actual display captures and test reports are retained locally under
`app/build/reports/input-publish/`. Screenshot publication is subject to the
user's review. No new latency measurement was made for this UI revision.
The follow-up router-ownership fix passed **47 tests** in `InputSourceOwnershipTest`,
`InputSurfaceSourceTest`, and `InputRouterTest`, including two new regressions.
An obsolete activity (or one without a router) cannot detach the replacement
activity's Deck routes; the current owner still releases its held keys on cleanup.
GitHub's complete Build APK workflow passed for `7b7722bd2`.
### Review follow-up
- Upgrade binding IDs are deterministic across unsuccessful reads. Editor lookup
from an earlier read remains valid when legacy screen-key import fails.
- An edit of malformed supported-version data commits both the restored library
and a recovery copy of the original text. Reads alone preserve the original;
newer unsupported versions remain protected against overwrite. Settings show
that recovery data was retained.
- Focus/capture loss releases hardware, Android keyboard and PC-keyboard owners
without releasing another source's hold. Secondary settings handle an invalid
presentation display at initial show and resume.
- Physical-button capture waits for release before confirmation/navigation.
Back and B remain assignable; another Back/B press cancels, A confirms, and
Retry starts a new capture. The existing dialog-window key hook is reused.
- Steam Controller BLE teardown synchronously closes its routed source and
gates state/feature publication. Queued callbacks and starts cannot reactivate
it after stop returns; disconnect work remains on the existing BLE thread.
The input/UI/global-reset run passed **959 tests in 127 suites**. After the
additional BLE finding, **5 BLE/routing tests passed**, including two regressions
for queued callbacks, synchronous release, source detachment and shared holds.
The final APK passed packaging, runtime/native dependency and signature checks:
`a5ec9120be120b829748f8489782a0c7158360d7f8deb61777fe75d601708d20`.
It was installed over Input Test; saved profile preferences remained byte-for-byte
identical and the existing environment was retained.
The button-capture dialog opened and cancelled on Thor without a crash. No
physical BLE-controller check or new latency measurement was made for these
follow-ups.
### Replacement-controller and mouse-backend follow-up
The Triton slot now has a single owner across activity instances. Late reports,
subscription callbacks, feature replies and teardown from a replaced controller
cannot overwrite the current controller's report or connection flag. Every
instance still closes its own routed source. Claiming the slot happens once at
start; a queued or repeated old start cannot take it back.
Legacy Select-to-Steam also migrates to the Desktop physical profile when there
is no saved Desktop profile. An existing Desktop assignment is preserved.
Mouse backend selection and publication share the button-owner lock with raw
channel open, soft release and destruction. Horizontal and vertical scrolling
use the same path and fall back to Wayland after leaving raw mode.
The complete input/UI/global-reset run passed **967 tests in 128 suites**.
The focused run passed **28 tests**, including replacement subscription before
old teardown, unstarted-instance teardown, saved/default Desktop migration, and
concurrent scroll versus raw-channel release/destruction. These changes add no
input queue or timer. Physical BLE hardware and latency remain unmeasured for
this follow-up.
### Upstream integration, 2026-11-07
Merged upstream through `542bbf89c`. The shared stick renderer retains the new
fast-double-tap rule from #2173, including cancelled, dragged, long-held and slow
second taps; the six upstream regressions were adapted to the profile-backed
renderer. WoW chat uses the existing custom lower-screen presentation and shared
key ownership, preserving Ctrl held by another input source. The new platform
pointer, PiP, component-update and store-launch changes are retained.
The merged candidate passed **1017 tests in 133 suites**, with no failures,
errors or skips: all input/UI tests, global reset, component updates, PiP policy,
runtime auto-update and Proton menu auto-selection. The gamescope p22 archive
was downloaded from the newly pinned component repository and verified against
upstream's SHA-256 before refreshing the local runtime inputs.
### Second-screen refresh follow-up, 2026-11-07
Refreshing a profile now updates the existing custom control surface in place.
Terminal and dashboard modes are untouched, and the custom surface keeps its
keyboard host, field-keyboard state and WoW chat panel. Actual mode changes
continue to use the existing full teardown/rebuild path.
All **35 tests in 5 suites passed** across `SessionSecondScreenTest`, the
surface UI suites and `InputSurfaceSourceTest`. Three new regressions cover
terminal/dashboard identity, changed elements/style with an open field keyboard,
and preservation of the lower-screen chat view and its dismissal callback.
## Input settings polish, 2026-11-06
The final input/UI host run passed **941 tests in 124 suites, zero failures**.
New coverage includes retained route/action drafts, centered element creation
without a placement tap, collapsed quick-profile settings, schema-4 migration
with a layout shared across both screens, target/scroll serialization, and
horizontal scrolling through both guest mouse backends. Existing physical native
fast-path and ownership tests remain green.
The settings use stable DroidDeck Input labels, contextual axis/contact hints,
color swatches, opacity sliders and rendered layout thumbnails. Diagnostics now
open a result dialog and explain how to record a sample and what it measures.
Shortcut rows show the actual assignment and offer Restore to default.
No new latency comparison was taken for this UI revision. The normal native
controller path is unchanged. New horizontal wheel work happens only for a local
scroll action. The user's environment installation is retained. Device verification is recorded below.
### Screen-key removal and Thor verification
The old screen-key overlay, standalone editor, layout panel and separate runtime
key sender are removed. Schema 6 imports the old active set and every named set
as editable profiles. Current defaults and selections are kept. Tests cover
idempotent import, chords, normalized geometry, per-element opacity, shapes,
image references, symbols, custom diagonals, and retaining the original data
when migration cannot complete. A backup and the converted library are committed
together before the old preferences are removed.
New shared element tests exercise latched keyboard and native controller buttons,
shared ownership, cancellation, menu suspension and profile replacement. Separate
diagonal bindings release previous keys and preserve neighboring-direction
fallbacks. No additional input queue or thread is used.
Final APK installed over `com.droiddeck.launcher.inputtest` on AYN Thor:
`6618be55c28a52746b6504656a480845d25d8a34114daa1829aad5dbb5b5e729`.
Packaging checked runtime inputs, session assets, native library dependencies and
v1/v2/v3 signatures. As with the earlier test packages, Rust store engines are
omitted; the Steam/Desktop environment is retained.
On-device checks:
- The expanded Profile has readable text and one visibly grouped block.
- Profile source rows and saved-layout rows show bounded thumbnails. The previous
black rectangle overflowing the list no longer appears.
- The original anchored color menu shows its color swatches.
- Steam → Controller → Edit opens settings on the spare display.
- Main-screen editing opens on the activity display over live Steam. A second
on-device crash was found and fixed: returning from lower-display settings must
use the existing activity dialog, not a Presentation on the default display.
- Adding a button immediately centers and selects it. Its properties and symbol
picker open on the main display.
- Secondary-screen editing leaves only the canvas below and tools above. Adding
a trackpad centers and selects it without requiring another placement touch.
- Temporary edits were discarded. Comparing the before/after saved libraries,
excluding new default schema fields, found no changes to the user's profiles,
geometry, bindings or selections. Schema changed from 5 to 6.
- Steam remained READY after closing the editors; the final device log contained
no new fatal exception.
The test device had no old screen-key sets to import. Legacy content migration
and toggle ownership were verified by host regression tests, not by a new
physical-input latency or end-to-end game test. Evidence is under
`app/build/reports/input-polish/`; the final installable copy is
`app/build/input-ui/DroidDeck-Input-Test-unified-screenkeys-2026-11-06.apk`.
## Unified editor and session profile composition, 2026-11-06
The fixed gamepad and its separate editor are replaced with a saved editable
layout rendered by `InputSurfaceView` on either display. The old fixed lower
panel, tab switches and separate mouse trackpad view are removed. Deck pads,
grips, single pads, the mouse trackpad and a compact gamepad are editable presets;
their menu, keyboard and click buttons are normal elements. The compact preset
reuses the previous panel geometry calculation.
Schema 4 migrates version-2 overlay references, mappings, saved positions,
appearance and stick-click preferences, and version-3 fixed lower layouts.
Other resolution-specific positions remain saved layouts. Future or invalid
libraries are not overwritten. Physical mappings retain their identities and
assignments. Existing custom layouts remain intact.
On two displays, the activity hosts Add, Undo, Snap, Done and full-size properties;
the other display contains only the canvas, without a reserved toolbar strip.
On the main display a draggable Tools bubble expands into the same tools,
including Properties. The editor snaps centers and edges while moving and
resizing. Both source pages list their actual elements. Device settings are
inline below the profiles.
The quick menu selects a whole profile or overrides Physical controller, Main
screen and Second screen independently. Game and Desktop selections are separate.
Changes do not mutate the source profiles; saving a combination creates independent
copies. Returning to Auto clears the overrides. Tests cover serialization,
reference validation, deletion protection and preservation of source profiles.
The complete input/UI host run passed **931 tests in 124 suites, zero failures**.
Coverage includes full-size shared editing, floating tools, display removal,
geometry migration, source ownership during detach/layout replacement, quick-menu
composition and saving, native Deck clicks and mouse gestures. Tap and hold use
the configured element click; ordinary taps do not reposition the cursor. Profile
changes and menu suspension suppress delayed gesture presses until neutral.
The neutral-stick/physical-input regression remains covered.
No new end-to-end latency measurement was performed for this revision. The
physical controller path remains unchanged. Touch output uses the existing
owned router and guest transports without an additional event queue. Mouse
movement is delivered synchronously; tap/hold recognition reuses the existing
`TouchpadGestures` timings. The local APK omits the Rust store engines and uses
the separate Input Test application ID.
Device verification on AYN Thor confirmed the four quick selectors and saved
second-screen presets, draggable/expandable tools, full-screen properties and a
canvas-only secondary display. WindowManager reports the main editor at
`[0,0][1920,1080]`; the secondary canvas uses the full `1240×1080` display.
The installed library migrated to schema 4 with existing profile references,
bindings and custom layouts preserved. A previous test migration's unchanged
main overlay geometry was corrected from its window-size copy to its saved
1920×1080 version. No user-edited geometry was replaced.
Final APK SHA-256: `c13326d11f5e8de3b878d83b296ba2f77495baf5c35d1d00dbb0327333c0277d`.
Runtime assets, native library dependencies and v1/v2/v3 APK signatures passed
verification. Focused editor/input checks passed again after fixing Compose 1.6
fullscreen dialog sizing. Live game/desktop input and perceived latency with this
final build still need user testing; the device checks above cover UI and migration.
## Controller integration and upstream update, 2026-11-06
Merged upstream `bf693d7f7` into the working branch as `ba5876bf6` before the UI
integration. This includes 90 upstream commits, including radial stick dead zones
and mouse-button handling. The merge keeps routed source ownership and the new
upstream raw-stick behavior; the targeted input merge checks passed.
Controller now contains two default-profile selectors, one profile list and a
Device settings entry. Source-specific action editing and saved layouts are
inside each profile. New profiles are independent by default, with an explicit
option to share physical controls. Direct editing and Hide tools / Show tools
are retained. No input-event queue or preference lookup was added by this UI
reorganization.
The first full host run executed 3,580 tests in 490 suites: 3,555 passed and 25
failed, with no skips. All 926 input and UI tests in that run passed. The failures
are in unchanged upstream log-export tests (24) and WoW add-on import tests (1).
The log fixtures require `SecureDirectoryStream`, unavailable in this macOS JDK;
the WoW test assumes `Foo` and `foo` refer to different paths, while this host
filesystem is case insensitive. A direct Java filesystem probe confirmed both
conditions. The log-export failures also reproduce when run separately. The
affected production and test sources match `origin/main`. These host limitations
have not been hidden by disabling tests or changing production code.
The final input/UI-only run passed all 927 tests in 122 suites.
Additional final checks cover integration in the existing Setup tabs, the lack
of duplicate Layout/Configure entries, back navigation, independent creation,
explicit physical sharing, and preserving elements while hiding editor tools.
Rendered UI checks cover 360×800, 640×360 and 960×600 dp. Earlier latency results
below are not a fresh measurement of this revision.
The final 26 profile/library/editor checks passed again after the source-specific
control lists and sharing labels were finalized. The APK passed session-asset
matching, runtime-input attestation, native dependency/export checks, alignment
and signing validation. It uses the same local HTTPS/direct-release download
adaptations as the previous build and still omits the store Rust engines.
Installed the update over `com.droiddeck.launcher.inputtest` on AYN Thor without
clearing data. Verified Setup → Controller → Desktop → Second screen → Edit on
screen; the canvas opened on the lower display, Hide tools / Show tools worked,
and selecting the right trackpad opened large properties on the main display.
Returned without saving; the stored input-profile preferences matched the backup
exactly. Device settings had no persisted overrides before or after this check.
No fresh gameplay or latency measurement was performed in this UI pass.
Final APK SHA-256:
`f1d16a503661055a0abe8cc954d3fc94f6b9a1c1c626ac38b6c88dba478810d1`.
## Local UI revision, 2026-11-05
This revision adds the Profiles / Layouts / Bindings libraries, migration of the
previous settings, independently identified touch elements, and a direct editor
on either display with properties on the activity display.
- The complete JVM/Compose run passed 3,306 tests across 467 suites, with no
failures, errors or skipped tests. The native identity fast-path checks remain
in that suite. After the final editor-window adjustment, all nine editor and
adaptive-layout UI tests passed again.
- New coverage includes legacy mapping/geometry/selection migration, shared
physical presets with separate touch actions, explicit private copies, safe
deletion/reset, unsupported schema preservation, independent element actions,
overlapping held outputs, suppression across profile/menu changes, and inert
editor previews, independent stick/trackpad click bindings, same-view
detach/reattach, and saving a draft after the auxiliary display disappears.
- Both Steam and Desktop quick settings expose the same input library entry.
- Compose navigation was exercised at 360×800, 640×360 and 960×600 dp. Rendered
images are generated under `app/build/reports/input-ui/`. The properties Save
control stays outside the scrollable content on the short landscape screen.
- Updated the existing `com.droiddeck.launcher.inputtest` installation on AYN
Thor from 0.6.0 to the local 0.6.9 build without clearing its data. The official
package was not replaced. A local backup of the two input preference files was
taken before migration; all 61 saved Desktop bindings matched after migration.
- On Thor, exercised the quick-settings entry, library navigation, full-size
editing on both displays, selection on the lower display with properties on
the main display, moving an element, Undo and Done. After undoing the move and
saving, the complete stored library matched the pre-edit data.
- The earlier Thor latency measurements below apply to the earlier candidate,
not to this library/editor revision. No new end-to-end or dispatch latency
claim is made.
The complete local runtime build passed session-asset matching, runtime-input
attestation, native export/dependency validation, alignment and signature checks.
The local build-script copy used HTTPS for the Debian archive and direct GitHub
release downloads when the unauthenticated API was rate limited; checksum and
revocation checks remained enabled. No build-script changes are included in the
source diff. As before, the test APK excludes the Epic/GOG/Amazon Rust engines.
The installed APK SHA-256 is
`3e3f41589d45b70294f2d59404f591ae76f2ceba8488657e8821aab1bca224b3`.
Before publishing, also test on Thor: create two Desktop profiles sharing physical
bindings; give them different touch actions; edit and resize on the actual lower
display; open properties on the top display; save/cancel; swap screens; return
from Android Home; disconnect/reconnect the auxiliary display; and repeat the
paired physical-button latency measurement. Verify primary custom overlays with
on-screen controls enabled and secondary layouts in Custom controls mode.
## Revisions and environment
The Thor candidate was `5021e7ac` on base `ac0909ea`, with the local temporary-file,
cursor visibility/handoff/frame cadence, scroll direction and button label fixes
subsequently committed on this branch. Its final APK SHA-256 was
`bb211095d6443e9f85ed1f5d8c5674f527df2937ac6b0efe7d6fc4adcb9dcf62`.
The branch was then rebased onto upstream `906e0d1f`. Review added regressions and
fixes for final touch-source release, concurrent native trackpad contact ownership,
and resetting the new preferences. Those final changes and the rebase have host
test coverage; the installed Thor APK was not silently replaced and those exact
changes have not yet been revalidated on the device.
Local Android builds use Java 17 and NDK `28.2.13676358`. The Thor test APK used a
local Gradle init script to install alongside the official app; no application ID
or app-name changes are part of this PR. Local builds use `-PskipRust=true`, so
Steam/Desktop checks do not establish the behavior of the Epic/GOG/Amazon store
engines in a complete release build.
## Automated checks
On the rebased implementation, the complete `:app:testDebugUnitTest` run passed:
**2,230 tests across 333 suites, zero failures/errors/skips**.
The supported `tools/build_local.sh` debug build also passed, including session
asset matching, runtime-input attestation, native export/dependency validation,
ZIP alignment and signature verification. The APK SHA-256 was
`a7d440b4029f9ad67ca6f44c92ccdd6b9eba9e9a758b28ab0e432256697d49b9`.
The only local environment repair was moving a Finder `.DS_Store` file out of the
NDK's host-toolchain directory; the upstream script counted it as another host.
No build checks were disabled to get past the stale-runtime check after rebasing.
Commands, with Java/SDK paths configured for the local machine:
```sh
./gradlew :app:testDebugUnitTest -PskipRust=true -PndkVersion=28.2.13676358
DROIDDECK_BUILD_VARIANT=debug DROIDDECK_SKIP_RUST=1 \
DROIDDECK_NDK_VERSION=28.2.13676358 bash tools/build_local.sh
# Linux with gcc and g++ available:
python3 -m unittest tools.tests.test_fakeinput_open -v
```
Coverage includes profile persistence and migration, invalid bindings, shared
holds, key order, trigger/direction hysteresis, neutral gating on profile changes,
source disposal, raw BLE report preservation, physical key capture, global reset,
existing controller/gesture/keyboard paths, pointer handoff, and Compose settings
and layout editor behavior.
Two additional ownership regressions were demonstrated failing before their
fixes, then passed in the full suite:
- Removing the last local touch surface must send the final release to the legacy
Epic mouse route, while removing only one of two holds must preserve the other.
- Concurrent native Deck pad surfaces must retain the first contact's coordinates
until it ends, then transfer ownership and release correctly.
The native preload regression probe compiles the actual library and compares
`open`, `openat`, `open64` and `openat64` with libc. It covers ordinary file
creation, `O_TMPFILE` permissions, and opening a directory without a mode argument
(12 combinations). The temporary-file fix addresses the KDE cache permissions
problem encountered during the Desktop test; it does not change controller
transport or publication ordering.
All three test methods passed on Linux x86-64 after rebasing. The same 12
combinations also passed in the earlier i386 probe.
Earlier native testing also exposed three i386 ring-test failures, reproduced on
the unmodified pre-feature baseline. These are not presented as passing or fixed
by this PR. Physical BLE coverage is likewise not implied by synthetic tests.
### PR review follow-up
After the first CodeRabbit review of PR #1439, nine regression tests were added
for deferred pad-ring opening, releases to an already-open ring, system-owned
Steam/VR actions, and retained input when replacing a router. Seven of the new
scenarios reproduced failures before the fixes. The retained-input tests cover
physical and touch button ownership, native sticks, Deck grips and trackpad
contacts; a separate regression verifies return to the native identity fast path
without scheduling output after held controls reach neutral.
The full `:app:testDebugUnitTest` run now passes **2,239 tests across 334 suites,
zero failures/errors/skips**. `:app:assembleDebug` also passes with the existing
verified native inputs and the same Java/SDK/NDK and `-PskipRust=true` settings.
The resulting APK passes `check_session_assets.py` and `runtime_inputs.py apk`;
its SHA-256 is
`1c77c85d5d08c4a054db0528ade33d60eaf8755c09a56a598f342207b27c85d4`.
These follow-up changes have not been installed on Thor or benchmarked on the
device. Retained-state initialization runs when sources attach; ordinary input
keeps its existing synchronous routing and native identity path. The VR command
uses the existing chord handler for its release, retaining the 120 ms pulse.
### SELECT and panel ownership follow-up
The next review covered SELECT-to-GUIDE routing and child-view ownership. The
existing one-time Select-as-Steam migration is exercised through real controller
key events, including simultaneous physical GUIDE ownership. Explicit native,
disabled and local SELECT assignments, and resetting the Game profile, remain
authoritative. Removing the obsolete bridge flag keeps raw SELECT state stable
when a router is attached or removed while the button is held.
Panel regressions use an attached Android view hierarchy and a two-finger
trackpad contact. Switching the other stick to a trackpad, or removing the D-pad
or face buttons, must release only that child's controls and preserve the other
trackpad's contact, coordinates and click. Whole-panel removal still releases
the shared source. These three panel scenarios and the retained SELECT scenario
failed before the fixes; all seven new tests pass after them.
The complete JVM run passes **2,246 tests across 336 suites, zero failures/errors/
skips**. The debug APK build, session-asset check and runtime-input verification
pass with the same local build settings. APK SHA-256:
`0d85b4003cd6b31084e50d603e8b00f239775c9c97f3ec17327264db5dd50612`.
These changes have host coverage only; Thor validation and new latency
measurements remain pending.
### Panel tab teardown follow-up
Tab changes now remove the old child views before releasing the panel's touch
source. Child detach callbacks can publish button releases, so source cleanup
must run after those callbacks to avoid retaining a neutral source.
The new tab-switch regression fails before this fix and passes after it. It
checks held-button release, removal of the old source before later physical
input, and touch input after returning to the Gamepad tab. A second regression
confirms that whole-panel removal already releases the source correctly.
All **22 targeted tests** in the panel ownership, input ownership and second-screen
suites pass. The debug APK build, session-asset check and runtime-input
verification also pass. This follow-up has not been tested on Thor or benchmarked
on the device.
### Upstream integration (2026-10-27)
Merged upstream `43a0ce93` while preserving the published PR history. The merge
retains upstream screen keys, portrait controls and Home/Back shortcuts alongside
the input profiles and custom second-screen layouts. Screen keys have their own
held-key owner, and explicit physical Back bindings take precedence over the
default QAM shortcut.
Queued Home/Back chord reports now use current routed axes, including touch
sources, rather than raw physical axes. Router/profile changes and disabled
routing discard queued reports. Cancelling a chord or releasing all input keeps
held controls blocked until neutral, including Deck grips. Ordinary routed input
still publishes synchronously without waiting for the chord queue.
Eleven integration regressions cover these interactions. Four routed-axis and
cancellation scenarios failed before their fixes. The existing panel ownership
fixture now explicitly selects a Deck controller, matching upstream's restriction
of native trackpad controls to that controller type.
The complete `:app:testDebugUnitTest` run passes **2,632 tests across 385 suites,
zero failures/errors/skips**, using Java 17, NDK `28.2.13676358` and
`-PskipRust=true`. This compiles the Android Java/Kotlin code and resources; a new
APK build, Thor validation and device latency measurements were not performed
for this merge.
### QAM and repeated enable-state review (2026-10-27)
A routed QAM press during an existing emulated chord now publishes simultaneous
button and stick changes immediately. Repeated enable/disable requests no longer
republish unchanged pad or Deck reports; the router reports whether its state
changed, so the bridge does not keep a second copy of that state.
Both regressions failed before the fixes. Two additional tests cover replacing a
disabled router and enabling a router that was disabled before attachment. The
complete `:app:testDebugUnitTest` run passes **2,636 tests across 385 suites, zero
failures/errors/skips**, with the same Java/SDK/NDK settings and `-PskipRust=true`.
These fixes have not been tested or benchmarked on Thor.
### Upstream Home/player-slot integration (2026-10-28)
Merged upstream `213c2da5` without rewriting the published branch. Home/Guide now
follows upstream's primary-player slot selection when the built-in pad is off,
while its report retains routed buttons and axes. The input-settings overlay
continues to gate Home shortcuts and now also gates upstream's Steam volume-key
handling.
A new regression combines Home on player slot 1 with button/stick remapping and
disabled routing. The complete `:app:testDebugUnitTest` run passes **2,707 tests
across 389 suites, zero failures/errors/skips**, using the same Java/SDK/NDK
settings and `-PskipRust=true`. A new APK build, Thor validation and latency
measurement were not performed for this merge.
### Upstream shortcuts and dual-screen integration (2026-10-30)
Merged upstream `6ffe675a` without rewriting the published branch. The merge
preserves upstream's docked second-screen menu, keyboard routing, remembered
panel selection, touch routing and physical shortcuts from #1202, together with
the active input profiles and custom lower-screen layouts.
Captured profile keys take precedence over global physical shortcuts, which in
turn precede default Back/QAM handling. Both use the original device/key identity
before Joy-Con normalization. Input-profile editing blocks shortcut execution;
consumed releases remain consumed when a shortcut opens or closes the menu.
The upstream shortcut configuration remains accessible from the input settings.
Mapped mouse output uses upstream's existing Wayland/raw-mouse helpers, retaining
shared button ownership. Changing the raw-mouse backend releases held buttons on
the outgoing backend. Touchpad zoom has an independent Ctrl owner, so it cannot
release Ctrl held by the profile or screen keys. Mapped pointer movement retains
the frame-callback rendering update and cursor visibility behavior.
Eight new regressions cover these interactions and reopening a custom panel with
either controller type. The upstream shortcut-page UI test now exercises the
replacement Input profiles page. The complete `:app:testDebugUnitTest` run passes
**2,918 tests across 413 suites, zero failures/errors/skips**, using Java 17,
NDK `28.2.13676358` and `-PskipRust=true`. The PR diff against upstream also passes
`git diff --check origin/main`.
This run compiles Android Java/Kotlin and resources. A new APK build, Thor test
and latency measurement were not performed for this merge; the full release
build remains an upstream CI check.
### Upstream PC input and bottom-screen integration (2026-11-04 UTC)
Merged upstream `f0052415` without rewriting the published branch. The merge
retains the common bottom-screen bar, dashboard, KDE desktop panel, game-screen
swap, hardware mouse/keyboard fixes and on-screen stick redraw optimization.
Custom controls use the same bar and preserve their secondary-source bindings
when the game and panel swap displays. Pointer handoff now lives in upstream's
shared `SessionTrackpadView`, used in either display arrangement.
Physical/IME key translation reuses `GuestKeys`, including keyboard Escape scan
codes and the Desktop right-Ctrl alias, while preserving shared key ownership.
The input-profile editor releases upstream's hardware mouse capture. Routed
on-screen stick movement uses the same redraw throttling as the native overlay;
pad publication remains synchronous, and mapped cursor frame updates keep their
existing cadence mechanism.
The pointer-handoff regression now exercises the extracted trackpad view. Menu
tests cover the custom panel alongside upstream's panel choices. A new ownership
regression checks that releasing a hardware right-Ctrl alias cannot release Ctrl
still held by the active profile. Panel teardown tests now use `ComponentActivity`
to supply the lifecycle required by upstream's Compose bar.
The complete `:app:testDebugUnitTest` run passes **3,281 tests across 464 suites,
zero failures/errors/skips**, using Java 17, NDK `28.2.13676358` and
`-PskipRust=true`. Upstream's pinned MaliBridge submodule is initialized and its
required Android server build also succeeds. The PR diff passes
`git diff --check origin/main`.
No new APK installation, Thor regression pass or latency measurement was performed
for this merge. The earlier device measurements do not validate this exact tree.
## Manual AYN Thor results
The user performed the following checks in the installed candidate. The desktop
application was KWrite; native gamepad input was checked in Steam's controller
input test. A loaded game's input-to-display response was not measured.
| Area | Observed result |
| --- | --- |
| Native Game profile | Physical buttons, D-pad, both sticks/clicks and triggers work without observed missed input or stuck controls |
| Desktop defaults | Left stick moves pointer; A/B click; X/Y provide Enter/Escape; D-pad supplies arrows; right stick scrolls and stops at neutral |
| Scroll direction | Stick down advances to higher line numbers; stick up returns to earlier lines |
| Cursor visibility | Controller buttons and stick movement no longer hide the mapped pointer |
| Pointer handoff | Lower trackpad and stick can alternate without a jump; click position matches cursor |
| Cursor cadence | User confirmed smooth movement after the frame update fix; measurements below |
| Android keyboard | Text entry works on the lower screen, including while moving the pointer; controls continue after hiding the keyboard |
| Configured shortcut | R3 assigned to Ctrl+F through the settings UI repeatedly opens search; Y/Escape closes it; subsequent typing has no stuck Ctrl |
| Combined keyboard input | Physical modifier with Android keyboard supports Ctrl+A, Ctrl+X, Ctrl+V and Ctrl+S |
| Custom lower screen | Right pad moves pointer, left pad scrolls, A/B click, and View/Select opens the keyboard |
| Shared mouse hold | Physical A and screen A can hold together; releasing the physical button preserves selection until the screen button is released |
| Menu/held input | User reported successful menu/Ctrl release and ordinary typing afterward |
| Android Home/resume | Physical stick, lower trackpad, clicks and keyboard shortcuts work after returning |
| Layout persistence | Removing/readding a button through the editor persists across APK update and restart; defaults restored through the editor |
| Label rendering | Long View/Select and Menu/Start labels fit within their buttons |
| Native Deck lower screen | Both trackpads, clicks, L4/L5/R4/R5 and physical controls work in Steam's input test |
The editor's individual compatible-native and disabled options are host-tested,
not an exhaustive manual hardware matrix. Menu is currently an open command;
Back/B closes it. Multi-button activation chords are not implemented.
## Relative latency measurement
### What was measured
On the same Thor, timestamps from the built-in controller's evdev stream using
`CLOCK_MONOTONIC` were matched to a read-only guest observer of the published
controller ring. The observer polled with a requested 250 microsecond sleep,
roughly 0.31 ms in practice. Each observation bounds receipt between polls; the
table reports the **upper bound**, not an exact physical latency.
This path includes Android dispatch and publication. It excludes controller
firmware, Steam Input/game processing, rendering and display scanout. USB was
used for control and collection, not as the timed input transport. This is not a
button-to-photon test.
The public 0.6.0 build (`22eac083`) was sampled first. A locally instrumented
baseline at `ac0909ea` was then compared with the feature candidate; their relevant
baseline Java/native input implementation matched. Both used Steam Deck emulation,
1280×720 at 60 Hz, Steam's main screen, and the same disabled low-latency options.
The manual samples used physical B presses. Automatic samples injected B edges
into the built-in controller's evdev path, continuing through normal Android
input dispatch. Settled paired runs used a 150-press warmup before 150 measured
presses (300 edges) at approximately 33°C battery temperature. Earlier preliminary
runs were affected by background preparation and are kept separate.
### Results
All values below are milliseconds, kernel event to observer upper bound.
| Run | Matched edges | p50 | p95 | p99 | Max |
| --- | ---: | ---: | ---: | ---: | ---: |
| Public 0.6.0, manual | 84 | 4.379 | 8.155 | 9.334 | 9.729 |
| Local baseline, manual | 82 | 3.876 | 8.322 | 10.667 | 12.447 |
| Baseline automatic, preliminary | 300 | 3.213 | 6.774 | 7.960 | 8.688 |
| Candidate automatic, preliminary | 300 | 2.694 | 5.197 | 6.725 | 8.164 |
| Candidate automatic, settled 1 | 300 | 3.403 | 6.503 | 9.114 | 10.167 |
| Baseline automatic, settled | 300 | 3.061 | 6.269 | 7.160 | 7.850 |
| Candidate automatic, settled 2 | 300 | 2.999 | 5.620 | 8.171 | 13.653 |
| Final installed candidate, extra check | 300 | 3.039 | 5.905 | 7.470 | 11.684 |
The extra final-APK check matched all 300 edges without an observed overrun or
extra/missing edge. Its p50 interval was 2.692–3.039 ms. It ran at 32°C with internal
timing disabled, without another paired baseline or the explicit 150-press warmup;
it is supporting regression evidence, not a controlled speed comparison.
A separate internal sample of 600 dispatch-to-publication observations reported:
| Build | p50 (µs) | p95 (µs) | p99 (µs) |
| --- | ---: | ---: | ---: |
| Baseline | 112.6 | 274.2 | 442.2 |
| Candidate | 116.9 | 315.2 | 561.7 |
The 4.3 µs median difference is the difference between those runs, **not an
isolated measurement of router cost**. The tested typical native path remained
within baseline variation. Candidate tail values were higher in some runs and
were not isolated to a cause. No claim of a speedup, zero cost, or unchanged worst
case is supported by these samples.
### Cursor frame cadence
The initial stick-driven cursor updated at a median 33.333 ms interval, while
the lower trackpad updated at 16.667 ms. Publishing pending Compose snapshot
notifications before the frame's draw traversal changed the stick median to
16.667 ms. Of 119 intervals, 115 were 14–20 ms and none were 30–37 ms. The remaining
four included longer start/stop gaps, so these are not a continuous-load frame-time
distribution. The user confirmed the visible improvement. Speed, sensitivity and
the stick response curve were unchanged.
This is a rendering cadence check, separate from event propagation latency.
## Remaining validation
- Repeat the short Thor regression pass on the rebased final branch.
- Measure simultaneous BLE and touch input, contention and allocations on hardware.
- Compare Desktop key/pointer and native Deck touch latency quantitatively.
- Repeat under a loaded game/rendering workload and, if needed, measure display
response with high-speed video.
- Exercise physical Bluetooth devices and other device-specific adapters.
- Let upstream CI build and test the complete release configuration, including
store engines omitted from the local Steam/Desktop APK.
These limits should remain visible in the PR while maintainers review the
architecture. Host tests and a successful APK build do not replace those checks.