* feat: adjustable RAM and VRAM caps for the container
Performance > Memory gets two sliders:
- RAM cap (soft, Steam sessions): when the session is resident above the
cap, the session watcher compresses the Steam client into zRAM, waits for
its page-out acknowledgment, then compresses the rest of the session
(processes that loaded libblsession) if still over. Nothing is closed.
Measured at most every 30 s; the client is read back once usage drops
below the cap. Invalid values leave the cap off. Needs zRAM swap.
- VRAM cap: sets dxgi.maxDeviceMemory ahead of the game's own DXVK_CONFIG,
so Direct3D games budget for less video memory. A game profile still
wins. Applies from the next session.
* fix: back off the RAM cap when paging out cannot get under it
A cap below the game's working set was never met, so the watcher paged out
the client and the whole session every 30 s for the rest of the session.
Usage is still measured every 30 s, but after a page-out that leaves the
session over the cap the next one waits twice as long (60 s up to 10 min),
resetting once usage is under the cap. The client step is skipped while the
client is already under 512 MB resident.
* fix: RAM cap covers desktop Steam, signals each group once, no idle churn
- Steam opened from the desktop runs the same session watcher, so the cap
is exported whenever the Steam client runs, not only in Steam mode.
- The second page-out goes to the rest of the session only; Steam was
signalled again even when its own step had been skipped as already small.
- A cap page-out while no game runs stays paged out. Reading the client back
right away pushed it over the cap again and repeated every minute; only a
game closing while over the cap still defers a read-back.
* fix: RAM cap marks the client packed only when it compressed it
When the cap skipped the client as already small during a game, it still
recorded the client as packed, so the regular in-game client page-out
waited up to 300 s for a page-out that never happened.
---------
Co-authored-by: jazzyjames <182448902+jazzyjamesgames@users.noreply.github.com>
GE-based Protons, the default with the x86-64 Steam client, run umu-protonfixes. Its Black Ops III fix switches the prefix to Wine's ALSA driver, which has no device here and replaces DirectAudio, and its Dark Souls Remastered fix runs the VC++ 2017 installer, which fails under ARM64 Proton.
Xbox Cloud Gaming: stream from the store and Games tab, native player with DroidDeck frame generation
Stream Xbox games with Xbox Cloud Gaming (Game Pass) from DroidDeck, outside
the Linux session.
- Backend: `xodus-cli cloud-token` (tools/xbox/droiddeck-cloud-token.patch)
gets the streaming tokens; they stay in memory.
- Web player (default): the Greenlight player page (tools/xcloud-player, MIT)
in a WebView, plus Microsoft's own xbox.com/play player. In-stream menu with
screen effects (sharpen, colour, deband, FXAA, CRT, NTSC, Toon, fps cap),
stream quality, performance overlay (level, layout, top right / top left)
and rumble.
- Native player (opt-in, Android 10+, Player engine: Native): Google's WebRTC
(io.github.webrtc-sdk 125.6422.07, BSD) with the Greenlight protocol ported
to Kotlin, in its own process (:xcloudnative) with its own instance of
DroidDeck's compositor. Decoded frames reach the compositor as
AHardwareBuffers (new stream mode, VK_ANDROID_external_memory_android_
hardware_buffer import), so DDFG 2x and LSFG 2x/3x/4x run on them. Live
frame generation, frame-rate target (presets and a 10-165 fps slider, even
pacing), zero-copy picture, small network buffer, fast decoder, instant
controller input, overlay with engine and on-screen fps, background end,
restart and web fallback. It takes the main process's tokens through
XboxCloudTokenService; they are fetched ahead while a cloud game is shown.
- The compositor requests the dma-buf import extensions only where the driver
has them (the system driver has none).
- Catalog: cloud availability, Game Pass plan CFQ7TTC0K6L8 counted, library
filter All / Owned / Game Pass / Cloud, Stream and Add to Games on the store
page, stream-only Games-tab entries (Stream + Download / Play + Stream).
- Xbox games get VC++ 2015-2022 and the ARM64 runtime registry entries
(vcredist2022_arm64, built in until the catalog has it) before each launch.
- Controller focus reaches pickers opened from a game page (install location,
uninstall); Add / Remove from Games keep the focus.
Credits: Greenlight by UnknownSKL (Jim Kroon), MIT, player and protocol;
Better xCloud for the resolution / OS mapping; MGamer404's Bannerlator #627
for the ARM64 runtime approach. Ported from the412banner's Bannerlator.
On some phones (tester: Honor BKQ-N49, Android 17) Xbox sign-in stopped with
"the Xbox helper stopped with exit code 1" before Microsoft's page opened.
The backend asks the app to read and save credentials over a socket and waits
10 s for each reply. The app sealed credentials with a StrongBox-backed
Keystore key, and each StrongBox operation took 7-15 s on that phone, so the
backend gave up. The app still finished those saves afterwards, leaving a
record without a username that every later attempt had to read back slowly.
- Seal new store credentials with a TEE Keystore key (v2 envelopes); v1
envelopes still open with the old key and move to v2 on their next write.
- A sign-in discards an Xbox record without a username (an unfinished
sign-in) before starting the backend.
- The backend waits 30 s for the app's credential reply.
- The backend names the failing step instead of only credential-io (socket
connect/exchange, session lock, missing socket name/port/key), and falls
back to a keyed 127.0.0.1 port when the abstract socket is refused.
- The app's credential listener logs and retries accept errors, serves each
connection on its own thread, and logs slow requests with per-stage times.
Refs #1337, #1300
Co-authored-by: KeegantheVeganat0r <58369898+KeegantheVeganat0r@users.noreply.github.com>
Steam's ARM64 client has no desktop-portal file chooser, so
SteamClient.System.OpenFileDialog failed at once and every Browse button
(Add a Non-Steam Game, shortcut target, library folder) did nothing.
The Steam UI bridge now answers it with a GTK file chooser run under
Steam's reaper, so gamescope shows it over Steam.
droiddeck-steam-shortcuts and AddedGames.accountConfig took the account
marked "MostRecent" in loginusers.vdf, else the only userdata folder.
The arm64 client here writes no MostRecent, so with two accounts in one
install nothing was picked: no added or store game reached
shortcuts.vdf ("no signed-in account yet; nothing written") and Steam
answered their launch with "App configuration unavailable".
Without MostRecent, take the account marked AutoLogin, then the one with
the latest timestamp. Both sides use the same order so the app reads the
config the guest script writes.
Steam installs Proton EasyAntiCheat Runtime (1826330) and Proton BattlEye Runtime (1161040) as tool depots. The launcher's Games tab and the desktop's game entries now skip them like the other runtimes.
The manifest's arguments apply only when its build matches the installed version, only arguments the merge inserted are tracked as its own, escaped quotes split like a Windows command line, and the REDprelauncher rule needs Cyberpunk2077.exe beside it.
Feature level 12_2 makes vkd3d-proton report mesh shaders and ray tracing that Turnip doesn't provide. Caravan SandWitch (Unreal Engine 5) then builds mesh shader pipelines without a pipeline layout and the driver crashes; Cyberpunk 2077 stops at ray tracing initialization. Only these two games get 12_1, and a value set in a game's own settings still wins.
The redistributable seeder marked every process in a game's install scripts as already run, so the Ubisoft Connect installer that Ubisoft games list was skipped and they stopped with "Unable to locate Uplay PC". It is now left to run like the Vulkan runtime installer, and a marker an earlier seeding wrote is taken back out.
## Summary
Follow-up to #1412, focused on repeat work observed in completed CI
logs.
- Give APK compilation and Android tests the same build-label and
API-key inputs. Previously `generateReleaseBuildConfig` changed between
invocations and caused Kotlin/Java recompilation before tests.
- Continue executing the Android test suite on every CI run: disable
reuse of the test task's results in CI while retaining compilation
caching. Keep the existing heap and process isolation settings.
- Cache the four remote checksum-pinned model/AAR downloads used by
add-on packs. Hash-check every cache hit, atomically replace corrupt
entries only after a verified download, and verify the destination
bytes. Packs are still regenerated from current prepared components and
their checksums remain part of the APK.
- Bound download time and retry connection resets as well as HTTP
errors; keep checksum mismatches fatal.
- Include the pack recipes in runtime source freshness checks.
- Restrict PR report updates to the workflow bot's own marker comment,
addressing the minor extraction review finding from #1412.
## Evidence and validation
The previous successful run explicitly recorded changed
`BuildConfig.java` as the reason for repeated Kotlin/Java compilation
between assembly and tests. The final #1412 normal run spent 38 seconds
building/downloading packs, 5m59s building the APK, and 3m03s in the
Android test step. A forced run also hit a connection reset while
fetching pack inputs and passed on retry.
Focused tests cover cold/warm input reuse, corrupt-cache repair,
changed-pin invalidation, rejected download bytes, actual local HTTP
connection failure/retry, identical build metadata and bot-only comment
selection. Actionlint passed. The clean Linux full suite passed 1,149
tests (18 skipped) before the final Gradle test-execution policy block.
Host, APK/ABI and Android checks are green on the final head, including
the ordinary PR run against the updated main branch and both controlled
branch attempts below.
Validation will compare cold and warm runs on the same commit, confirm
cache-hit hash verification, confirm app compilation remains up-to-date
in the test invocation, and confirm the Android suite actually executes
on both runs. Source-tree, preload, APK/ABI, all-test, signing and
publishing gates remain.
Existing Gradle, CMake and Cargo caching remains. Native compilation was
tens of seconds in the inspected baseline; no speculative native cache
changes or unmeasured speed claim are included. Android test execution
itself was about 2m30s; this PR removes avoidable setup/recompilation
rather than weakening coverage or changing concurrency.
## Follow-up correction
The first CI run failed two artwork-key tests that assumed the
compiled-in API key was empty. Keeping build inputs consistent exposed
that assumption. Tests now check the configured build key and its
trimmed fallback after clearing a user key; existing pure-function
empty-key and precedence coverage remains. No key values are included in
assertion messages. The failed run confirmed BuildConfig generation and
Kotlin/Java compilation were UP-TO-DATE in the test invocation;
subsequent full runs passed. The merged #1412 main run40007656633 passed
signing, pack publishing, preview announcement and catalog updates.
## Gradle cache audit fixes
Completed-run logs showed setup-gradle was read-only on PRs.
Same-repository PRs and branch runs can now persist their own cache
entries; fork PRs remain read-only. GitHub cache ref isolation is
retained. See [Gradle cache
guidance](https://github.com/gradle/actions/blob/main/docs/setup-gradle.md)
and [GitHub cache
scopes](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching).
PR caches consume additional shared quota; no claim of reuse between
different PR refs is made.
COMMIT_TIME now comes from SOURCE_COMMIT, matching the provenance field
and avoiding timestamp changes from temporary PR merge commits. The
source tree and pack validation remain unchanged. This enables a
meaningful same-commit warm check. Per-commit generated Java metadata
may still cause cross-commit recompilation; moving metadata into
packaged data is a separate candidate, not part of this patch. Head
9835c4e3 passed full CI and the same-commit warm rerun below.
## Confirmed final results
Both controlled runs use exactly
9835c4e3e1 and workflow_dispatch, avoiding
PR merge-tree changes between attempts.
| Measurement | First attempt | Warm attempt | Reduction |
| --- | ---: | ---: | ---: |
| Request to required gate green | 13m09s | 7m11s | 5m58s (45%) |
| Gradle APK build step | 6m11s | 2m08s | 4m03s (66%) |
| Android test step | 2m57s | 2m08s | 49s |
| Add-on pack build step | 40s | 31s | 9s |
[First
attempt](https://github.com/Droid-Deck/DroidDeck/actions/runs/40015059221/attempts/1)
and [warm
attempt](https://github.com/Droid-Deck/DroidDeck/actions/runs/40015059221/attempts/2)
both passed. Initial queue delay was 3 seconds in each. This is one
observed pair, not a guaranteed latency or isolated attribution of every
timing difference to cache changes. The first attempt already had some
dependency/build cache reuse.
Warm logs confirm the exact head's Gradle-home cache restored; release
Kotlin and Java compilation were FROM-CACHE during assembly, then
UP-TO-DATE during tests. Assembly reported 22 tasks FROM-CACHE. The
Android test executor actually ran; test results were neither cached nor
skipped. All four remote pack input hits were hash-verified; packs were
regenerated. Runtime inputs and APK native exports/dependencies
verified. The ordinary [PR
workflow](https://github.com/Droid-Deck/DroidDeck/actions/runs/40013127993/attempts/2)
also passed against the updated main merge tree.
A fresh Gradle daemon and "no cached configuration" remain expected on a
new runner: configuration/task-graph persistence is distinct from
build-output caching and currently has no configured encryption key. The
measured Kotlin/Java FROM-CACHE results demonstrate output reuse despite
that message.
This proves same-commit warm reuse. New pushes can still invalidate
compilation through changed source, toolchain, generated provenance or
pack metadata, and cache eviction/ref scope can cause misses.
Cross-commit compilation reuse is not established by this pair. Branch
validation intentionally skips main-only publishing; merged #1412's main
run40007656633 separately passed signing and publication. The user
authorized merging after the final cache-recovery correction passed.
## Final cache-recovery correction
Commit c95e1147 addresses the remaining minor review finding: after a
download passes its SHA check, an unexpected directory at the exact
hash-named cache entry is removed before the verified file is promoted.
Invalid downloads leave that directory untouched. A regression test
covers failed verification, successful repair and subsequent warm reuse.
All five focused download tests passed in a clean Linux export. All
final-head PR checks passed in
[run40018498031](https://github.com/Droid-Deck/DroidDeck/actions/runs/40018498031):
host suite ran 1,164 tests (49 platform skips), APK build and native
validation passed, Android tests executed successfully, and the
source/build validation status is green. Benchmark results above remain
measurements of 9835c4e3.
The default FEX preset (Performance + TSO) is fine for Unity. But when
the Default profile is switched to a preset
that turns TSO off (Performance, Extreme, Extreme-gn, Denuvo), Unity
players break. DroidDeck already keeps full TSO
for .NET games (`coreclr.dll`) whatever the preset; this gives Unity
players the same safety net for TSO.
Tested on a Pixel 10 Pro Fold (PowerVR DXT-48-1536), shared preset
Performance (TSO off):
| Game | Build | TSO off | TSO on |
| --- | --- | --- | --- |
| Megabonk | Unity 2023.2 IL2CPP, native Linux, "Linux x86 (FEX)" |
froze on about half of launches (every thread parked on a futex) | 5 of
5 starts, played run stayed responsive |
| Crab Game | Unity IL2CPP, Windows, Proton arm64 | 9 of 10 starts; 1
access violation in `UnityPlayer.dll` (read of a garbage pointer) | 10
of 10 starts |
Measured cost of TSO on Megabonk: none (+2% fps with TSO on, within
noise).
What changes:
- `droiddeck-game-env`: `unity_memory_ordering` sets `FEX_TSOENABLED=1`
when `UnityPlayer.dll`, `UnityPlayer.so` or
`GameAssembly.so` sits next to the executable (or one folder below). A
FEX preset chosen for the game still wins
(fixes sit between the shared preset and the per-game entry, as for
.NET).
- `droiddeck-fex`: the FEX tool now applies the environment half of
these game-file fixes, so native Linux Unity builds
get it too. Launch arguments some fixes add stay Proton-only.
- Docs paragraph in `docs/development/game-environment.md`; tests in
`test_game_environment.py` and
`test_droiddeck_fex.py`.
Device check of the change itself: with the session's
`droiddeck-game-env` patched, shared TSO off and no per-game
entry, Crab Game started with `FEX_TSOENABLED=1`.
Related to #1386, but not a fix for its Risk of Rain 2 crash: that game
is Unity Mono, which with the default preset
already has TSO on; the report points at multiblock instead (as the .NET
fix turns off). Both games tested here are
IL2CPP, so this PR doesn't change multiblock. #1386's other points
(`steam_appid.txt`, `device.txt`) aren't covered.
Tests (WSL, on main `16bedf36` with the patch): `test_game_environment`
+ `test_droiddeck_fex` 119 OK. Full `tools/tests`: 1114 tests, one
failure,
`test_appimage_run…test_a_failed_program_shows_the_log_it_wrote`, which
fails the same way without the patch.
Refs #1248
## Problem
Users report that changing a game's compatibility tool in Steam's game
properties doesn't stick: it goes back to DroidDeck's default. #1248 was
closed by #1250, but its reporter still saw it after clearing the game's
DroidDeck profile.
A game's tool can be chosen in two places, DroidDeck's per-game Proton
(`games.json`, written by `ProtonDefault.requestGame`) and Steam.
`plan_mapping` treats every `games.json` entry as an override that
always wins, so the live helper (`droiddeck-steam-compat`, on
`CompatManager.NotifyStateChanged`) sets it back within a second, and
`steam-compatibility` does again at every session start. A `null` entry
("use the default", also written by a profile reset) is forced as the
default, so resetting the profile made it worse.
Reproduced with the real `plan_mapping` (game 220, Steam set to
`droiddeck-fex`):
| `games.json` entry | before |
|---|---|
| none | kept |
| `null` (profile reset / "Default") | reverted to default |
| a DroidDeck Proton | reverted to it |
## Change
Whichever side changed a game's tool last wins:
- A DroidDeck choice is applied once (as now). Once applied, a different
pick in Steam stands:
- a pick of one of DroidDeck's Protons becomes the DroidDeck choice too
(`games.json` updated, so the app shows it);
- any other pick (FEX, a Steam Linux Runtime, clearing the forced tool)
removes the game from `games.json`.
- `null` is applied once and then dropped, so the game follows the
default like one DroidDeck never managed.
- What was applied is kept in `state.json` as `games_applied`. Choices
an earlier build already applied count as applied (when `state.json` is
from an earlier build), so upgrading respects the Steam picks users
already made.
- `games.json` is rewritten only if the app didn't replace it after it
was read, so a newer app choice is never overwritten.
- An x86-only Valve Proton picked in Steam is still replaced
(`unrunnable`), unchanged.
Both paths share the logic: `register_default` (session start, reads
config.vdf; with the client stopped, an app missing from the mapping
counts as cleared) and the live helper's `reconcile`.
## Tests
- `test_steam_compat_live.py` + `test_game_environment.py`: 109 pass
locally (103 existing + 6 new). The 6 new tests fail on the old scripts.
- Steam pick after an app choice is kept live, and a new app choice
applies again
- Steam pick of a DroidDeck Proton becomes the app's choice
- "use the default" applies once, then Steam picks stand
- upgrade: choices an earlier build applied give way to Steam picks
- an app choice written during a sync is not overwritten
- session start keeps a Steam pick from config.vdf
Not device-tested.
## Side note: `test_authorship` writes into the global git hooks
`InstalledHooksTest.setUp` runs
`tools/release/install_authorship_hooks.py` in a temp repo, but the
installer reads `core.hooksPath` from the **global** config. On a
machine with a global hooks path, running the suite replaced the user's
global `commit-msg`/`prepare-commit-msg`/`pre-push` with hooks pointing
into the deleted temp repo (every later commit failed), and the saved
originals were deleted with the temp repo. The test should isolate
`HOME`/`GIT_CONFIG_GLOBAL`. Not changed in this PR.
## Summary
`droiddeck-fex` kept an `LD_PRELOAD` entry only if `os.path.exists()`
found it. A bare soname that the loader resolves through
`LD_LIBRARY_PATH` doesn't exist relative to the working directory, so it
was dropped before the x86 game started. MelonLoader's bootstrap in
Drova is one example.
Names without a `/` are now passed through. Paths that don't exist and
arm64 libraries are still dropped, as before.
## Proof (REDMAGIC tablet, Stardew Valley on x86 Linux (FEX), launch
options `LD_PRELOAD=libz.so.1 %command%`)
| | before | after |
|---|---|---|
| `LD_PRELOAD` in the FEX game process | missing | `libz.so.1` |
| `libz.so` mappings in the game process | 4 (the arm64 host's) | 10
(the x86 one loaded too) |
| Game reaches its title screen | yes | yes |
## Tests
`tools/tests/test_droiddeck_fex.py` now has a bare soname in the
guest-environment case. It fails on main and passes here; all 51 tests
pass.
## Credits
Reported with the fix idea on the DroidDeck Discord by whatsupbros.
droiddeck-fex dropped every LD_PRELOAD entry that was not an existing path, so a bare soname the loader finds through LD_LIBRARY_PATH (MelonLoader's bootstrap in Drova, for one) never reached the game. Names without a slash are now kept; missing paths and arm64 libraries are still dropped.
Enabling GPU clock pinning froze Thor's kernel utilization counters,
leaving the HUD stuck at an arbitrary value (10–11% in the reproduced
sessions). KGSL_PROP_PWRCTRL disables both the governor and its
utilization sampling.
Use a dedicated KGSL context with a maximum-power constraint, renewed by
a small NOP submission every 25 ms. Keep devfreq running, bound the
queue to one submission, and release the context, mapped command buffer
and GPU allocation on stop or failure. Preserve recovery of legacy
global pins and report native setup failures instead of claiming
success.
Validation:
- AYN Thor: reproduced frozen counters with the old pin. The replacement
native implementation, run as a standalone ARM64 harness alongside the
installed Steam/Bulletstorm session, kept 680 MHz while utilization
changed from 17% in Steam to about 4% loading and 44% at the animated
title. A final-source smoke run also returned changing 60–66% samples at
680 MHz.
- Native lifecycle regression passed on Linux and as an ARM64 executable
on Thor: setup failure cleanup, renewal, bounded queue, runtime failure,
restart, and type-3/type-7 NOP selection for older/newer Adreno cores.
- Android native build and Kotlin compilation passed locally (NDK
27.0.12077973).
- CI build, Android tests, native regressions, and signing passed.
Installed the release-signed PR APK on Thor with app data preserved.
With GPU pin enabled, the normal Steam/Bulletstorm session held 680 MHz
while the HUD changed from 3% loading to 53% in the animated game scene
at 60 FPS; raw gpubusy samples also continued changing. This checks
utilization reporting, not long-term gameplay stability.
The earlier session exit was recorded by Android as LOW_MEMORY; this PR
addresses the utilization freeze.
## Summary
droiddeck-ntsync (the userspace stand-in for the Linux ntsync driver,
Wine sync = ntsync) was slower than droiddeck-esync on every cheap
operation. On the tablet a `SetEvent` cost 0.94 µs against esync's 0.21
µs, and a wait on a signaled event 0.45 µs against 0.08 µs. Four things
made it slow:
- **Signal masking:** every operation blocked and restored the signal
mask, so each call made 2 `rt_sigprocmask` syscalls, and a blocking wait
made 4. The masking itself is needed: if Wine's suspend signal stopped a
thread while it held an object lock, a GC thread that suspended it and
then touched the same object would deadlock.
- **Queueing for objects that were already free:** a wait linked itself
onto every object and unlinked again even when the first object could
simply be taken.
- **Wakes nobody needed:** a futex wake syscall went out for every
handoff, even when the waiter was still spinning and would have seen the
change anyway.
- **One shared counter:** every call took and dropped a reference on a
single process-wide device counter, so threads working on unrelated
objects bounced one cache line between cores.
### Changes
- **Signals are deferred instead of masked:** handlers installed by
Wine's `ntdll.so` run behind a small trampoline. If such a signal
arrives inside a critical section (a per-thread depth count), it is
saved with its `siginfo` and sent again with `rt_tgsigqueueinfo`
(falling back to `tgkill`) as soon as the section ends.
- `sigaction` queries still return Wine's own handler.
- Synchronous signals (SEGV, BUS, ILL, FPE, TRAP, SYS) are never
wrapped, as before.
- Handlers installed by anything other than `ntdll.so` are left alone.
- **Wait-any fast path:** an object that is signaled and has nobody
queued on it is taken directly, in index order. A wait whose timeout has
already passed returns `ETIMEDOUT` without building a queue.
- **Short spin before sleeping:** a waiter spins for up to 4 µs, bounded
by time rather than by iteration count, before it sleeps on the futex.
- **Wake only sleeping waiters:** the waiter publishes a `sleeping`
flag, and the waker only calls `FUTEX_WAKE` when it is set. Both sides
use seq_cst ordering, so a wake can't be lost. The queue layout changed,
so `NS_VERSION` is now 2.
- **Per-thread device reference:** a thread takes one reference on the
device the first time it uses it and keeps it until it exits or moves to
another device, instead of two atomics on the shared counter per call.
- **More than four signals in one section:** a fifth distinct signal is
held pending with the thread's signal mask, then released once the
section ends. Its handler never runs while an object lock is held.
- **Replay:** deferred signals are replayed with every signal blocked,
so each one is sent exactly once. A held signal is released at the same
signal-nesting level that held it. Handlers installed with
`SA_RESETHAND` are not wrapped, so a one-shot handler keeps its kernel
behaviour.
- **Waits from a handler during another wait's sleep:** they get their
own queue and, for a second device, a reference for that call only. The
outer wait keeps its queue and can't pick up the inner wait's result.
### ntsync becomes the default
- **New installs:** start on ntsync.
- **Existing installs:** move to ntsync once, on the first start of this
version, whatever sync they were on before (a one-time `ntsyncDefaultV1`
marker). Anything picked after that is kept, including esync, fsync and
wineserver.
- **Wording:** the ntsync switch in Performance no longer says
"experimental" or "for testing", in all 33 languages.
- **Unchanged:** per-game sync choices on a game's settings page still
win over the app-wide choice. Proton builds without ntsync still fall
back to droiddeck-esync, then droiddeck-fsync.
- **Tests:** `SyncBackendTest` covers a fresh install, each earlier
choice moving to ntsync once, and a later choice surviving the next
start. All 2,160 unit tests pass.
## Performance (REDMAGIC tablet)
Windows microbenchmark (`syncbench.exe`, x86-64 under FEX), µs per
operation:
| | esync | ntsync before | ntsync after |
|---|---|---|---|
| SetEvent | 0.57 / 0.70 | 0.51 / 0.65 | 0.38 / 0.35 |
| wait on signaled event | 0.13 / 0.13 | 0.44 / 0.43 | 0.27 / 0.20 |
| wait, timeout 0 | 0.27 / 0.25 | 0.52 / 0.51 | 0.19 / 0.18 |
| semaphore release + wait | 0.72 / 0.68 | 0.66 / 0.63 | 0.12 / 0.16 |
| mutex wait + release | 0.77 / 0.73 | 0.66 / 0.64 | 0.12 / 0.12 |
| wait on 8 objects | 0.10 / 0.09 | 0.59 / 0.57 | 0.15 / 0.15 |
| alertable wait | 0.06 / 0.05 | 0.43 / 0.42 | 0.07 / 0.07 |
| event ping-pong, 2 threads | 25.6 / 24.5 | 24.4 / 29.3 | 1.97 / 1.66 |
| 4 threads on one semaphore | 9.4 / 4.6 | 20.3 / 19.1 | 7.3 / 7.6 |
The backend each run actually used was checked from its file
descriptors:
- **esync:** wineserver held eventfds.
- **ntsync:** wineserver held the `droiddeck-ntsync` device and its
timerfd objects.
Native arm64 benchmark against the device directly, µs (before / after):
- set: 0.31–0.36 → 0.06–0.08
- signaled wait: 0.37–0.45 → 0.04–0.08
- semaphore pair: 0.56–0.58 → 0.06–0.07
- mutex pair: 0.56–0.61 → 0.04
- ping-pong: 15–30 → 1.7–2.5
Removing the shared device counter on its own takes 4 threads working on
their own objects from 0.21 to 0.04 µs per operation (x86-64 host).
Games, default Steam launch, The Forest and PEAK at the same scene each
run, 20 s pacing. Columns are esync, ntsync before and ntsync after;
each cell gives two runs.
| | esync | ntsync before | ntsync after |
|---|---|---|---|
| The Forest Low, fps | 64.5 / 61.9 | 70.1 / 74.8 | 72.1 / 72.1 |
| The Forest Low, CPU | 489 / 511% | 434 / 419% | 436 / 426% |
| The Forest High, fps | 58.3 / 57.8 | 72.0 / 69.8 | 72.5 / 69.9 |
| The Forest High, CPU | 506 / 504% | 417 / 429% | 425 / 435% |
| PEAK, fps | 50.4 / 48.6 | — | 47.0 |
| RDR2, launch to signed in / game process | — | — | 29.2 / 42.0 s and
28.3 / 41.1 s |
Two more games, esync against ntsync after, three runs each, default
settings:
| | esync | ntsync after |
|---|---|---|
| Among Us, menu fps (59 cap) | 59.0 / 58.9 / 59.3 | 59.0 / 59.0 / 58.7
|
| Among Us, launch to main menu | 6.65 / 6.84 / 6.85 s | 6.39 / 6.52 /
6.64 s |
| Among Us, CPU | 123 / 132 / 133% | 133 / 133 / 124% |
| Mortal Kombat X attract fight, fps | 56.7 / 54.1 / 56.3 | 61.1 / 56.9
/ 56.2 |
| Mortal Kombat X, 1% low | 20.0 / 30.3 / 30.3 | 30.3 / 30.3 / 30.3 |
| Mortal Kombat X, launch to window | 7.75 / 7.79 / 7.75 s | 7.81 / 7.71
/ 7.63 s |
Mortal Kombat X's CPU moves between 140% and 275% in both modes because
the attract mode changes fighters and stage, so it isn't compared.
The Forest's 1% low was 41.7 fps in every run. PEAK is GPU-bound and
stays in run-to-run noise. Its second ntsync run stopped at the title
menu because the rig's menu input didn't register; the game was still
running at 87 fps, and the same rig miss happened on esync before. RDR2
signs in and reaches the game on the new code.
## Tests
- **`tools/tests/test_ntsync_preload.py` (new):** builds the preload and
checks the emulation against the semantics documented for the kernel
driver:
- semaphore counts and `EOVERFLOW`;
- mutex owners, recursion, `EPERM`, kill/abandon with `EOWNERDEAD` on
read and on the next wait;
- auto and manual events, set/reset/pulse previous state;
- wait-any index order, duplicates, the alert object, `EINVAL` limits,
timeouts;
- wait-all atomicity, duplicates and alert;
- wakes across threads and across processes (inherited fds and fds sent
with `SCM_RIGHTS`);
- 20,000-round ping-pong and 4-thread semaphore stress with infinite
timeouts;
- two signal cases. In one, a handler that re-enters the same object is
fired 2,000 times at a busy thread. In the other, a handler parks the
thread while another thread uses the same object, as Wine's suspend does
under a GC.
- eight real-time signals fired in bursts at a thread busy on one
object, each handler touching that object, so more signals arrive inside
a section than can be deferred;
- a handler that waits on the same device and on a second device while
its thread sleeps in another wait;
- a one-shot (`SA_RESETHAND`) handler installed and fired 300 times at a
busy thread, every one of which must run.
- **The signal tests catch a broken implementation:** with signal
protection removed, the program hangs and the test fails. Before the
last fix, the burst test hung and the outer wait returned the nested
wait's result.
- **The suite passes:** 8 of 8 runs natively on arm64 under proot on the
tablet, and 40 of 40 on the host.
- **On the final build:** The Forest Low ran at 77.7 fps, PEAK at 47.5
fps, and Among Us and two Mortal Kombat X runs played through.
- **The new paths stay out of the way in games:** a build that logged
them saw about 1,400 Wine signals land during ntsync sleeps over two
Mortal Kombat X runs and one The Forest run. None of those handlers
waited on ntsync and no signal was held back, so games run the same code
as before the last fix. The Forest Low ran at 71.6 fps and PEAK at 47.2
fps on the final build.
- **Existing preload tests still pass:** fsync, openat2, usr2 (also with
`BL_SYNC=1`), tracer, dircache, noexec, binfmt, faultreport and esync
packs.
## Credits
- [Linux](https://github.com/torvalds/linux)
`Documentation/userspace-api/ntsync.rst` (GPL-2.0): the object, wait and
error semantics the new test checks. Also added to the README credits.
- Handlers installed with SA_RESETHAND are no longer wrapped, so a deferred one-shot signal can't fall through to the default action.
- Deferred signals are replayed with every signal blocked, so a signal landing mid-replay can't send an entry twice.
- Taking a deferral slot and holding a signal are safe when one signal interrupts another, and a held signal is released only at the nesting level that held it, so a handler's return can't block it again.
A FEX preset that turns TSO off (Performance, Extreme, Extreme-gn, Denuvo)
breaks Unity players under FEX:
- Megabonk (Unity 2023.2, IL2CPP, native Linux build, "Linux x86 (FEX)")
froze on about half of its launches, every thread parked on a futex.
With TSO on: 5 of 5 starts, and a played run stayed responsive.
- Crab Game (Windows IL2CPP build under Proton arm64) crashed on 1 of 5
starts with an access violation in UnityPlayer.dll on a garbage pointer.
With TSO on: 5 of 5 starts.
A Unity player (UnityPlayer.dll, UnityPlayer.so or GameAssembly.so beside
the executable) now keeps FEX_TSOENABLED=1 over the shared preset, the way
.NET games already keep full TSO. The FEX tool applies the environment half
of the game-file fixes too, so native Linux builds get it as well. A FEX
preset chosen for the game itself still wins.
Related: #1386 (its Mono crash happens with TSO already on)
Charles DuBose