ESync, FSync, NTSync + OpenNow + FEX for Desktop Apps (#197)

* Make the Compatible Proton ARM64 the default and label compatible builds

The CompatToolMapping "0" default always points at DroidDeck's Proton
ARM64 tool, and per-game picks of adopted GE/CachyOS builds are kept,
including when the shortcuts writer remaps.

The tool is named after the Proton build it runs, Proton Experimental
ARM64 (Compatible), or Proton 11.0 ARM64 (Compatible) once that newer build
is installed. Adopted builds carry the same Compatible mark, read from a
string resource the app stages into the runtime, and the Proton page
explains which builds have the prefix fixes.

The registrar is renamed steam-compatibility and the tool
droiddeck-proton-arm64. Existing installs migrate on the next session: the
old tool directory is removed, every mapping that named it moves to the new
tool, and the retired bannerlator-steam-compat script is removed from the
runtime. CI and local packaging include the renamed script.

* Drop the duplicate R import left by merging main

* Add userspace ntsync for Proton on Android

Android kernels have no /dev/ntsync and its app policy refuses futex_waitv,
so every Proton 11 build here fell back to wineserver for every wait.
libblsession now serves /dev/ntsync itself: wineserver opens a memfd region,
each sync object is a timerfd that carries its slot, and the kernel ntsync
ioctls are answered in shared memory with futexes, following the kernel
driver's wake order. Valve's Proton Experimental ARM64, GE-Proton and
proton-cachyos use it unchanged.

Locks record their owner so a killed process cannot wedge the others,
signals are held off inside each critical section, dead processes and
exiting threads are swept, and object lifetimes follow close, dup2, dup3,
close_range and received descriptors.

Fast sync is on by default under Performance > Session fixes, and
PROTON_NO_NTSYNC=1 turns it off per game.

* Add droiddeck-esync as the default sync for Proton on Android

Without a kernel ntsync driver, Proton 11 builds here fell back to
wineserver for every wait. droiddeck-esync brings eventfd and shared-memory
sync to the exact Proton build a game runs on: a pack holds that build's
ntdll.so and wineserver with the patch series in tools/droiddeck-esync
applied, and the guest runs Proton from a shadow tree that links the stock
tool and swaps in those two files. A wineserver from a pack still prefers
/dev/ntsync when a kernel driver or droiddeck-ntsync is there.

Packs are built in Valve's SDK image from each release's own wine source
(Proton Experimental ARM64 tags, GE-Proton and proton-cachyos releases); the
control build must reproduce the release binaries byte for byte, and a
signed index lists the packs. The APK bundles the packs pinned in
release.env, the app fetches newer ones, and Steam's compatibility tool
picks the pack for each Proton build, so new Proton releases get packs
without an app update.

The ntdll side reads, writes and polls its eventfds with raw system calls,
so the guest's LD_PRELOAD wrappers stay out of the sync path. The server
abandons a mutex once a thread killed by TerminateThread has really exited
and when its process dies, and a mutex's owner and recursion count change
in one atomic step, so a killed owner cannot touch the next owner's count.

droiddeck-esync is on by default and PROTON_NO_ESYNC=1 turns it off per
game; droiddeck-ntsync, the userspace ntsync, is now an opt-in toggle.

* Watch Valve's Proton releases and build droiddeck-esync packs for them

A scheduled workflow reads the Proton ARM64 builds Steam ships (anonymous
steamcmd app info) and the upstream source tags, compares them with the
previous watch state and starts the pack build for any build that has no
pack yet. The pack build applies the patch series with --fuzz=0 and runs a
smoke test of each new pack against the esstress suite before it is
published.

* Add droiddeck-fsync, keep Steam and the app on one Proton, fix DirectAudio

droiddeck-fsync: libblsession answers futex_waitv(2), which the app
sandbox refuses, in userspace, so Proton's own fsync runs on the stock
Proton files with no pack. Waiters register the words they wait on in a
table beside the fsync shared memory and sleep on a futex of their own;
FUTEX_WAKE on the fsync memory wakes the registered waiters. Slots carry
one owner word that the sweeper and the owner claim with a CAS, and the
mapping table is rebuilt under a lock held across the real munmap, so a
reused address or a recycled slot cannot lose a waiter. It stands in for
a Proton build without a pack (droiddeck-ntsync where fsync is off), takes
over for games with esync off, and the "droiddeck-fsync first" toggle
(BL_FSYNC_FIRST, also per game) puts it ahead of the packs. The games
Proton keeps off fsync and esync never get it. A launch into a prefix
whose wineserver is still running keeps that server's sync choice.

One Proton for Steam and the app: the Components page's Proton is the
default Steam uses, applied live through the client while Steam runs
(droiddeck-steam-compat) and at the next start otherwise; a choice made in
Steam's menu comes back to the app. Requests are ordered by a sequence that
never goes backwards, and the page no longer jumps back to the default
after a swap or restore.

DirectAudio: the wrapper wrote the driver key with its backslashes
stripped, so no game ever used DirectAudio. Wine 11 also loads a builtin
only from the prefix, so the driver is now linked into system32 and
syswow64 before "directaudio,pulse" is selected; mmdevapi falls back to
winepulse whenever DirectAudio cannot start.

* Pick esync, ntsync or fsync from the Steam settings' Games group

A three-way tab row above the FEX preset sets the sync Proton games use. It
writes the existing switches together (droiddeck-ntsync, droiddeck-fsync
first, droiddeck-esync), so the Performance toggles and the tabs always agree,
starts on esync and keeps the choice until it is changed again.

* Make droiddeck-fsync the default sync for Proton games

A fresh install, or one that never picked a sync, starts on fsync; a choice made
in the Steam settings' Games group still sticks. Games Proton keeps off fsync
still fall back to droiddeck-esync.

* Add wineserver to the Wine sync tabs and order them ntsync, fsync, esync, wineserver

wineserver turns every in-process sync off so games wait through wineserver;
picking any other tab turns sync back on. The tabs shrink to fit a narrow pane.

* Shorten the Wine sync description

* Answer openat2 and execveat for FEX in the session preload

FEX opens every file of its x86 rootfs with openat2(RESOLVE_IN_ROOT) and
only falls back to the host path on EXDEV. Under proot on Android openat2
answers ENOSYS, so each lookup of an x86 library silently went to the host
and missed. The exported syscall() now turns openat2 into openat once the
real call reports ENOSYS: RESOLVE_IN_ROOT paths are taken relative to the
root fd (FEX has already resolved the links inside the rootfs), and an
absolute path under RESOLVE_BENEATH fails with EXDEV as the kernel would.

execveat with an absolute path, or relative to AT_FDCWD, and without
AT_EMPTY_PATH or AT_SYMLINK_NOFOLLOW, is plain execve; it is sent as such
because proot refuses execveat with a directory fd, which FEX uses when it
re-executes itself for a child x86 program.

* Run x86 Linux games and programs through FEX

Steam: Linux-only titles that have no ARM64 build (from the client's
appinfo cache: type game, demo, application or beta) are mapped to a new
compatibility tool, droiddeck-fex ("Linux x86 (FEX)", linux to linux,
no required runtime). Valve's own "fex" tool needs pressure-vessel, which
cannot run under proot, so a mapping to it or to a Steam Linux Runtime is
replaced; any other pick the user made stays. Titles that also ship for
Windows stay on Proton. When such a title is installed and FEX or the
sniper runtime is missing, the client is asked to install them.

droiddeck-fex runs the program under the FEX that Steam installs (app
3127680), with Steam Linux Runtime 3.0 (sniper, app 1628350) as the x86
rootfs:
- the sniper platform is linked in place from its usr-mtree (relative
  links only, files/usr -> .), once per runtime version;
- the x86_64 and i386 builds of libblsession and libfakeinput go into the
  runtime's lib/<triplet>/droiddeck and its etc/ld.so.preload, mirroring
  the session's own list, so x86 programs get the pads and the session
  fixes (FEX passes their system calls straight to the kernel, so the
  ARM64 preloads cannot reach them);
- Config.json sets RootFS and the GL and Vulkan thunks, so x86 games draw
  with the device's Turnip and Mesa;
- the FEX preset and the game's own environment from the Steam settings
  apply, and ARM64 entries are dropped from LD_PRELOAD/LD_LIBRARY_PATH.
It also links ~/.steam/sdk64 and sdk32 to the client's x86 steamclient
when the client has one and nothing is there yet.

Desktop: x86_64 and i386 AppImages can be imported now; they unpack
through FEX (or unsquashfs) and record their architecture. Scripts,
AppImages and programs started in a run session use FEX automatically
when they are x86, and each added app has a "Use FEX" Auto/On/Off choice
in its edit dialog. The launchers' "still running" checks also look at
FEX processes, whose exe is FEX rather than the program. A GitHub release
without an ARM64 AppImage falls back to its x86_64 one. Decky's x86
loader starts through droiddeck-fex instead of the fex-mesa rootfs path,
which no runtime ships.

The x86 preloads are built against glibc 2.31 (Debian 11) so they load
in sniper: tools/linuxfs/build-x86-preloads.sh, run in CI and by
build_local.sh. fakeinput's stat hooks fall back to __xstat on a glibc
without a stat export, and mmap64 takes off64_t for the i386 build.

* Keep a FEX unixlib from outliving its DLL

A FEX package or original bundle without the aarch64-unix helpers left
the previous version's libarm64ecfex.so / libwow64fex.so in the Proton,
so the new DLLs loaded a unixlib built for another FEX. A swap now removes
the helpers the package does not carry; FEX then runs DLL-only, which it
already falls back to when Wine cannot load the unixlib.

* Fix the Wine sync choices an audit found

- wineserver tab: it passed Proton's environment through, so a build with
  esync built in (GE, cachyos) still ran esync and a kernel ntsync would
  still be used. It now sets PROTON_NO_ESYNC, PROTON_NO_FSYNC and
  PROTON_NO_NTSYNC (unless the game's own environment sets them).
- ntsync tab: a Proton whose wineserver has no /dev/ntsync client got
  wineserver-only sync although fsync would work. The hash cache records
  whether the wineserver has one; without it the launch goes to
  droiddeck-fsync, and no stand-in or pack turns droiddeck-ntsync on.
- A build with esync built in returned before the per-game fallbacks, so
  a game that turns esync off ran with no in-process sync. It now gets
  droiddeck-fsync, or droiddeck-ntsync where fsync is off too.
- When a pack's proton cannot start and the stock one is used, or the
  selection fails, the prefix's record now says stock, so a second
  launch into the same running prefix no longer picks the pack's ntdll.
- droiddeck-fsync: its mutex is held across fork and re-made in the
  child, a closed or replaced shm fd is forgotten (close, dup2, dup3,
  close_range, closefrom), and a waiter woken while it was leaving on a
  timeout or signal reports the wake instead of losing it.
- BL_SYNC_MIN_NOFILE applies per game to every verb, debug logs carry
  prefixes.json, the device report names the chosen sync instead of the
  three raw switches, and PATCHES.md and the proot notes describe the
  selector, the fsync default and droiddeck-fsync's futex_waitv.

* Name the runtime's scripts and files droiddeck, migrating old installs

The guest scripts are droiddeck-* now (droiddeck-session and the other 19),
as are the files the app and scripts generate (etc/droiddeck, the network
state files, the components dir, the Steam install stamp, the redist marker,
the extra-Proton wrapper and backups) and the compositor's own names
(libdroiddeckwayland, droiddeck_* sources, DROIDDECK_* switches, the
wl_output and seat names).

Installed runtimes carry on: LegacyNames moves etc/bannerlator and
share/bannerlator over and leaves relative links at the old paths before
every proot command, turns any old script into a link to its new name,
moves the generated files and points menu entries at the new scripts.
The session scripts and steam-compatibility move their markers on first
use, the components dir's desired.tsv is rewritten with it, and the
runtime counts as installed under either session script.

Kept: the banner_ahb_v1/banner_desktop_v1 wire names and BANNER_AUDIO_DIRECT
(read by prebuilt Turnip and Wine builds), Banners-Turnip and other
third-party names, BL_* switches, libblsession, and credits to Bannerlator.

* Unpack x86 AppImages without FEX, then ask Steam for FEX

Importing an x86 AppImage failed when FEX was not installed yet ("needs FEX
to unpack and no unsquashfs is installed"). AppImages now unpack with
uruntime (VHSgunzo, MIT, v0.8.1 aarch64, pinned by sha256 and shipped
unmodified with its licence), which reads SquashFS and DwarFS images of any
architecture natively. FEX's own --appimage-extract and unsquashfs stay as
fallbacks, and an ARM64 image whose own runtime cannot unpack it goes the
same way.

An x86 import with FEX missing records that FEX is wanted, so the next
Steam session requests FEX and the Steam Linux Runtime, and the program's
page says FEX is not set up yet and how to get it.

* Run Vulkan Qt Quick AppImages on Wayland

OpenNOW (Qt 6.8, Qt Quick on Vulkan) closed within two seconds in a run
mode. The session sends toolkits to gamescope's Xwayland, and Qt's xcb
backend only presents with Vulkan when Mesa's X11 WSI does, which needs
DRI3; Xwayland on KGSL has none, so Qt found no queue that can present and
aborted ("Failed to create RHI").

An AppImage that bundles Qt Quick, the Vulkan loader and Qt's Wayland
plugin now runs on gamescope's Wayland socket (the desktop's, outside
gamescope), where Turnip presents as it does for games, with Qt's xcb
backend as the fallback. SDL_VIDEODRIVER and XDG_SESSION_TYPE follow, so
helpers that pick their window system from them (OpenNOW's streamer) agree
with Qt. Everything else stays on X11; a qt file in the app's directory
(wayland or xcb) overrides the choice.

* Keep gamescope's WSI layer away from Qt apps on its Wayland socket

OpenNOW still closed at once: its FFmpeg (the Raspberry Pi fork, libavutil
61) creates its Vulkan queues internally synchronized whenever the driver
offers VK_KHR_internally_synchronized_queues, which DD-Turnip does. The
gamescope WSI layer registers every queue of a new device through
vkGetDeviceQueue, which only returns queues created without flags, so it
got VK_NULL_HANDLE and vkroots aborted on assert(obj) before Qt opened a
window.

Programs that draw on gamescope's Wayland socket do not need the layer,
which only carries X11 windows past Xwayland, so it is turned off for
them. OpenNOW bundles no Vulkan loader (it reaches Vulkan through Qt and
dlopen), so a binary that uses QVulkanInstance counts as a Vulkan app too,
and a Qt Quick app left on X11 says why in the session log.

* Keep the end of the session log, and say when FEX is missing

The session sends its output through head -c into session.log. head
writes through stdio, so it held up to 4 KB that proot's --kill-on-exit
threw away when the guest ended: every log stopped mid-line at a 4096-byte
boundary, losing exactly the lines that say why a program quit (an x86
AppImage's "x86 Linux programs need FEX" among them). dd writes each read
as it comes and keeps the same cap.

A run mode always ends with status 0 (gamescope does not pass its child's
on), so a program that could not start just closed. When the log says FEX
is missing, the ended screen now says so and how to get it.

The app reads FEX's readiness from status.json, which only a launch or an
import wrote, so it still said "not set up" after Steam installed FEX.
droiddeck-fex status rewrites it, and a Steam session ends with it while
FEX is wanted. FEX is also looked for at the tool's root and under files/,
and a tool folder without it lists what it holds.

* Let FEX and its runtime install runnable on an SD card library

A Steam library on shared storage loses the executable bits, so native
tools are staged in private directories bound over it; only Proton and
SteamLinuxRuntime folders were. FEX (3127680) installed there could never
run, droiddeck-fex saw no FEX and kept asking Steam for it. FEX and the
sniper runtime (1628350) now get private directories from their first
download, whatever their install folder is called.

* Keep fsync off in the pack smoke test

A runner whose kernel allows futex_waitv started stock Proton with fsync,
so the server-side baseline never said what the test waits for and every
pack was waved through as untestable; the esync run would have measured
fsync too.

* Judge a pack against stock's own esstress failures

Stock Proton fails esstress's dup wait all under server-side sync as well
(a Wine bug, seen on the phone too), and any stock failure made the test
call the runner unusable. A pack now fails only when it crashes, hangs or
fails a check stock passes.

* Ask Steam for FEX and its runtime one at a time

Both steam://install requests were started together, and the running client
took only the first: FEX installed, the sniper runtime FEX needs never did.
Each request now waits for the client to take it before the next.

* Bundle the first published droiddeck-esync packs in the apk

* Log how an x86 program under FEX ends

FEX keeps its own errors quiet unless asked, so a program it could not run
left nothing in the session log. Its log now goes to stderr, and the AppImage
launcher records the program's exit status or the signal that ended it.

* Show the log a failed AppImage program wrote

Programs like Eden stop themselves on a failed check and say why only in
their own log under the home folder. When the program ends badly, the end of
any log it wrote during the run is copied into the session log.

* Report where an x86 program under FEX hit an illegal instruction

A tiny x86_64 preload, built without libc, catches SIGILL and writes the
address, the instruction bytes and the mapping they belong to before the
program dies as it would have. droiddeck-fex preloads it into x86_64 programs.

* Load the fault reporter from the x86 runtime

Programs under FEX look libraries up in the x86 runtime, so the reporter is
copied there with the other preloads and named by its path inside it.

* Hand FEX its x86 runtime and thunks through the environment

Valve's FEX runs portable and reads its config beside itself, so the
Config.json droiddeck-fex writes was not seen: programs ran without the
runtime. The runtime, thunk libraries and thunk list now also go in FEX's
own environment variables, and each launch says what it was given.

* Use the sync pack built for the installed Proton release

Some Proton releases ship byte-identical ntdll.so and wineserver (Valve
11.0-2b/2c, CachyOS 0702/0703), so a pack built for a newer release
matched an older install exactly and the older install never fetched its
own pack. A pack from a newer release no longer counts as an exact match;
among matches the one built for the installed version wins, then rev.

The app ranks the pack a tool already uses the same way and only fetches
a pack that ranks higher, reads the release epoch from tools.tsv and
refetches when tools.tsv changes. A prefix whose server is still running
re-checks its recorded pack on waitforexitandrun and starts fresh when a
better one now applies.

* Keep published sync packs and revocations consistent

The publish step never replaces a pack that has an entry beside it; a
pack left without its entry by an interrupted run is replaced so the
build stops repeating. Discovery counts revocations the signed index does
not show yet, including superseded packs only the flavor releases list,
so the index is signed again. The release watch always builds every
flavor.

* Fetch the droiddeck-esync packs from DroidDeck-Components

The packs, the signed index and the apk bundle now live in the releases
of Droid-Deck/DroidDeck-Components, together with the pipeline that
builds them, so this repository keeps only the app and the guest code
that consumes them. The app's index and asset URLs point there,
release.env names the repository its bundle comes from, and the apk
builds drop bundled packs that the published index revokes.

* Stop the apk build when the published sync index cannot be read

* Ask Steam for one FEX dependency at a time

The client shows a confirmation dialog for each steam://install, and a
second request replaces the first dialog, so FEX was never installed
when the runtime was requested right after it. The next dependency is
now requested once the previous one has an app manifest.

* Use generic names in the AppImage test fixtures

* Give x86-64 apps under FEX the Vulkan thunk

Apps that bring their own glibc and Vulkan loader skipped FEX's thunk
overlay and loaded their bundled desktop Mesa drivers, which crash on the
phone's GPU during Vulkan setup. droiddeck-fex now adds an x86-64 audit
module when a program's bundled glibc and libstdc++ meet the thunk's
version needs. The module sends libvulkan.so.1 lookups to a nodelete stub
that needs FEX's thunk path. That keeps the thunk mapped when an app
unloads Vulkan and loads it again, since FEX keeps its first trampoline
links and the old addresses get reused.

The fault reporter now restores the default action before re-raising,
because FEX ignores SA_RESETHAND and the handler looped.
This commit is contained in:
MaxsTechReview authored and GitHub committed 2026-10-04 12:43:57 -04:00
1 parent 4cc9f2a3f3
commit 49a5f4aa94
165 files changed
+11709 -9667

No files matched your search

+46 -6
View File
@@ -39,9 +39,9 @@ for tool in curl tar zstd shasum unzip; do
exit 1
fi
done
if [[ -f "${repo_root}/tools/gamescope/release.env" || -f "${repo_root}/tools/wlroots/release.env" ]] \
&& ! command -v gh >/dev/null 2>&1; then
echo "GitHub CLI is required to download the pinned Gamescope and wlroots release assets." >&2
if [[ -f "${repo_root}/tools/gamescope/release.env" || -f "${repo_root}/tools/wlroots/release.env" \
|| -f "${repo_root}/tools/droiddeck-esync/release.env" ]] && ! command -v gh >/dev/null 2>&1; then
echo "GitHub CLI is required to download the pinned Gamescope, wlroots and droiddeck-esync release assets." >&2
exit 1
fi
@@ -129,7 +129,7 @@ docker run --rm --platform linux/amd64 \
mkdir -p "$d/usr/local/bin"
aarch64-linux-gnu-gcc -O2 -Wall -Wextra -o "$d/usr/local/bin/droiddeck-clipboard" tools/linuxfs/clipboard/clipboard.c -ldl
aarch64-linux-gnu-strip --strip-unneeded "$d/usr/local/bin/droiddeck-clipboard"
for script in tools/linuxfs/overlay/usr/local/bin/bannerlator-*; do
for script in tools/linuxfs/overlay/usr/local/bin/droiddeck-* tools/linuxfs/overlay/usr/local/bin/steam-compatibility; do
install -Dm644 "$script" "$d/usr/local/bin/$(basename "$script")"
done
for f in tools/linuxfs/overlay/usr/bin/* tools/linuxfs/overlay/usr/bin/steamos-polkit-helpers/*; do
@@ -166,8 +166,18 @@ docker run --rm --platform linux/amd64 \
exit 1
}
done
test -f "$d/usr/local/bin/bannerlator-session"
test -f "$d/usr/local/bin/bannerlator-proton-extra"
test -f "$d/usr/local/bin/droiddeck-session"
test -f "$d/usr/local/bin/droiddeck-proton-extra"
'
docker run --rm --platform linux/amd64 \
-v "${repo_root}:/src" -w /src debian:bullseye bash -c '
set -euo pipefail
printf "deb http://archive.debian.org/debian bullseye main\ndeb http://archive.debian.org/debian-security bullseye-security main\n" > /etc/apt/sources.list
apt-get -o Acquire::Check-Valid-Until=false update -qq
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends gcc g++ gcc-multilib g++-multilib binutils >/dev/null
tools/linuxfs/build-x86-preloads.sh app/src/main/assets/linuxfs
chown -R '"$(id -u):$(id -g)"' app/src/main/assets/linuxfs
'
github_repo=${DROIDDECK_GITHUB_REPOSITORY:-}
@@ -223,6 +233,36 @@ if [[ -f "${repo_root}/tools/wlroots/release.env" ]]; then
test -f "${linuxfs_dir}/usr/local/lib/droiddeck-wlroots/libwlroots-0.20.so"
fi
. "${repo_root}/tools/linuxfs/uruntime.env"
uruntime_binary=$(cached "${URUNTIME_SHA256}" "${URUNTIME_ASSET}" \
bash -c 'curl -fsSL --retry 3 -o "$out" "$0"' "https://github.com/VHSgunzo/uruntime/releases/download/${URUNTIME_VERSION}/${URUNTIME_ASSET}")
install -Dm644 "${uruntime_binary}" "${linuxfs_dir}/usr/local/lib/droiddeck/uruntime"
install -Dm644 "${repo_root}/tools/linuxfs/licenses/uruntime-LICENSE" "${linuxfs_dir}/usr/local/share/licenses/uruntime/LICENSE"
sync_assets="${repo_root}/app/src/main/assets/droiddeck-esync"
if [[ -f "${repo_root}/tools/droiddeck-esync/release.env" ]]; then
. "${repo_root}/tools/droiddeck-esync/release.env"
sync_archive=$(cached "${SYNC_BUNDLE_SHA256}" "${SYNC_BUNDLE_ASSET}" \
bash -c 'gh release download "$0" -R "$2" -p "$1" -O "$out"' "${SYNC_BUNDLE_TAG}" "${SYNC_BUNDLE_ASSET}" "${SYNC_BUNDLE_REPO}")
rm -rf "${sync_assets}"
mkdir -p "${sync_assets}"
zstd -dc "${sync_archive}" | tar -xf - -C "${sync_assets}"
test -f "${sync_assets}/index.json"
test -f "${sync_assets}/index.json.sig"
sync_index=$(mktemp)
gh release download "${SYNC_BUNDLE_TAG}" -R "${SYNC_BUNDLE_REPO}" -p index.json -O "${sync_index}" --clobber
revoked=$(python3 -c 'import json, sys; print(" ".join(p["id"] for p in json.load(open(sys.argv[1]))["packs"] if p.get("revoked") is True))' "${sync_index}")
for id in ${revoked}; do
if [[ -e "${sync_assets}/packs/${id}.tzst" ]]; then
echo "${SYNC_BUNDLE_ASSET} carries revoked pack ${id}; it is left out of the APK" >&2
rm -f "${sync_assets}/packs/${id}.tzst"
fi
done
rm -f "${sync_index}"
elif [[ -d "${sync_assets}" ]]; then
echo "No tools/droiddeck-esync/release.env: the APK bundles the droiddeck-esync packs already in ${sync_assets}." >&2
fi
mango_dir="${linuxfs_dir}/usr/local/lib/mangoapp"
mango_pkgs="${staging_dir}/mango-pkgs"
mkdir -p "${mango_dir}" "${mango_pkgs}"
-170
View File
@@ -1,170 +0,0 @@
# droiddeck-esync packs
DroidDeck gives Proton three in-process sync backends, picked with the Wine sync tabs in the
Steam settings' Games group: ntsync, fsync (the default), esync and wineserver. The wineserver tab
sets `PROTON_NO_ESYNC=1`, `PROTON_NO_FSYNC=1` and `PROTON_NO_NTSYNC=1` (unless a game's own
environment sets them), so every sync object goes through the server.
- droiddeck-esync: eventfd and shared memory sync. Stock Wine 11 / Proton 11 builds
have no eventfd-based sync, so the patch series under `patches/<series>` adds it to wineserver
and the unix side of ntdll. A Proton build without a pack (a release newer than the published
packs, or one whose pack failed to install) runs stock with droiddeck-fsync standing in, or
droiddeck-ntsync where fsync is off for the game (`PROTON_NO_FSYNC=1`, or one of the games
Proton itself keeps off fsync and esync), so it still gets in-process sync; `PROTON_NO_NTSYNC=1`
as well leaves it on wineserver-only sync. A Proton with esync built in (GE, cachyos) runs its
own esync, and a game that turns esync off moves to droiddeck-fsync or droiddeck-ntsync there
too.
- droiddeck-fsync: Proton's own fsync on the stock binaries. fsync waits with `futex_waitv(2)`,
which the app sandbox refuses, so `libblsession.so` (`BL_FSYNC=1`) answers that system call in
userspace: every waiter registers the futexes it waits on in a shared table next to the fsync
shared memory and sleeps on a futex of its own, and every `FUTEX_WAKE` on the fsync shared memory
wakes the waiters registered on that word. No pack is involved, so it keeps working the moment
Steam updates Proton. It is the default (the fsync tab, `BL_FSYNC_FIRST=1`, also per game),
ahead of droiddeck-esync; on the esync tab it stands in where no pack fits, and it also takes
over for a game whose environment turns esync off (`PROTON_NO_ESYNC=1`, `WINEESYNC=0`, or too
few file descriptors); where fsync is off as well, such a game gets droiddeck-ntsync, with the
pack's wineserver when a pack fits. A launch into a prefix whose wineserver is still running
keeps the sync choice that server was started with (`prefixes.json` in the store), because a
Wine client that disagrees with its server about fsync exits. It inherits stock fsync's behaviour, the same as on a Linux kernel:
a wait-all on a mutex abandoned by a dead thread is not satisfied, a pending user APC is
delivered ahead of an object that is already signaled, a mutex owned by a process killed with
`TerminateProcess` is not abandoned, and `PulseEvent` can miss a waiter that has not run yet.
- droiddeck-ntsync: a userspace implementation of the `/dev/ntsync` interface in
`libblsession.so` (`BL_SYNC=1`), which stock Proton 11 builds use as they would the kernel
driver. It is the ntsync tab. A pack still supplies the binaries when one fits, and its
wineserver then picks ntsync over esync; a Proton whose wineserver has no ntsync client (no
`/dev/ntsync` in it) gets droiddeck-fsync instead.
wineserver picks the backend at startup: `PROTON_NO_NTSYNC` unset -> `/dev/ntsync`
(droiddeck-ntsync, or a kernel driver the device provides) -> fsync (droiddeck-fsync, or a kernel
and sandbox that allow `futex_waitv`) -> droiddeck-esync (needs `WINEESYNC` nonzero) ->
server-side sync. When a pack is in use and droiddeck-esync is on, the guest's `droiddeck-esync`
script also sets `PROTON_NO_FSYNC=1` unless the game's environment sets it, so droiddeck-esync
comes before fsync unless droiddeck-fsync was chosen. A wineserver from a pack reports the
backend on stderr as `droiddeck-ntsync: up and running.` or `droiddeck-esync: up and running.`;
with droiddeck-fsync the stock wineserver prints `fsync: up and running.` after
`droiddeck-fsync: up and running.`. A pack's wineserver also abandons, when a process dies, every
mutex still owned by one of its threads, and, once a thread killed by `TerminateThread` has really
exited, every mutex it still owns. A killed thread keeps running until the kill signal reaches it
and can take a mutex after the server abandoned its mutexes; without these passes that mutex would
stay locked for good (they cover droiddeck-ntsync and kernel ntsync as well). The ntdll side
reads, writes and polls its eventfds with raw system calls, so no `LD_PRELOAD` wrapper of
`read`/`write`/`poll` (the guest preloads several) sits in the sync path or in the unwind of a
thread Wine kills.
The patch changes no wineserver request: `make_pack.py` refuses a build whose
`server_protocol.h`, `request_handlers.h` or `request_trace.h` differ from the unpatched
control build, or whose ntdll.so exports other symbols. Only `files/lib/wine/aarch64-unix/ntdll.so`
and `files/bin-arm64/wineserver` are replaced, so every Proton build needs its own pair, built
from that build's exact wine source. That pair plus `pack.json` is a pack (`<id>.tzst`). On the
device the guest runs Proton from a shadow tree that links the stock tool and holds the two
patched files (`droiddeck-esync`, store in `~/.local/share/droiddeck-esync`).
## Credits
droiddeck-esync follows the esync design and droiddeck-ntsync implements the ntsync interface;
both are the work of Elizabeth Figura for Wine, Proton and the Linux kernel. droiddeck-fsync runs
fsync, by Elizabeth Figura and Paul Gofman for Proton, over `futex_waitv(2)` by André Almeida.
DroidDeck builds on that work for Android, where neither the ntsync driver, `futex_waitv` nor a
Proton build with esync is available.
## Matching
- `ge`, `cachyos`: built against the release tarball's own binaries. `pack.json` `stock` holds
their sha256; the guest uses the pack only when both files hash the same. `make_pack.py` also
requires the control build to reproduce both files byte for byte, so the patched pair differs
from the release only by the patch (`--allow-unreproduced` lifts this for local experiments).
- `valve`: the depots cannot be downloaded without a Steam login, so `stock` holds the control
build's hashes and `source_match` is true: the guest also accepts a tool whose version token
(second field of `<tool>/version`) and ntdll.so exports hash equal the pack's. With the optional
`STEAM_DEPOT_USER` / `STEAM_DEPOT_TOKEN` secrets (and the `DEPOT_DOWNLOADER_SHA256` variable)
the job downloads the depot instead and, when the depot is at the tag being built, makes an
exact pack. The guest's launch line says which kind matched (`exact match` or `source match`).
- A stock wineserver that already has esync built in (it prints `esync: up and running`) never
gets a pack.
## Building and publishing
`.github/workflows/watch-proton-releases.yml` runs every 20 minutes and decides when a build is
due. It asks Steam anonymously (`steamcmd +app_info_print`) which public build of each Valve ARM64
depot app in `flavors.json` is live, and `discover.py watch` compares that, the newest `keep` tags
and releases of every flavor, each flavor's `rev` and its patch series digest with the state of the
last run (`watch-state.json` in the `droiddeck-esync-index` release). A flavor whose builds, rev or
patch series changed, or whose Steam build moved, starts
`.github/workflows/build-droiddeck-esync-packs.yml` for that flavor; nothing else starts a build.
A run that finds no change takes about a minute and builds nothing. If steamcmd fails, the run
watches the tags only and keeps the last Steam state.
`.github/workflows/build-droiddeck-esync-packs.yml` also runs once a day as a safety net, on pushes
to `tools/droiddeck-esync/**` and by hand (flavor, one tag, publish yes/no). Only runs on `main` of
`Droid-Deck/DroidDeck` publish packs; the schedule and pushes do nothing in other repositories,
and manual runs elsewhere only build.
1. `discover.py matrix` lists the newest `keep` builds of each family in `flavors.json` (Valve
tags, GE and CachyOS releases with an arm64 tarball) and drops the ones the published index,
or the `droiddeck-esync-<flavor>` release, already has at the flavor's `rev`, the ones
`revoked.txt` names, and the ones that failed in the last day at the same `rev` and patch
series (a `<key>-r<rev>-<series digest>.failed` asset in the flavor's release). Naming the tag
in a manual run builds it anyway.
2. Each build runs on an arm64 runner with a read-only token, since it runs upstream code: the
wine source is checked out at the tag (GE also runs its `protonprep-valve-staging.sh`),
`build-pack.sh` builds a control and a patched pair inside the SDK image named by the
source's own `Makefile.in` (arm64-llvm), at the flavor's `build_dir` in `flavors.json` (the
path the release itself was built at, which ends up in the binaries), and `make_pack.py` makes
the pack. Patches apply with `--fuzz=0`: a hunk whose context moved fails the build instead of
landing somewhere else. For GE and CachyOS, whose release tarball holds the whole Proton,
`smoke-test.sh` then boots a prefix with that stock Proton and runs `tests/esstress.c` (built
with the SDK's llvm-mingw) under server-side sync as a baseline, and again with the pack under
droiddeck-esync. A pack that fails while stock passes is not published; when stock itself cannot
run on the runner the step only warns.
3. The publish job checks each build's artifact against the matrix (`discover.py packs`) and
uploads it to the `droiddeck-esync-valve`, `droiddeck-esync-ge` or `droiddeck-esync-cachyos`
release as `<id>.tzst` with its index entry `<id>.json`. A build that made no pack gets its
`.failed` marker there (`discover.py failures`), so a patch that does not apply or a stock
wineserver with esync built in is not rebuilt on every run.
4. The index job (environment `droiddeck-esync-signing`) runs only when the run built a pack or
a published pack is not in the index yet. It collects every `<id>.json`, keeps the
highest `rev` per stock pair and version, marks revoked ids, signs `index.json` (ECDSA P-256,
`index-key.pub`) and publishes `bundle-<YYYYMMDD>-<HHMMSS>.tzst` (named after `generated`, so
every signed index has its own), `index.json` and `index.json.sig` to `droiddeck-esync-index`.
`generated` only grows. Packs stay in the index until they are revoked, also when their
entry asset disappears; the previous index counts only when its signature verifies.
A pack is rebuilt only for a new `rev`: bump `rev` of the flavor in `flavors.json` whenever the
patch series or the build changes. Local builds: `BUILD_DIR=<the flavor's build_dir> SYNC_PROFILE=<the
flavor's profile> build-local.sh <sdk-rootfs> <wine-source> <work-dir> <out-dir> [patch-dir]`,
then `make_pack.py`.
## Setup
- Signing key: create the repository environment `droiddeck-esync-signing` (Settings ->
Environments, limit it to `main`) and store the PEM private key as its secret
`SYNC_INDEX_KEY`: `gh secret set SYNC_INDEX_KEY --env droiddeck-esync-signing < index-key.pem`.
The key never enters the repository; the app trusts only `index-key.pub` (also compiled into
`EsyncPacks.kt`), so a new key needs an app release.
- Optional: `STEAM_DEPOT_USER` and `STEAM_DEPOT_TOKEN` repository secrets for exact Valve packs,
plus the repository variable `DEPOT_DOWNLOADER_SHA256`: the sha256 of
`DepotDownloader-linux-arm64.zip` from the DepotDownloader release the workflow pins
(`DEPOT_DOWNLOADER_TAG`). The depot step is skipped while it is unset.
- APK bundle: after an index run, the job summary prints three lines. Put them in
`tools/droiddeck-esync/release.env`:
```
SYNC_BUNDLE_TAG=droiddeck-esync-index
SYNC_BUNDLE_ASSET=bundle-20261002-061733.tzst
SYNC_BUNDLE_SHA256=<sha256>
```
`build.yml` then unpacks that bundle into `app/src/main/assets/droiddeck-esync/` (gitignored). A
published bundle is never overwritten, so the pinned sha256 stays valid; bump `release.env`
to ship newer packs inside the apk. The app fetches newer packs on its own. A pack the bundle
carries that `revoked.txt` names is left out of the apk, with a warning to bump `release.env`.
## Revocation
Add the pack id as the first word of a line in `revoked.txt` and push to `main`. The workflow
re-signs the index with `"revoked": true` for that id; the app deletes installed copies and
never installs it again, and `discover.py` does not rebuild it at the same `rev`. The apk build
also leaves it out of the bundled packs. To ship a fixed pack for the same Proton build, bump the
flavor's `rev`. Runs that only build (other refs, or manual runs without publish) never queue in
front of a publishing run, so a revocation is signed by the next run on `main`.
-27
View File
@@ -1,27 +0,0 @@
#!/bin/bash
set -euo pipefail
if [ $# -lt 4 ]; then
echo "usage: $0 <sdk-rootfs> <wine-source> <work-dir> <out-dir> [patch-dir]" >&2
echo "env: BUILD_DIR (path the build runs at inside the SDK), SYNC_PROFILE (valve or cachyos), JOBS, SOURCE_DATE_EPOCH" >&2
exit 2
fi
here=$(cd "$(dirname "$0")" && pwd)
sdk=$(realpath "$1")
src=$(realpath "$2")
work=$(realpath -m "$3")
out=$(realpath -m "$4")
patches=${5:+$(realpath "$5")}
build_dir=${BUILD_DIR:-/builds/proton/proton/build-dir}
rm -rf "$work"
mkdir -p "$work" "$out"
"$here/build-pack.sh" prepare "$src" "$work" ${patches:+"$patches"}
bwrap --unshare-user --uid 0 --gid 0 --unshare-ipc --unshare-pid \
--bind "$sdk" / --proc /proc --dev /dev --tmpfs /tmp \
--bind "$work" "$build_dir" --bind "$out" /out --ro-bind "$here/build-pack.sh" /build-pack.sh \
--setenv PATH /opt/llvm-mingw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
--setenv HOME /tmp --setenv LANG C.UTF-8 --setenv JOBS "${JOBS:-$(nproc)}" \
--setenv SOURCE_DATE_EPOCH "${SOURCE_DATE_EPOCH:-1700000000}" --setenv SYNC_PROFILE "${SYNC_PROFILE:-valve}" \
--chdir "$build_dir" \
/bin/bash /build-pack.sh build "$build_dir" /out
sha256sum "$out/files/lib/wine/aarch64-unix/ntdll.so" "$out/files/bin-arm64/wineserver"
-125
View File
@@ -1,125 +0,0 @@
#!/bin/bash
set -euo pipefail
usage() {
echo "usage: $0 prepare <wine-source> <build-dir> [patch-dir]" >&2
echo " $0 build <build-dir> <out-dir>" >&2
echo " $0 all <wine-source> <out-dir> [patch-dir]" >&2
exit 2
}
prepare() {
local origin=$1 b=$2 patches=${3:-}
local s=$b/src-wine
mkdir -p "$s"
rsync --filter=:C --exclude '*~' --exclude .git --exclude compile_commands.json \
--exclude configure --exclude autom4te.cache --exclude include/config.h.in --exclude include/config.h.ink \
--exclude include/wine/vulkan.h --exclude dlls/vulkan-1/vulkan-1.spec \
--exclude dlls/winevulkan/loader_thunks.c --exclude dlls/winevulkan/loader_thunks.h \
--exclude dlls/winevulkan/vulkan_thunks.c --exclude dlls/winevulkan/vulkan_thunks.h \
--exclude dlls/winevulkan/winevulkan.json --exclude dlls/winevulkan/winevulkan.spec \
--exclude dlls/ntdll/ntsyscalls.h --exclude dlls/win32u/win32syscalls.h \
--exclude include/wine/server_protocol.h --exclude server/request_handlers.h --exclude server/request_trace.h \
-Oarx --delete "$origin/" "$s/"
for f in include/wine/server_protocol.h server/request_handlers.h server/request_trace.h; do
cp -a "$origin/$f" "$s/$f"
done
if [ -n "$patches" ]; then
local p found=0
for p in "$patches"/*.patch; do
[ -f "$p" ] || continue
patch -d "$s" -p1 --forward --fuzz=0 --no-backup-if-mismatch -s < "$p"
found=1
done
[ "$found" = 1 ] || { echo "no patches in $patches" >&2; exit 1; }
fi
(
cd "$s"
XDG_CACHE_HOME=$b/cache dlls/winevulkan/make_vulkan -x vk.xml -X video.xml
tools/make_specfiles || true
tools/make_requests
) > "$b/generate.log" 2>&1
touch "$b/.prepared"
}
build() {
local b=$1 out=$2
local s=$b/src-wine o=$b/obj-wine-aarch64 d=$b/dst-wine-aarch64
[ -f "$b/.prepared" ] || { echo "$b is not prepared" >&2; exit 1; }
local jobs=${JOBS:-$(nproc)}
local epoch=$(( ${SOURCE_DATE_EPOCH:-1700000000} - 20 ))
local flags args pe_common i386_flags x86_64_flags conf
case ${SYNC_PROFILE:-valve} in
valve)
flags="-Wno-discarded-qualifiers -march=armv8.2-a -mtune=cortex-x3 -O2 -fwrapv -fno-strict-aliasing -ggdb -ffunction-sections -fdata-sections -fno-omit-frame-pointer -ffile-prefix-map=$s=."
args=(--enable-werror --with-mingw=clang --disable-tests --enable-archs=arm64ec,aarch64,i386,x86_64)
pe_common="-O2 -fwrapv -fno-strict-aliasing -ggdb -ffunction-sections -fdata-sections -fno-omit-frame-pointer -ffile-prefix-map=$s=."
i386_flags="-Wno-discarded-qualifiers -mstackrealign -march=nocona -mtune=core-avx2 -mfpmath=sse $pe_common"
x86_64_flags="-Wno-discarded-qualifiers -mcmodel=small -march=nocona -mtune=core-avx2 -mfpmath=sse $pe_common"
conf="../$(basename "$s")/configure"
;;
cachyos)
flags="-Wno-discarded-qualifiers -Wno-stringop-overflow -Wno-incompatible-pointer-types -O2 -march=armv8.2-a -mtune=cortex-x3 -fwrapv -fno-strict-aliasing -ffunction-sections -fdata-sections -fno-omit-frame-pointer -ffile-prefix-map=$s=."
args=(--with-mingw=clang --enable-build-id --disable-tests --enable-archs=arm64ec,aarch64,i386,x86_64)
pe_common="-fwrapv -fno-strict-aliasing -ffunction-sections -fdata-sections -fno-omit-frame-pointer -ffile-prefix-map=$s=."
i386_flags="-Wno-discarded-qualifiers -Wno-stringop-overflow -Wno-incompatible-pointer-types -mstackrealign -O2 -march=nocona -mtune=core-avx2 -pipe -mfpmath=sse $pe_common"
x86_64_flags="-Wno-discarded-qualifiers -Wno-stringop-overflow -Wno-incompatible-pointer-types -mcmodel=small -O2 -march=nocona -mtune=core-avx2 -pipe -mfpmath=sse $pe_common"
conf="$s/configure"
;;
*)
echo "unknown SYNC_PROFILE ${SYNC_PROFILE}" >&2
exit 2
;;
esac
local env=(STRIP=llvm-strip AR=aarch64-linux-gnu-ar RANLIB=aarch64-linux-gnu-ranlib
CC=aarch64-linux-gnu-gcc CXX=aarch64-linux-gnu-g++ LD=aarch64-linux-gnu-ld
RC=aarch64-w64-mingw32-windres WIDL=aarch64-w64-mingw32-widl PKG_CONFIG=aarch64-linux-gnu-pkg-config
"PKG_CONFIG_LIBDIR=/usr/lib/aarch64-linux-gnu/pkgconfig:/usr/share/pkgconfig"
"CFLAGS=$flags" "CPPFLAGS=$flags" "CXXFLAGS=-std=c++17 $flags" "LDFLAGS=" "SOURCE_DATE_EPOCH=$epoch"
"CROSSCFLAGS=$flags" "CROSSLDFLAGS="
"aarch64_CFLAGS=$flags" "aarch64_CPPFLAGS=$flags" "aarch64_CXXFLAGS=-std=c++17 $flags" "aarch64_LDFLAGS="
i386_AR=i686-w64-mingw32-ar i386_RANLIB=i686-w64-mingw32-ranlib i386_CC=i686-w64-mingw32-gcc
i386_CXX=i686-w64-mingw32-g++ i386_LD=i686-w64-mingw32-ld
"i386_CFLAGS=$i386_flags" "i386_CPPFLAGS=$i386_flags" "i386_CXXFLAGS=-std=c++17 $i386_flags" "i386_LDFLAGS="
"i386_PKG_CONFIG_LIBDIR=/usr/lib/i386-w64-mingw32/pkgconfig:/usr/share/pkgconfig"
x86_64_AR=x86_64-w64-mingw32-ar x86_64_RANLIB=x86_64-w64-mingw32-ranlib x86_64_CC=x86_64-w64-mingw32-gcc
x86_64_CXX=x86_64-w64-mingw32-g++ x86_64_LD=x86_64-w64-mingw32-ld
"x86_64_CFLAGS=$x86_64_flags" "x86_64_CPPFLAGS=$x86_64_flags" "x86_64_CXXFLAGS=-std=c++17 $x86_64_flags" "x86_64_LDFLAGS="
"x86_64_PKG_CONFIG_LIBDIR=/usr/lib/x86_64-w64-mingw32/pkgconfig:/usr/share/pkgconfig")
export PATH=/opt/llvm-mingw/bin:$PATH
( cd "$s" && autoreconf -fi && rm -rf autom4te.cache ) > "$b/autoreconf.log" 2>&1
rm -rf "$o" "$d"
mkdir -p "$o" "$d"
( cd "$o" && env "${env[@]}" "$conf" -C --prefix="$d" --libdir="$d/lib" --host=aarch64-linux-gnu "${env[@]}" \
"${args[@]}" ) > "$b/configure.log" 2>&1
( cd "$o" && env "${env[@]}" make -j"$jobs" __tooldeps__ ) > "$b/tools.log" 2>&1
( cd "$o" && env "${env[@]}" make -j"$jobs" dlls/ntdll/ntdll.so server/wineserver ) > "$b/make.log" 2>&1
mkdir -p "$out/files/lib/wine/aarch64-unix" "$out/files/bin-arm64"
llvm-objcopy --strip-debug --set-section-flags .text=contents,alloc,load,readonly,code \
"$o/dlls/ntdll/ntdll.so" "$out/files/lib/wine/aarch64-unix/ntdll.so"
llvm-objcopy --strip-debug "$o/server/wineserver" "$out/files/bin-arm64/wineserver"
chmod 0755 "$out/files/lib/wine/aarch64-unix/ntdll.so" "$out/files/bin-arm64/wineserver"
cp "$s/include/wine/server_protocol.h" "$s/server/request_handlers.h" "$s/server/request_trace.h" "$out/"
sed -n 's/^#define SERVER_PROTOCOL_VERSION \([0-9]*\)$/\1/p' "$out/server_protocol.h" > "$out/protocol"
}
[ $# -ge 1 ] || usage
case $1 in
prepare)
[ $# -ge 3 ] || usage
prepare "$(realpath "$2")" "$(realpath -m "$3")" "${4:+$(realpath "$4")}"
;;
build)
[ $# -eq 3 ] || usage
build "$(realpath "$2")" "$(realpath -m "$3")"
;;
all)
[ $# -ge 3 ] || usage
b=${BUILD_DIR:-/builds/proton/proton/build-dir}
prepare "$(realpath "$2")" "$b" "${4:+$(realpath "$4")}"
build "$b" "$(realpath -m "$3")"
;;
*)
usage
;;
esac
-654
View File
@@ -1,654 +0,0 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import hashlib
import json
import os
import re
import subprocess
import sys
import urllib.parse
from datetime import datetime, timedelta, timezone
from pathlib import Path
from make_index import check_entry
from make_pack import FLAVORS, HERE, PATCHES, REVOKED, SERIES, PackError, patch_digest, read_revoked, sha256_file, slug
FLAVORS_FILE = HERE / "flavors.json"
ARCHIVE = re.compile(r"\.tar\.(gz|xz)$")
ASSIGNMENT = re.compile(r"(?:export\s+|override\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*(?:\?|\+|!|:{1,3})?=\s*(.*)")
REFERENCE = re.compile(r"\$[({]([A-Za-z_][A-Za-z0-9_]*)[)}]")
IMAGE = re.compile(r"[A-Za-z0-9][A-Za-z0-9.-]*(?::[0-9]+)?(?:/[A-Za-z0-9._-]+)+(?::[A-Za-z0-9._-]+)?(?:@sha256:[0-9a-f]{64})?")
REPO = re.compile(r"[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+")
CLONES = ("shallow", "treeless")
PACK_REV = re.compile(r"-r([0-9]+)$")
ARTIFACT_PREFIX = "droiddeck-esync-pack-"
FAILED = ".failed"
RETRY_AFTER = timedelta(days=1)
VDF_TOKEN = re.compile(r'"((?:[^"\\]|\\.)*)"|([{}])')
APP_HEADER = re.compile(r"^AppID : ([0-9]+),", re.M)
ANSI = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
def gh(*args: str) -> subprocess.CompletedProcess:
try:
return subprocess.run(["gh", *args], capture_output=True, text=True, check=False)
except FileNotFoundError as e:
raise PackError("the GitHub CLI (gh) is not installed") from e
def json_values(text: str) -> list:
decoder = json.JSONDecoder()
values = []
position = 0
while True:
while position < len(text) and text[position].isspace():
position += 1
if position == len(text):
return values
value, position = decoder.raw_decode(text, position)
values.append(value)
def gh_api(path: str, paginate: bool = False, missing_ok: bool = False):
result = gh("api", *(["--paginate"] if paginate else []), path)
if result.returncode != 0:
if missing_ok and "HTTP 404" in result.stderr:
return None
raise PackError(f"gh api {path}: {result.stderr.strip() or f'exit {result.returncode}'}")
try:
values = json_values(result.stdout)
except ValueError as e:
raise PackError(f"gh api {path}: {e}") from e
if len(values) == 1:
return values[0]
if values and all(isinstance(value, list) for value in values):
return [item for value in values for item in value]
raise PackError(f"gh api {path}: unexpected output")
def quote(value: str) -> str:
return urllib.parse.quote(value, safe="")
def load_flavors(path: Path) -> dict:
try:
data = json.loads(path.read_text())
except (OSError, ValueError) as e:
raise PackError(f"cannot read {path}: {e}") from e
def fail(why: str):
raise PackError(f"{path}: {why}")
def compiles(pattern: str, where: str):
try:
re.compile(pattern)
except re.error as e:
fail(f"{where}: {pattern!r} is not a regular expression: {e}")
if not isinstance(data, dict) or data.get("format") != 1 or not isinstance(data.get("flavors"), dict):
fail("is not a format 1 flavor list")
image = data.get("sdk_image")
if not isinstance(image, dict) or not all(isinstance(image.get(key), str) and image[key] for key in ("file", "variable", "match")):
fail("sdk_image needs file, variable and match")
for name, cfg in data["flavors"].items():
if name not in FLAVORS:
fail(f"unknown flavor {name!r}")
if not isinstance(cfg, dict) or not isinstance(cfg.get("repo"), str) or not REPO.fullmatch(cfg["repo"]):
fail(f"{name}: repo must be owner/name")
for key in ("keep", "rev"):
if type(cfg.get(key)) is not int or cfg[key] < 1:
fail(f"{name}: {key} must be a positive number")
if not isinstance(cfg.get("series"), str) or not SERIES.fullmatch(cfg["series"]):
fail(f"{name}: series is not a patch series name")
if cfg.get("profile") not in ("valve", "cachyos"):
fail(f"{name}: profile must be valve or cachyos")
if type(cfg.get("source_match")) is not bool:
fail(f"{name}: source_match must be a boolean")
build_dir = cfg.get("build_dir")
if not isinstance(build_dir, str) or not re.fullmatch(r"(?:/[A-Za-z0-9._-]+)+", build_dir) or ".." in build_dir.split("/"):
fail(f"{name}: build_dir must be the absolute path the release was built at")
wine = cfg.get("wine")
submodules = wine.get("submodules") if isinstance(wine, dict) else None
if not isinstance(submodules, list) or "wine" not in submodules \
or not all(isinstance(item, str) and re.fullmatch(r"[A-Za-z0-9._-]+(?:/[A-Za-z0-9._-]+)*", item) for item in submodules):
fail(f"{name}: wine.submodules must list the submodules to check out, wine among them")
if wine.get("clone") not in CLONES:
fail(f"{name}: wine.clone must be one of {', '.join(CLONES)}")
prep = wine.get("prep")
if not isinstance(prep, str) or (prep and not re.fullmatch(r"[A-Za-z0-9._-]+(?:/[A-Za-z0-9._-]+)*", prep)) or ".." in prep.split("/"):
fail(f"{name}: wine.prep must be a path inside the checkout or empty")
if cfg.get("list") == "tags":
families = cfg.get("families")
if not isinstance(families, dict) or not families:
fail(f"{name}: families must name at least one tag family")
for family, settings in families.items():
if not isinstance(settings, dict) or not isinstance(settings.get("tag"), str) \
or not isinstance(settings.get("prefix"), str) or not re.fullmatch(r"[A-Za-z0-9._-]+", settings["prefix"]) \
or not isinstance(settings.get("depot_app", ""), str):
fail(f"{name}: family {family} needs a tag prefix and a tag pattern")
compiles(settings["tag"], f"{name}: family {family}")
if not isinstance(cfg.get("version_token"), str) or "{tag}" not in cfg["version_token"]:
fail(f"{name}: version_token must contain {{tag}}")
elif cfg.get("list") == "releases":
if not isinstance(cfg.get("tag"), str) or not isinstance(cfg.get("assets"), list) or not cfg["assets"] \
or not all(isinstance(pattern, str) for pattern in cfg["assets"]):
fail(f"{name}: releases need a tag pattern and asset patterns")
for pattern in (cfg["tag"], *cfg["assets"]):
compiles(pattern, name)
else:
fail(f"{name}: list must be tags or releases")
return data
def natural_key(text: str) -> list[tuple]:
return [(0, int(part), "") if part[0] in "0123456789" else (1, 0, part) for part in re.findall(r"[0-9]+|[^0-9]+", text)]
def candidate(flavor: str, cfg: dict, tag: str, asset: str = "", token: str = "", depot_app: str = "") -> dict:
return {
"key": slug(f"{flavor}-{ARCHIVE.sub('', asset) if asset else tag}"),
"flavor": flavor,
"repo": cfg["repo"],
"tag": tag,
"asset": asset,
"token": token,
"depot_app": depot_app,
"rev": cfg["rev"],
"series": cfg["series"],
"profile": cfg["profile"],
"source_match": cfg["source_match"],
"submodules": " ".join(cfg["wine"]["submodules"]),
"clone": cfg["wine"]["clone"],
"prep": cfg["wine"]["prep"],
"build_dir": cfg["build_dir"],
}
def tag_candidates(flavor: str, cfg: dict, ref: str) -> list[dict]:
found = []
for family, settings in sorted(cfg["families"].items()):
if ref and not ref.startswith(settings["prefix"]):
continue
refs = gh_api(f"repos/{cfg['repo']}/git/matching-refs/tags/{quote(settings['prefix'])}", paginate=True)
if not isinstance(refs, list):
raise PackError(f"{cfg['repo']}: the tag list is not a list")
pattern = re.compile(settings["tag"])
names = set()
for item in refs:
name = item.get("ref", "") if isinstance(item, dict) else ""
if isinstance(name, str) and name.startswith("refs/tags/") and pattern.search(name[len("refs/tags/"):]):
names.add(name[len("refs/tags/"):])
ordered = sorted(names, key=natural_key, reverse=True)
chosen = [name for name in ordered if name == ref] if ref else ordered[:cfg["keep"]]
for name in chosen:
found.append(candidate(flavor, cfg, name, token=cfg["version_token"].format(tag=name), depot_app=settings.get("depot_app", "")))
return found
def pick_assets(assets: list, patterns: list[re.Pattern]) -> list[str]:
chosen = {}
for asset in assets:
name = asset.get("name") if isinstance(asset, dict) else None
if not isinstance(name, str) or not any(pattern.search(name) for pattern in patterns):
continue
stem = ARCHIVE.sub("", name)
if stem not in chosen or name.endswith(".tar.xz"):
chosen[stem] = name
return sorted(chosen.values())
def release_candidates(flavor: str, cfg: dict, ref: str) -> list[dict]:
repo = cfg["repo"]
if ref:
release = gh_api(f"repos/{repo}/releases/tags/{quote(ref)}", missing_ok=True)
releases = [release] if release else []
else:
releases = gh_api(f"repos/{repo}/releases?per_page=100")
if not isinstance(releases, list):
raise PackError(f"{repo}: the release list is not a list")
pattern = re.compile(cfg["tag"])
patterns = [re.compile(item) for item in cfg["assets"]]
usable = []
for release in releases:
tag = release.get("tag_name") if isinstance(release, dict) else None
if not isinstance(tag, str) or release.get("draft") or not pattern.search(tag):
continue
assets = pick_assets(release.get("assets") or [], patterns)
if assets:
usable.append((release.get("published_at") or release.get("created_at") or "", natural_key(tag), tag, assets))
usable.sort(reverse=True)
if not ref:
usable = usable[:cfg["keep"]]
return [candidate(flavor, cfg, tag, asset=asset) for _, _, tag, assets in usable for asset in assets]
def built(index: dict) -> dict[tuple[str, str, str], int]:
revs = {}
packs = index.get("packs") if isinstance(index, dict) else None
for pack in packs if isinstance(packs, list) else []:
if not isinstance(pack, dict) or not isinstance(pack.get("source"), dict):
continue
rev = pack.get("rev") if type(pack.get("rev")) is int else 0
key = (pack.get("flavor"), pack["source"].get("ref"), pack["source"].get("asset", ""))
revs[key] = max(revs.get(key, 0), rev)
return revs
def revoked_build(entry: dict, revoked: set[str]) -> bool:
prefixes = {f"{entry['flavor']}-{slug(name)}-" for name in (entry["token"], entry["tag"]) if name}
suffix = f"-r{entry['rev']}"
return any(ident.endswith(suffix) and ident.startswith(prefix) for ident in revoked for prefix in prefixes)
def chosen_flavors(flavors: dict, selected: str) -> list[str]:
return sorted(flavors["flavors"]) if selected == "all" else [selected]
def failure_marker(entry: dict, digests: dict[str, str] | None = None) -> str:
series = entry["series"]
if not SERIES.fullmatch(series):
raise PackError(f"{series!r} is not a patch series name")
digest = digests.get(series) if digests is not None else None
if digest is None:
digest = patch_digest(PATCHES / series)
if digests is not None:
digests[series] = digest
return f"{entry['key']}-r{entry['rev']}-{digest[:12]}{FAILED}"
def discover(flavors: dict, selected: str, ref: str, index: dict, revoked: set[str],
failed: frozenset[str] = frozenset()) -> list[dict]:
names = chosen_flavors(flavors, selected)
done = built(index)
digests = {}
entries = []
seen = set()
matched = False
for flavor in names:
cfg = flavors["flavors"].get(flavor)
if cfg is None:
raise PackError(f"flavors.json has no {flavor}")
found = tag_candidates(flavor, cfg, ref) if cfg["list"] == "tags" else release_candidates(flavor, cfg, ref)
matched = matched or bool(found)
for entry in found:
if entry["key"] in seen:
continue
seen.add(entry["key"])
label = f"{flavor} {entry['tag']}{' ' + entry['asset'] if entry['asset'] else ''}"
if revoked_build(entry, revoked):
print(f"discover: {label}: revoked at rev {entry['rev']}", file=sys.stderr)
elif done.get((flavor, entry["tag"], entry["asset"]), 0) >= entry["rev"]:
print(f"discover: {label}: in the index at rev {entry['rev']}", file=sys.stderr)
elif not ref and failure_marker(entry, digests) in failed:
print(f"discover: {label}: failed at rev {entry['rev']} with this patch series in the last "
f"{RETRY_AFTER.days} days; skipped (bump rev, change the series or build the tag by name)", file=sys.stderr)
else:
print(f"discover: {label}: to build", file=sys.stderr)
entries.append(entry)
if ref and not matched:
raise PackError(f"no {'' if selected == 'all' else selected + ' '}build is tagged {ref}")
return entries
def vdf(text: str) -> dict:
stack = [{}]
key = None
for match in VDF_TOKEN.finditer(text):
word, brace = match.group(1), match.group(2)
if brace == "{":
if key is None:
raise PackError("app info: a block has no name")
child = {}
stack[-1][key] = child
stack.append(child)
key = None
elif brace == "}":
if len(stack) == 1:
raise PackError("app info: unbalanced braces")
stack.pop()
key = None
elif key is None:
key = word
else:
stack[-1][key] = word
key = None
if len(stack) != 1:
raise PackError("app info: unbalanced braces")
return stack[0]
def steam_builds(text: str, apps: list[str]) -> dict[str, dict]:
found = {}
text = ANSI.sub("", text)
starts = [(m.start(), m.group(1)) for m in APP_HEADER.finditer(text)]
for number, (start, app) in enumerate(starts):
end = starts[number + 1][0] if number + 1 < len(starts) else len(text)
if app not in apps:
continue
body = text[text.find("\n", start) + 1:end]
info = vdf(body).get(app)
depots = info.get("depots") if isinstance(info, dict) else None
public = depots.get("branches", {}).get("public") if isinstance(depots, dict) else None
if not isinstance(public, dict) or not str(public.get("buildid", "")).isdigit():
continue
manifests = sorted(depot["manifests"]["public"]["gid"] for name, depot in depots.items()
if name.isdigit() and isinstance(depot, dict) and isinstance(depot.get("manifests"), dict)
and isinstance(depot["manifests"].get("public"), dict) and depot["manifests"]["public"].get("gid"))
found[app] = {"buildid": public["buildid"], "built": public.get("timebuildupdated", ""), "manifests": manifests}
return found
def steam_apps(flavors: dict) -> list[str]:
return sorted({family["depot_app"] for cfg in flavors["flavors"].values() if cfg["list"] == "tags"
for family in cfg["families"].values() if family.get("depot_app")})
def watch(flavors: dict, previous: dict, steam: dict[str, dict] | None) -> tuple[dict, list[str], list[str]]:
state = {}
changed = []
notes = []
for flavor in sorted(flavors["flavors"]):
cfg = flavors["flavors"][flavor]
found = tag_candidates(flavor, cfg, "") if cfg["list"] == "tags" else release_candidates(flavor, cfg, "")
builds = sorted(f"{entry['tag']} {entry['asset']}".strip() for entry in found)
ships = {}
if cfg["list"] == "tags" and steam is not None:
for family in cfg["families"].values():
app = family.get("depot_app")
if app and app in steam:
ships[app] = steam[app]
old = previous.get(flavor) if isinstance(previous.get(flavor), dict) else {}
if steam is None and isinstance(old.get("steam"), dict):
ships = old["steam"]
mark = {"builds": builds, "rev": cfg["rev"], "series": cfg["series"], "patch": patch_digest(PATCHES / cfg["series"])}
fingerprint = hashlib.sha256(json.dumps(mark, sort_keys=True).encode()).hexdigest()
state[flavor] = {**mark, "fingerprint": fingerprint, "steam": ships}
if old.get("fingerprint") != fingerprint:
changed.append(flavor)
fresh = sorted(set(builds) - set(old.get("builds") or []))
notes.append(f"{flavor}: {', '.join(fresh) if fresh else 'the patch series or rev changed'}")
for app, ship in sorted(ships.items()):
before = (old.get("steam") or {}).get(app) if isinstance(old.get("steam"), dict) else None
if isinstance(before, dict) and before.get("buildid") != ship.get("buildid"):
notes.append(f"{flavor}: Steam app {app} moved from build {before.get('buildid')} to {ship.get('buildid')}")
if flavor not in changed:
changed.append(flavor)
return state, changed, notes
def asset_text(repo: str, asset_id, what: str) -> str:
result = gh("api", "-H", "Accept: application/octet-stream", f"repos/{repo}/releases/assets/{asset_id}")
if result.returncode != 0:
raise PackError(f"cannot download {what}: {result.stderr.strip()}")
return result.stdout
def release_index(repo: str, tag: str) -> dict:
release = gh_api(f"repos/{repo}/releases/tags/{quote(tag)}", missing_ok=True)
if release is None:
return {}
for asset in release.get("assets") or []:
if isinstance(asset, dict) and asset.get("name") == "index.json":
text = asset_text(repo, asset["id"], f"the index of {repo}")
try:
return json.loads(text)
except ValueError as e:
raise PackError(f"the index of {repo} is not JSON: {e}") from e
return {}
def published(repo: str, flavors: dict, selected: str, index: dict) -> list[dict]:
packs = index.get("packs") if isinstance(index, dict) else None
listed = {pack.get("id") for pack in packs if isinstance(pack, dict)} if isinstance(packs, list) else set()
found = []
for flavor in chosen_flavors(flavors, selected):
cfg = flavors["flavors"].get(flavor)
tag = f"droiddeck-esync-{flavor}"
release = gh_api(f"repos/{repo}/releases/tags/{quote(tag)}", missing_ok=True) if cfg else None
if not isinstance(release, dict) or type(release.get("id")) is not int:
continue
assets = gh_api(f"repos/{repo}/releases/{release['id']}/assets?per_page=100", paginate=True)
if not isinstance(assets, list):
raise PackError(f"{repo} {tag}: the asset list is not a list")
for asset in assets:
name = asset.get("name") if isinstance(asset, dict) else None
if not isinstance(name, str) or not name.endswith(".json"):
continue
ident = name[:-len(".json")]
rev = PACK_REV.search(ident)
if ident in listed or not rev or int(rev.group(1)) < cfg["rev"]:
continue
try:
entry = json.loads(asset_text(repo, asset.get("id"), f"{tag} {name}"))
except ValueError:
entry = None
if not isinstance(entry, dict) or entry.get("id") != ident or entry.get("flavor") != flavor:
print(f"discover: {tag} {name} is not a pack entry", file=sys.stderr)
continue
print(f"discover: {tag} {name}: published, not in the index yet", file=sys.stderr)
found.append(entry)
return found
def parse_time(value) -> datetime | None:
if not isinstance(value, str):
return None
try:
moment = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError:
return None
return moment if moment.tzinfo else moment.replace(tzinfo=timezone.utc)
def recent_failures(repo: str, flavors: dict, selected: str, now: datetime) -> frozenset[str]:
found = set()
for flavor in chosen_flavors(flavors, selected):
tag = f"droiddeck-esync-{flavor}"
release = gh_api(f"repos/{repo}/releases/tags/{quote(tag)}", missing_ok=True) if flavor in flavors["flavors"] else None
if not isinstance(release, dict) or type(release.get("id")) is not int:
continue
assets = gh_api(f"repos/{repo}/releases/{release['id']}/assets?per_page=100", paginate=True)
if not isinstance(assets, list):
raise PackError(f"{repo} {tag}: the asset list is not a list")
for asset in assets:
name = asset.get("name") if isinstance(asset, dict) else None
if not isinstance(name, str) or not name.endswith(FAILED):
continue
moment = parse_time(asset.get("updated_at")) or parse_time(asset.get("created_at"))
if moment is not None and now - moment < RETRY_AFTER:
found.add(name)
return frozenset(found)
def failed_builds(matrix: dict, directory: Path) -> list[tuple[str, str]]:
rows = matrix.get("include") if isinstance(matrix, dict) else None
if not isinstance(rows, list) or not all(isinstance(row, dict) and isinstance(row.get("key"), str) for row in rows):
raise PackError("the build matrix has no include list")
made = {row["key"] for row in rows if (directory / f"{ARTIFACT_PREFIX}{row['key']}").is_dir()}
digests = {}
return [(row["flavor"], failure_marker(row, digests)) for row in rows if row["key"] not in made]
def artifact_packs(matrix: dict, directory: Path) -> list[tuple[str, Path, str]]:
rows = matrix.get("include") if isinstance(matrix, dict) else None
if not isinstance(rows, list) or not all(isinstance(row, dict) and isinstance(row.get("key"), str) for row in rows):
raise PackError("the build matrix has no include list")
builds = {row["key"]: row for row in rows}
found = []
for folder in sorted(directory.iterdir()) if directory.is_dir() else []:
row = builds.get(folder.name[len(ARTIFACT_PREFIX):]) if folder.name.startswith(ARTIFACT_PREFIX) else None
if row is None or folder.is_symlink() or not folder.is_dir():
raise PackError(f"{folder.name} is not the artifact of a build in this run")
names = sorted(path.name for path in folder.iterdir())
entries = [name for name in names if name.endswith(".json")]
if len(entries) != 1 or names != sorted([entries[0], entries[0][:-len(".json")] + ".tzst"]):
raise PackError(f"{folder.name} must hold exactly <id>.json and <id>.tzst")
ident = entries[0][:-len(".json")]
entry_path, archive = folder / entries[0], folder / f"{ident}.tzst"
if entry_path.is_symlink() or archive.is_symlink() or not entry_path.is_file() or not archive.is_file():
raise PackError(f"{folder.name} holds links or special files")
try:
data = json.loads(entry_path.read_text())
except ValueError as e:
raise PackError(f"{entry_path}: {e}") from e
entry = check_entry(data, str(entry_path))
source = data.get("source") if isinstance(data.get("source"), dict) else {}
if entry["id"] != ident or entry["flavor"] != row["flavor"] or entry["rev"] != row["rev"] \
or source.get("ref") != row["tag"] or source.get("asset", "") != row["asset"]:
raise PackError(f"{entry_path} is not the pack of {row['flavor']} {row['tag']} at rev {row['rev']}")
if archive.stat().st_size != entry["asset"]["size"] or sha256_file(archive) != entry["asset"]["sha256"]:
raise PackError(f"{archive} is not the asset its entry describes")
found.append((row["flavor"], folder, ident))
return found
def emit(entries: list[dict], unindexed: int = 0) -> None:
matrix = json.dumps({"include": entries}, sort_keys=True, separators=(",", ":"))
path = os.environ.get("GITHUB_OUTPUT")
if path:
with open(path, "a") as stream:
stream.write(f"matrix={matrix}\ncount={len(entries)}\nunindexed={unindexed}\n")
print(matrix)
def make_variables(text: str) -> tuple[dict[str, str], list[tuple[str, str]]]:
variables = {}
assignments = []
for raw in text.splitlines():
match = ASSIGNMENT.fullmatch(raw.split("#", 1)[0].strip())
if match:
name, value = match.group(1), match.group(2).strip()
variables.setdefault(name, value)
assignments.append((name, value))
return variables, assignments
def expand(value: str, variables: dict[str, str], depth: int = 0) -> str:
if depth > 16:
raise PackError("Makefile variables nest too deeply")
def lookup(match: re.Match) -> str:
if match.group(1) not in variables:
raise PackError(f"$({match.group(1)}) is not set in the Makefile")
return expand(variables[match.group(1)], variables, depth + 1)
return REFERENCE.sub(lookup, value)
def sdk_image(text: str, variable: str = "STEAMRT_IMAGE", match: str = "arm64-llvm") -> str:
variables, assignments = make_variables(text)
problem = None
for name, value in assignments:
if not name.startswith(variable):
continue
try:
image = expand(value, variables)
except PackError as e:
problem = problem or e
continue
if match not in image:
continue
if not IMAGE.fullmatch(image):
raise PackError(f"{name} = {image!r} is not an image reference")
return image
raise problem or PackError(f"no {variable} assignment names an {match} image")
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Find the Proton builds that have no droiddeck-esync pack yet, as a GitHub Actions matrix.")
parser.add_argument("--flavors", default=str(FLAVORS_FILE))
commands = parser.add_subparsers(dest="command", required=True)
matrix = commands.add_parser("matrix")
matrix.add_argument("--flavor", default="all", choices=("all", *FLAVORS))
matrix.add_argument("--ref", default="")
matrix.add_argument("--index", default="")
matrix.add_argument("--index-release", default="")
matrix.add_argument("--repo", default=os.environ.get("GITHUB_REPOSITORY", "Droid-Deck/DroidDeck"))
matrix.add_argument("--revoked", default=str(REVOKED))
image = commands.add_parser("sdk-image")
image.add_argument("makefile")
artifacts = commands.add_parser("packs")
artifacts.add_argument("--matrix", required=True)
artifacts.add_argument("directory")
failures = commands.add_parser("failures")
failures.add_argument("--matrix", required=True)
failures.add_argument("directory")
commands.add_parser("steam-apps")
watching = commands.add_parser("watch")
watching.add_argument("--state", default="")
watching.add_argument("--steam-info", default="")
watching.add_argument("--out", required=True)
args = parser.parse_args(argv)
try:
flavors = load_flavors(Path(args.flavors))
if args.command == "sdk-image":
settings = flavors["sdk_image"]
print(sdk_image(Path(args.makefile).read_text(), settings["variable"], settings["match"]))
return 0
if args.command == "packs":
for flavor, folder, ident in artifact_packs(json.loads(args.matrix), Path(args.directory)):
print(f"{flavor}\t{folder}\t{ident}")
return 0
if args.command == "failures":
for flavor, marker in failed_builds(json.loads(args.matrix), Path(args.directory)):
print(f"{flavor}\t{marker}")
return 0
if args.command == "steam-apps":
print(" ".join(steam_apps(flavors)))
return 0
if args.command == "watch":
previous = {}
if args.state and Path(args.state).is_file():
try:
previous = json.loads(Path(args.state).read_text())
except ValueError:
print("discover: the previous watch state is not JSON; every flavor counts as changed", file=sys.stderr)
previous = previous.get("flavors", {}) if isinstance(previous, dict) and previous.get("format") == 1 else {}
steam = None
if args.steam_info:
try:
steam = steam_builds(Path(args.steam_info).read_text(errors="replace"), steam_apps(flavors))
except (OSError, PackError) as e:
print(f"discover: Steam app info unusable ({e}); watching the source tags only", file=sys.stderr)
if steam is not None and len(steam) != len(steam_apps(flavors)):
print(f"discover: Steam app info covers {sorted(steam)} of {steam_apps(flavors)}", file=sys.stderr)
state, changed, notes = watch(flavors, previous, steam)
Path(args.out).write_text(json.dumps({"format": 1, "flavors": state}, indent=1, sort_keys=True) + "\n")
for note in notes:
print(f"discover: {note}", file=sys.stderr)
path = os.environ.get("GITHUB_OUTPUT")
if path:
with open(path, "a") as stream:
stream.write(f"changed={' '.join(changed)}\n")
print(" ".join(changed))
return 0
if args.index and args.index_release:
raise PackError("pass --index or --index-release, not both")
failed = frozenset()
unindexed = 0
if args.index:
path = Path(args.index)
index = json.loads(path.read_text()) if path.exists() else {}
elif args.index_release:
if not REPO.fullmatch(args.repo):
raise PackError(f"{args.repo!r} is not owner/name")
index = release_index(args.repo, args.index_release)
listed = index.get("packs") if isinstance(index, dict) and isinstance(index.get("packs"), list) else []
waiting = published(args.repo, flavors, args.flavor, index)
unindexed = len(waiting)
index = {"packs": [*listed, *waiting]}
failed = recent_failures(args.repo, flavors, args.flavor, datetime.now(timezone.utc))
else:
index = {}
revoked = read_revoked(Path(args.revoked)) if args.revoked else set()
emit(discover(flavors, args.flavor, args.ref.strip(), index, revoked, failed), unindexed)
except (PackError, OSError, ValueError) as e:
print(f"discover: {e}", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())
-84
View File
@@ -1,84 +0,0 @@
{
"format": 1,
"sdk_image": {
"file": "Makefile.in",
"variable": "STEAMRT_IMAGE",
"match": "arm64-llvm"
},
"flavors": {
"valve": {
"repo": "ValveSoftware/Proton",
"list": "tags",
"families": {
"experimental": {
"prefix": "experimental-11.0-",
"tag": "^experimental-11\\.0-\\d{8}[a-z]?$",
"depot_app": "4427310"
},
"proton": {
"prefix": "proton-11.0-",
"tag": "^proton-11\\.0-\\d+[a-z]?$",
"depot_app": "4628740"
}
},
"version_token": "{tag}-arm64",
"keep": 2,
"rev": 1,
"series": "valve-11",
"profile": "valve",
"source_match": true,
"build_dir": "/builds/proton/proton/build-dir",
"wine": {
"submodules": [
"wine"
],
"clone": "shallow",
"prep": ""
}
},
"ge": {
"repo": "GloriousEggroll/proton-ge-custom",
"list": "releases",
"tag": "^GE-Proton11-\\d+$",
"assets": [
"aarch64\\.tar\\.(gz|xz)$",
"arm64\\.tar\\.(gz|xz)$"
],
"keep": 3,
"rev": 1,
"series": "valve-11",
"profile": "valve",
"source_match": false,
"build_dir": "/home/tcrider/build",
"wine": {
"submodules": [
"wine",
"wine-staging"
],
"clone": "treeless",
"prep": "patches/protonprep-valve-staging.sh"
}
},
"cachyos": {
"repo": "CachyOS/proton-cachyos",
"list": "releases",
"tag": "^cachyos-11\\.0-",
"assets": [
"arm64\\.tar\\.(gz|xz)$"
],
"keep": 3,
"rev": 1,
"series": "cachyos-11",
"profile": "cachyos",
"source_match": false,
"build_dir": "/home/runner/work/proton-cachyos/proton-cachyos/build",
"wine": {
"submodules": [
"wine"
],
"clone": "shallow",
"prep": ""
}
}
}
}
-4
View File
@@ -1,4 +0,0 @@
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEFyqZQAMh3g3nsEGLU+kBGVPBEkXr
Dv5F3kPtuQ+FV1DkTCGmYUQ1LS9wJtHGt0Ht4wToHGGEp9yzDibYZ4/zhg==
-----END PUBLIC KEY-----
-285
View File
@@ -1,285 +0,0 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import datetime as dt
import json
import os
import re
import subprocess
import sys
import tempfile
import time
from pathlib import Path
from make_pack import (
BINARIES,
EPOCH,
FLAVORS,
HERE,
NTDLL,
PACK_ID,
REVOKED,
SHA256,
WINESERVER,
PackError,
compress,
dump_json,
read_revoked,
sha256_bytes,
sha256_file,
tar_bytes,
write_atomic,
)
PUBKEY = HERE / "index-key.pub"
SCHEMA = 1
INDEX_KEYS = ("id", "flavor", "rev", "version", "version_line", "stock", "exports", "source_match", "files", "protocol", "source", "patch")
def check_entry(entry, origin: str) -> dict:
def fail(why: str):
raise PackError(f"{origin}: {why}")
if not isinstance(entry, dict):
fail("is not an object")
ident = entry.get("id")
if not isinstance(ident, str) or not PACK_ID.fullmatch(ident):
fail(f"{ident!r} is not a pack id")
if entry.get("flavor") not in FLAVORS:
fail(f"unknown flavor {entry.get('flavor')!r}")
if type(entry.get("rev")) is not int or entry["rev"] < 1:
fail("rev is not a positive number")
version, line = entry.get("version"), entry.get("version_line")
fields = line.split() if isinstance(line, str) else []
if not isinstance(version, str) or len(fields) < 2 or fields[1] != version or not EPOCH.fullmatch(fields[0]):
fail("version and version_line disagree")
for field in ("stock", "files"):
value = entry.get(field)
if not isinstance(value, dict) or set(value) != set(BINARIES) or not all(isinstance(v, str) and SHA256.fullmatch(v) for v in value.values()):
fail(f"{field} must give the sha256 of {' and '.join(BINARIES)}")
if not isinstance(entry.get("exports"), str) or not SHA256.fullmatch(entry["exports"]):
fail("exports is not a sha256")
if type(entry.get("source_match")) is not bool:
fail("source_match is not a boolean")
asset = entry.get("asset")
if not isinstance(asset, dict) or not isinstance(asset.get("sha256"), str) or not SHA256.fullmatch(asset["sha256"]) \
or type(asset.get("size")) is not int or asset["size"] <= 0:
fail("asset needs its sha256 and size")
clean = {key: entry[key] for key in INDEX_KEYS if key in entry}
clean["asset"] = {"sha256": asset["sha256"], "size": asset["size"]}
return clean
def load_entries(directory: Path) -> list[dict]:
if not directory.is_dir():
raise PackError(f"{directory} is not a directory")
entries = []
for path in sorted(directory.glob("*.json")):
try:
data = json.loads(path.read_text())
except ValueError as e:
raise PackError(f"{path}: {e}") from e
entry = check_entry(data, str(path))
if path.stem != entry["id"]:
raise PackError(f"{path} holds the entry of {entry['id']}")
entries.append(entry)
return entries
def load_previous(path: Path) -> tuple[int, list[dict], list]:
try:
data = json.loads(path.read_text())
except (OSError, ValueError) as e:
raise PackError(f"cannot read the previous index {path}: {e}") from e
if not isinstance(data, dict) or data.get("schema") != SCHEMA or type(data.get("generated")) is not int \
or not isinstance(data.get("packs"), list):
raise PackError(f"{path} is not a schema {SCHEMA} index")
packs = []
for number, pack in enumerate(data["packs"]):
entry = check_entry(pack, f"{path} pack {number}")
entry["revoked"] = pack.get("revoked") is True
packs.append(entry)
return data["generated"], packs, data["packs"]
def epoch(entry: dict) -> int:
return int(entry["version_line"].split()[0])
def rank(entry: dict) -> tuple:
return entry["rev"], epoch(entry), entry["id"]
def choose(entries: list[dict], previous: list[dict], revoked: set[str]) -> list[dict]:
known = {entry["id"]: entry for entry in entries}
for entry in previous:
known.setdefault(entry["id"], entry)
active = {}
withdrawn = []
for entry in known.values():
if entry["id"] in revoked:
withdrawn.append(dict(entry, revoked=True))
continue
key = (entry["stock"][NTDLL], entry["stock"][WINESERVER], entry["version"])
if key not in active or rank(entry) > rank(active[key]):
active[key] = dict(entry, revoked=False)
return sorted([*active.values(), *withdrawn], key=lambda entry: entry["id"])
def build_index(packs: list[dict], base_url: str, generated: int) -> dict:
listed = []
for pack in packs:
entry = dict(pack)
entry["asset"] = {
"url": f"{base_url}/droiddeck-esync-{pack['flavor']}/{pack['id']}.tzst",
"sha256": pack["asset"]["sha256"],
"size": pack["asset"]["size"],
}
listed.append(entry)
return {"schema": SCHEMA, "generated": generated, "packs": listed}
def family(entry: dict) -> tuple[str, str, str]:
token = entry["version"]
major = re.search(r"[0-9]+", token)
return entry["flavor"], re.match(r"[A-Za-z]*", token).group(0).lower(), major.group(0) if major else ""
def bundle_choice(packs: list[dict], available: set[str] | None = None) -> list[dict]:
newest = {}
for pack in packs:
if pack["revoked"] or (available is not None and pack["id"] not in available):
continue
key = family(pack)
if key not in newest or (epoch(pack), pack["rev"], pack["id"]) > (epoch(newest[key]), newest[key]["rev"], newest[key]["id"]):
newest[key] = pack
return sorted(newest.values(), key=lambda pack: pack["id"])
def openssl(*args: str) -> subprocess.CompletedProcess:
try:
return subprocess.run(["openssl", *args], capture_output=True, text=True, check=False)
except FileNotFoundError as e:
raise PackError("the openssl command is not installed") from e
def verified(index: Path, signature: Path, pubkey: Path) -> bool:
result = openssl("dgst", "-sha256", "-verify", str(pubkey), "-signature", str(signature), str(index))
return result.returncode == 0 and "Verified OK" in result.stdout
def trusted_previous(path: Path, signature: Path | None, pubkey: Path) -> tuple[int, list[dict], list | None]:
if signature is not None and not (path.is_file() and signature.is_file() and verified(path, signature, pubkey)):
print(f"make_index: the previous index {path} does not verify with {signature} and {pubkey}; it is ignored", file=sys.stderr)
return 0, [], None
return load_previous(path)
def sign(index: Path, signature: Path, key: Path | None, pubkey: Path) -> None:
with tempfile.TemporaryDirectory() as tmp:
if key is None:
pem = os.environ.get("SYNC_INDEX_KEY", "")
if not pem.strip():
raise PackError("no signing key: pass --key or set SYNC_INDEX_KEY")
key = Path(tmp) / "key.pem"
with os.fdopen(os.open(key, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600), "w") as stream:
stream.write(pem if pem.endswith("\n") else pem + "\n")
result = openssl("dgst", "-sha256", "-sign", str(key), "-out", str(signature), str(index))
if result.returncode != 0:
raise PackError(f"openssl could not sign the index: {result.stderr.strip()}")
if not verified(index, signature, pubkey):
raise PackError(f"the signature does not verify with {pubkey}: the signing key is not the one the app trusts")
def bundle_name(generated: int) -> str:
return f"bundle-{dt.datetime.fromtimestamp(generated, dt.timezone.utc):%Y%m%d-%H%M%S}.tzst"
def bundle(out: Path, packs: list[dict], index: bytes, signature: bytes, packs_dir: Path, generated: int) -> tuple[str, str]:
if not packs_dir.is_dir():
raise PackError(f"{packs_dir} is not a directory")
available = {path.name[:-len(".tzst")] for path in packs_dir.glob("*.tzst") if path.is_file()}
members = {"index.json": (index, 0o644), "index.json.sig": (signature, 0o644)}
for pack in bundle_choice(packs, available):
path = packs_dir / f"{pack['id']}.tzst"
if path.stat().st_size != pack["asset"]["size"] or sha256_file(path) != pack["asset"]["sha256"]:
raise PackError(f"{path} is not the asset the index lists for {pack['id']}")
members[f"packs/{pack['id']}.tzst"] = (path.read_bytes(), 0o644)
name = bundle_name(generated)
archive = compress(tar_bytes(members))
write_atomic(out / name, archive)
return name, sha256_bytes(archive)
def report(values: dict[str, str]) -> None:
path = os.environ.get("GITHUB_OUTPUT")
if path:
with open(path, "a") as stream:
for key, value in values.items():
stream.write(f"{key}={value}\n")
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Make, sign and bundle the droiddeck-esync pack index from the published pack entries.")
parser.add_argument("--entries", required=True)
parser.add_argument("--base-url", required=True)
parser.add_argument("--key", default="")
parser.add_argument("--pubkey", default=str(PUBKEY))
parser.add_argument("--out", default="")
parser.add_argument("--bundle-packs-dir", default="")
parser.add_argument("--revoked", default=str(REVOKED))
parser.add_argument("--previous", default="")
parser.add_argument("--previous-sig", default="")
parser.add_argument("--list-bundle", action="store_true")
args = parser.parse_args(argv)
try:
base_url = args.base_url.rstrip("/")
if not base_url.startswith("https://"):
raise PackError("--base-url must be an https URL")
revoked = read_revoked(Path(args.revoked)) if args.revoked else set()
if args.previous_sig and not args.previous:
raise PackError("--previous-sig needs --previous")
previous_signature = Path(args.previous_sig) if args.previous_sig else None
previous_generated, previous, previous_raw = trusted_previous(Path(args.previous), previous_signature, Path(args.pubkey)) \
if args.previous else (0, [], None)
packs = choose(load_entries(Path(args.entries)), previous, revoked)
if args.list_bundle:
for pack in bundle_choice(packs):
print(f"{pack['flavor']}\t{pack['id']}")
return 0
if not args.out:
raise PackError("--out is required")
out = Path(args.out)
out.mkdir(parents=True, exist_ok=True)
generated = max(int(time.time()), previous_generated + 1)
index = build_index(packs, base_url, generated)
data = dump_json(index)
changed = bool(packs) if previous_raw is None else previous_raw != index["packs"]
temp_index = out / f".index.json.tmp-{os.getpid()}"
temp_signature = out / f".index.json.sig.tmp-{os.getpid()}"
try:
temp_index.write_bytes(data)
sign(temp_index, temp_signature, Path(args.key) if args.key else None, Path(args.pubkey))
signature = temp_signature.read_bytes()
os.replace(temp_index, out / "index.json")
os.replace(temp_signature, out / "index.json.sig")
finally:
temp_index.unlink(missing_ok=True)
temp_signature.unlink(missing_ok=True)
outputs = {"changed": "true" if changed else "false", "generated": str(generated), "index_sha256": sha256_bytes(data)}
live = sum(1 for pack in packs if not pack["revoked"])
print(f"index.json {outputs['index_sha256']} {live} pack(s), {len(packs) - live} revoked, generated {generated}, {'changed' if changed else 'unchanged'}")
if args.bundle_packs_dir:
name, digest = bundle(out, packs, data, signature, Path(args.bundle_packs_dir), generated)
outputs.update(bundle=name, bundle_sha256=digest)
print(f"{name} {digest}")
report(outputs)
except (PackError, OSError) as e:
print(f"make_index: {e}", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())
-367
View File
@@ -1,367 +0,0 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import functools
import hashlib
import importlib.machinery
import importlib.util
import io
import json
import os
import re
import subprocess
import sys
import tarfile
from pathlib import Path
HERE = Path(__file__).resolve().parent
ROOT = HERE.parents[1]
GUEST = ROOT / "tools/linuxfs/overlay/usr/local/bin/droiddeck-esync"
PATCHES = HERE / "patches"
REVOKED = HERE / "revoked.txt"
NTDLL = "files/lib/wine/aarch64-unix/ntdll.so"
WINESERVER = "files/bin-arm64/wineserver"
BINARIES = (NTDLL, WINESERVER)
COPY = ("files/bin-arm64/wine", "files/lib/wine/aarch64-unix/wine", "files/lib/wine/aarch64-unix/wine-preloader")
HEADERS = ("server_protocol.h", "request_handlers.h", "request_trace.h")
FLAVORS = ("valve", "ge", "cachyos")
SOURCE_KEYS = ("repo", "ref", "commit", "wine_commit", "sdk_image")
ENTRY_KEYS = ("id", "flavor", "rev", "version", "version_line", "stock", "exports", "source_match", "files", "protocol", "source", "patch")
PACK_ID = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,127}")
SERIES = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]*")
SHA256 = re.compile(r"[0-9a-f]{64}")
EPOCH = re.compile(r"[0-9]{1,12}")
PROTOCOL = re.compile(rb"^#define SERVER_PROTOCOL_VERSION ([0-9]+)[ \t]*$", re.M)
BUILTIN_ESYNC = b"esync: up and running"
ELF_MAGIC = b"\x7fELF"
GUEST_MODULE = "droiddeck_esync_tools"
class PackError(RuntimeError):
pass
@functools.lru_cache(maxsize=None)
def guest():
loader = importlib.machinery.SourceFileLoader(GUEST_MODULE, str(GUEST))
module = importlib.util.module_from_spec(importlib.util.spec_from_loader(GUEST_MODULE, loader))
sys.modules[GUEST_MODULE] = module
write_bytecode = sys.dont_write_bytecode
sys.dont_write_bytecode = True
try:
loader.exec_module(module)
except Exception as e:
sys.modules.pop(GUEST_MODULE, None)
raise PackError(f"cannot load {GUEST}: {e}") from e
finally:
sys.dont_write_bytecode = write_bytecode
return module
def sha256_bytes(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1 << 20), b""):
digest.update(block)
return digest.hexdigest()
def slug(text: str) -> str:
return re.sub(r"[^A-Za-z0-9._-]", "-", text)
def dump_json(value) -> bytes:
return (json.dumps(value, indent=1, sort_keys=True, ensure_ascii=True) + "\n").encode()
def write_atomic(path: Path, data: bytes) -> None:
temp = path.with_name(f".{path.name}.tmp-{os.getpid()}")
try:
with temp.open("wb") as stream:
stream.write(data)
stream.flush()
os.fsync(stream.fileno())
os.replace(temp, path)
except BaseException:
temp.unlink(missing_ok=True)
raise
def read_revoked(path: Path | None) -> set[str]:
if path is None:
return set()
try:
text = path.read_text()
except OSError as e:
raise PackError(f"cannot read the revocation list {path}: {e}") from e
revoked = set()
for raw in text.splitlines():
line = raw.strip()
if not line or line.startswith("#"):
continue
ident = line.split()[0]
if not PACK_ID.fullmatch(ident):
raise PackError(f"{path}: {ident!r} is not a pack id")
revoked.add(ident)
return revoked
def tar_bytes(members: dict[str, tuple[bytes, int]]) -> bytes:
buffer = io.BytesIO()
with tarfile.open(fileobj=buffer, mode="w", format=tarfile.USTAR_FORMAT) as tar:
for name in sorted(members):
data, mode = members[name]
info = tarfile.TarInfo(name)
info.size = len(data)
info.mode = mode
info.mtime = 0
info.uid = 0
info.gid = 0
info.uname = ""
info.gname = ""
tar.addfile(info, io.BytesIO(data))
return buffer.getvalue()
def zstd(data: bytes, *args: str) -> bytes:
try:
result = subprocess.run(["zstd", "-q", *args], input=data, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False)
except FileNotFoundError as e:
raise PackError("the zstd command is not installed") from e
if result.returncode != 0:
raise PackError(f"zstd {' '.join(args)} failed: {result.stderr.decode(errors='replace').strip()}")
return result.stdout
def compress(data: bytes) -> bytes:
return zstd(data, "-19", "-T1", "-c")
def unpack(archive: bytes) -> dict[str, tuple[bytes, int]]:
members = {}
try:
with tarfile.open(fileobj=io.BytesIO(zstd(archive, "-d", "-c")), mode="r:") as tar:
for info in tar:
if not info.isreg() or info.name in members:
raise PackError(f"unexpected entry {info.name!r} in the archive")
members[info.name] = (tar.extractfile(info).read(), info.mode)
except tarfile.TarError as e:
raise PackError(f"the archive does not read back: {e}") from e
return members
def binary(base: Path, rel: str) -> Path:
path = base / rel
if path.is_symlink() or not path.is_file():
raise PackError(f"{path} is not a regular file")
with path.open("rb") as stream:
if stream.read(len(ELF_MAGIC)) != ELF_MAGIC:
raise PackError(f"{path} is not an ELF file")
return path
def exports_hash(path: Path) -> str:
try:
value = guest().exports_hash(path)
except PackError:
raise
except Exception as e:
raise PackError(f"{path}: cannot read its exported symbols: {e}") from e
if not isinstance(value, str) or not SHA256.fullmatch(value):
raise PackError(f"{path}: has no exported symbols")
return value
def parse_version_line(line: str) -> tuple[int, str]:
fields = line.split()
if "\n" in line or len(fields) < 2 or not EPOCH.fullmatch(fields[0]):
raise PackError(f"version line {line!r} is not '<build epoch> <version>'")
token = guest().version_token(line)
if not isinstance(token, str) or not token:
raise PackError(f"version line {line!r} has no version token")
return int(fields[0]), token
def parse_source(text: str) -> dict[str, str]:
try:
source = json.loads(text)
except ValueError as e:
raise PackError(f"--source-json is not JSON: {e}") from e
if not isinstance(source, dict) or not all(isinstance(key, str) and isinstance(value, str) for key, value in source.items()):
raise PackError("--source-json must be an object of strings")
missing = [key for key in SOURCE_KEYS if not source.get(key)]
if missing:
raise PackError(f"--source-json lacks {', '.join(missing)}")
return source
def patch_digest(directory: Path) -> str:
patches = sorted(path for path in directory.glob("*.patch") if path.is_file())
if not patches:
raise PackError(f"{directory} holds no *.patch files")
digest = hashlib.sha256()
for path in patches:
digest.update(f"{path.name}\0{sha256_file(path)}\n".encode())
return digest.hexdigest()
def protocol_version(control: Path, patched: Path) -> int:
for name in HEADERS:
before, after = control / name, patched / name
if not before.is_file() or not after.is_file():
raise PackError(f"{name} is missing from the control or the patched build")
if before.read_bytes() != after.read_bytes():
raise PackError(f"the patch changes {name}: the droiddeck-esync patch must not change the wineserver protocol")
found = PROTOCOL.findall((control / HEADERS[0]).read_bytes())
if len(found) != 1:
raise PackError("server_protocol.h does not define SERVER_PROTOCOL_VERSION once")
version = int(found[0])
stamp = control / "protocol"
if stamp.is_file() and stamp.read_text().strip() != str(version):
raise PackError(f"{stamp} disagrees with server_protocol.h ({version})")
return version
def make_pack(
flavor: str,
rev: int,
stock_dir: Path | None,
version_line: str,
control: Path,
patched: Path,
source: dict[str, str],
series: str,
out: Path,
source_match: bool,
patch_dir: Path | None = None,
allow_unreproduced: bool = False,
) -> dict:
if flavor not in FLAVORS:
raise PackError(f"unknown flavor {flavor!r}")
if type(rev) is not int or rev < 1:
raise PackError("--rev must be a positive number")
if not SERIES.fullmatch(series):
raise PackError(f"{series!r} is not a patch series name")
if stock_dir is None and not source_match:
raise PackError("without --stock-dir a pack can only match by source: pass --source-match")
if stock_dir is not None and source_match:
raise PackError("a pack made against stock binaries matches them exactly: drop --source-match")
patch_dir = patch_dir or PATCHES / series
if not patch_dir.is_dir():
raise PackError(f"the patch series {patch_dir} does not exist")
version_line = version_line.strip()
if stock_dir is not None:
stock_line = guest().version_line(stock_dir)
if stock_line and version_line and stock_line != version_line:
raise PackError(f"{stock_dir}/version says {stock_line!r}, not {version_line!r}")
version_line = version_line or stock_line
epoch, token = parse_version_line(version_line)
stock_paths = {rel: binary(stock_dir or control, rel) for rel in BINARIES}
control_paths = {rel: binary(control, rel) for rel in BINARIES}
patched_paths = {rel: binary(patched, rel) for rel in BINARIES}
if BUILTIN_ESYNC in stock_paths[WINESERVER].read_bytes():
raise PackError(f"{stock_paths[WINESERVER]} has esync built in and needs no pack")
stock = {rel: sha256_file(path) for rel, path in stock_paths.items()}
built = {rel: sha256_file(path) for rel, path in control_paths.items()}
files = {rel: sha256_file(path) for rel, path in patched_paths.items()}
for rel in BINARIES:
if files[rel] == built[rel]:
raise PackError(f"the patch left {rel} unchanged")
exports = exports_hash(stock_paths[NTDLL])
control_exports = exports_hash(control_paths[NTDLL])
if exports_hash(patched_paths[NTDLL]) != control_exports:
raise PackError("the patch changes the symbols ntdll.so exports")
if exports != control_exports:
raise PackError("the control build of ntdll.so exports other symbols than the stock one: the wine source does not match this release")
protocol = protocol_version(control, patched)
if stock_dir is not None:
for rel in BINARIES:
state = "reproduces" if stock[rel] == built[rel] else "differs from"
print(f"make_pack: the control build {state} the stock {rel}", file=sys.stderr)
unreproduced = [rel for rel in BINARIES if stock[rel] != built[rel]]
if unreproduced and not allow_unreproduced:
raise PackError(f"the control build does not reproduce the stock {', '.join(unreproduced)}: the patched files would differ "
"from the release in more than the patch (check the build directory and the SDK image, or pass --allow-unreproduced)")
pack_id = f"{flavor}-{slug(token)}-{epoch}-{stock[NTDLL][:12]}-r{rev}"
if not PACK_ID.fullmatch(pack_id):
raise PackError(f"{pack_id!r} is not a valid pack id")
pack = {
"format": 1,
"id": pack_id,
"flavor": flavor,
"rev": rev,
"version": token,
"version_line": version_line,
"stock": stock,
"exports": exports,
"source_match": source_match,
"files": files,
"copy": list(COPY),
"protocol": protocol,
"source": dict(source),
"patch": {"series": series, "sha256": patch_digest(patch_dir)},
}
members = {
"pack.json": (dump_json(pack), 0o644),
NTDLL: (patched_paths[NTDLL].read_bytes(), 0o755),
WINESERVER: (patched_paths[WINESERVER].read_bytes(), 0o755),
}
archive = compress(tar_bytes(members))
if unpack(archive) != members:
raise PackError("the archive does not read back as written")
entry = {key: pack[key] for key in ENTRY_KEYS}
entry["asset"] = {"sha256": sha256_bytes(archive), "size": len(archive)}
entry["revoked"] = False
out.mkdir(parents=True, exist_ok=True)
write_atomic(out / f"{pack_id}.tzst", archive)
write_atomic(out / f"{pack_id}.json", dump_json(entry))
return entry
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Make a droiddeck-esync pack from a control and a patched wine build of one Proton release.")
parser.add_argument("--flavor", required=True, choices=FLAVORS)
parser.add_argument("--rev", required=True, type=int)
parser.add_argument("--stock-dir", default="")
parser.add_argument("--version-line", default="")
parser.add_argument("--control", required=True)
parser.add_argument("--patched", required=True)
parser.add_argument("--source-json", required=True)
parser.add_argument("--patch-series", required=True)
parser.add_argument("--patch-dir", default="")
parser.add_argument("--out", required=True)
parser.add_argument("--source-match", action="store_true")
parser.add_argument("--allow-unreproduced", action="store_true")
args = parser.parse_args(argv)
try:
entry = make_pack(
flavor=args.flavor,
rev=args.rev,
stock_dir=Path(args.stock_dir) if args.stock_dir else None,
version_line=args.version_line,
control=Path(args.control),
patched=Path(args.patched),
source=parse_source(args.source_json),
series=args.patch_series,
out=Path(args.out),
source_match=args.source_match,
patch_dir=Path(args.patch_dir) if args.patch_dir else None,
allow_unreproduced=args.allow_unreproduced,
)
except (PackError, OSError) as e:
print(f"make_pack: {e}", file=sys.stderr)
return 1
print(entry["id"])
return 0
if __name__ == "__main__":
raise SystemExit(main())
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+4
View File
@@ -0,0 +1,4 @@
SYNC_BUNDLE_REPO=Droid-Deck/DroidDeck-Components
SYNC_BUNDLE_TAG=droiddeck-esync-index
SYNC_BUNDLE_ASSET=bundle-20261004-142603.tzst
SYNC_BUNDLE_SHA256=7e311e92444f0709de7779ef5334708511345d60556aa429e6761b86d711a9b2
View File
Whitespace-only changes.
-62
View File
@@ -1,62 +0,0 @@
#!/bin/bash
set -euo pipefail
if [ $# -ne 4 ]; then
echo "usage: $0 <proton-dir> <pack.tzst> <esstress.exe> <work-dir>" >&2
exit 2
fi
tool=$(realpath "$1")
archive=$(realpath "$2")
exe=$(realpath "$3")
work=$(realpath -m "$4")
here=$(cd "$(dirname "$0")" && pwd)
guest=$here/../linuxfs/overlay/usr/local/bin/droiddeck-esync
rm -rf "$work"
mkdir -p "$work/pack"
tar --zstd -xf "$archive" -C "$work/pack"
python3 "$guest" build-shadow "$tool" "$work/pack" "$work/dist" >/dev/null
[ -e "$work/dist/files/bin" ] || ln -s bin-arm64 "$work/dist/files/bin"
export WINEDEBUG=-all WINEDLLOVERRIDES="mscoree,mshtml="
run() {
local mode=$1 bin=$2 want=$3
shift 3
local prefix=$work/prefix-$mode
mkdir -p "$prefix"
(
export WINEPREFIX=$prefix WINESERVER=$bin/wineserver "$@"
timeout "${BOOT_TIMEOUT:-900}" "$bin/wine" wineboot -i > "$work/$mode-boot.log" 2>&1 || true
"$bin/wineserver" -w || true
if ! grep -q "$want" "$work/$mode-boot.log"; then
echo "$mode: wine never said \"$want\""
tail -n 30 "$work/$mode-boot.log"
exit 1
fi
cp "$exe" "$prefix/drive_c/esstress.exe"
status=0
timeout "${STRESS_TIMEOUT:-1800}" "$bin/wine" 'C:\esstress.exe' > "$work/$mode-stress.log" 2>&1 || status=$?
"$bin/wineserver" -k 2>/dev/null || true
if ! grep -q '^esstress:' "$work/$mode-stress.log"; then
echo "$mode: esstress did not finish (status $status)"
tail -n 30 "$work/$mode-stress.log"
exit 1
fi
grep -o '^FAIL [^ ]*:[0-9]*:' "$work/$mode-stress.log" | sort -u > "$work/$mode-failures" || true
echo "$mode: $(grep -E '^esstress:' "$work/$mode-stress.log" | tail -n 1)"
)
}
if ! run stock "$tool/files/bin-arm64" "server-side synchronization" PROTON_NO_NTSYNC=1 WINEESYNC=0 WINEFSYNC=0; then
echo "stock Proton does not run here either; the pack cannot be judged on this machine"
exit 3
fi
run esync "$work/dist/files/bin-arm64" "esync: up and running" PROTON_NO_NTSYNC=1 WINEESYNC=1 WINEFSYNC=0
if [ -s "$work/stock-failures" ]; then
echo "stock fails these as well, so they do not count against the pack: $(tr '\n' ' ' < "$work/stock-failures")"
fi
new=$(comm -13 "$work/stock-failures" "$work/esync-failures")
if [ -n "$new" ]; then
echo "esync: fails checks stock passes:"
grep -F "$new" "$work/esync-stress.log" || echo "$new"
exit 1
fi
-618
View File
@@ -1,618 +0,0 @@
#include <windows.h>
#include <winternl.h>
#include <stdio.h>
#include <stdlib.h>
static int failures;
#define CHECK(c, ...) do { if (!(c)) { failures++; printf("FAIL %s:%d: ", __FILE__, __LINE__); printf(__VA_ARGS__); printf("\n"); fflush(stdout); } } while (0)
static volatile LONG counter;
static HANDLE mutex, sem, ev_auto, ev_manual, start_ev;
static volatile LONG stop;
static DWORD WINAPI mutex_worker(void *arg)
{
int i;
WaitForSingleObject(start_ev, INFINITE);
for (i = 0; i < 20000; i++)
{
DWORD r = WaitForSingleObject(mutex, INFINITE);
LONG v;
if (r != WAIT_OBJECT_0) { CHECK(0, "mutex wait %lu", r); break; }
v = counter;
if (i % 7 == 0) WaitForSingleObject(mutex, 0), ReleaseMutex(mutex);
counter = v + 1;
ReleaseMutex(mutex);
}
return 0;
}
static void test_mutex_contention(void)
{
HANDLE th[8];
int i;
counter = 0;
mutex = CreateMutexA(NULL, FALSE, NULL);
start_ev = CreateEventA(NULL, TRUE, FALSE, NULL);
for (i = 0; i < 8; i++) th[i] = CreateThread(NULL, 0, mutex_worker, NULL, 0, NULL);
SetEvent(start_ev);
WaitForMultipleObjects(8, th, TRUE, INFINITE);
CHECK(counter == 8 * 20000, "mutex counter %ld", counter);
for (i = 0; i < 8; i++) CloseHandle(th[i]);
CloseHandle(mutex);
CloseHandle(start_ev);
printf("mutex_contention ok counter=%ld\n", counter);
}
static volatile LONG produced, consumed;
static DWORD WINAPI sem_producer(void *arg)
{
int i;
for (i = 0; i < 50000; i++)
{
LONG prev;
while (!ReleaseSemaphore(sem, 1, &prev))
{
CHECK(GetLastError() == ERROR_TOO_MANY_POSTS, "release err %lu", GetLastError());
Sleep(0);
}
InterlockedIncrement(&produced);
}
return 0;
}
static DWORD WINAPI sem_consumer(void *arg)
{
for (;;)
{
DWORD r = WaitForSingleObject(sem, 2000);
if (r == WAIT_TIMEOUT) break;
CHECK(r == WAIT_OBJECT_0, "sem wait %lu", r);
InterlockedIncrement(&consumed);
}
return 0;
}
static void test_semaphore(void)
{
HANDLE th[8];
LONG prev;
int i;
produced = consumed = 0;
sem = CreateSemaphoreA(NULL, 0, 16, NULL);
for (i = 0; i < 4; i++) th[i] = CreateThread(NULL, 0, sem_producer, NULL, 0, NULL);
for (i = 4; i < 8; i++) th[i] = CreateThread(NULL, 0, sem_consumer, NULL, 0, NULL);
WaitForMultipleObjects(8, th, TRUE, INFINITE);
CHECK(produced == 200000 && consumed == 200000, "sem produced %ld consumed %ld", produced, consumed);
CHECK(!ReleaseSemaphore(sem, 17, &prev) && GetLastError() == ERROR_TOO_MANY_POSTS, "limit");
CHECK(ReleaseSemaphore(sem, 16, &prev) && prev == 0, "prev %ld", prev);
CHECK(!ReleaseSemaphore(sem, 1, &prev), "over max");
for (i = 0; i < 8; i++) CloseHandle(th[i]);
CloseHandle(sem);
printf("semaphore ok %ld/%ld\n", produced, consumed);
}
static volatile LONG auto_hits;
static DWORD WINAPI auto_waiter(void *arg)
{
while (!stop)
{
DWORD r = WaitForSingleObject(ev_auto, 50);
if (r == WAIT_OBJECT_0) InterlockedIncrement(&auto_hits);
}
return 0;
}
static void test_auto_event(void)
{
HANDLE th[6];
int i;
LONG expected = 0;
stop = 0; auto_hits = 0;
ev_auto = CreateEventA(NULL, FALSE, FALSE, NULL);
for (i = 0; i < 6; i++) th[i] = CreateThread(NULL, 0, auto_waiter, NULL, 0, NULL);
for (i = 0; i < 30000; i++)
{
LONG before = auto_hits;
SetEvent(ev_auto);
expected++;
while (auto_hits == before) SwitchToThread();
}
stop = 1;
WaitForMultipleObjects(6, th, TRUE, INFINITE);
CHECK(auto_hits == expected, "auto hits %ld expected %ld", auto_hits, expected);
CHECK(WaitForSingleObject(ev_auto, 0) == WAIT_TIMEOUT, "auto event left signaled");
SetEvent(ev_auto); SetEvent(ev_auto);
CHECK(WaitForSingleObject(ev_auto, 0) == WAIT_OBJECT_0, "auto set");
CHECK(WaitForSingleObject(ev_auto, 0) == WAIT_TIMEOUT, "auto coalesce");
for (i = 0; i < 6; i++) CloseHandle(th[i]);
CloseHandle(ev_auto);
printf("auto_event ok hits=%ld\n", auto_hits);
}
static volatile LONG manual_seen;
static DWORD WINAPI manual_waiter(void *arg)
{
DWORD r = WaitForSingleObject(ev_manual, 10000);
if (r == WAIT_OBJECT_0) InterlockedIncrement(&manual_seen);
return 0;
}
static void test_manual_event(void)
{
HANDLE th[16];
int i, round;
ev_manual = CreateEventA(NULL, TRUE, FALSE, NULL);
for (round = 0; round < 50; round++)
{
manual_seen = 0;
ResetEvent(ev_manual);
for (i = 0; i < 16; i++) th[i] = CreateThread(NULL, 0, manual_waiter, NULL, 0, NULL);
Sleep(2);
SetEvent(ev_manual);
WaitForMultipleObjects(16, th, TRUE, INFINITE);
CHECK(manual_seen == 16, "manual round %d seen %ld", round, manual_seen);
for (i = 0; i < 16; i++) CloseHandle(th[i]);
}
for (i = 0; i < 20000; i++)
{
SetEvent(ev_manual);
if (i & 1) ResetEvent(ev_manual);
}
CHECK(WaitForSingleObject(ev_manual, 0) == WAIT_TIMEOUT, "manual final reset");
SetEvent(ev_manual);
CHECK(WaitForSingleObject(ev_manual, 0) == WAIT_OBJECT_0, "manual stays");
CHECK(WaitForSingleObject(ev_manual, 0) == WAIT_OBJECT_0, "manual stays 2");
CloseHandle(ev_manual);
printf("manual_event ok\n");
}
static HANDLE wa_objs[3];
static volatile LONG wa_count;
static DWORD WINAPI wait_all_worker(void *arg)
{
int i;
for (i = 0; i < 3000; i++)
{
DWORD r = WaitForMultipleObjects(3, wa_objs, TRUE, 5000);
if (r != WAIT_OBJECT_0) { CHECK(0, "wait all %lu", r); break; }
InterlockedIncrement(&wa_count);
ReleaseMutex(wa_objs[0]);
ReleaseSemaphore(wa_objs[1], 1, NULL);
SetEvent(wa_objs[2]);
}
return 0;
}
static void test_wait_all(void)
{
HANDLE th[6], dup[2];
int i;
DWORD r;
wa_count = 0;
wa_objs[0] = CreateMutexA(NULL, FALSE, NULL);
wa_objs[1] = CreateSemaphoreA(NULL, 1, 1, NULL);
wa_objs[2] = CreateEventA(NULL, FALSE, TRUE, NULL);
for (i = 0; i < 6; i++) th[i] = CreateThread(NULL, 0, wait_all_worker, NULL, 0, NULL);
WaitForMultipleObjects(6, th, TRUE, INFINITE);
CHECK(wa_count == 18000, "wait all count %ld", wa_count);
dup[0] = wa_objs[2];
DuplicateHandle(GetCurrentProcess(), wa_objs[2], GetCurrentProcess(), &dup[1], 0, FALSE, DUPLICATE_SAME_ACCESS);
r = WaitForMultipleObjects(2, dup, TRUE, 0);
CHECK(r == WAIT_FAILED && GetLastError() == ERROR_INVALID_PARAMETER, "dup wait all %lu %lu", r, GetLastError());
CloseHandle(dup[1]);
for (i = 0; i < 6; i++) CloseHandle(th[i]);
for (i = 0; i < 3; i++) CloseHandle(wa_objs[i]);
printf("wait_all ok %ld\n", wa_count);
}
static DWORD WINAPI hold_and_die(void *arg)
{
WaitForSingleObject((HANDLE)arg, INFINITE);
return 0;
}
static void test_abandoned(void)
{
HANDLE m = CreateMutexA(NULL, FALSE, NULL);
HANDLE th = CreateThread(NULL, 0, hold_and_die, m, 0, NULL);
HANDLE objs[2];
DWORD r;
WaitForSingleObject(th, INFINITE);
r = WaitForSingleObject(m, 1000);
CHECK(r == WAIT_ABANDONED, "abandoned %lu", r);
CHECK(ReleaseMutex(m), "release after abandon");
th = CreateThread(NULL, 0, hold_and_die, m, 0, NULL);
WaitForSingleObject(th, INFINITE);
objs[0] = CreateEventA(NULL, TRUE, TRUE, NULL);
objs[1] = m;
r = WaitForMultipleObjects(2, objs, TRUE, 1000);
CHECK(r == WAIT_ABANDONED_0, "abandoned all %lu", r);
ReleaseMutex(m);
r = WaitForMultipleObjects(2, objs, FALSE, 1000);
CHECK(r == WAIT_OBJECT_0, "any after %lu", r);
CloseHandle(th); CloseHandle(objs[0]); CloseHandle(m);
printf("abandoned ok\n");
}
static volatile LONG apc_ran;
static void CALLBACK apc_proc(ULONG_PTR p) { InterlockedIncrement(&apc_ran); }
static HANDLE apc_ready;
static DWORD WINAPI alert_waiter(void *arg)
{
SetEvent(apc_ready);
return WaitForSingleObjectEx((HANDLE)arg, 5000, TRUE);
}
static void test_apc(void)
{
HANDLE ev = CreateEventA(NULL, TRUE, FALSE, NULL);
HANDLE th;
DWORD r, code;
int i;
apc_ready = CreateEventA(NULL, FALSE, FALSE, NULL);
for (i = 0; i < 200; i++)
{
apc_ran = 0;
th = CreateThread(NULL, 0, alert_waiter, ev, 0, NULL);
WaitForSingleObject(apc_ready, INFINITE);
if (i & 1) Sleep(1);
QueueUserAPC(apc_proc, th, 0);
WaitForSingleObject(th, INFINITE);
GetExitCodeThread(th, &code);
CHECK(code == WAIT_IO_COMPLETION && apc_ran == 1, "apc round %d code %lu ran %ld", i, code, apc_ran);
CloseHandle(th);
}
apc_ran = 0;
QueueUserAPC(apc_proc, GetCurrentThread(), 0);
r = SleepEx(1000, TRUE);
CHECK(r == WAIT_IO_COMPLETION && apc_ran == 1, "sleepex %lu %ld", r, apc_ran);
r = SleepEx(10, TRUE);
CHECK(r == 0, "sleepex timeout %lu", r);
QueueUserAPC(apc_proc, GetCurrentThread(), 0);
r = WaitForSingleObjectEx(ev, 0, FALSE);
CHECK(r == WAIT_TIMEOUT, "non alertable %lu", r);
r = WaitForSingleObjectEx(ev, 0, TRUE);
CHECK(r == WAIT_IO_COMPLETION, "alertable pending %lu", r);
CloseHandle(ev);
printf("apc ok\n");
}
static DWORD WINAPI close_waiter(void *arg)
{
return WaitForSingleObject((HANDLE)arg, 300);
}
static void test_close_while_waiting(void)
{
int i;
for (i = 0; i < 300; i++)
{
HANDLE ev = CreateEventA(NULL, TRUE, FALSE, NULL), ev2, th;
DWORD code;
th = CreateThread(NULL, 0, close_waiter, ev, 0, NULL);
Sleep(0);
CloseHandle(ev);
ev2 = CreateEventA(NULL, TRUE, TRUE, NULL);
WaitForSingleObject(th, INFINITE);
GetExitCodeThread(th, &code);
CHECK(code == WAIT_TIMEOUT || code == WAIT_FAILED || code == WAIT_OBJECT_0, "close round %d code %lu", i, code);
CHECK(WaitForSingleObject(ev2, 0) == WAIT_OBJECT_0, "reused handle state");
CloseHandle(ev2);
CloseHandle(th);
}
printf("close_while_waiting ok\n");
}
static void test_pulse(void)
{
HANDLE ev = CreateEventA(NULL, TRUE, FALSE, NULL), th[4];
DWORD code;
int i, woke = 0;
for (i = 0; i < 4; i++) th[i] = CreateThread(NULL, 0, close_waiter, ev, 0, NULL);
Sleep(50);
PulseEvent(ev);
WaitForMultipleObjects(4, th, TRUE, INFINITE);
for (i = 0; i < 4; i++) { GetExitCodeThread(th[i], &code); if (code == WAIT_OBJECT_0) woke++; CloseHandle(th[i]); }
CHECK(WaitForSingleObject(ev, 0) == WAIT_TIMEOUT, "pulse left signaled");
CHECK(woke == 4, "pulse woke %d of 4", woke);
CloseHandle(ev);
ev = CreateEventA(NULL, FALSE, FALSE, NULL);
PulseEvent(ev);
CHECK(WaitForSingleObject(ev, 0) == WAIT_TIMEOUT, "auto pulse without waiter left signaled");
th[0] = CreateThread(NULL, 0, close_waiter, ev, 0, NULL);
th[1] = CreateThread(NULL, 0, close_waiter, ev, 0, NULL);
Sleep(50);
PulseEvent(ev);
WaitForMultipleObjects(2, th, TRUE, INFINITE);
woke = 0;
for (i = 0; i < 2; i++) { GetExitCodeThread(th[i], &code); if (code == WAIT_OBJECT_0) woke++; CloseHandle(th[i]); }
CHECK(woke == 1, "auto pulse woke %d of 2", woke);
CHECK(WaitForSingleObject(ev, 0) == WAIT_TIMEOUT, "auto pulse left signaled");
CloseHandle(ev);
printf("pulse ok\n");
}
static void test_semantics(void)
{
HANDLE ev = CreateEventA(NULL, TRUE, TRUE, NULL), sem = CreateSemaphoreA(NULL, 2, 3, NULL), h[2];
NTSTATUS (WINAPI *pNtWaitForSingleObject)(HANDLE, BOOLEAN, const LARGE_INTEGER *);
LARGE_INTEGER when;
LONG prev = -1;
DWORD r, start;
int i;
pNtWaitForSingleObject = (void *)GetProcAddress(GetModuleHandleA("ntdll.dll"), "NtWaitForSingleObject");
apc_ran = 0;
QueueUserAPC(apc_proc, GetCurrentThread(), 0);
r = WaitForSingleObjectEx(ev, INFINITE, TRUE);
CHECK(r == WAIT_OBJECT_0 && apc_ran == 0, "object before apc %lu ran %ld", r, apc_ran);
r = SleepEx(0, TRUE);
CHECK(r == WAIT_IO_COMPLETION && apc_ran == 1, "apc after object %lu ran %ld", r, apc_ran);
r = SleepEx(0, TRUE);
CHECK(r == 0, "sleepex zero %lu", r);
ResetEvent(ev);
GetSystemTimeAsFileTime((FILETIME *)&when);
when.QuadPart += 2000000;
start = GetTickCount();
r = pNtWaitForSingleObject(ev, FALSE, &when);
CHECK(r == STATUS_TIMEOUT, "absolute timeout status %lx", r);
CHECK(GetTickCount() - start >= 150 && GetTickCount() - start < 1500, "absolute timeout took %lu", GetTickCount() - start);
CHECK(!ReleaseSemaphore(sem, 2, &prev) && GetLastError() == ERROR_TOO_MANY_POSTS, "semaphore limit");
CHECK(ReleaseSemaphore(sem, 1, &prev) && prev == 2, "semaphore prev %ld", prev);
for (i = 0; i < 3; i++) CHECK(WaitForSingleObject(sem, 0) == WAIT_OBJECT_0, "semaphore take %d", i);
CHECK(WaitForSingleObject(sem, 0) == WAIT_TIMEOUT, "semaphore empty");
h[0] = sem; h[1] = ev;
ReleaseSemaphore(sem, 1, NULL);
CHECK(WaitForMultipleObjects(2, h, TRUE, 50) == WAIT_TIMEOUT, "wait all partial");
CHECK(ReleaseSemaphore(sem, 0, &prev) && prev == 1, "wait all rollback lost token prev %ld", prev);
SetEvent(ev);
CHECK(WaitForMultipleObjects(2, h, TRUE, 50) == WAIT_OBJECT_0, "wait all ready");
CHECK(ReleaseSemaphore(sem, 0, &prev) && prev == 0, "wait all consumed prev %ld", prev);
CHECK(WaitForSingleObject(ev, 0) == WAIT_OBJECT_0, "manual stays set");
CloseHandle(sem);
CloseHandle(ev);
printf("semantics ok\n");
}
static HANDLE token_sem;
static volatile LONG token_count;
static DWORD WINAPI token_worker(void *arg)
{
HANDLE h[2] = { token_sem, (HANDLE)arg };
int i;
for (i = 0; i < 20000; i++)
{
DWORD r = (i & 1) ? WaitForSingleObject(token_sem, INFINITE) : WaitForMultipleObjects(2, h, TRUE, INFINITE);
if (r != WAIT_OBJECT_0) { CHECK(0, "token wait %lu", r); break; }
if (InterlockedIncrement(&token_count) > 3) CHECK(0, "token overcommit");
InterlockedDecrement(&token_count);
if (!(i & 1)) SetEvent((HANDLE)arg);
ReleaseSemaphore(token_sem, 1, NULL);
}
return 0;
}
static void test_semaphore_tokens(void)
{
HANDLE th[6], ev = CreateEventA(NULL, FALSE, TRUE, NULL);
LONG prev = -1;
int i;
token_sem = CreateSemaphoreA(NULL, 3, 3, NULL);
for (i = 0; i < 6; i++) th[i] = CreateThread(NULL, 0, token_worker, ev, 0, NULL);
WaitForMultipleObjects(6, th, TRUE, INFINITE);
for (i = 0; i < 6; i++) CloseHandle(th[i]);
CHECK(ReleaseSemaphore(token_sem, 0, &prev) && prev == 3, "token final count %ld", prev);
for (i = 0; i < 3; i++) CHECK(WaitForSingleObject(token_sem, 0) == WAIT_OBJECT_0, "token drain %d", i);
CHECK(WaitForSingleObject(token_sem, 0) == WAIT_TIMEOUT, "token extra");
CHECK(WaitForSingleObject(ev, 0) == WAIT_OBJECT_0, "token event lost");
CloseHandle(token_sem);
CloseHandle(ev);
printf("semaphore_tokens ok\n");
}
static volatile LONG churn_ready;
static DWORD WINAPI churn_mutex(void *arg)
{
HANDLE h[2] = { (HANDLE)arg, ev_manual };
unsigned int i = 0;
if (WaitForMultipleObjects(2, h, TRUE, INFINITE) <= WAIT_ABANDONED_0 + 1) ReleaseMutex(h[0]);
InterlockedIncrement(&churn_ready);
for (;;)
{
DWORD r = (i++ & 1) ? WaitForSingleObject(h[0], INFINITE) : WaitForMultipleObjects(2, h, TRUE, INFINITE);
if (r == WAIT_OBJECT_0 || r == WAIT_ABANDONED || r == WAIT_ABANDONED_0) ReleaseMutex(h[0]);
}
return 0;
}
static DWORD WINAPI blocked_waiter(void *arg)
{
WaitForSingleObject((HANDLE)arg, 0);
InterlockedIncrement(&churn_ready);
return WaitForSingleObject((HANDLE)arg, INFINITE);
}
static void test_terminate(void)
{
HANDLE th[4], m, ev;
DWORD r, start;
int round, i, stuck = 0;
ev_manual = CreateEventA(NULL, TRUE, TRUE, NULL);
for (round = 0; round < 150; round++)
{
m = CreateMutexA(NULL, FALSE, NULL);
churn_ready = 0;
for (i = 0; i < 4; i++) th[i] = CreateThread(NULL, 0, churn_mutex, m, 0, NULL);
while (churn_ready < 4) Sleep(0);
Sleep(round % 4);
for (i = 0; i < 4; i++) TerminateThread(th[i], 7);
WaitForMultipleObjects(4, th, TRUE, INFINITE);
r = WaitForSingleObject(m, 3000);
if (r == WAIT_TIMEOUT) stuck++;
else ReleaseMutex(m);
for (i = 0; i < 4; i++) CloseHandle(th[i]);
CloseHandle(m);
}
CHECK(!stuck, "mutex stuck after terminate in %d rounds", stuck);
ev = CreateEventA(NULL, TRUE, FALSE, NULL);
churn_ready = 0;
for (i = 0; i < 4; i++) th[i] = CreateThread(NULL, 0, blocked_waiter, ev, 0, NULL);
while (churn_ready < 4) Sleep(0);
Sleep(50);
for (i = 0; i < 4; i++) TerminateThread(th[i], 7);
WaitForMultipleObjects(4, th, TRUE, INFINITE);
for (i = 0; i < 4; i++) CloseHandle(th[i]);
start = GetTickCount();
for (i = 0; i < 100; i++) PulseEvent(ev);
CHECK(GetTickCount() - start < 400, "pulse after killed waiters took %lu ms", GetTickCount() - start);
CHECK(WaitForSingleObject(ev, 0) == WAIT_TIMEOUT, "pulse left event set");
CloseHandle(ev);
CloseHandle(ev_manual);
printf("terminate ok\n");
}
static void test_cross_process(const char *self)
{
HANDLE ev = CreateEventA(NULL, FALSE, FALSE, "Local\\esstress_ev");
HANDLE done = CreateEventA(NULL, TRUE, FALSE, "Local\\esstress_done");
HANDLE sm = CreateSemaphoreA(NULL, 0, 100000, "Local\\esstress_sem");
PROCESS_INFORMATION pi;
STARTUPINFOA si = { sizeof(si) };
char cmd[MAX_PATH + 16];
DWORD r, code;
int i;
sprintf(cmd, "\"%s\" child", self);
CHECK(CreateProcessA(NULL, cmd, NULL, NULL, FALSE, 0, NULL, NULL, &si, &pi), "createprocess %lu", GetLastError());
for (i = 0; i < 5000; i++)
{
SetEvent(ev);
r = WaitForSingleObject(sm, 5000);
if (r != WAIT_OBJECT_0) { CHECK(0, "xproc wait %lu at %d", r, i); break; }
}
SetEvent(done);
r = WaitForSingleObject(pi.hProcess, 10000);
CHECK(r == WAIT_OBJECT_0, "child exit wait %lu", r);
GetExitCodeProcess(pi.hProcess, &code);
CHECK(code == 0, "child code %lu", code);
CloseHandle(pi.hProcess); CloseHandle(pi.hThread);
CloseHandle(ev); CloseHandle(done); CloseHandle(sm);
printf("cross_process ok\n");
}
static int child(void)
{
HANDLE ev = OpenEventA(SYNCHRONIZE | EVENT_MODIFY_STATE, FALSE, "Local\\esstress_ev");
HANDLE done = OpenEventA(SYNCHRONIZE, FALSE, "Local\\esstress_done");
HANDLE sm = OpenSemaphoreA(SEMAPHORE_MODIFY_STATE | SYNCHRONIZE, FALSE, "Local\\esstress_sem");
HANDLE objs[2];
objs[0] = done; objs[1] = ev;
if (!ev || !done || !sm) return 1;
for (;;)
{
DWORD r = WaitForMultipleObjects(2, objs, FALSE, 10000);
if (r == WAIT_OBJECT_0) return 0;
if (r != WAIT_OBJECT_0 + 1) return 2;
ReleaseSemaphore(sm, 1, NULL);
}
}
static void test_kill_holder(const char *self)
{
HANDLE m = CreateMutexA(NULL, FALSE, "Local\\esstress_mutex");
PROCESS_INFORMATION pi;
STARTUPINFOA si = { sizeof(si) };
char cmd[MAX_PATH + 16];
DWORD r;
sprintf(cmd, "\"%s\" holder", self);
CreateProcessA(NULL, cmd, NULL, NULL, FALSE, 0, NULL, NULL, &si, &pi);
Sleep(500);
TerminateProcess(pi.hProcess, 7);
WaitForSingleObject(pi.hProcess, INFINITE);
r = WaitForSingleObject(m, 3000);
CHECK(r == WAIT_ABANDONED, "killed holder %lu", r);
ReleaseMutex(m);
CloseHandle(pi.hProcess); CloseHandle(pi.hThread); CloseHandle(m);
printf("kill_holder ok\n");
}
static int holder(void)
{
HANDLE m = OpenMutexA(SYNCHRONIZE, FALSE, "Local\\esstress_mutex");
WaitForSingleObject(m, INFINITE);
Sleep(INFINITE);
return 0;
}
static void test_many_objects(void)
{
static HANDLE h[20000];
int i, n = 0;
for (i = 0; i < 20000; i++)
{
if (!(h[i] = CreateEventA(NULL, i & 1, i & 2, NULL))) break;
n++;
}
CHECK(n == 20000, "created %d", n);
for (i = 0; i < n; i++) CHECK(WaitForSingleObject(h[i], 0) == ((i & 2) ? WAIT_OBJECT_0 : WAIT_TIMEOUT), "obj %d", i);
for (i = 0; i < n; i++) CloseHandle(h[i]);
for (i = 0; i < 100000; i++)
{
HANDLE e = CreateEventA(NULL, FALSE, TRUE, NULL);
if (WaitForSingleObject(e, 0) != WAIT_OBJECT_0) { CHECK(0, "churn %d", i); CloseHandle(e); break; }
CloseHandle(e);
}
printf("many_objects ok\n");
}
static void test_timers_and_threads(void)
{
HANDLE t = CreateWaitableTimerA(NULL, FALSE, NULL);
LARGE_INTEGER due;
DWORD start = GetTickCount(), r;
int i;
due.QuadPart = -10 * 1000 * 20;
SetWaitableTimer(t, &due, 10, NULL, NULL, FALSE);
for (i = 0; i < 20; i++)
{
r = WaitForSingleObject(t, 1000);
CHECK(r == WAIT_OBJECT_0, "timer %d %lu", i, r);
}
CHECK(GetTickCount() - start < 2000, "timer slow %lu", GetTickCount() - start);
CancelWaitableTimer(t);
CloseHandle(t);
r = WaitForSingleObject(GetCurrentProcess(), 10);
CHECK(r == WAIT_TIMEOUT, "process self %lu", r);
printf("timers ok\n");
}
int main(int argc, char **argv)
{
if (argc > 1 && !strcmp(argv[1], "child")) return child();
if (argc > 1 && !strcmp(argv[1], "holder")) return holder();
test_mutex_contention();
test_semaphore();
test_auto_event();
test_manual_event();
test_wait_all();
test_abandoned();
test_apc();
test_close_while_waiting();
test_pulse();
test_semantics();
test_semaphore_tokens();
if (argc > 1 && !strcmp(argv[1], "terminate")) test_terminate();
test_cross_process(argv[0]);
test_kill_holder(argv[0]);
test_many_objects();
test_timers_and_threads();
printf("esstress: %d failures\n", failures);
return failures ? 1 : 0;
}
+48
View File
@@ -0,0 +1,48 @@
#!/bin/sh
set -eu
out=${1:?usage: build-x86-preloads.sh <linuxfs asset directory>}
for arch in x86_64 i386; do
case $arch in
x86_64) bits=-m64 ;;
i386) bits=-m32 ;;
esac
mkdir -p "$out/$arch"
gcc $bits -shared -fPIC -O2 -Wall -pthread \
-o "$out/$arch/libblsession.so" tools/linuxfs/preload/*.c -ldl
g++ $bits -shared -fPIC -O2 -Wall -Wno-attributes -Wno-nonnull-compare \
-pthread -std=c++17 -static-libstdc++ -static-libgcc -Wl,--exclude-libs,ALL \
-o "$out/$arch/libfakeinput.so" app/src/main/cpp/fakeinput_steam.cpp -ldl
strip --strip-unneeded "$out/$arch/libblsession.so" "$out/$arch/libfakeinput.so"
for lib in "$out/$arch/libblsession.so" "$out/$arch/libfakeinput.so"; do
newest=$(objdump -T "$lib" | grep -o 'GLIBC_2\.[0-9]*' | sort -u -t. -k2,2n | tail -1)
minor=${newest#GLIBC_2.}
if [ "$minor" -gt 31 ]; then
echo "$lib needs $newest; Steam Linux Runtime 3.0 has glibc 2.31" >&2
exit 1
fi
if objdump -p "$lib" | grep -q 'NEEDED.*lib\(stdc++\|gcc_s\)'; then
echo "$lib links the C++ runtime dynamically" >&2
exit 1
fi
done
done
for name in faultreport thunkaudit; do
lib=$out/x86_64/lib$name.so
gcc -m64 -shared -fPIC -O2 -Wall -Wextra -nostdlib -fno-stack-protector -fno-builtin -fvisibility=hidden \
-o "$lib" tools/linuxfs/fex/$name.c
strip --strip-unneeded "$lib"
if objdump -p "$lib" | grep -q NEEDED || [ -n "$(nm -D --undefined-only "$lib" 2>/dev/null)" ]; then
echo "$lib must not need any library" >&2
exit 1
fi
done
stub=$(mktemp -d)
gcc -m64 -shared -nostdlib -Wl,-soname,/usr/lib/x86_64-linux-gnu/libvulkan.so.1 -o "$stub/libvulkan.so.1" -x c /dev/null
gcc -m64 -shared -nostdlib -Wl,-z,nodelete -Wl,--no-as-needed \
-o "$out/x86_64/libvulkan-thunk.so" -x c /dev/null -x none "$stub/libvulkan.so.1"
rm -rf "$stub"
if ! objdump -p "$out/x86_64/libvulkan-thunk.so" | grep -q 'NEEDED *\/usr/lib/x86_64-linux-gnu/libvulkan\.so\.1$' \
|| ! objdump -p "$out/x86_64/libvulkan-thunk.so" | grep -q 'FLAGS_1 *0x0*8$'; then
echo "$out/x86_64/libvulkan-thunk.so must be nodelete and need only the Vulkan thunk" >&2
exit 1
fi
+1 -1
View File
@@ -39,7 +39,7 @@ run lxqt-panel lxqt-panel
# The desktop was asked for with Steam in it (the front end's "Steam Desktop UI", Big Picture's
# "Switch to Desktop"): the client's desktop UI, as the menu's Steam entry opens it. A running
# client just gets the request; on a software desktop it goes back to the app as a Steam session.
[ "${BL_DESKTOP_STEAM:-0}" = 1 ] && run steam /usr/local/bin/bannerlator-steam-launch desktop
[ "${BL_DESKTOP_STEAM:-0}" = 1 ] && run steam /usr/local/bin/droiddeck-steam-launch desktop
# Remote runs, for driving the desktop from a debugger: with ~/.droiddeck-remote-run present (only
# the app or root can create it), each script dropped into ~/.cache/droiddeck/run/ is run once,
# inside the session with its environment, its output tagged into the desktop log.
+7 -7
View File
@@ -1,7 +1,7 @@
#!/bin/bash
# The desktop session: labwc nested as a client of the app's compositor, with LXQt's panel and
# file manager started from its autostart, and Xwayland for the X11 programs (the Steam client,
# most emulators). Run by bannerlator-session in place of gamescope for mode "lxqt".
# most emulators). Run by droiddeck-session in place of gamescope for mode "lxqt".
# Staged by the app over the desktop package's copy on every launch, so a fix here reaches an
# installed desktop without re-downloading the package.
set -u
@@ -114,16 +114,16 @@ if [ ! -f /var/cache/fontconfig/.droiddeck-built ] && command -v fc-cache >/dev/
echo "== fontconfig: cache rebuilt, default sans is $(fc-match sans-serif 2>/dev/null)"
fi
# The menu's "Steam" / "Steam Big Picture" entries and one entry per installed game: all of them
# hand off to the app (bannerlator-steam-launch), since the client cannot draw on this desktop.
# hand off to the app (droiddeck-steam-launch), since the client cannot draw on this desktop.
export BL_DESKTOP=1
ln -sf /usr/local/bin/bannerlator-steam-launch /usr/local/bin/steamdeck-steam 2>/dev/null
ln -sf /usr/local/bin/bannerlator-steam-shim /usr/local/bin/steam 2>/dev/null
python3 /usr/local/bin/bannerlator-desktop-games 2>&1 | sed 's/^/[games] /'
ln -sf /usr/local/bin/droiddeck-steam-launch /usr/local/bin/steamdeck-steam 2>/dev/null
ln -sf /usr/local/bin/droiddeck-steam-shim /usr/local/bin/steam 2>/dev/null
python3 /usr/local/bin/droiddeck-desktop-games 2>&1 | sed 's/^/[games] /'
# Player 1 on the controller in every emulator that has no profile of its own yet - now, and again
# every half minute: an emulator started for the first time writes its own config, which is then
# given the controller for its next start without waiting for the next desktop.
python3 /usr/local/bin/bannerlator-pad-defaults 2>&1 | sed 's/^/[pads] /'
( while sleep 30; do python3 /usr/local/bin/bannerlator-pad-defaults 2>&1 | sed 's/^/[pads] /'; done ) &
python3 /usr/local/bin/droiddeck-pad-defaults 2>&1 | sed 's/^/[pads] /'
( while sleep 30; do python3 /usr/local/bin/droiddeck-pad-defaults 2>&1 | sed 's/^/[pads] /'; done ) &
# Games and emulators from the menu run through droiddeck-gpu (their own gamescope on the app's
# compositor) while the desktop is on pixman, where they could not draw with the GPU.
# BL_DESKTOP_GPU=on/off overrides; each entry keeps "Run in a desktop window" either way.
+2 -2
View File
@@ -18,7 +18,7 @@ LOCAL_DIR = "/usr/local/share/applications"
FLATPAK_DIR = os.path.join(HOME, ".local/share/flatpak/exports/share/applications")
USER_DIR = os.path.join(HOME, ".local/share/applications")
MARK = "X-DroidDeck-GPU=1"
# The client and its games already hand themselves to the app (bannerlator-steam-launch).
# The client and its games already hand themselves to the app (droiddeck-steam-launch).
SKIP = re.compile(r"^(steam|droiddeck-game-|droiddeck-steam|steamdeck-)", re.I)
@@ -45,7 +45,7 @@ def gpu_exec(exec_line):
"""The Exec that runs a program on the GPU; a Flatpak app goes through its launcher, which
keeps it on gamescope's X server instead of the desktop's Wayland socket."""
if exec_line.startswith("/usr/bin/flatpak run ") or exec_line.startswith("flatpak run "):
exec_line = "/usr/local/bin/bannerlator-flatpak-run " + exec_line.split(" run ", 1)[1]
exec_line = "/usr/local/bin/droiddeck-flatpak-run " + exec_line.split(" run ", 1)[1]
return "droiddeck-gpu " + exec_line
+1 -1
View File
@@ -20,7 +20,7 @@
</keyboard>
<mouse><doubleClickTime>500</doubleClickTime></mouse>
<!-- Programs opened on the desktop at a size that suits it. melonDS remembers a 1x window for the
front end's full-screen toggle (bannerlator-pad-defaults), too small for a desktop: here it
front end's full-screen toggle (droiddeck-pad-defaults), too small for a desktop: here it
opens at 800x600, centred (AutoPlace alone kept the top-left corner). Only the desktop reads this file, so the front end is unaffected. -->
<windowRules>
<windowRule title="melonDS*">
+144
View File
@@ -0,0 +1,144 @@
#if !defined(__x86_64__)
#error faultreport is built for x86_64 guests only
#endif
#define SYS_read 0
#define SYS_write 1
#define SYS_close 3
#define SYS_rt_sigaction 13
#define SYS_getpid 39
#define SYS_gettid 186
#define SYS_tgkill 234
#define SYS_openat 257
#define AT_FDCWD (-100)
#define SIGILL 4
#define SA_SIGINFO 0x4UL
#define SA_RESTORER 0x04000000UL
#define SA_NODEFER 0x40000000UL
#define SA_RESETHAND 0x80000000UL
struct kernel_sigaction {
void (*handler)(int, void *, void *);
unsigned long flags;
void (*restorer)(void);
unsigned long mask;
};
static long sys(long n, long a, long b, long c, long d)
{
long r;
register long r10 __asm__("r10") = d;
__asm__ volatile("syscall" : "=a"(r) : "a"(n), "D"(a), "S"(b), "d"(c), "r"(r10) : "rcx", "r11", "memory");
return r;
}
static char out[1024];
static unsigned long used;
static char maps[65536];
static void put(const char *s)
{
while (*s && used < sizeof(out) - 1)
out[used++] = *s++;
}
static void put_hex(unsigned long v, int digits)
{
char buf[17];
for (int i = digits - 1; i >= 0; i--) {
buf[i] = "0123456789abcdef"[v & 15];
v >>= 4;
}
buf[digits] = 0;
put(buf);
}
static unsigned long parse_hex(const char **p)
{
unsigned long v = 0;
for (;;) {
char c = **p;
if (c >= '0' && c <= '9') v = v * 16 + (c - '0');
else if (c >= 'a' && c <= 'f') v = v * 16 + (c - 'a' + 10);
else return v;
(*p)++;
}
}
static int find_mapping(unsigned long addr, const char **line, unsigned long *length, unsigned long *start, int *readable)
{
long fd = sys(SYS_openat, AT_FDCWD, (long)"/proc/self/maps", 0, 0);
if (fd < 0) return 0;
unsigned long have = 0;
long n;
while (have < sizeof(maps) - 1 && (n = sys(SYS_read, fd, (long)(maps + have), sizeof(maps) - 1 - have, 0)) > 0)
have += n;
sys(SYS_close, fd, 0, 0, 0);
maps[have] = 0;
const char *p = maps;
while (*p) {
const char *begin = p;
unsigned long lo = parse_hex(&p);
if (*p == '-') p++;
unsigned long hi = parse_hex(&p);
while (*p == ' ') p++;
int r = *p == 'r';
const char *end = begin;
while (*end && *end != '\n') end++;
if (addr >= lo && addr < hi) {
*line = begin;
*length = end - begin;
*start = lo;
*readable = r && addr + 16 <= hi;
return 1;
}
p = *end ? end + 1 : end;
}
return 0;
}
static void report(int sig, void *info, void *context)
{
(void)context;
unsigned long addr = *(unsigned long *)((char *)info + 16);
int code = *(int *)((char *)info + 8);
used = 0;
put("droiddeck-fex: SIGILL at 0x");
put_hex(addr, 12);
put(" (si_code ");
put_hex((unsigned long)code, 2);
put(")");
const char *line;
unsigned long length, start;
int readable;
if (find_mapping(addr, &line, &length, &start, &readable)) {
put(", offset 0x");
put_hex(addr - start, 8);
if (readable) {
put(", bytes");
for (int i = 0; i < 16; i++) {
put(" ");
put_hex(((unsigned char *)addr)[i], 2);
}
}
put("\n in ");
for (unsigned long i = 0; i < length && used < sizeof(out) - 2; i++)
out[used++] = line[i];
} else {
put(", outside any mapping");
}
put("\n");
sys(SYS_write, 2, (long)out, used, 0);
struct kernel_sigaction fallback = {0, 0, 0, 0};
sys(SYS_rt_sigaction, sig, (long)&fallback, 0, sizeof(fallback.mask));
sys(SYS_tgkill, sys(SYS_getpid, 0, 0, 0, 0), sys(SYS_gettid, 0, 0, 0, 0), sig, 0);
}
void faultreport_restore(void);
__asm__(".text\n.type faultreport_restore,@function\nfaultreport_restore:\nmov $15, %eax\nsyscall\nhlt\n");
__attribute__((constructor)) static void install(void)
{
struct kernel_sigaction action = {report, SA_SIGINFO | SA_NODEFER | SA_RESETHAND | SA_RESTORER, faultreport_restore, 0};
sys(SYS_rt_sigaction, SIGILL, (long)&action, 0, sizeof(action.mask));
}
+40
View File
@@ -0,0 +1,40 @@
#if !defined(__x86_64__)
#error thunkaudit is built for x86_64 guests only
#endif
#define LA_SER_ORIG 0x01
static const char vulkan_thunk[] = "/usr/lib/x86_64-linux-gnu/libvulkan.so.1";
static const char vulkan_pin[] = "/usr/lib/x86_64-linux-gnu/droiddeck/libvulkan-thunk.so";
static int same(const char *a, const char *b)
{
while (*a && *a == *b) {
a++;
b++;
}
return *a == *b;
}
static const char *base_name(const char *path)
{
const char *name = path;
for (; *path; path++)
if (*path == '/')
name = path + 1;
return name;
}
__attribute__((visibility("default"))) unsigned int la_version(unsigned int version)
{
return version;
}
__attribute__((visibility("default"))) char *la_objsearch(const char *name, unsigned long *cookie, unsigned int flag)
{
const char *file = base_name(name);
(void)cookie;
if (flag == LA_SER_ORIG && !same(name, vulkan_thunk) && (same(file, "libvulkan.so.1") || same(file, "libvulkan.so")))
return (char *)vulkan_pin;
return (char *)name;
}
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 VHSgunzo
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
@@ -3,7 +3,7 @@
# session: "plasma", "plasma-wayland-persistent", ...) and waits for Game Mode to end; with nothing
# here it sat on "Switching to Desktop..." for ever. Here the desktop is DroidDeck's own: the app is
# asked for a desktop session with Steam's desktop client open in it (the request file
# bannerlator-steam-launch uses; the app ends this session and starts that one). Anything else -
# droiddeck-steam-launch uses; the app ends this session and starts that one). Anything else -
# "gamescope", the session Game Mode already is - has nothing to do.
echo "steamos-session-select: $*" >&2
case "${1:-}" in
@@ -1,69 +0,0 @@
#!/bin/bash
# An AppImage imported from the front end (AppImageManager), extracted once at import into
# /opt/appimages/user/<id>/app
# and started from there: proot has no FUSE to mount the image, and extracting it at every
# start (APPIMAGE_EXTRACT_AND_RUN) would cost the whole image in /tmp each time.
# bannerlator-appimage-run /opt/appimages/user/<id> [args...]
set -u
dir=${1:?usage: bannerlator-appimage-run <imported AppImage directory> [args...]}
shift
export APPDIR=$dir/app
# APPIMAGE is left unset on purpose: with it an app believes it runs from an image file and offers
# to add itself to the menu (DuckStation) or update the file in place - DroidDeck already does the
# one, and there is no image file for the other.
unset APPIMAGE
export ARGV0=$(cat "$dir/name" 2>/dev/null || basename "$dir")
if [ ! -x "$APPDIR/AppRun" ]; then
echo "bannerlator-appimage-run: $APPDIR/AppRun is missing; import the AppImage again" >&2
exit 127
fi
# Browsers sandbox their own children with namespaces and a setuid helper, neither of which proot
# can give them (the Flatpak sandboxes get the same; see BwrapSpawner): Firefox and its forks (Zen,
# LibreWolf) by these switches, Electron and Chromium builds by --no-sandbox. Chromium on Wayland
# also asks the render node for a DRM version KGSL cannot give, so with an X server up it stays on
# X11, which it picks from the session type.
export MOZ_DISABLE_CONTENT_SANDBOX=1 MOZ_DISABLE_GMP_SANDBOX=1 MOZ_DISABLE_RDD_SANDBOX=1
export MOZ_DISABLE_SOCKET_PROCESS_SANDBOX=1 MOZ_DISABLE_UTILITY_SANDBOX=1
# sharun images start the app through their own ld.so, patched not to read /etc/ld.so.preload, and
# drop LD_PRELOAD unless allowed: without this the app never gets the session's pads.
if [ -s /etc/ld.so.preload ]; then
preload=$(tr -s '\n' ':' < /etc/ld.so.preload)
export LD_PRELOAD="${preload%:}${LD_PRELOAD:+:$LD_PRELOAD}" SHARUN_ALLOW_LD_PRELOAD=1
fi
flags=()
if [ -e "$APPDIR/chrome-sandbox" ] || compgen -G "$APPDIR/*/chrome-sandbox" >/dev/null; then
flags+=(--no-sandbox)
if [ -n "${DISPLAY:-}" ]; then
export XDG_SESSION_TYPE=x11
unset WAYLAND_DISPLAY
flags+=(--ozone-platform=x11)
fi
fi
echo "== appimage $ARGV0 ($APPDIR) ${flags[*]}"
cd "$APPDIR" || exit 1
# Apps expect a session bus; Electron's main process traps (SIGTRAP) on an address it cannot
# parse. A run mode has none, so one is started for the app, as for Flatpak apps.
case ${DBUS_SESSION_BUS_ADDRESS:-} in
unix:*) "$APPDIR/AppRun" "${flags[@]}" "$@" ;;
*) unset DBUS_SESSION_BUS_ADDRESS
if command -v dbus-run-session >/dev/null 2>&1; then
dbus-run-session -- "$APPDIR/AppRun" "${flags[@]}" "$@"
else
"$APPDIR/AppRun" "${flags[@]}" "$@"
fi ;;
esac
rc=$?
# The program a session was started for may hand over to another process and exit: Electron apps
# relaunch themselves (Obsidian on its first start), launchers fork their real program. The
# session ends with this script, and takes the rest with it - so it waits while anything running
# from the image is still up.
image_running() {
local p exe
for p in /proc/[0-9]*; do
exe=$(readlink "$p/exe" 2>/dev/null) || continue
case $exe in "$APPDIR"/*) return 0 ;; esac
done
return 1
}
while image_running; do sleep 2; done
exit $rc
@@ -1,37 +0,0 @@
#!/bin/bash
# A script added on the Desktop page (UserApps), run from its own folder:
# bannerlator-script-run /opt/droiddeck-apps/<id> [args...]
set -u
dir=${1:?usage: bannerlator-script-run <added app directory> [args...]}
shift
entry=$(cat "$dir/entry" 2>/dev/null) || { echo "bannerlator-script-run: $dir has no entry" >&2; exit 127; }
cd -P "$dir/files" 2>/dev/null || { echo "bannerlator-script-run: $dir/files is missing; add the script again" >&2; exit 127; }
folder=$PWD
# Shared storage is noexec, so the script is handed to its interpreter rather than executed.
first=
IFS= read -r first < "$entry"
first=${first%$'\r'}
interp=(/bin/bash)
case $first in '#!'*) read -r -a interp <<< "${first#'#!'}" ;; esac
echo "== script $entry in $folder (${interp[*]})"
case ${DBUS_SESSION_BUS_ADDRESS:-} in
unix:*) "${interp[@]}" "./$entry" "$@" ;;
*) unset DBUS_SESSION_BUS_ADDRESS
if command -v dbus-run-session >/dev/null 2>&1; then
dbus-run-session -- "${interp[@]}" "./$entry" "$@"
else
"${interp[@]}" "./$entry" "$@"
fi ;;
esac
rc=$?
# A launcher script often starts the game and returns; the session lasts while it runs.
running() {
local p exe
for p in /proc/[0-9]*; do
exe=$(readlink "$p/exe" 2>/dev/null) || continue
case $exe in "$folder"/*) return 0 ;; esac
done
return 1
}
while running; do sleep 2; done
exit $rc
@@ -1,641 +0,0 @@
#!/usr/bin/python3
# Makes the ARM64 Proton the client already downloaded usable here, and sets it as the default tool
# for Windows titles (CompatToolMapping "0" in config.vdf) when the user has not chosen one. Run
# before the client starts; it rewrites config.vdf on exit.
#
# bannerlator-steam-compat <steam root>
#
# Valve ships a native aarch64 Proton (arm64ec Wine with FEX as the x86 layer) as an ordinary depot,
# which is exactly what this runtime wants. Its toolmanifest declares require_tool_appid 4185400 -
# Steam Linux Runtime 4 for arm64 - so Steam stacks pressure-vessel underneath it, and
# pressure-vessel needs unprivileged user namespaces, which an Android app does not get. Steam then
# drops the launch on the floor: no window, no log, straight back to the library.
#
# So the depot is re-registered here as a compatibility tool of our own: every entry symlinked so
# nothing is copied and Steam still updates the real depot, but with a toolmanifest that leaves out
# require_tool_appid. Proton takes its base directory from dirname(sys.argv[0]) without resolving
# it, so it reads files/ through those symlinks and runs directly on this rootfs.
import os
import re
import shutil
import sys
TOOL = "bannerlator-proton-arm64"
# Valve's own ARM64 Proton, by the name the client registers it under. Used only to bootstrap a
# fresh install: naming it is what makes the client offer it and fetch the depot.
SEED_TOOL = "proton-experimental-arm64"
# The ARM64 Proton depot the client downloads for itself, as a compatibility-tool appid.
# The session asks the client to install it on the command line the first time (steam://install),
# so a new client has it before a Windows title is first launched. It is deliberately NOT named as
# this tool's dependency: a dependency is composed with its own dependencies, and this depot
# requires 4185400, the arm64 Steam Linux Runtime, whose container needs user namespaces Android
# does not give - the launch would be wrapped in it and die before this tool ever ran.
PROTON_ARM64_APPID = "4427310"
LAUNCHER = "bannerlator-proton"
# The extra Steam library the app mounts the installed games into (see bannerlator-steam-library).
# installed_apps() scans it too; it lives only in the sibling script, so define it here as well.
LIBRARY = "/mnt/bannerlator" # the card library is LIBRARY + "-sd"; internal storage is steam_root itself
# The client lists its own "Proton Experimental (ARM64)" and "Proton 11.0 (ARM64)" in the same
# Compatibility dropdown, and those are the two that cannot start a game here. Ours has to be
# unmistakable at a glance, or a user picks a neighbour and gets a launch that fails silently.
# Newest first: whichever the client has downloaded is used.
SOURCES = ("Proton Experimental (ARM64)", "Proton 11.0 (ARM64)")
# Written by us; everything else in the depot is symlinked through.
OWN_FILES = ("toolmanifest.vdf", "compatibilitytool.vdf", "bannerlator-proton")
# The client assigns a Proton to a Windows title by itself, as a per-app CompatToolMapping entry at
# priority 250 - which outranks the "0" default, so setting that default alone changes nothing. Every
# Proton it can pick is one this runtime cannot start: the ARM64 ones want pressure-vessel (see
# above) and the x86_64 ones want pressure-vessel and FEX on top. Ours is the only one that runs
# here, so every Proton mapping is repointed at it. Tools that are not Proton are left alone.
COMPAT_TOOL_VDF = """"compatibilitytools"
{
"compat_tools"
{
"%s"
{
"install_path" "."
"display_name" "Bannerlator Proton (ARM64)"
"from_oslist" "windows"
"to_oslist" "linux"
}
}
}
"""
TOOL_MANIFEST_VDF = """"manifest"
{
"version" "2"
"commandline" "/%s %%verb%%"
"use_sessions" "1"
"compatmanager_layer_name" "proton"
}
""" % LAUNCHER
# Steam runs its own graphics probe through EVERY registered tool at once, ~40 s after logging in,
# all with STEAM_COMPAT_DATA_PATH=.../compatdata/0, and those concurrent Protons can write their
# builtin DLLs through each other's prefix links into each other's trees (seen: GE's FEX DLLs became
# Proton Experimental's). Moving those runs to a prefix per tool was tried and crashed the Steam
# client right after the probe started (segfault, rc 139, first ever in 25 sessions), so the client
# relies on compatdata/0 itself. The launch-time check below is the protection instead.
# The Components page's choices, enforced in the one place every game launch passes: the app keeps
# desired.tsv (tool dir, Proton build, component, file, sha256, source copy) and an unpacked copy of
# each package in use. A file that no longer matches - whatever changed it - is put back before
# Proton starts. Only for the Proton build the choice was made for: after a Steam update the build
# differs and the Proton's own new files are left alone until the user chooses again.
BL_COMPONENTS_SETUP = """
bl_components() {
base=$1
cm_dir="${HOME:-/root}/.local/share/bannerlator-components"
cm_list="$cm_dir/desired.tsv"
# A real game launch only: the verb the client starts a game with and the game's own prefix. Every
# other run - above all the client's own probes right after login, which all share compatdata/0 -
# is left exactly as it was: with this check running through them, the client segfaulted (rc 139)
# within seconds of starting its probes, four sessions out of four. The prefix decides rather than
# the app id because a non-Steam shortcut launches with app id 0 but still gets a prefix of its own.
[ "${verb:-}" = waitforexitandrun ] || return 0
case "${STEAM_COMPAT_DATA_PATH:-}" in ''|*/compatdata/0|*/compatdata/0/|*/compatdata/0-*) return 0 ;; esac
[ -f "$cm_list" ] || return 0
cm_app=${STEAM_COMPAT_DATA_PATH%/}
cm_app=${cm_app##*/}
cm_tool=${base##*/}
cm_build=$(awk 'NR == 1 { print $2 }' "$base/version" 2>/dev/null)
cm_fixed=0
cm_checked=0
while IFS=$'\\t' read -r t want comp rel sum src; do
[ "$t" = "$cm_tool" ] && [ "$want" = "$cm_build" ] || continue
cm_checked=$((cm_checked + 1))
f="$base/$rel"
if [ -f "$f" ] && [ "$(sha256sum "$f" | cut -d' ' -f1)" = "$sum" ]; then continue; fi
if [ ! -f "$src" ]; then
echo "bannerlator-proton: components: $comp source missing ($src), left as it is" >&2
continue
fi
mkdir -p "$(dirname "$f")"
cp -f "$src" "$f.bl-new" && mv -f "$f.bl-new" "$f" || continue
case "$rel" in *.so) chmod 755 "$f" ;; *) chmod 644 "$f" ;; esac
cm_fixed=$((cm_fixed + 1))
done < "$cm_list"
[ "$cm_checked" -gt 0 ] && echo "bannerlator-proton: components: checked $cm_checked file(s) in $cm_tool for app $cm_app" >&2
if [ "$cm_fixed" -gt 0 ]; then
echo "bannerlator-proton: components: re-applied $cm_fixed of $cm_checked file(s) in $cm_tool before this launch" >&2
printf '%s\\t%s\\t%s\\n' "$(date +%s)" "$cm_tool" "$cm_fixed" >> "$cm_dir/reapplied.log"
fi
}
"""
# Resolves the depot at launch time, so the tool can be registered before the depot exists and
# survives the client updating or replacing it. Proton takes its base directory from
# dirname(sys.argv[0]) without resolving it, so it is run from the depot itself.
# Cores for a game the client launches, kept separate from the client's own. One mask suits the
# client's menus and another suits a game, and they are wanted at the same time - so the game's is
# applied here, in the launch path every game goes through, rather than to the session as a whole.
BL_GAME_CPUS_SETUP = """
bl_game_cpus() {
[ -n "${BL_GAME_CPUS:-}" ] || return 0
command -v taskset >/dev/null 2>&1 || return 0
# exec'd through taskset rather than set afterwards: a mask applied to a pid races the process
# into spawning its own threads, and this way every thread inherits it from the first instruction.
BL_TASKSET="taskset -c $BL_GAME_CPUS"
echo "bannerlator-proton: game cores $BL_GAME_CPUS" >&2
}
"""
# Quake III engine games (id Tech 3) set their own resolution and switch the display to it. Under
# gamescope with the ARM64 Proton that switch is refused ("calling CDS: failed, bad mode") and the
# game never gets OpenGL; left at its own size in a window, it draws small in a corner. So these
# titles are told to stay windowed at the session's own size - unless the user's launch options
# already say something about the video mode, which are left as they are.
# The session puts the client's own libraries (steamrtarm64, its panorama/) ahead of everything for
# the client, and a game inherits that: Wine then resolved gnutls, SDL, FFmpeg and the rest against
# the client's private copies instead of the rootfs it was built for. Inside Steam's runtime
# container that path never reaches Proton either. (WinNative's winnative-proton-launch.)
BL_LIBPATH_SETUP = """
bl_libpath() {
# A Proton built to prefer PipeWire (CachyOS's among the adopted ones) hung its audio thread
# here, where only PulseAudio answers. (WinNative, winnative-proton-launch.)
export PROTON_USE_PIPEWIRE=${PROTON_USE_PIPEWIRE:-0}
[ -n "${LD_LIBRARY_PATH:-}" ] || return 0
local kept= dir old_ifs=$IFS
IFS=:
for dir in $LD_LIBRARY_PATH; do
case $dir in
"" | */steamrtarm64 | */steamrtarm64/*) ;;
*) kept=${kept:+$kept:}$dir ;;
esac
done
IFS=$old_ifs
if [ -n "$kept" ]; then export LD_LIBRARY_PATH="$kept"; else unset LD_LIBRARY_PATH; fi
}
"""
BL_IDTECH3_SETUP = """
bl_idtech3() {
BL_EXTRA_ARGS=()
case "${SteamAppId:-${STEAM_COMPAT_APP_ID:-}}" in
# Quake III Arena, Quake III: Team Arena, Return to Castle Wolfenstein, Jedi Academy.
2200|2350|9010|6020) ;;
*) return 0 ;;
esac
case " $* " in
*r_fullscreen*|*r_mode*|*r_customwidth*|*r_customheight*)
echo "bannerlator-proton: Quake-engine title; its launch options set the video mode, left as they are" >&2
return 0 ;;
esac
w=${BL_WIDTH:-1280}
h=${BL_HEIGHT:-720}
BL_EXTRA_ARGS=(+set r_fullscreen 0 +set r_mode -1 +set r_customwidth "$w" +set r_customheight "$h")
echo "bannerlator-proton: Quake-engine title: windowed at ${w}x${h}" >&2
}
"""
BL_DIRECTAUDIO_SETUP = """
bl_directaudio() {
base=$1
[ -n "${BL_DIRECTAUDIO:-}" ] || return 0
[ -d "$BL_DIRECTAUDIO/lib/wine" ] || return 0
version=$("$base/files/bin-arm64/wine" --version 2>/dev/null)
major=${version#wine-}
major=${major%%.*}
if [ "$major" != "11" ]; then
echo "bannerlator-proton: DirectAudio left off - it is built for Wine 11 and this Proton is '${version:-unknown}'. Pairing them would be silent rather than broken, so it is not worth guessing." >&2
return 0
fi
# Wine appends the per-architecture subdirectory to each entry, so the directory holding
# aarch64-unix/ and aarch64-windows/ is what goes here.
WINEDLLPATH="$BL_DIRECTAUDIO/lib/wine${WINEDLLPATH:+:$WINEDLLPATH}"
export WINEDLLPATH
# mmdevapi only tries the drivers named in the registry; its built-in list is pulse, alsa, oss and
# coreaudio and knows nothing about this one. The key is appended to the prefix's user.reg rather
# than written with `wine reg`, which would start a wineserver here just to set one value.
reg="${STEAM_COMPAT_DATA_PATH:-}/pfx/user.reg"
if [ -f "$reg" ]; then
if ! grep -q '"Audio"="directaudio"' "$reg" 2>/dev/null; then
printf '\n[Software\\\\Wine\\\\Drivers] %s\n"Audio"="directaudio"\n' "$(date +%s)" >> "$reg"
echo "bannerlator-proton: DirectAudio selected in the prefix" >&2
fi
else
# A prefix that does not exist yet is created by Proton moments from now, after this runs, so
# the key lands on the NEXT launch of that game. Said out loud because silent audio on a first
# launch would otherwise look like a fault.
echo "bannerlator-proton: no prefix yet - DirectAudio takes effect the next time this game starts" >&2
fi
echo "bannerlator-proton: DirectAudio ready (Wine $major)" >&2
}
"""
# Resolves the depot at launch time, so the tool can be registered before the depot exists and
# survives the client updating or replacing it. Proton takes its base directory from
# dirname(sys.argv[0]) without resolving it, so it is run from the depot itself.
LAUNCHER_SH = """#!/bin/bash
# Bannerlator Proton (ARM64): runs the ARM64 Proton the Steam client downloaded, without the
# Steam Linux Runtime container its own manifest asks for (no user namespaces on Android).
verb=$1; shift
steam_root=${STEAM_COMPAT_CLIENT_INSTALL_PATH:-${XDG_DATA_HOME:-$HOME/.local/share}/Steam}
# Steam's overlay is preloaded into every game Valve launches, and used to be dropped here: it was
# landing in front of open() and read() on /dev/input, letting Wine open the session's pad and
# answering every read of it with nothing - a controller that never reports, and Wine's HID thread
# spinning a core on it.
#
# Removing it was the wrong cure for an ordering problem. glibc processes LD_PRELOAD BEFORE
# /etc/ld.so.preload, and the interposer lives in the latter, so the overlay was simply ahead of it.
# Naming the interposer first in LD_PRELOAD puts it back in front and the overlay loads as Valve
# intends. That matters beyond the overlay itself: every deviation from the launch Valve ships is a
# difference this runtime would otherwise have to justify.
fake=/usr/local/lib/libfakeinput.so
if [ -f "$fake" ]; then
case ":${LD_PRELOAD:-}:" in
*":$fake:"*) ;;
*) LD_PRELOAD="$fake${LD_PRELOAD:+:$LD_PRELOAD}"; export LD_PRELOAD ;;
esac
fi
# The newest of the ARM64 depots the client has, by the build number that opens each one's version
# file: taking the first one found ran a stale Experimental over a newer 11.0. (WinNative.)
depot=
best=0
for name in %s; do
for library in "$steam_root" /mnt/droiddeck-sd; do
candidate="$library/steamapps/common/$name"
[ -d "$candidate/files/bin-arm64" ] && [ -x "$candidate/proton" ] || continue
build=0
read -r build _ 2>/dev/null < "$candidate/version"
case $build in "" | *[!0-9]*) build=0 ;; esac
if [ -z "$depot" ] || [ "$build" -gt "$best" ]; then
depot=$candidate
best=$build
fi
done
done
if [ -n "$depot" ]; then
bl_components "$depot"
bl_directaudio "$depot"
bl_game_cpus
bl_idtech3 "$@"
bl_libpath
exec ${BL_TASKSET:-} /usr/local/bin/bannerlator-game-env "$depot/proton" "$verb" "$@" "${BL_EXTRA_ARGS[@]}"
fi
echo "bannerlator-proton: no ARM64 Proton depot yet - the client downloads one as this tool's dependency; retry the launch once it has" >&2
exit 1
""" % " ".join('"%s"' % n for n in SOURCES)
# Spliced in after the formatting above, never through it: the function body contains ${major%%.*}
# and a % format would eat that into a single %, leaving a shell expansion that silently does
# something else.
LAUNCHER_SH = LAUNCHER_SH.replace(
"verb=$1; shift", "verb=$1; shift\n" + BL_DIRECTAUDIO_SETUP + BL_GAME_CPUS_SETUP + BL_IDTECH3_SETUP
+ BL_COMPONENTS_SETUP + BL_LIBPATH_SETUP, 1)
# A third-party Proton dropped into compatibilitytools.d - GE-Proton, proton-cachyos, anything of
# that shape - is adopted rather than used as shipped, for the same two reasons Valve's ARM64 depot
# is: its manifest declares require_tool_appid, and the client's overlay library has to come out of
# LD_PRELOAD. Its own entry point is left untouched and called through this wrapper.
EXTRA_WRAPPER = "bannerlator-proton-wrap"
EXTRA_MANIFEST_VDF = """"manifest"
{
"version" "2"
"commandline" "/%s %%verb%%"
"use_sessions" "1"
"compatmanager_layer_name" "proton"
}
""" % EXTRA_WRAPPER
# %s is the tool's own entry point, read out of the manifest it shipped with.
EXTRA_WRAPPER_SH = """#!/bin/bash
# Bannerlator: runs a Proton installed into compatibilitytools.d without the Steam Linux Runtime
# container its own manifest asks for - pressure-vessel needs unprivileged user namespaces, which
# an Android app does not get, and the launch dies before Proton is reached.
verb=$1; shift
here=$(dirname "$0")
# See bannerlator-proton: the interposer goes first in LD_PRELOAD rather than the overlay being
# removed, because glibc reads LD_PRELOAD before /etc/ld.so.preload and the overlay was simply
# ahead of it on /dev/input.
fake=/usr/local/lib/libfakeinput.so
if [ -f "$fake" ]; then
case ":${LD_PRELOAD:-}:" in
*":$fake:"*) ;;
*) LD_PRELOAD="$fake${LD_PRELOAD:+:$LD_PRELOAD}"; export LD_PRELOAD ;;
esac
fi
bl_components "$here"
bl_directaudio "$here"
bl_game_cpus
bl_idtech3 "$@"
bl_libpath
# Proton takes its base directory from dirname(sys.argv[0]) without resolving it, so it has to be
# started from its own tree.
exec ${BL_TASKSET:-} /usr/local/bin/bannerlator-game-env "$here/%s" "$verb" "$@" "${BL_EXTRA_ARGS[@]}"
"""
EXTRA_COMPAT_TOOL_VDF = """"compatibilitytools"
{
"compat_tools"
{
"%s"
{
"install_path" "."
"display_name" "%s (Bannerlator)"
"from_oslist" "windows"
"to_oslist" "linux"
}
}
}
"""
TOKEN = re.compile(r'\s*(?:"((?:[^"\\]|\\.)*)"|([{}])|//[^\n]*)')
def tokenize(text):
tokens = []
pos = 0
while pos < len(text):
m = TOKEN.match(text, pos)
if not m or m.end() == pos:
break
pos = m.end()
if m.group(2):
tokens.append(m.group(2))
elif m.group(1) is not None:
tokens.append('"' + m.group(1) + '"')
return tokens
def find_block(tokens, path):
"""Index of the '{' opening the block at path, or None."""
depth, want, i = 0, 0, 0
while i < len(tokens):
t = tokens[i]
if t == "{":
depth += 1
elif t == "}":
depth -= 1
if depth < want:
return None
elif depth == want and i + 1 < len(tokens) and tokens[i + 1] == "{":
if t.lower() == '"%s"' % path[want].lower():
if want == len(path) - 1:
return i + 1
want += 1
i += 1
return None
def block_end(tokens, start):
depth = 0
for i in range(start, len(tokens)):
if tokens[i] == "{":
depth += 1
elif tokens[i] == "}":
depth -= 1
if depth == 0:
return i
return len(tokens)
def serialize(tokens):
out, depth, i = [], 0, 0
while i < len(tokens):
t = tokens[i]
if t == "}":
depth -= 1
out.append("\t" * depth + "}")
i += 1
elif i + 1 < len(tokens) and tokens[i + 1] == "{":
out.append("\t" * depth + t)
out.append("\t" * depth + "{")
depth += 1
i += 2
else:
out.append("\t" * depth + t + "\t\t" + tokens[i + 1])
i += 2
return "\n".join(out) + "\n"
def write_changed(path, text, mode=None):
"""Writes `text` to `path` only when it differs, through a rename so a reader never sees half a
file. The session runs this script every 15 s for as long as the client does; rewriting files
that have not changed cost a python start's worth of writes each time, and config.vdf is the
client's own, which it may be writing at that moment. Returns whether anything was written."""
try:
with open(path, encoding="utf-8", errors="replace") as f:
same = f.read() == text
except OSError:
same = False
if not same:
tmp = path + ".bannerlator"
with open(tmp, "w", encoding="utf-8") as f:
f.write(text)
if mode is not None:
os.chmod(tmp, mode)
os.replace(tmp, path)
elif mode is not None and (os.stat(path).st_mode & 0o7777) != mode:
os.chmod(path, mode)
return not same
def first_key(path, section):
"""The single key naming a block inside `section`, or None. Used to keep a third-party tool's
own internal name, so a mapping the user already made against it keeps working."""
try:
with open(path, encoding="utf-8", errors="replace") as f:
tokens = tokenize(f.read())
except OSError:
return None
block = find_block(tokens, section)
if block is None:
return None
for i in range(block + 1, block_end(tokens, block)):
if tokens[i] == "}":
break
if i + 1 < len(tokens) and tokens[i + 1] == "{":
return tokens[i].strip('"')
return None
def manifest_entry_point(path):
"""The script a toolmanifest's commandline runs, relative to the tool directory."""
try:
with open(path, encoding="utf-8", errors="replace") as f:
tokens = tokenize(f.read())
except OSError:
return None
for i in range(len(tokens) - 1):
if tokens[i].lower() == '"commandline"':
command = tokens[i + 1].strip('"').split()
if command:
return command[0].lstrip("/")
return None
def adopt_extras(tools):
"""Makes every other Proton in compatibilitytools.d launchable here, and returns the tool names
to leave alone in CompatToolMapping - the whole point of installing one is to pick it per game,
so register_default must not repoint it back at ours."""
adopted = {}
try:
entries = sorted(os.listdir(tools))
except OSError:
return adopted
for entry in entries:
if entry == TOOL or entry.startswith("."):
continue
tool_dir = os.path.join(tools, entry)
manifest = os.path.join(tool_dir, "toolmanifest.vdf")
if not os.path.isdir(tool_dir) or not os.path.isfile(manifest):
continue
# The shipped manifest is kept, once, and stays the source of the entry point: re-running
# must not read back our own rewritten commandline and wrap the wrapper.
original = manifest + ".bannerlator-orig"
if not os.path.isfile(original):
try:
shutil.copy2(manifest, original)
except OSError:
continue
entry_point = manifest_entry_point(original)
if not entry_point or entry_point == EXTRA_WRAPPER:
entry_point = "proton"
if not os.path.exists(os.path.join(tool_dir, entry_point)):
print("bannerlator-steam-compat: %s has no %s; left alone" % (entry, entry_point))
continue
compat_vdf = os.path.join(tool_dir, "compatibilitytool.vdf")
name = first_key(compat_vdf, ["compatibilitytools", "compat_tools"]) or entry
wrapper = os.path.join(tool_dir, EXTRA_WRAPPER)
try:
# Formatted first, spliced second: the function carries a printf %s and a
# ${major%%.*} of its own, and putting it through the format would eat both.
body = (EXTRA_WRAPPER_SH % entry_point).replace(
'here=$(dirname "$0")',
'here=$(dirname "$0")\n' + BL_DIRECTAUDIO_SETUP + BL_GAME_CPUS_SETUP + BL_IDTECH3_SETUP
+ BL_COMPONENTS_SETUP + BL_LIBPATH_SETUP, 1)
write_changed(wrapper, body, 0o755)
write_changed(manifest, EXTRA_MANIFEST_VDF)
write_changed(compat_vdf, EXTRA_COMPAT_TOOL_VDF % (name, entry))
except OSError as e:
print("bannerlator-steam-compat: could not adopt %s (%s)" % (entry, e))
continue
adopted[name] = entry
return adopted
# Steam's own tools and runtimes live in steamapps beside the games; they are not titles and must
# not be mapped to a compatibility tool.
NOT_GAMES = frozenset((
"228980", # Steamworks Common Redistributables
"1070560", # Steam Linux Runtime 1.0 (scout)
"1391110", # Steam Linux Runtime 2.0 (soldier)
"1493710", # Proton Experimental
"3127680", # FEX
"4183110", # Steam Linux Runtime 4.0
"4185400", # Steam Linux Runtime 4.0 - Arm64
"4427310", # Proton Experimental (ARM64)
"4628740", # Proton 11.0 (ARM64)
))
def installed_apps(steam_root):
"""App ids of the titles the client has installed, tools and runtimes excluded."""
apps = set()
for library in (steam_root, LIBRARY + "-sd"):
steamapps = os.path.join(library, "steamapps")
try:
names = os.listdir(steamapps)
except OSError:
continue
for name in names:
m = re.fullmatch(r"appmanifest_(\d+)\.acf", name)
if m and m.group(1) not in NOT_GAMES:
apps.add(m.group(1))
return sorted(apps, key=int)
def register_default(config_path, apps=(), name=None, protect=()):
"""Writes the mapping. `name` defaults to our own tool; the first-run seed passes Valve's."""
tool = name or TOOL
# Tool names the user may deliberately have chosen (see adopt_extras); never repointed.
keep = {n.lower() for n in protect}
with open(config_path, encoding="utf-8", errors="replace") as f:
original = f.read()
tokens = tokenize(original)
steam = find_block(tokens, ["InstallConfigStore", "Software", "Valve", "Steam"])
if steam is None:
return
mapping = find_block(tokens[steam + 1:], ["CompatToolMapping"])
entry = ['"0"', "{", '"name"', '"%s"' % tool, '"config"', '""', '"priority"', '"75"', "}"]
if mapping is None:
end = block_end(tokens, steam)
tokens[end:end] = ['"CompatToolMapping"', "{"] + entry + ["}"]
else:
mapping += steam + 1
# Repoint every per-app entry the client aimed at a Proton it cannot actually start.
for i in range(mapping, block_end(tokens, mapping)):
named = tokens[i + 1].strip('"').lower()
if (tokens[i] == '"name"' and named.startswith("proton")
and named != tool.lower() and named not in keep):
tokens[i + 1] = '"%s"' % tool
if find_block(tokens[mapping + 1:], ["0"]) is None:
tokens[mapping + 1:mapping + 1] = entry
# The "0" default only decides how a title is installed, so it settles new downloads and
# leaves anything already installed as a Linux build running as one. An entry per installed
# app is what the client's own Compatibility checkbox writes, and is what makes it switch.
for app in apps:
if find_block(tokens[mapping + 1:], [app]) is not None:
continue
tokens[mapping + 1:mapping + 1] = [
'"%s"' % app, "{", '"name"', '"%s"' % tool, '"config"', '""', '"priority"', '"250"', "}"]
# Nothing to change is the usual case on the session's beat: the client's own file is left alone.
text = serialize(tokens)
if text != serialize(tokenize(original)):
write_changed(config_path, text)
def depot_build(depot):
"""The build number a depot's version file starts with; 0 when there is none."""
try:
with open(os.path.join(depot, "version"), encoding="utf-8", errors="replace") as f:
first = f.read().split(None, 1)
except OSError:
return 0
return int(first[0]) if first and first[0].isdigit() else 0
def find_source(steam_root):
"""The newest ARM64 Proton depot the client has downloaded (the one the launcher runs), or None."""
# bin-arm64 is what makes it the native build rather than the x86_64 one.
found = [os.path.join(root, "steamapps", "common", name)
for root in (steam_root, "/mnt/droiddeck-sd") for name in SOURCES
if os.path.isdir(os.path.join(root, "steamapps", "common", name, "files", "bin-arm64"))]
return max(found, key=depot_build) if found else None
def build_tool(tool_dir):
"""Our own manifests and the launcher. The depot is found at launch time, not here."""
if os.path.islink(tool_dir):
os.unlink(tool_dir)
elif os.path.isdir(tool_dir) and set(os.listdir(tool_dir)) - set(OWN_FILES):
# Earlier versions mirrored the depot in here as symlinks; start clean. A directory holding
# only our own files is kept, and each file rewritten only when it changed.
shutil.rmtree(tool_dir)
os.makedirs(tool_dir, exist_ok=True)
write_changed(os.path.join(tool_dir, "toolmanifest.vdf"), TOOL_MANIFEST_VDF)
write_changed(os.path.join(tool_dir, "compatibilitytool.vdf"), COMPAT_TOOL_VDF % TOOL)
write_changed(os.path.join(tool_dir, LAUNCHER), LAUNCHER_SH, 0o755)
def main():
steam_root = sys.argv[1]
tools = os.path.join(steam_root, "compatibilitytools.d")
os.makedirs(tools, exist_ok=True)
build_tool(os.path.join(tools, TOOL))
extras = adopt_extras(tools)
for name, directory in sorted(extras.items()):
print("bannerlator-steam-compat: adopted %s as %s" % (directory, name))
depot = find_source(steam_root)
print("bannerlator-steam-compat: registered %s (%s)" % (
TOOL, "depot %s present" % os.path.basename(depot) if depot else
"no ARM64 Proton depot yet; the session asks the client to fetch it"))
config = os.path.join(steam_root, "config", "config.vdf")
if os.path.isfile(config):
apps = installed_apps(steam_root)
register_default(config, apps, protect=extras)
print("bannerlator-steam-compat: default and %d installed title(s) set to %s" % (len(apps), TOOL))
if __name__ == "__main__":
main()
+122
View File
@@ -0,0 +1,122 @@
#!/bin/bash
# An AppImage imported from the front end (AppImageManager), extracted once at import into
# /opt/appimages/user/<id>/app
# and started from there: proot has no FUSE to mount the image, and extracting it at every
# start (APPIMAGE_EXTRACT_AND_RUN) would cost the whole image in /tmp each time.
# droiddeck-appimage-run /opt/appimages/user/<id> [args...]
set -u
dir=${1:?usage: droiddeck-appimage-run <imported AppImage directory> [args...]}
shift
export APPDIR=$dir/app
# APPIMAGE is left unset on purpose: with it an app believes it runs from an image file and offers
# to add itself to the menu (DuckStation) or update the file in place - DroidDeck already does the
# one, and there is no image file for the other.
unset APPIMAGE
export ARGV0=$(cat "$dir/name" 2>/dev/null || basename "$dir")
if [ ! -x "$APPDIR/AppRun" ]; then
echo "droiddeck-appimage-run: $APPDIR/AppRun is missing; import the AppImage again" >&2
exit 127
fi
# Browsers sandbox their own children with namespaces and a setuid helper, neither of which proot
# can give them (the Flatpak sandboxes get the same; see BwrapSpawner): Firefox and its forks (Zen,
# LibreWolf) by these switches, Electron and Chromium builds by --no-sandbox. Chromium on Wayland
# also asks the render node for a DRM version KGSL cannot give, so with an X server up it stays on
# X11, which it picks from the session type.
export MOZ_DISABLE_CONTENT_SANDBOX=1 MOZ_DISABLE_GMP_SANDBOX=1 MOZ_DISABLE_RDD_SANDBOX=1
export MOZ_DISABLE_SOCKET_PROCESS_SANDBOX=1 MOZ_DISABLE_UTILITY_SANDBOX=1
# sharun images start the app through their own ld.so, patched not to read /etc/ld.so.preload, and
# drop LD_PRELOAD unless allowed: without this the app never gets the session's pads.
if [ -s /etc/ld.so.preload ]; then
preload=$(tr -s '\n' ':' < /etc/ld.so.preload)
export LD_PRELOAD="${preload%:}${LD_PRELOAD:+:$LD_PRELOAD}" SHARUN_ALLOW_LD_PRELOAD=1
fi
flags=()
if [ -e "$APPDIR/chrome-sandbox" ] || compgen -G "$APPDIR/*/chrome-sandbox" >/dev/null; then
flags+=(--no-sandbox)
if [ -n "${DISPLAY:-}" ]; then
export XDG_SESSION_TYPE=x11
unset WAYLAND_DISPLAY
flags+=(--ozone-platform=x11)
fi
fi
qt=$(cat "$dir/qt" 2>/dev/null)
case $qt in wayland|xcb) ;; *) qt=auto ;; esac
wayland=${GAMESCOPE_WAYLAND_DISPLAY:-${WAYLAND_DISPLAY:-}}
if [ "$qt" = auto ] && [ -n "$wayland" ]; then
bundled=$(find "$APPDIR" -maxdepth 6 \( -name 'libQt6Quick.so.6*' -o -name libvulkan.so.1 -o -name 'libqwayland*.so' \) -printf '%f\n' 2>/dev/null)
if grep -q '^libQt6Quick\.so\.6' <<<"$bundled"; then
if grep -q '^libqwayland' <<<"$bundled" \
&& { grep -qx libvulkan.so.1 <<<"$bundled" || grep -qs QVulkanInstance "$APPDIR"/usr/bin/* "$APPDIR"/bin/*; }; then
qt=wayland
else
echo "== appimage: Qt Quick without Vulkan or Qt's Wayland plugin stays on X11 ($(tr '\n' ' ' <<<"$bundled"))"
fi
fi
fi
if [ "$qt" = wayland ] && [ -n "$wayland" ]; then
export QT_QPA_PLATFORM='wayland;xcb' WAYLAND_DISPLAY=$wayland SDL_VIDEODRIVER=wayland XDG_SESSION_TYPE=wayland
[ "$wayland" = "${GAMESCOPE_WAYLAND_DISPLAY:-}" ] && export QT_WAYLAND_DISABLE_WINDOWDECORATION=1 DISABLE_GAMESCOPE_WSI=1
elif [ "$qt" = xcb ] && [ -n "${DISPLAY:-}" ]; then
export QT_QPA_PLATFORM=xcb
fi
fex=$(cat "$dir/fex" 2>/dev/null)
case $fex in on|off) ;; *) fex=auto ;; esac
run=("$APPDIR/AppRun")
if [ -x /usr/local/bin/droiddeck-fex ]; then
case $fex:$(cat "$dir/arch" 2>/dev/null) in
auto:x86_64|auto:i386) run=(/usr/local/bin/droiddeck-fex run --mode on -- "$APPDIR/AppRun") ;;
auto:arm64) ;;
*) run=(/usr/local/bin/droiddeck-fex run --mode "$fex" --for "$APPDIR" -- "$APPDIR/AppRun") ;;
esac
fi
echo "== appimage $ARGV0 ($APPDIR) ${flags[*]} FEX $fex Qt $qt${QT_QPA_PLATFORM:+ ($QT_QPA_PLATFORM)}"
cd "$APPDIR" || exit 1
started=$(mktemp "${TMPDIR:-/tmp}/droiddeck-appimage.XXXXXX")
# Apps expect a session bus; Electron's main process traps (SIGTRAP) on an address it cannot
# parse. A run mode has none, so one is started for the app, as for Flatpak apps.
case ${DBUS_SESSION_BUS_ADDRESS:-} in
unix:*) "${run[@]}" "${flags[@]}" "$@" ;;
*) unset DBUS_SESSION_BUS_ADDRESS
if command -v dbus-run-session >/dev/null 2>&1; then
dbus-run-session -- "${run[@]}" "${flags[@]}" "$@"
else
"${run[@]}" "${flags[@]}" "$@"
fi ;;
esac
rc=$?
if [ "$rc" -gt 128 ]; then
echo "== appimage $ARGV0: ended by signal $((rc - 128)) ($(kill -l $((rc - 128)) 2>/dev/null))"
else
echo "== appimage $ARGV0: exited with status $rc"
fi
if [ "$rc" -ne 0 ] && [ -n "$started" ]; then
find "${HOME:-/root}/.local/share" "${HOME:-/root}/.config" "${HOME:-/root}/.cache" -maxdepth 4 -type f \
\( -iname '*.log' -o -iname '*log*.txt' \) -newer "$started" -size -64M 2>/dev/null | head -n 3 |
while IFS= read -r log; do
echo "== appimage $ARGV0: end of $log"
tail -n 40 "$log" | LC_ALL=C tr -cd '\11\12\40-\176' | cut -c1-400
done
fi
rm -f "$started"
# The program a session was started for may hand over to another process and exit: Electron apps
# relaunch themselves (Obsidian on its first start), launchers fork their real program. The
# session ends with this script, and takes the rest with it - so it waits while anything running
# from the image is still up.
image_running() {
local p exe arg cwd
for p in /proc/[0-9]*; do
exe=$(readlink "$p/exe" 2>/dev/null) || continue
case $exe in "$APPDIR"/*) return 0 ;; esac
case $exe in
*/FEX|*/FEXInterpreter)
arg=$(tr '\0' '\n' < "$p/cmdline" 2>/dev/null | sed -n 2p)
cwd=$(readlink "$p/cwd" 2>/dev/null)
case $arg in "$APPDIR"/*) return 0 ;; esac
case $cwd in "$APPDIR"|"$APPDIR"/*) return 0 ;; esac
;;
esac
done
return 1
}
while image_running; do sleep 2; done
exit $rc
@@ -4,7 +4,7 @@
# the app - so an entry of ours exists only for a game the client wrote none for. Plus two links in
# the home folder to the game files. Run at every desktop start; stale entries go.
#
# bannerlator-desktop-games [steam root]
# droiddeck-desktop-games [steam root]
import os
import re
import sys
@@ -90,7 +90,7 @@ def write_entries():
safe = name.replace("\n", " ").strip() or str(appid)
body = (
"[Desktop Entry]\nType=Application\nName=%s\nComment=Steam game %d - opens in Steam under gamescope\n"
"Exec=/usr/local/bin/bannerlator-steam-launch desktop %d\nIcon=%s\nTerminal=false\n"
"Exec=/usr/local/bin/droiddeck-steam-launch desktop %d\nIcon=%s\nTerminal=false\n"
"Categories=Game;\nX-DroidDeck-AppId=%d\n" % (safe, appid, appid, icon_for(appid), appid)
)
path = os.path.join(APPS_DIR, "%s%d.desktop" % (PREFIX, appid))
@@ -123,7 +123,7 @@ def link_libraries():
def main():
wanted = write_entries()
link_libraries()
print("bannerlator-desktop-games: %d entr%s of ours, %d the client's" % (
print("droiddeck-desktop-games: %d entr%s of ours, %d the client's" % (
len(wanted), "y" if len(wanted) == 1 else "ies", len(client_entries())))
@@ -364,11 +364,21 @@ def load_packs(root):
return packs
def line_epoch(line):
fields = (line or "").split()
try:
return int(fields[0]) if fields else 0
except ValueError:
return 0
def match_kind(pack, measured):
if pack.get("revoked") or pack.get("ignored") or measured["builtin"]:
return None
stock = pack["stock"]
if stock[NTDLL] == measured["ntdll"] and stock[WINESERVER] == measured["wineserver"]:
if pack.get("version") != measured["token"] and line_epoch(pack.get("version_line")) > line_epoch(measured["version_line"]) > 0:
return None
return "exact"
if (pack.get("source_match") is True and measured["token"] and pack.get("version") == measured["token"]
and pack.get("exports") == measured["exports"]):
@@ -383,7 +393,7 @@ def find_pack(packs, measured):
kind = match_kind(pack, measured)
if kind is None:
continue
key = (kind == "exact", pack["rev"], pack["id"])
key = (kind == "exact", bool(measured["token"]) and pack.get("version") == measured["token"], pack["rev"], pack["id"])
if best_key is None or key > best_key:
best, best_key = pack, key
return best
@@ -789,6 +799,18 @@ def keep_live(root, command, env, tool):
return None
if not os.path.isfile(kept_command) or not server_running(prefix):
return None
kept_pack = row["env"].get("BL_SYNC_PACK")
if isinstance(kept_pack, str) and kept_pack and command[1:2] == ["waitforexitandrun"]:
try:
cache = HashCache(root)
try:
pack = find_pack(load_packs(root), measure(tool, cache))
finally:
cache.save()
except (OSError, ValueError):
pack = {"id": kept_pack}
if pack is None or pack["id"] != kept_pack:
return None
command, env = list(command), dict(env)
command[0] = kept_command
for name in LIVE_KEYS:
+659
View File
@@ -0,0 +1,659 @@
#!/usr/bin/python3
import contextlib
import fcntl
import filecmp
import gzip
import json
import os
import re
import runpy
import shutil
import struct
import subprocess
import sys
HOME = os.environ.get("HOME") or "/root"
STEAM_ROOT = os.path.join(HOME, ".local", "share", "Steam")
LIBRARIES = ("/mnt/droiddeck", "/mnt/droiddeck-sd")
STATE = os.path.join(HOME, ".local", "share", "droiddeck-fex")
CONFIG_DIR = os.path.join(STATE, "config")
DATA_DIR = os.path.join(STATE, "data")
CACHE_DIR = os.path.join(HOME, ".cache", "droiddeck-fex")
STATUS_FILE = os.path.join(STATE, "status.json")
WANTED = os.path.join(HOME, ".config", "droiddeck", "fex-wanted")
GAME_ENVIRONMENT = os.path.join(HOME, ".config", "droiddeck", "game-environment.json")
BIN_DIR = os.path.dirname(os.path.abspath(__file__))
PRELOADS = "/usr/local/lib/droiddeck-fex"
URUNTIME = "/usr/local/lib/droiddeck/uruntime"
HOST_PRELOAD = "/etc/ld.so.preload"
FEX_APPID = "3127680"
SNIPER_APPID = "1628350"
INSTALL_DIRS = {FEX_APPID: "FEX-Emu", SNIPER_APPID: "SteamLinuxRuntime_sniper"}
MARKER = ".droiddeck-fex"
LAYOUT = 1
GUEST_LIBDIRS = (("x86_64", "lib/x86_64-linux-gnu"), ("i386", "lib/i386-linux-gnu"))
PRELOAD_NAMES = ("libblsession.so", "libfakeinput.so")
FAULT_REPORT = "libfaultreport.so"
FAULT_REPORT_GUEST = "/usr/lib/x86_64-linux-gnu/droiddeck/" + FAULT_REPORT
THUNK_AUDIT = "libthunkaudit.so"
THUNK_AUDIT_GUEST = "/usr/lib/x86_64-linux-gnu/droiddeck/" + THUNK_AUDIT
VULKAN_PIN = "libvulkan-thunk.so"
VULKAN_THUNK = "libvulkan-guest.so"
RUNTIME_LIBRARIES = (("libc.so.6", b"GLIBC_"), ("libstdc++.so.6", b"GLIBCXX_"))
VERSION_NAME = re.compile(rb"GLIBC(?:XX)?_[0-9][0-9.]*[0-9]")
BUNDLE_DEPTH = 3
THUNKS = {"GL": 1, "Vulkan": 1}
X86 = {3: "i386", 62: "x86_64"}
ARM = {40: "arm", 183: "arm64"}
SCAN_DEPTH = 4
SCAN_LIMIT = 4000
MODES = ("auto", "on", "off")
STEAM_VERBS = ("run", "waitforexitandrun")
ESCAPE = re.compile(rb"\\([0-7]{3})")
def say(message):
print("droiddeck-fex: %s" % message, file=sys.stderr, flush=True)
def read_text(path):
try:
with open(path, encoding="utf-8", errors="replace") as f:
return f.read()
except OSError:
return None
def write_if_changed(path, data, mode=0o644):
if isinstance(data, str):
data = data.encode("utf-8")
try:
with open(path, "rb") as f:
if f.read() == data:
return False
except OSError:
pass
os.makedirs(os.path.dirname(path), exist_ok=True)
tmp = "%s.%d.tmp" % (path, os.getpid())
with open(tmp, "wb") as f:
f.write(data)
os.chmod(tmp, mode)
os.replace(tmp, path)
return True
def copy_if_changed(source, target):
if os.path.isfile(target) and filecmp.cmp(source, target, shallow=False):
return False
os.makedirs(os.path.dirname(target), exist_ok=True)
tmp = "%s.%d.tmp" % (target, os.getpid())
shutil.copyfile(source, tmp)
os.chmod(tmp, 0o755)
os.replace(tmp, target)
return True
@contextlib.contextmanager
def locked():
os.makedirs(STATE, exist_ok=True)
with open(os.path.join(STATE, "lock"), "w") as handle:
fcntl.flock(handle, fcntl.LOCK_EX)
try:
yield
finally:
fcntl.flock(handle, fcntl.LOCK_UN)
def libraries():
roots = [STEAM_ROOT]
text = read_text(os.path.join(STEAM_ROOT, "steamapps", "libraryfolders.vdf")) or ""
roots += [path.replace("\\\\", "\\") for path in re.findall(r'"path"\s+"((?:[^"\\]|\\.)*)"', text)]
roots += LIBRARIES
seen, found = set(), []
for root in roots:
real = os.path.realpath(root)
if real not in seen and os.path.isdir(os.path.join(root, "steamapps")):
seen.add(real)
found.append(root)
return found
def app_dir(appid):
for library in libraries():
steamapps = os.path.join(library, "steamapps")
match = re.search(r'"installdir"\s+"([^"]+)"', read_text(os.path.join(steamapps, "appmanifest_%s.acf" % appid)) or "")
for name in filter(None, (match.group(1) if match else None, INSTALL_DIRS.get(appid))):
path = os.path.join(steamapps, "common", name)
if os.path.isdir(path):
return path
return None
def first_dir(prefix, candidates):
for candidate in candidates:
path = os.path.join(prefix, candidate)
if os.path.isdir(path):
return path
return None
def fex_install():
prefixes = []
tool = app_dir(FEX_APPID)
if tool:
prefixes += [(os.path.join(tool, sub), True) for sub in ("usr", "", "files/usr", "files")]
prefixes.append(("/usr", False))
for prefix, portable in prefixes:
binary = next((os.path.join(prefix, "bin", name) for name in ("FEX", "FEXInterpreter")
if os.access(os.path.join(prefix, "bin", name), os.X_OK)), None)
if not binary:
continue
server = os.path.join(prefix, "bin", "FEXServer")
thunks_db = os.path.join(prefix, "share", "fex-emu", "ThunksDB.json")
return {
"bin": binary,
"server": server if os.access(server, os.X_OK) else shutil.which("FEXServer"),
"portable": portable,
"host_thunks": first_dir(prefix, ("lib/aarch64-linux-gnu/fex-emu/HostThunks", "lib/fex-emu/HostThunks", "lib64/fex-emu/HostThunks")),
"guest_thunks": first_dir(prefix, ("share/fex-emu/GuestThunks",)),
"thunks_db": thunks_db if os.path.isfile(thunks_db) else None,
}
return None
def platform_version(path):
return tuple(int(part) for part in re.findall(r"\d+", os.path.basename(path)))
def runtime_platform():
runtime = app_dir(SNIPER_APPID)
if not runtime:
return None
try:
entries = os.listdir(runtime)
except OSError:
return None
platforms = [os.path.join(runtime, entry) for entry in entries if entry.startswith("sniper_platform_")]
platforms = [path for path in platforms if os.path.isdir(os.path.join(path, "files"))
and os.path.isfile(os.path.join(path, "usr-mtree.txt.gz"))]
return max(platforms, key=platform_version, default=None)
def unescape(raw):
return ESCAPE.sub(lambda match: bytes([int(match.group(1), 8)]), raw)
def in_root(root, link, target):
if not target.startswith(b"/"):
return target
relative = target.lstrip(b"/")
if relative == b"usr" or relative.startswith(b"usr/"):
relative = relative[4:]
return os.path.relpath(os.path.join(root, relative), os.path.dirname(link))
def materialize(platform):
mtree = os.path.join(platform, "usr-mtree.txt.gz")
info = os.stat(mtree)
stamp = "%d %d %d\n" % (LAYOUT, info.st_size, info.st_mtime_ns)
root = os.fsencode(os.path.join(platform, "files"))
marker = os.path.join(os.fsdecode(root), MARKER)
if read_text(marker) == stamp:
return False
links = []
with gzip.open(mtree, "rb") as source:
for line in source:
parts = line.split()
if not parts or parts[0].startswith(b"#"):
continue
fields = dict(part.split(b"=", 1) for part in parts[1:] if b"=" in part)
relative = os.path.normpath(unescape(parts[0]))
if os.path.isabs(relative) or relative == b".." or relative.startswith(b"../"):
continue
dest = os.path.join(root, relative)
kind = fields.get(b"type")
if kind == b"dir":
os.makedirs(dest, exist_ok=True)
elif kind == b"file" and b"contents" in fields:
blob = os.path.join(root, os.path.normpath(unescape(fields[b"contents"])))
links.append((dest, os.path.relpath(blob, os.path.dirname(dest))))
elif kind == b"link" and b"link" in fields:
links.append((dest, in_root(root, dest, unescape(fields[b"link"]))))
for dest, target in links:
if os.path.islink(dest):
if os.readlink(dest) == target:
continue
os.unlink(dest)
elif os.path.lexists(dest):
continue
os.makedirs(os.path.dirname(dest), exist_ok=True)
os.symlink(target, dest)
usr = os.path.join(root, b"usr")
if not os.path.lexists(usr):
os.symlink(b".", usr)
write_if_changed(marker, stamp)
return True
def host_preloads():
listed = read_text(HOST_PRELOAD) or ""
return [name for name in PRELOAD_NAMES if name in listed]
def install_preloads(root):
shipped = set()
for arch, libdir in GUEST_LIBDIRS:
for name in PRELOAD_NAMES:
source = os.path.join(PRELOADS, arch, name)
if os.path.isfile(source):
copy_if_changed(source, os.path.join(root, libdir, "droiddeck", name))
shipped.add(name)
for name in (FAULT_REPORT, THUNK_AUDIT, VULKAN_PIN):
source = os.path.join(PRELOADS, "x86_64", name)
if os.path.isfile(source):
copy_if_changed(source, os.path.join(root, "lib/x86_64-linux-gnu/droiddeck", name))
wanted = [name for name in host_preloads() if name in shipped]
write_if_changed(os.path.join(root, "etc", "ld.so.preload"), "".join("/usr/$LIB/droiddeck/%s\n" % name for name in wanted))
def write_config(fex, root):
config = {"Config": {"RootFS": root}}
if fex["host_thunks"] and fex["guest_thunks"] and fex["thunks_db"]:
config["Config"]["ThunkHostLibs"] = fex["host_thunks"]
config["Config"]["ThunkGuestLibs"] = fex["guest_thunks"]
config["ThunksDB"] = dict(THUNKS)
write_if_changed(os.path.join(CONFIG_DIR, "ThunksConfig.json"), json.dumps({"ThunksDB": dict(THUNKS)}, indent=2) + "\n")
write_if_changed(os.path.join(CONFIG_DIR, "Config.json"), json.dumps(config, indent=2) + "\n")
def link_steam_sdk():
for sdk, candidates in (("sdk64", ("linux64", "ubuntu12_64")), ("sdk32", ("linux32", "ubuntu12_32"))):
link = os.path.join(HOME, ".steam", sdk)
if os.path.lexists(link):
continue
want = "x86_64" if sdk == "sdk64" else "i386"
for candidate in candidates:
directory = os.path.join(STEAM_ROOT, candidate)
if elf_arch(os.path.join(directory, "steamclient.so")) == want:
os.makedirs(os.path.dirname(link), exist_ok=True)
os.symlink(directory, link)
break
def steam_running():
for pid in os.listdir("/proc"):
if not pid.isdigit():
continue
try:
with open("/proc/%s/cmdline" % pid, "rb") as f:
first = f.read().split(b"\0", 1)[0]
except OSError:
continue
if first.endswith(b"steamrtarm64/steam"):
return True
return False
def request_install():
os.makedirs(os.path.dirname(WANTED), exist_ok=True)
write_if_changed(WANTED, "1\n")
if not steam_running():
return []
try:
compat = runpy.run_path(os.path.join(BIN_DIR, "steam-compatibility"))
return compat["request_fex"](STEAM_ROOT, set())
except Exception as error:
say("could not ask the client to install FEX (%s)" % error)
return []
def describe(fex, platform):
return {
"fex": fex["bin"] if fex else None,
"server": fex["server"] if fex else None,
"thunks": bool(fex and fex["host_thunks"] and fex["guest_thunks"] and fex["thunks_db"]),
"runtime": platform,
"steamclient64": os.path.realpath(os.path.join(HOME, ".steam", "sdk64", "steamclient.so"))
if elf_arch(os.path.join(HOME, ".steam", "sdk64", "steamclient.so")) == "x86_64" else None,
"ready": bool(fex and fex["server"] and platform),
}
def prepare():
with locked():
fex = fex_install()
platform = runtime_platform()
status = describe(fex, platform)
if not status["ready"]:
missing = []
tool = app_dir(FEX_APPID)
if not fex and tool:
say("FEX is installed at %s but holds no FEX or FEXInterpreter in usr/bin, bin or files; it has: %s"
% (tool, ", ".join(sorted(os.listdir(tool))[:20])))
if not fex:
missing.append("FEX (Steam app %s)" % FEX_APPID)
elif not fex["server"]:
missing.append("FEXServer beside %s" % fex["bin"])
if not platform:
missing.append("Steam Linux Runtime 3.0 (sniper, Steam app %s)" % SNIPER_APPID)
requested = request_install()
say("x86 Linux programs need %s; %s" % (" and ".join(missing),
"the Steam client was asked to install it" if requested else "open Steam once and it is installed for you"))
write_if_changed(STATUS_FILE, json.dumps(status, indent=2) + "\n")
return None
root = os.path.join(platform, "files")
if materialize(platform):
say("prepared %s as the x86 runtime" % os.path.basename(platform))
install_preloads(root)
write_config(fex, root)
fex["rootfs"] = root
try:
link_steam_sdk()
except OSError:
pass
status = describe(fex, platform)
write_if_changed(STATUS_FILE, json.dumps(status, indent=2) + "\n")
with contextlib.suppress(OSError):
os.unlink(WANTED)
return fex
def elf_arch(path):
try:
with open(path, "rb") as f:
head = f.read(20)
except OSError:
return None
if len(head) < 20 or head[:4] != b"\x7fELF":
return None
machine = struct.unpack_from("<H" if head[5] == 1 else ">H", head, 18)[0]
return X86.get(machine) or ARM.get(machine) or "other"
def scan(folder):
counts = {"x86": 0, "arm": 0}
seen = 0
base = folder.rstrip("/").count("/")
for directory, dirs, files in os.walk(folder):
if directory.count("/") - base >= SCAN_DEPTH:
dirs[:] = []
for name in files:
seen += 1
if seen > SCAN_LIMIT:
return counts
arch = elf_arch(os.path.join(directory, name))
if arch in X86.values():
counts["x86"] += 1
elif arch in ARM.values():
counts["arm"] += 1
return counts
def needs_fex(path):
arch = elf_arch(path)
if arch is not None:
return arch in X86.values()
folder = path if os.path.isdir(path) else os.path.dirname(os.path.abspath(path))
counts = scan(folder)
return counts["x86"] > 0 and counts["arm"] == 0
def is_arm(path):
return elf_arch(path) in ARM.values()
def split_paths(value):
return [part for part in re.split(r"[:\s]+", value or "") if part]
def guest_environment(fex, env):
result = dict(env)
for directory in (CONFIG_DIR, DATA_DIR, CACHE_DIR):
os.makedirs(directory, exist_ok=True)
result["FEX_APP_CONFIG_LOCATION"] = CONFIG_DIR + "/"
result["FEX_APP_DATA_LOCATION"] = DATA_DIR + "/"
result["FEX_APP_CACHE_LOCATION"] = CACHE_DIR + "/"
result.setdefault("FEX_SILENTLOG", "0")
result.setdefault("FEX_OUTPUTLOG", "stderr")
if fex["portable"]:
result["FEX_PORTABLE"] = "1"
else:
result.pop("FEX_PORTABLE", None)
for name in ("FEX_ROOTFS", "FEX_THUNKHOSTLIBS", "FEX_THUNKGUESTLIBS"):
if name in result and not os.path.exists(result[name]):
result.pop(name)
if fex.get("rootfs"):
result["FEX_ROOTFS"] = fex["rootfs"]
if fex.get("rootfs") and fex.get("host_thunks") and fex.get("guest_thunks") and fex.get("thunks_db"):
result["FEX_THUNKHOSTLIBS"] = fex["host_thunks"]
result["FEX_THUNKGUESTLIBS"] = fex["guest_thunks"]
result["FEX_THUNKCONFIG"] = os.path.join(CONFIG_DIR, "ThunksConfig.json")
preload = [part for part in split_paths(result.get("LD_PRELOAD")) if os.path.exists(part) and not is_arm(part)]
library_path = [part for part in (result.get("LD_LIBRARY_PATH") or "").split(":")
if part and "steamrtarm64" not in part and "aarch64" not in part]
for name, parts in (("LD_PRELOAD", preload), ("LD_LIBRARY_PATH", library_path)):
if parts:
result[name] = ":".join(parts)
else:
result.pop(name, None)
return result
def version_names(path):
try:
with open(path, "rb") as f:
return set(VERSION_NAME.findall(f.read()))
except OSError:
return set()
def bundled_libraries(folder):
wanted = [name for name, _ in RUNTIME_LIBRARIES]
found = {}
seen = 0
base = folder.rstrip("/").count("/")
for directory, dirs, files in os.walk(folder):
seen += len(files)
if seen > SCAN_LIMIT:
break
if directory.count("/") - base >= BUNDLE_DEPTH:
dirs[:] = []
for name in wanted:
if name in files and name not in found:
found[name] = os.path.join(directory, name)
return found
def vulkan_thunk_loads(fex, folder):
needed = version_names(os.path.join(fex["guest_thunks"], VULKAN_THUNK))
if not needed:
return False
found = bundled_libraries(folder)
for name, prefix in RUNTIME_LIBRARIES:
library = found.get(name) or os.path.join(fex["rootfs"], "lib/x86_64-linux-gnu", name)
if not {version for version in needed if version.startswith(prefix)} <= version_names(library):
return False
return True
def resolve(program, env):
if "/" in program:
return os.path.abspath(program)
return shutil.which(program, path=env.get("PATH")) or program
def launch(command, env, mode, decide=None):
program = resolve(command[0], env)
use = mode == "on" or (mode == "auto" and needs_fex(decide or program))
if not use:
return os.execvpe(command[0], command, env)
fex = prepare()
if fex is None:
return 126
guest = guest_environment(fex, env)
x86_64 = elf_arch(program) != "i386"
if os.path.isfile(os.path.join(PRELOADS, "x86_64", FAULT_REPORT)) and x86_64:
guest["LD_PRELOAD"] = ":".join([FAULT_REPORT_GUEST] + split_paths(guest.get("LD_PRELOAD")))
thunks = "on" if "FEX_THUNKCONFIG" in guest else "off"
folder = decide if decide and os.path.isdir(decide) else os.path.dirname(program)
if thunks == "on" and x86_64 and all(os.path.isfile(os.path.join(PRELOADS, "x86_64", name)) for name in (THUNK_AUDIT, VULKAN_PIN)) \
and vulkan_thunk_loads(fex, folder):
guest["LD_AUDIT"] = ":".join([THUNK_AUDIT_GUEST] + split_paths(guest.get("LD_AUDIT")))
thunks = "on, its own Vulkan loader too"
say("running %s with %s (runtime %s, thunks %s)" % (program, fex["bin"], guest.get("FEX_ROOTFS", "none"), thunks))
return os.execve(fex["bin"], [fex["bin"], program] + command[1:], guest)
def game_environment(env):
try:
game_env = runpy.run_path(os.path.join(BIN_DIR, "droiddeck-game-env"))
app = game_env["game_id"](env) or env.get("SteamAppId") or env.get("SteamGameId")
if not app:
return env
with open(GAME_ENVIRONMENT, encoding="utf-8") as source:
return game_env["apply_config"](env, json.load(source), str(app))
except FileNotFoundError:
return env
except Exception as error:
say("game settings were not applied (%s)" % error)
return env
def appimage_offset(path):
with open(path, "rb") as f:
head = f.read(64)
if head[:4] != b"\x7fELF":
raise ValueError("not an ELF file")
order = "<" if head[5] == 1 else ">"
if head[4] == 2:
shoff, = struct.unpack_from(order + "Q", head, 0x28)
shentsize, shnum = struct.unpack_from(order + "HH", head, 0x3A)
else:
shoff, = struct.unpack_from(order + "I", head, 0x20)
shentsize, shnum = struct.unpack_from(order + "HH", head, 0x2E)
return shoff + shentsize * shnum
def clear_extracted():
for stale in ("squashfs-root", "AppDir"):
if os.path.islink(stale):
os.unlink(stale)
else:
shutil.rmtree(stale, ignore_errors=True)
def extracted():
if os.path.isdir("AppDir") and not os.path.lexists("squashfs-root"):
os.symlink("AppDir", "squashfs-root")
return os.path.isdir("squashfs-root")
def want_fex():
with locked():
status = describe(fex_install(), runtime_platform())
if not status["ready"]:
request_install()
write_if_changed(STATUS_FILE, json.dumps(status, indent=2) + "\n")
def extract(image):
image = os.path.abspath(image)
clear_extracted()
x86 = elf_arch(image) in X86.values()
if os.access(URUNTIME, os.X_OK):
status = subprocess.call([URUNTIME, "--appimage-extract"], stdout=subprocess.DEVNULL,
env=dict(os.environ, TARGET_APPIMAGE=image))
if status == 0 and extracted():
if x86:
want_fex()
return 0
say("uruntime could not extract it (status %d)" % status)
clear_extracted()
fex = prepare() if x86 else None
if fex is not None:
status = subprocess.call([fex["bin"], image, "--appimage-extract"], stdout=subprocess.DEVNULL,
env=guest_environment(fex, dict(os.environ)))
if status == 0 and extracted():
return 0
say("the image's own runtime could not extract it under FEX (status %d)" % status)
unsquashfs = shutil.which("unsquashfs")
if unsquashfs:
clear_extracted()
status = subprocess.call([unsquashfs, "-q", "-n", "-o", str(appimage_offset(image)), "-d", "squashfs-root", image],
stdout=subprocess.DEVNULL)
if status == 0:
if x86:
want_fex()
return 0
say("unsquashfs could not extract it (status %d)" % status)
elif fex is None:
say("this AppImage could not be unpacked here: no AppImage runtime for ARM64, no FEX yet and no unsquashfs")
return 1
def parse_run(args):
mode, decide = os.environ.get("DROIDDECK_FEX", "auto"), None
while args and args[0].startswith("--"):
option = args.pop(0)
if option == "--":
break
if option in ("--mode", "--for") and args:
value = args.pop(0)
if option == "--mode":
mode = value
else:
decide = value
else:
raise ValueError("unknown option %s" % option)
if mode not in MODES:
raise ValueError("--mode takes %s" % ", ".join(MODES))
return mode, decide, args
def main(argv):
if not argv:
say("usage: droiddeck-fex steam VERB CMD... | run [--mode auto|on|off] [--for PATH] [--] CMD... | needs PATH | extract IMAGE | prepare | status")
return 2
action, args = argv[0], list(argv[1:])
if action == "steam":
if not args:
return 2
verb, command = args[0], args[1:]
if verb not in STEAM_VERBS:
return 0
if not command:
return 2
return launch(command, game_environment(dict(os.environ)), os.environ.get("DROIDDECK_FEX", "auto"))
if action == "run":
try:
mode, decide, command = parse_run(args)
except ValueError as error:
say(str(error))
return 2
if not command:
return 2
return launch(command, dict(os.environ), mode, decide)
if action == "needs":
if len(args) != 1:
return 2
needed = needs_fex(args[0])
print("fex" if needed else "native")
return 0 if needed else 1
if action == "extract":
if len(args) != 1:
return 2
return extract(args[0])
if action == "prepare":
return 0 if prepare() else 1
if action == "status":
with locked():
status = describe(fex_install(), runtime_platform())
write_if_changed(STATUS_FILE, json.dumps(status, indent=2) + "\n")
print(json.dumps(status, indent=2))
return 0 if status["ready"] else 1
say("unknown action %s" % action)
return 2
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
@@ -3,10 +3,10 @@
# reaches the front end as numbers rather than a terminal's redrawn bar. Per-user installation,
# Flathub as the remote, this device's architecture.
#
# bannerlator-flatpak install <app-id> the app and whatever runtimes it needs
# bannerlator-flatpak uninstall <app-id> the app, then runtimes nothing uses any more
# bannerlator-flatpak update [<app-id>] one app, or everything with an update
# bannerlator-flatpak updates what has one (a "updates" line)
# droiddeck-flatpak install <app-id> the app and whatever runtimes it needs
# droiddeck-flatpak uninstall <app-id> the app, then runtimes nothing uses any more
# droiddeck-flatpak update [<app-id>] one app, or everything with an update
# droiddeck-flatpak updates what has one (a "updates" line)
#
# Lines: {"e":"op","i":1,"n":3,"kind":"install","ref":"..."} as each step starts,
# {"e":"progress","i":1,"n":3,"percent":45,"status":"..."} while it runs,
@@ -15,7 +15,7 @@ import json
import os
import sys
os.environ.setdefault("FLATPAK_BWRAP", "/usr/local/bin/bannerlator-bwrap")
os.environ.setdefault("FLATPAK_BWRAP", "/usr/local/bin/droiddeck-bwrap")
import gi # noqa: E402
@@ -216,7 +216,7 @@ def updates():
def main():
if len(sys.argv) < 2:
sys.stderr.write(__doc__ or "usage: bannerlator-flatpak install|uninstall|update|updates [app-id]\n")
sys.stderr.write(__doc__ or "usage: droiddeck-flatpak install|uninstall|update|updates [app-id]\n")
return 2
cmd, args = sys.argv[1], sys.argv[2:]
if cmd in ("install", "update"):
@@ -229,7 +229,7 @@ def main():
return update(args[0] if args else None)
if cmd == "updates":
return updates()
sys.stderr.write("usage: bannerlator-flatpak install|uninstall|update|updates [app-id]\n")
sys.stderr.write("usage: droiddeck-flatpak install|uninstall|update|updates [app-id]\n")
return 2
@@ -1,8 +1,8 @@
#!/bin/bash
# A Flatpak app from the front end, full screen under gamescope like an emulator:
# bannerlator-flatpak-run <app-id> [args...]
# droiddeck-flatpak-run <app-id> [args...]
set -u
export FLATPAK_BWRAP=/usr/local/bin/bannerlator-bwrap
export FLATPAK_BWRAP=/usr/local/bin/droiddeck-bwrap
case ":${XDG_DATA_DIRS:-}:" in
*"/flatpak/exports/share:"*) ;;
*) export XDG_DATA_DIRS=$HOME/.local/share/flatpak/exports/share:${XDG_DATA_DIRS:-/usr/local/share:/usr/share} ;;
@@ -9,7 +9,7 @@
# few packages Flatpak needs on top of this image, verifying each against the Arch Linux ARM
# keyring, and they are unpacked here without hooks or scripts, like the desktop package itself.
#
# bannerlator-flatpak-setup install or repair; prints "== STEP ..." milestones
# droiddeck-flatpak-setup install or repair; prints "== STEP ..." milestones
set -u
export LANG=C.UTF-8
step() { echo "== STEP $*"; }
@@ -17,7 +17,7 @@ fail() { echo "== FAIL $*"; exit 1; }
# Flatpak, the libraries it and appstream add to this image, and the D-Bus proxy flatpak run puts
# between an app and the session bus. bubblewrap is not among them: Android gives an app no user
# namespaces, and bannerlator-bwrap stands in for it (FLATPAK_BWRAP).
# namespaces, and droiddeck-bwrap stands in for it (FLATPAK_BWRAP).
PACKAGES="flatpak ostree appstream libxmlb libmalcontent libfyaml accountsservice composefs xdg-dbus-proxy"
# Libraries Flatpak and the store's helper also need, which the Desktop package brings but a
# runtime without it may lack: each is fetched only when its file is not already there.
@@ -97,7 +97,7 @@ python3 -c 'import gi; gi.require_version("Flatpak", "1.0"); from gi.repository
|| fail "the store helper cannot load libflatpak from Python"
step "Adding Flathub"
export FLATPAK_BWRAP=/usr/local/bin/bannerlator-bwrap
export FLATPAK_BWRAP=/usr/local/bin/droiddeck-bwrap
# Flathub's descriptor - its URL and the key its commits are signed with - is carried here rather
# than fetched: adding the remote then needs no network, and a descriptor that did not arrive
# whole cannot be what failed. Flathub's own copy: https://dl.flathub.org/repo/flathub.flatpakrepo
@@ -2,9 +2,13 @@
import json
import os
import re
import runpy
import sys
from pathlib import Path
SYNC_KEYS = ("BL_SYNC_FALLBACK", "BL_FSYNC_FIRST", "BL_SYNC_MIN_NOFILE", "PROTON_NO_ESYNC", "PROTON_NO_FSYNC", "PROTON_NO_NTSYNC", "WINEESYNC")
PACK_KEYS = ("WINESERVER", "BL_SYNC_PACK", "WINEESYNC", "PROTON_NO_ESYNC", "PROTON_NO_FSYNC")
def game_id(env):
prefix = env.get("STEAM_COMPAT_DATA_PATH", "").rstrip("/")
@@ -144,26 +148,91 @@ def apply_config(env, config, app_id, fixes=None):
return result
def sync_config(config, app_id):
games = config.get("games", {})
return {
"version": config.get("version"),
"shared": {name: value for name, value in config.get("shared", {}).items() if name in SYNC_KEYS},
"games": {app_id: {name: value for name, value in games.get(app_id, {}).items() if name in SYNC_KEYS}},
}
def load_sync():
try:
return runpy.run_path(str(Path(__file__).resolve().parent / "droiddeck-esync"))
except Exception as error:
print("droiddeck-game-env: sync selection failed (%s); stock launch" % error, file=sys.stderr)
return None
def select_sync(command, env, sync=None):
try:
sync = sync if sync is not None else load_sync()
if sync is None:
return command, env
chosen, chosen_env = sync["select"](command, env)
if chosen and all(isinstance(value, str) for value in chosen) and isinstance(chosen_env, dict):
return list(chosen), dict(chosen_env)
except Exception as error:
print("droiddeck-game-env: sync selection failed (%s); stock launch" % error, file=sys.stderr)
return command, env
def record_stock(sync, command, env):
try:
if sync is not None and "record" in sync:
sync["record"](command, env)
except Exception as error:
print("droiddeck-game-env: stock launch not recorded (%s)" % error, file=sys.stderr)
def without_pack(chosen_env, env):
result = dict(chosen_env)
for name in PACK_KEYS:
if name in env:
result[name] = env[name]
else:
result.pop(name, None)
return result
def main():
command = sys.argv[1:]
if not command:
return 2
env = os.environ.copy()
app_id = game_id(env)
if len(command) > 1 and command[1] == "waitforexitandrun" and app_id:
fixes, extra, notes = engine_fixes(command[2], command[3:]) if len(command) > 2 else ({}, [], [])
for note in notes:
print("bannerlator-game-env: " + note, file=sys.stderr)
command += extra
if app_id:
waiting = len(command) > 1 and command[1] == "waitforexitandrun"
fixes = {}
if waiting:
fixes, extra, notes = engine_fixes(command[2], command[3:]) if len(command) > 2 else ({}, [], [])
for note in notes:
print("droiddeck-game-env: " + note, file=sys.stderr)
command += extra
path = Path.home() / ".config/droiddeck/game-environment.json"
try:
with path.open(encoding="utf-8") as source:
env = apply_config(env, json.load(source), app_id, fixes)
config = json.load(source)
if waiting:
env = apply_config(env, config, app_id, fixes)
else:
env = apply_config(env, sync_config(config, app_id), app_id)
except FileNotFoundError:
env = apply_config(env, {"version": 1}, app_id, fixes)
if waiting:
env = apply_config(env, {"version": 1}, app_id, fixes)
except (OSError, ValueError, TypeError, AttributeError):
print("bannerlator-game-env: invalid configuration; using inherited environment", file=sys.stderr)
os.execvpe(command[0], command, env)
print("droiddeck-game-env: invalid configuration; using inherited environment", file=sys.stderr)
sync = load_sync()
launch, launch_env = select_sync(command, env, sync)
if launch[0] != command[0]:
try:
os.execvpe(launch[0], launch, launch_env)
except OSError as error:
print("droiddeck-game-env: %s could not start (%s); stock launch" % (launch[0], error), file=sys.stderr)
launch, launch_env = command, without_pack(launch_env, env)
record_stock(sync, launch, launch_env)
os.execvpe(launch[0], launch, launch_env)
if __name__ == "__main__":
@@ -126,7 +126,7 @@ class LoginManager:
def main():
directory = os.environ.get("BL_LAUNCH_DIR")
if not directory or not Path(directory).is_dir():
sys.exit("bannerlator-login1: no DroidDeck session directory")
sys.exit("droiddeck-login1: no DroidDeck session directory")
ready = Path(directory) / "steam-sleep-ready"
ready.unlink(missing_ok=True)
loop = GLib.MainLoop()
@@ -27,8 +27,8 @@ IP6 = ROOT + "/IP6Config/1"
AP = ROOT + "/AccessPoint/1"
PROFILE = SETTINGS + "/1"
UUID = "b1d0c8e4-57f0-4c0e-9d60-77696e6e6174"
SNAPSHOT = Path("/etc/bannerlator-network.json")
SCAN_REQUEST = Path("/etc/bannerlator-wifi-scan-request")
SNAPSHOT = Path("/etc/droiddeck-network.json")
SCAN_REQUEST = Path("/etc/droiddeck-wifi-scan-request")
OBJECT_MANAGER = "org.freedesktop.DBus.ObjectManager"
PROPERTIES = "org.freedesktop.DBus.Properties"
@@ -17,7 +17,7 @@ def seed(path, text, what, done="Player 1 on the controller"):
with open(path + ".tmp", "w", encoding="utf-8") as f:
f.write(text)
os.replace(path + ".tmp", path)
print(f"bannerlator-pad-defaults: {what}: {done} ({path})")
print(f"droiddeck-pad-defaults: {what}: {done} ({path})")
return True
@@ -97,7 +97,7 @@ def rpcs3_shader_mode():
with open(cfg + ".tmp", "w", encoding="utf-8") as f:
f.write(text)
os.replace(cfg + ".tmp", cfg)
print(f"bannerlator-pad-defaults: RPCS3: shader mode {RPCS3_RECOMPILER} ({cfg})")
print(f"droiddeck-pad-defaults: RPCS3: shader mode {RPCS3_RECOMPILER} ({cfg})")
open(marker, "w").close()
@@ -144,7 +144,7 @@ def patch_ini_section(path, section, body, what, needle="SDL-", require=None, do
with open(path + ".tmp", "w", encoding="utf-8") as f:
f.write("\n".join(lines) + "\n")
os.replace(path + ".tmp", path)
print(f"bannerlator-pad-defaults: {what}: {done} ({path})")
print(f"droiddeck-pad-defaults: {what}: {done} ({path})")
# DuckStation: SDL-<n>/<name>, axes signed (src/util/sdl_input_source.cpp); Pad1 keys from
@@ -285,12 +285,12 @@ def duckstation():
# One set up by an earlier build still has its first-time setup pending, in the way of a game.
if ini_get(ini, "Main", "SetupWizardIncomplete") == "true":
ini_set(ini, "Main", "SetupWizardIncomplete", "false")
print("bannerlator-pad-defaults: DuckStation: first-time setup already done here")
print("droiddeck-pad-defaults: DuckStation: first-time setup already done here")
# Keeping the screen awake goes through org.freedesktop.ScreenSaver, which nothing in the session
# provides (Android decides when the screen sleeps): DuckStation only warns it failed, every game.
if (ini_get(ini, "Main", "InhibitScreensaver") or "true") == "true":
ini_set(ini, "Main", "InhibitScreensaver", "false")
print("bannerlator-pad-defaults: DuckStation: no screensaver inhibit (no such service here)")
print("droiddeck-pad-defaults: DuckStation: no screensaver inhibit (no such service here)")
# Once, for a DuckStation set up before this: its own empty BIOS folder moves to the ROMs folder,
# and Automatic becomes Vulkan. A later choice in DuckStation's settings is kept.
marker = os.path.join(os.path.dirname(ini), ".droiddeck-setup")
@@ -300,10 +300,10 @@ def duckstation():
own = current if os.path.isabs(current) else os.path.join(os.path.dirname(ini), current)
if not (os.path.isdir(own) and os.listdir(own)):
ini_set(ini, "BIOS", "SearchDirectory", bios_folder(ps1))
print(f"bannerlator-pad-defaults: DuckStation: BIOS from {bios_folder(ps1)}")
print(f"droiddeck-pad-defaults: DuckStation: BIOS from {bios_folder(ps1)}")
if (ini_get(ini, "GPU", "Renderer") or "Automatic") == "Automatic":
ini_set(ini, "GPU", "Renderer", "Vulkan")
print("bannerlator-pad-defaults: DuckStation: renderer Vulkan")
print("droiddeck-pad-defaults: DuckStation: renderer Vulkan")
open(marker, "w").close()
@@ -451,7 +451,7 @@ def armsx2():
src = os.path.join(old, sub)
if os.path.isdir(src):
shutil.copytree(src, os.path.join(root, sub), dirs_exist_ok=True)
print(f"bannerlator-pad-defaults: ARMSX2: settings and memory cards copied from PCSX2 ({root})")
print(f"droiddeck-pad-defaults: ARMSX2: settings and memory cards copied from PCSX2 ({root})")
elif not os.path.isfile(ini):
# A new player: everything ARMSX2's first-time setup asks is answered here - the BIOS
# folder in the ROMs folder, the ps2 folder as its game list, the controller on Player 1 -
@@ -482,7 +482,7 @@ def armsx2():
require="SettingsVersion", done="games listed from the ROMs folder")
if ini_get(ini, "UI", "SetupWizardIncomplete") == "true":
ini_set(ini, "UI", "SetupWizardIncomplete", "false")
print("bannerlator-pad-defaults: ARMSX2: first-time setup already done here")
print("droiddeck-pad-defaults: ARMSX2: first-time setup already done here")
armsx2_bios(ini)
@@ -507,7 +507,7 @@ def armsx2_bios(ini):
us = [f for f in dumps if any(t in f.lower() for t in ("usa", "(us)", "(us,", "ntsc-u"))]
pick = (us or dumps)[0]
ini_set(ini, "Filenames", "BIOS", pick)
print(f"bannerlator-pad-defaults: ARMSX2: BIOS {pick}")
print(f"droiddeck-pad-defaults: ARMSX2: BIOS {pick}")
# Dolphin (PC) has no controller menu, so the guide button is its Toggle Fullscreen hotkey: out of full
@@ -596,7 +596,7 @@ def dolphin_data():
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=600)
src = os.path.join(work, "squashfs-root/usr/share/dolphin-emu")
if not os.path.isfile(os.path.join(src, "Sys/GC/font_western.bin")):
print("bannerlator-pad-defaults: Dolphin: could not unpack its data from the AppImage")
print("droiddeck-pad-defaults: Dolphin: could not unpack its data from the AppImage")
return
new = DOLPHIN_DATA + ".new"
shutil.rmtree(new, ignore_errors=True)
@@ -605,9 +605,9 @@ def dolphin_data():
f.write(stamp)
shutil.rmtree(DOLPHIN_DATA, ignore_errors=True)
os.rename(new, DOLPHIN_DATA)
print("bannerlator-pad-defaults: Dolphin: its data (fonts, per-game fixes) in " + DOLPHIN_DATA)
print("droiddeck-pad-defaults: Dolphin: its data (fonts, per-game fixes) in " + DOLPHIN_DATA)
except (OSError, subprocess.SubprocessError) as e:
print("bannerlator-pad-defaults: Dolphin: unpacking its data failed: %s" % e)
print("droiddeck-pad-defaults: Dolphin: unpacking its data failed: %s" % e)
finally:
shutil.rmtree(work, ignore_errors=True)
@@ -622,7 +622,7 @@ def dolphin():
wii_pad = os.path.join(d, "WiimoteNew.ini")
if ini_get(wii_pad, "Wiimote1", "Buttons/Home") == "`Guide`":
ini_set(wii_pad, "Wiimote1", "Buttons/Home", "`Shoulder R`")
print("bannerlator-pad-defaults: Dolphin (Wii): Home on the right shoulder button (guide = Dolphin's window)")
print("droiddeck-pad-defaults: Dolphin (Wii): Home on the right shoulder button (guide = Dolphin's window)")
hotkeys = os.path.join(d, "Hotkeys.ini")
if not os.path.isfile(hotkeys):
seed(hotkeys, DOLPHIN_HOTKEYS, "Dolphin", "the guide button shows Dolphin's window (Toggle Fullscreen)")
@@ -631,7 +631,7 @@ def dolphin():
if current is not None and DOLPHIN_GUIDE not in current:
ini_set(hotkeys, "Hotkeys", "General/Toggle Fullscreen",
(current + " | " if current else "") + DOLPHIN_GUIDE)
print("bannerlator-pad-defaults: Dolphin: the guide button shows Dolphin's window (Toggle Fullscreen)")
print("droiddeck-pad-defaults: Dolphin: the guide button shows Dolphin's window (Toggle Fullscreen)")
# Games from the ROMs folder's gamecube and wii folders, subfolders included; Vulkan; no
# NKit warning (it will not let a game start until a box is ticked - games run either way,
# the warning is about loading times, netplay and save states); no statistics question.
@@ -647,7 +647,7 @@ def dolphin():
"[Core]\nGFXBackend = Vulkan\n"
"[General]\nISOPaths = 2\nISOPath0 = %s\nISOPath1 = %s\nRecursiveISOPaths = True\n"
"[Interface]\nSkipNKitWarning = True\n" % (gc, wii))
print("bannerlator-pad-defaults: Dolphin: first-time setup (Vulkan, games from %s and %s)" % (gc, wii))
print("droiddeck-pad-defaults: Dolphin: first-time setup (Vulkan, games from %s and %s)" % (gc, wii))
return
try:
count = int(ini_get(ini, "General", "ISOPaths") or "0")
@@ -679,7 +679,7 @@ def dolphin():
ini_set(ini, "General", "RecursiveISOPaths", "True")
changed.append("subfolders")
if changed:
print("bannerlator-pad-defaults: Dolphin: games from " + ", ".join(changed))
print("droiddeck-pad-defaults: Dolphin: games from " + ", ".join(changed))
# Once, for a Dolphin set up before this: Vulkan unless it was chosen already, and the NKit box.
marker = os.path.join(d, ".droiddeck-setup")
if os.path.exists(marker):
@@ -691,7 +691,7 @@ def dolphin():
ini_set(ini, "Analytics", "Enabled", "False")
ini_set(ini, "Analytics", "PermissionAsked", "True")
open(marker, "w").close()
print("bannerlator-pad-defaults: Dolphin: Vulkan, no NKit warning")
print("droiddeck-pad-defaults: Dolphin: Vulkan, no NKit warning")
# Cemu: an SDLController profile for the GamePad. The uuid is "<index>_<SDL GUID>" and SDL's GUID
@@ -954,7 +954,7 @@ def melonds():
with open(path + ".tmp", "w", encoding="utf-8") as f:
f.write("\n".join(lines).rstrip("\n") + "\n")
os.replace(path + ".tmp", path)
print(f"bannerlator-pad-defaults: melonDS: {'; '.join(done)} ({path})")
print(f"droiddeck-pad-defaults: melonDS: {'; '.join(done)} ({path})")
# PPSSPP (SDL): its pad defaults leave the shoulders, the right stick and the triggers unbound on
@@ -1041,7 +1041,7 @@ def retroarch():
with open(cfg + ".tmp", "w", encoding="utf-8") as f:
f.write("\n".join(lines) + "\n")
os.replace(cfg + ".tmp", cfg)
print(f"bannerlator-pad-defaults: RetroArch: joypad driver udev -> sdl2 ({cfg})")
print(f"droiddeck-pad-defaults: RetroArch: joypad driver udev -> sdl2 ({cfg})")
# yuzu and its forks: Player 1 in [Controls] of <name>/qt-config.ini, which each writes at its first
@@ -1090,11 +1090,11 @@ def switch_emulators():
f.write("\n".join(lines) + "\n")
os.replace(cfg + ".tmp", cfg)
open(marker, "w").close()
print(f"bannerlator-pad-defaults: {name}: Player 1 on the controller ({cfg})")
print(f"droiddeck-pad-defaults: {name}: Player 1 on the controller ({cfg})")
for step in (rpcs3, duckstation, pcsx2, armsx2, dolphin, cemu, melonds, ppsspp, retroarch, switch_emulators):
try:
step()
except Exception as e:
print(f"bannerlator-pad-defaults: {step.__name__}: {e}")
print(f"droiddeck-pad-defaults: {step.__name__}: {e}")
@@ -1,12 +1,12 @@
#!/usr/bin/python3
# Installs a third-party Proton build into the client's compatibilitytools.d, so it appears in the
# Compatibility dropdown beside Bannerlator Proton (ARM64) and can be picked per game.
# Compatibility dropdown beside our own Proton ARM64 and can be picked per game.
#
# bannerlator-proton-extra <steam root> ge [--tag GE-Proton11-7]
# bannerlator-proton-extra <steam root> cachyos [--tag cachyos-11.0-20260703-slr]
# bannerlator-proton-extra <steam root> <url of an aarch64 tarball>
# bannerlator-proton-extra <steam root> /path/to/an/aarch64/tarball
# bannerlator-proton-extra <steam root> --list what is installed now
# droiddeck-proton-extra <steam root> ge [--tag GE-Proton11-7]
# droiddeck-proton-extra <steam root> cachyos [--tag cachyos-11.0-20260703-slr]
# droiddeck-proton-extra <steam root> <url of an aarch64 tarball>
# droiddeck-proton-extra <steam root> /path/to/an/aarch64/tarball
# droiddeck-proton-extra <steam root> --list what is installed now
#
# Both projects publish native ARM64 builds - GloriousEggroll as "*-aarch64.tar.gz", CachyOS as
# "*-arm64.tar.xz" - and both are ordinary Proton trees: a python entry point beside a files/
@@ -16,7 +16,7 @@
# What it cannot use is their toolmanifest. Like Valve's ARM64 depot they declare
# require_tool_appid, which makes Steam stack pressure-vessel underneath - and pressure-vessel
# needs unprivileged user namespaces, which an Android app does not get. Patching that out is
# bannerlator-steam-compat's job (adopt_extras); this script only puts the tree in place and then
# steam-compatibility's job (adopt_extras); this script only puts the tree in place and then
# calls it, so an install is usable without restarting the client.
import hashlib
import json
@@ -36,14 +36,14 @@ SOURCES = {
"ge": ("GloriousEggroll/proton-ge-custom", re.compile(r"aarch64\.tar\.(gz|xz)$")),
"cachyos": ("CachyOS/proton-cachyos", re.compile(r"arm64\.tar\.(gz|xz)$")),
}
USER_AGENT = "bannerlator-proton-extra"
USER_AGENT = "droiddeck-proton-extra"
# Room for the tarball and the tree it unpacks to. GE-Proton's aarch64 build is ~650 MB packed and
# unpacks to about three times that; refusing early beats a half-written tool directory.
NEED_BYTES = 4 * 1024 * 1024 * 1024
def log(*args):
print("bannerlator-proton-extra:", *args, flush=True)
print("droiddeck-proton-extra:", *args, flush=True)
def fetch_json(url):
@@ -77,7 +77,7 @@ def release_asset(source, tag):
sums = [u for n, u in assets.items()
if n.startswith(name.split(".tar")[0]) and "sha512" in n]
return tag, assets[name], (sums[0] if sums else None)
raise SystemExit("bannerlator-proton-extra: %s has no ARM64 asset" % tag)
raise SystemExit("droiddeck-proton-extra: %s has no ARM64 asset" % tag)
def download(url, path):
@@ -88,7 +88,7 @@ def download(url, path):
"curl", "-fL", "--retry", "5", "--retry-delay", "3", "--retry-connrefused",
"-C", "-", "-A", USER_AGENT, "-o", path, url])
if rc != 0:
raise SystemExit("bannerlator-proton-extra: download failed (curl %d)" % rc)
raise SystemExit("droiddeck-proton-extra: download failed (curl %d)" % rc)
log("downloaded %.0f MB" % (os.path.getsize(path) / 1048576.0))
@@ -109,7 +109,7 @@ def verify(path, sums_url):
h.update(chunk)
if h.hexdigest() != want:
os.unlink(path)
raise SystemExit("bannerlator-proton-extra: checksum mismatch; the download was deleted")
raise SystemExit("droiddeck-proton-extra: checksum mismatch; the download was deleted")
log("checksum ok")
@@ -118,7 +118,7 @@ def extract(tarball, into):
log("unpacking (a few minutes; nothing is shown while tar runs)")
rc = subprocess.call(["tar", "-xf", tarball, "-C", into])
if rc != 0:
raise SystemExit("bannerlator-proton-extra: tar failed (%d)" % rc)
raise SystemExit("droiddeck-proton-extra: tar failed (%d)" % rc)
# A Proton tarball holds one top-level directory. Take whichever entry has the entry point so a
# stray ._ file or a nested layout does not decide it.
entries = [e for e in sorted(os.listdir(into)) if not e.startswith(".")]
@@ -128,7 +128,7 @@ def extract(tarball, into):
return path
if len(entries) == 1 and os.path.isdir(os.path.join(into, entries[0])):
return os.path.join(into, entries[0])
raise SystemExit("bannerlator-proton-extra: no Proton tree in the archive")
raise SystemExit("droiddeck-proton-extra: no Proton tree in the archive")
def installed(tools):
@@ -144,7 +144,7 @@ def installed(tools):
def main():
argv = sys.argv[1:]
if not argv:
raise SystemExit(__doc__ or "usage: bannerlator-proton-extra <steam root> ge|cachyos|<url>")
raise SystemExit(__doc__ or "usage: droiddeck-proton-extra <steam root> ge|cachyos|<url>")
steam_root = argv.pop(0)
tools = os.path.join(steam_root, "compatibilitytools.d")
os.makedirs(tools, exist_ok=True)
@@ -167,7 +167,7 @@ def main():
if what in SOURCES:
tag, url, sums = release_asset(what, tag) if tag else pick_asset(what)
if not url:
raise SystemExit("bannerlator-proton-extra: no ARM64 build published for %s" % what)
raise SystemExit("droiddeck-proton-extra: no ARM64 build published for %s" % what)
log("%s -> %s" % (what, tag))
elif what.startswith("http"):
url, sums = what, None
@@ -176,14 +176,15 @@ def main():
log("using %s" % what)
else:
raise SystemExit(
"bannerlator-proton-extra: expected ge, cachyos, a URL or a tarball path, got %r" % what)
"droiddeck-proton-extra: expected ge, cachyos, a URL or a tarball path, got %r" % what)
free = shutil.disk_usage(tools).free
if free < NEED_BYTES:
raise SystemExit("bannerlator-proton-extra: needs about %d GB free, has %.1f GB" % (
raise SystemExit("droiddeck-proton-extra: needs about %d GB free, has %.1f GB" % (
NEED_BYTES // (1024 ** 3), free / float(1024 ** 3)))
staging = os.path.join(tools, ".bannerlator-download")
shutil.rmtree(os.path.join(tools, ".bannerlator-download"), ignore_errors=True)
staging = os.path.join(tools, ".droiddeck-download")
os.makedirs(staging, exist_ok=True)
if local:
tarball = local
@@ -210,7 +211,7 @@ def main():
# Patches out require_tool_appid and wraps the entry point, which is what makes it launchable
# here. Doing it now means the client picks the tool up on its next restart with no extra step.
subprocess.call(["bannerlator-steam-compat", steam_root])
subprocess.call(["steam-compatibility", steam_root])
if __name__ == "__main__":
+51
View File
@@ -0,0 +1,51 @@
#!/bin/bash
# A script added on the Desktop page (UserApps), run from its own folder:
# droiddeck-script-run /opt/droiddeck-apps/<id> [args...]
set -u
dir=${1:?usage: droiddeck-script-run <added app directory> [args...]}
shift
entry=$(cat "$dir/entry" 2>/dev/null) || { echo "droiddeck-script-run: $dir has no entry" >&2; exit 127; }
cd -P "$dir/files" 2>/dev/null || { echo "droiddeck-script-run: $dir/files is missing; add the script again" >&2; exit 127; }
folder=$PWD
# Shared storage is noexec, so the script is handed to its interpreter rather than executed.
first=
IFS= read -r first < "$entry"
first=${first%$'\r'}
interp=(/bin/bash)
case $first in '#!'*) read -r -a interp <<< "${first#'#!'}" ;; esac
fex=$(cat "$dir/fex" 2>/dev/null)
case $fex in on|off) ;; *) fex=auto ;; esac
run=("${interp[@]}" "./$entry")
if [ -x /usr/local/bin/droiddeck-fex ]; then
run=(/usr/local/bin/droiddeck-fex run --mode "$fex" --for "$folder" -- "${interp[@]}" "./$entry")
fi
echo "== script $entry in $folder (${interp[*]}, FEX $fex)"
case ${DBUS_SESSION_BUS_ADDRESS:-} in
unix:*) "${run[@]}" "$@" ;;
*) unset DBUS_SESSION_BUS_ADDRESS
if command -v dbus-run-session >/dev/null 2>&1; then
dbus-run-session -- "${run[@]}" "$@"
else
"${run[@]}" "$@"
fi ;;
esac
rc=$?
# A launcher script often starts the game and returns; the session lasts while it runs.
running() {
local p exe arg cwd
for p in /proc/[0-9]*; do
exe=$(readlink "$p/exe" 2>/dev/null) || continue
case $exe in "$folder"/*) return 0 ;; esac
case $exe in
*/FEX|*/FEXInterpreter)
arg=$(tr '\0' '\n' < "$p/cmdline" 2>/dev/null | sed -n 2p)
cwd=$(readlink "$p/cwd" 2>/dev/null)
case $arg in "$folder"/*) return 0 ;; esac
case $cwd in "$folder"|"$folder"/*) return 0 ;; esac
;;
esac
done
return 1
}
while running; do sleep 2; done
exit $rc
@@ -81,7 +81,11 @@ for reg in "$steam_root"/steamapps/common/Proton*/files/share/default_pfx*/syste
"$steam_root"/steamapps/compatdata/*/pfx/system.reg \
/mnt/droiddeck-sd/steamapps/compatdata/*/pfx/system.reg; do
[ -f "$reg" ] || continue
seeded=$(dirname "$reg")/.bannerlator-redists
seeded=$(dirname "$reg")/.droiddeck-redists
legacy=$(dirname "$reg")/.bannerlator-redists
if [ -f "$legacy" ] && [ ! -f "$seeded" ]; then
mv -f "$legacy" "$seeded"
fi
[ "$(cat "$seeded" 2>/dev/null)" = "$stamp" ] && continue
dated=$(mktemp) || continue
touch -r "$reg" "$dated"
@@ -1,10 +1,10 @@
#!/bin/bash
# The Linux runtime's session. Bannerlator execs this under proot with the output size and mode;
# The Linux runtime's session. DroidDeck execs this under proot with the output size and mode;
# the first pass starts gamescope, which execs this script again inside it for the program:
# bannerlator-session desktop the file manager
# bannerlator-session run <path> a Linux program (AppImage, script or ELF)
# bannerlator-session steam [steam-url] the native Steam client
# bannerlator-session lxqt the LXQt/labwc desktop (DroidDeck's desktop package)
# droiddeck-session desktop the file manager
# droiddeck-session run <path> a Linux program (AppImage, script or ELF)
# droiddeck-session steam [steam-url] the native Steam client
# droiddeck-session lxqt the LXQt/labwc desktop (DroidDeck's desktop package)
# BL_WIDTH / BL_HEIGHT size the output, BL_FPS limits it, BL_LOG collects everything said here.
set -u
export HOME=${HOME:-/root}
@@ -22,8 +22,8 @@ if [ -n "${BL_LOG:-}" ] && [ -z "${BL_INSIDE:-}" ]; then
# A guest looping on an error writes gigabytes in minutes (an Xwayland proot had lost track of
# managed 9 GB of one line in 19 min): the log keeps its first BL_LOG_CAP bytes and the rest is
# drained, not refused - a writer that filled it must not die of SIGPIPE.
exec > >({ head -c "${BL_LOG_CAP:-268435456}" >>"$BL_LOG"; cat >/dev/null; }) 2>&1
echo "== bannerlator-session $(date) mode=$mode $*"
exec > >({ dd of="$BL_LOG" oflag=append conv=notrunc iflag=count_bytes count="${BL_LOG_CAP:-268435456}" bs=65536 status=none; cat >/dev/null; }) 2>&1
echo "== droiddeck-session $(date) mode=$mode $*"
echo "== uid=$(id -u) size=${BL_WIDTH:-?}x${BL_HEIGHT:-?} fps=${BL_FPS:-0} icd=${VK_ICD_FILENAMES:-none}"
echo "== gamescope: $(gamescope --version 2>&1 | head -1) at $(command -v gamescope)"
echo "== rootfs: $(cat /.version 2>/dev/null || echo unversioned)"
@@ -42,6 +42,16 @@ collect_steam_logs() {
[ -f "$HOME/.steam/steam/logs/steamwebhelper.log" ] && cp "$HOME/.steam/steam/logs/steamwebhelper.log" "$BL_DEBUG_DIR/steam/" 2>/dev/null
# NOT copied, ever: loginusers.vdf, config.vdf and the ssfn* files, which hold the credentials
# themselves rather than a mention of them.
local sync_store=$HOME/.local/share/droiddeck-esync sync_file
if [ -d "$sync_store" ] && mkdir -p "$BL_DEBUG_DIR/droiddeck-esync" 2>/dev/null; then
for sync_file in launches.log tools.tsv wanted.tsv prefixes.json; do
[ -f "$sync_store/$sync_file" ] && cp "$sync_store/$sync_file" "$BL_DEBUG_DIR/droiddeck-esync/" 2>/dev/null
done
fi
local fex_state=$HOME/.local/share/droiddeck-fex
if [ -d "$fex_state" ] && mkdir -p "$BL_DEBUG_DIR/droiddeck-fex" 2>/dev/null; then
cp "$fex_state/status.json" "$fex_state/config/Config.json" "$BL_DEBUG_DIR/droiddeck-fex/" 2>/dev/null
fi
echo "== steam logs copied to $BL_DEBUG_DIR/steam"
}
# A session that is killed never reaches the copy below, and those are the sessions worth reading.
@@ -93,11 +103,11 @@ export XDG_CONFIG_HOME=${XDG_CONFIG_HOME:-$HOME/.config}
export XDG_CACHE_HOME=${XDG_CACHE_HOME:-$HOME/.cache}
mkdir -p "$XDG_DATA_HOME" "$XDG_CONFIG_HOME" "$XDG_CACHE_HOME"
if [ "$mode" = lxqt ] || [ "$mode" = desktop ]; then
/usr/local/bin/bannerlator-desktop-bookmarks || true
/usr/local/bin/droiddeck-desktop-bookmarks || true
fi
# Flatpak (the store's apps): its sandboxes through the app instead of bubblewrap, which needs the
# user namespaces Android keeps from apps, and its apps' menu entries and icons on the desktop.
export FLATPAK_BWRAP=/usr/local/bin/bannerlator-bwrap
export FLATPAK_BWRAP=/usr/local/bin/droiddeck-bwrap
export XDG_DATA_DIRS=$XDG_DATA_HOME/flatpak/exports/share:${XDG_DATA_DIRS:-/usr/local/share:/usr/share}
# The cores a program runs on outside Steam (the desktop and everything started from it, a
@@ -238,32 +248,41 @@ case $mode in
taskset -cp "$cores" $$ >/dev/null 2>&1 || true
fi
# Player 1 on the controller in an emulator that has no profile of its own yet.
python3 /usr/local/bin/bannerlator-pad-defaults 2>&1 || true
python3 /usr/local/bin/droiddeck-pad-defaults 2>&1 || true
fex=()
if [ -x /usr/local/bin/droiddeck-fex ]; then
case $(od -An -tu2 -j18 -N2 "$target" 2>/dev/null | tr -d ' ') in
3|62) fex=(/usr/local/bin/droiddeck-fex run --mode on --) ;;
esac
fi
case ${target,,} in
*.appimage)
export APPIMAGE_EXTRACT_AND_RUN=1
chmod +x "$target" 2>/dev/null
exec "$target" "$@"
exec "${fex[@]}" "$target" "$@"
;;
*.sh)
if [ -x /usr/local/bin/droiddeck-fex ]; then
exec /usr/local/bin/droiddeck-fex run --for "$target" -- bash "$target" "$@"
fi
exec bash "$target" "$@"
;;
*)
chmod +x "$target" 2>/dev/null
cd "$(dirname "$target")" && exec "$target" "$@"
cd "$(dirname "$target")" && exec "${fex[@]}" "$target" "$@"
;;
esac
;;
steam)
step "checking the Steam client (first run downloads it, ~1-2 min with nothing on screen)"
bannerlator-steam-install || exit $?
droiddeck-steam-install || exit $?
step "Steam client ready"
steam_root=$XDG_DATA_HOME/Steam
# Register the card library (bound at /mnt/droiddeck-sd) with the client - internal storage is
# its main library already - and the ARM64 Proton that lets it launch the Windows ones.
# A request left in ~/.bl-proton-extra installs a third-party Proton before the client starts,
# so it is in the Compatibility dropdown the first time the user looks. One per line, each the
# arguments to bannerlator-proton-extra: "ge", "cachyos", "ge --tag GE-Proton11-7", or a URL.
# arguments to droiddeck-proton-extra: "ge", "cachyos", "ge --tag GE-Proton11-7", or a URL.
# Hundreds of megabytes each, so it is only ever done on request, and the line is dropped once
# it has succeeded - a failed download stays and is retried next session.
if [ -s "$HOME/.bl-proton-extra" ]; then
@@ -272,7 +291,7 @@ case $mode in
while read -r _bl_extra || [ -n "$_bl_extra" ]; do
case ${_bl_extra# } in ""|"#"*) continue ;; esac
step "installing Proton '$_bl_extra' (large download, several minutes)"
if bannerlator-proton-extra "$steam_root" $_bl_extra; then
if droiddeck-proton-extra "$steam_root" $_bl_extra; then
step "installed Proton '$_bl_extra'"
else
step "Proton '$_bl_extra' failed; will retry next session"
@@ -323,27 +342,27 @@ case $mode in
fi
step "registering the app's game library with Steam"
bannerlator-steam-library "$steam_root" || true
droiddeck-steam-library "$steam_root" || true
step "setting up the ARM64 Proton compatibility tool"
bannerlator-steam-compat "$steam_root" || true
steam-compatibility "$steam_root" || true
if [ -z "${BL_DESKTOP:-}" ] && [ "${BL_STEAM_UI:-}" != desktop ]; then
bannerlator-steam-ui-scale "$steam_root" "${BL_WIDTH:-1280}" "${BL_HEIGHT:-720}" || true
droiddeck-steam-ui-scale "$steam_root" "${BL_WIDTH:-1280}" "${BL_HEIGHT:-720}" || true
fi
# The user's own games, as non-Steam shortcuts under that tool; the list is written by the app
# each session (empty = the app's earlier entries are removed, the client's own are kept).
if [ -n "${BL_ADDED_GAMES:-}" ]; then
step "adding the app's own games to the client's library"
bannerlator-steam-shortcuts "$steam_root" "$BL_ADDED_GAMES" || true
droiddeck-steam-shortcuts "$steam_root" "$BL_ADDED_GAMES" || true
fi
# Steam runs a game's shared redistributables inside its prefix on first launch; marking
# them as already run keeps those x86 installers from starting under emulation at all.
bannerlator-seed-redists "$steam_root" || true
droiddeck-seed-redists "$steam_root" || true
# The compatibility tool runs Valve's arm64 Proton, which the client fetches like any depot.
# Asked for on the command line the first time, so a Windows title has it before it is needed.
if [ ! -f "$steam_root/steamapps/appmanifest_4427310.acf" ]; then
step "asking the client to fetch the ARM64 Proton"
set -- "$@" steam://install/4427310
# Ported from Bannerlator (bannerlator-session, device-proven 2026-09-22). The client decides
# Ported from Bannerlator's session script (device-proven 2026-09-22). The client decides
# a title's compatibility tool when the title is installed and keeps that decision for the
# rest of its run, and the tool is set up before the client comes up. So once the depot has
# fully landed, restart the client once - its own -shutdown, then the loop below, which knows
@@ -357,7 +376,7 @@ case $mode in
# StateFlags 4 = fully installed; a moment more for the client to finish its bookkeeping.
if grep -q '"StateFlags"[[:space:]]*"4"' "$_bl_acf" 2>/dev/null; then
sleep 10
bannerlator-steam-compat "$steam_root" >/dev/null 2>&1 || true
steam-compatibility "$steam_root" >/dev/null 2>&1 || true
step "compatibility layer installed: restarting the Steam client once"
: > "$HOME/.bl-restart-once"
"$steam_root/steamrtarm64/steam" -shutdown >/dev/null 2>&1 || true
@@ -514,10 +533,10 @@ case $mode in
nap 5
beat=$((beat + 1))
[ "$((beat % 3))" -eq 0 ] || continue
bannerlator-seed-redists "$steam_root" >/dev/null 2>&1 || true
droiddeck-seed-redists "$steam_root" >/dev/null 2>&1 || true
if compat_due; then
: > "$compat_stamp"
bannerlator-steam-compat "$steam_root" >/dev/null 2>&1 || true
steam-compatibility "$steam_root" >/dev/null 2>&1 || true
fi
done
) &
@@ -542,9 +561,9 @@ case $mode in
# refuses what it cannot do, since the app has no say over Wi-Fi - answer what the client asks.
# The bus config is written here rather than shipped, so a runtime installed earlier gets this
# from the app's copy of the session scripts without a re-host. (WinNative, maxjivi05, e7a224ae.)
if command -v dbus-daemon >/dev/null 2>&1 && command -v bannerlator-netmanager >/dev/null 2>&1; then
mkdir -p /run/dbus /usr/local/share/bannerlator
cat > /usr/local/share/bannerlator/system-bus.conf <<'BUSCONF'
if command -v dbus-daemon >/dev/null 2>&1 && command -v droiddeck-netmanager >/dev/null 2>&1; then
mkdir -p /run/dbus /usr/local/share/droiddeck
cat > /usr/local/share/droiddeck/system-bus.conf <<'BUSCONF'
<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-Bus Bus Configuration 1.0//EN"
"http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd">
<busconfig>
@@ -560,21 +579,21 @@ case $mode in
</busconfig>
BUSCONF
rm -f /run/dbus/system_bus_socket
dbus-daemon --config-file=/usr/local/share/bannerlator/system-bus.conf --nofork --nopidfile &
dbus-daemon --config-file=/usr/local/share/droiddeck/system-bus.conf --nofork --nopidfile &
system_bus=$!
for _ in $(seq 1 50); do
[ -S /run/dbus/system_bus_socket ] && break
sleep 0.1
done
if [ -S /run/dbus/system_bus_socket ]; then
bannerlator-netmanager &
droiddeck-netmanager &
netmanager=$!
echo "== network: system bus up, NetworkManager stand-in running"
# Steam blanks its main UI before calling login1.Suspend. Without the
# matching resume notification it stays black although the menus work.
# Own the name before Steam constructs its login1 proxy at startup.
rm -f "$BL_LAUNCH_DIR/steam-sleep-ready"
bannerlator-login1 &
droiddeck-login1 &
login_manager=$!
for ((_i = 0; _i < 50; _i++)); do
[ -f "$BL_LAUNCH_DIR/steam-sleep-ready" ] && break
@@ -598,26 +617,20 @@ BUSCONF
export STEAM_RUNTIME=1
export LD_LIBRARY_PATH=$steam_root/steamrtarm64:$steam_root/steamrtarm64/panorama${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}
export SDL_VIDEO_X11_DGAMOUSE=0
# Native-Linux x86 games go through Valve's FEX compat tool, which runs FEX with FEX_PORTABLE=1
# - that makes FEX look for its config next to the binary, while the tool writes one into the
# app's compatdata instead. FEX therefore starts with no rootfs, every x86_64 library lookup
# falls through to the host, and the guest shell dies on libreadline before the game is reached.
# FEX_ROOTFS names it outright and is read whatever the config situation.
[ -d /usr/share/guestos/fex-mesa ] && export FEX_ROOTFS=/usr/share/guestos/fex-mesa
# Decky is optional and session scoped: the app's marker opts in, and this supervisor is
# reaped with the client session. Native ARM64 runs directly. A legacy x86_64 asset is used
# only when its recorded architecture and this guest's existing FEX installation agree.
# only when its recorded architecture says so and droiddeck-fex has FEX and its runtime ready.
decky_loader=$HOME/homebrew/services/PluginLoader
if [ -f "$HOME/.droiddeck-decky-enabled" ] && [ -x "$decky_loader" ]; then
decky_arch=$(cat "$HOME/homebrew/services/.droiddeck-decky-arch" 2>/dev/null || echo unknown)
if [ "$decky_arch" = arm64 ] || { [ "$decky_arch" = x86_64 ] && [ -x /usr/bin/FEX ] && [ -n "${FEX_ROOTFS:-}" ]; }; then
if [ "$decky_arch" = arm64 ] || { [ "$decky_arch" = x86_64 ] && /usr/local/bin/droiddeck-fex prepare >/dev/null; }; then
(
decky_child=
decky_stop() { [ -z "$decky_child" ] || kill "$decky_child" 2>/dev/null; [ -z "$decky_child" ] || wait "$decky_child" 2>/dev/null; exit 0; }
trap decky_stop TERM INT HUP
decky_exits=0
while :; do
if [ "$decky_arch" = x86_64 ]; then /usr/bin/FEX "$decky_loader" >>"$HOME/homebrew/services/PluginLoader.log" 2>&1 &
if [ "$decky_arch" = x86_64 ]; then /usr/local/bin/droiddeck-fex run --mode on -- "$decky_loader" >>"$HOME/homebrew/services/PluginLoader.log" 2>&1 &
else "$decky_loader" >>"$HOME/homebrew/services/PluginLoader.log" 2>&1 &
fi
decky_child=$!
@@ -640,6 +653,20 @@ BUSCONF
echo "== Decky: installed loader architecture is unsupported by this guest; not starting"
fi
fi
_bl_cdp=()
if [ -f "$steam_root/.cef-enable-remote-debugging" ]; then
export BL_CDP_PORT=8080
else
export BL_CDP_PORT=$((20000 + RANDOM % 30000))
_bl_cdp=(-cef-enable-debugging -devtools-port "$BL_CDP_PORT")
fi
if [ -f "$HOME/.droiddeck-decky-enabled" ] && [ "${decky_arch:-}" = x86_64 ]; then
echo "== steam: devtools port $BL_CDP_PORT left open for the x86_64 Decky loader"
else
export BL_CDP_GUARD=$BL_CDP_PORT
echo "== steam: devtools port $BL_CDP_PORT, guest processes only"
fi
droiddeck-steam-compat "$steam_root" 2>&1 &
# The client exits 42 when it has updated itself and wants a fresh start (steam.sh does this).
# BL_STEAM_UI=desktop = the client's desktop UI (a window, the library as a list), which the
# desktop's launchers ask for; gamescope shows one window at a time and the game takes over.
@@ -682,7 +709,7 @@ BUSCONF
steam_channel=${BL_STEAM_CHANNEL:-steamdeck_publicbeta}
steam_beta=$steam_root/package/beta
if [ -n "${BL_ADDED_GAMES:-}" ] && [ -f "$BL_ADDED_GAMES" ]; then
bannerlator-steam-games watch "$steam_root" "$BL_ADDED_GAMES" &
droiddeck-steam-games watch "$steam_root" "$BL_ADDED_GAMES" &
games_watcher=$!
fi
# Lossless Scaling (Steam app 993090): whether this account owns it, asked of the signed-in
@@ -694,7 +721,7 @@ BUSCONF
for ((_i = 0; _i < 60; _i++)); do
nap 30
[ -e "$owned" ] && exit 0
answer=$(bannerlator-steam-games probe "$steam_root" 993090 2>/dev/null) || continue
answer=$(droiddeck-steam-games probe "$steam_root" 993090 2>/dev/null) || continue
case $answer in
*'"993090": true'*) mkdir -p "${owned%/*}" && : > "$owned" && echo "== Lossless Scaling: owned" ;;
*) echo "== Lossless Scaling: not owned by this account" ;;
@@ -703,7 +730,7 @@ BUSCONF
done
) &
fi
echo "== steam args: ${_bl_ui[*]} -clientbeta $steam_channel -no-cef-sandbox -cef-force-gpu -cef-ozone-platform=x11 -cef-use-gl=angle -cef-use-angle=vulkan $*"
echo "== steam args: ${_bl_ui[*]} ${_bl_cdp[*]} -clientbeta $steam_channel -no-cef-sandbox -cef-force-gpu -cef-ozone-platform=x11 -cef-use-gl=angle -cef-use-angle=vulkan $*"
# An update that cannot install asks for a restart (rc=42) every time; the count is capped so a
# session can never again loop in silence, which reads as a menu opening and closing by itself.
steam_updates=0
@@ -749,14 +776,17 @@ BUSCONF
echo "$steam_channel" > "$steam_beta"
fi
step "starting the Steam client${BL_STEAM_UI:+ ($BL_STEAM_UI UI)}"
"$steam_root/steamrtarm64/steam" "${_bl_ui[@]}" -clientbeta "$steam_channel" \
"$steam_root/steamrtarm64/steam" "${_bl_ui[@]}" "${_bl_cdp[@]}" -clientbeta "$steam_channel" \
-no-cef-sandbox -cef-force-gpu -cef-ozone-platform=x11 \
-cef-use-gl=angle -cef-use-angle=vulkan "$@"
rc=$?
echo "== steam exited rc=$rc"
collect_steam_logs
if [ -f "$HOME/.config/droiddeck/fex-wanted" ] && [ -x /usr/local/bin/droiddeck-fex ]; then
/usr/local/bin/droiddeck-fex status >/dev/null 2>&1 || true
fi
if [ -n "${BL_ADDED_GAMES:-}" ] && [ -f "$BL_ADDED_GAMES" ]; then
bannerlator-steam-shortcuts "$steam_root" "$BL_ADDED_GAMES" || true
droiddeck-steam-shortcuts "$steam_root" "$BL_ADDED_GAMES" || true
fi
if [ -f "$HOME/.bl-stop-requested" ]; then
rm -f "$HOME/.bl-stop-requested" "$HOME/.bl-restart-once"
@@ -787,7 +817,7 @@ BUSCONF
wait "$games_watcher" 2>/dev/null || true
fi
if [ -n "${BL_DESKTOP:-}" ]; then
# Started from the desktop (bannerlator-steam-launch): the desktop's own programs share this
# Started from the desktop (droiddeck-steam-launch): the desktop's own programs share this
# script's parent, so reap_orphans - right under gamescope, where everything is the client's -
# is not for here. The helpers this run started (the cores and redistributables beats, the
# system bus and the NetworkManager stand-in) go with the client instead.
@@ -812,7 +842,7 @@ BUSCONF
exit "$rc"
;;
*)
echo "bannerlator-session: unknown mode '$mode'" >&2
echo "droiddeck-session: unknown mode '$mode'" >&2
exit 64
;;
esac
+495
View File
@@ -0,0 +1,495 @@
#!/usr/bin/python3
import base64
import ctypes
import json
import os
import runpy
import select
import signal
import socket
import struct
import sys
import time
HERE = os.path.dirname(os.path.realpath(__file__))
COMPAT = runpy.run_path(os.path.join(HERE, "steam-compatibility"))
BINDING = "droiddeckCompatChanged"
INSTALL_JS = r"""
(() => {
if (window.__droiddeckCompat) return "present";
const enc = new TextEncoder(), dec = new TextDecoder();
const varint = (value, out) => { let n = BigInt(value); do { let b = Number(n & 0x7fn); n >>= 7n; if (n) b |= 0x80; out.push(b); } while (n); };
const key = (field, wire, out) => varint(field * 8 + wire, out);
const str = (field, s, out) => { const b = enc.encode(s); key(field, 2, out); varint(b.length, out); for (const x of b) out.push(x); };
const uint = (field, v, out) => { key(field, 0, out); varint(v, out); };
const fixed64 = (field, v, out) => { key(field, 1, out); let n = BigInt(v); for (let i = 0; i < 8; i++) { out.push(Number(n & 0xffn)); n >>= 8n; } };
const parse = (bytes) => {
const fields = {};
let i = 0;
const read = () => { let r = 0n, s = 0n, b; do { b = bytes[i++]; r |= BigInt(b & 0x7f) << s; s += 7n; } while (b & 0x80); return r; };
while (i < bytes.length) {
const k = Number(read()), field = k >> 3, wire = k & 7;
let v;
if (wire === 0) v = read();
else if (wire === 1) { v = 0n; for (let j = 0; j < 8; j++) v |= BigInt(bytes[i + j]) << BigInt(8 * j); i += 8; }
else if (wire === 2) { const n = Number(read()); v = bytes.subarray(i, i + n); i += n; }
else if (wire === 5) { v = bytes.subarray(i, i + 4); i += 4; }
else throw new Error("unexpected wire type " + wire);
(fields[field] = fields[field] || []).push(v);
}
return fields;
};
const text = (f, n) => f[n] ? dec.decode(f[n][0]) : "";
const self = { seq: 0, pending: new Map(), ws: null, opening: null, changes: 0 };
const frame = (method, body, auth) => {
const h = [];
fixed64(10, ++self.seq, h);
str(12, method, h);
if (auth) str(40, auth, h);
const out = new Uint8Array(8 + h.length + body.length);
const view = new DataView(out.buffer);
view.setUint32(0, (0x80000000 | 146) >>> 0, true);
view.setUint32(4, h.length, true);
out.set(h, 8);
out.set(body, 8 + h.length);
return [self.seq, out];
};
const onMessage = (event) => {
const bytes = new Uint8Array(event.data);
const length = new DataView(event.data).getUint32(4, true);
const header = parse(bytes.subarray(8, 8 + length));
const target = header[11] ? Number(header[11][0] & 0xffffffffn) : -1;
if (self.pending.has(target)) {
const done = self.pending.get(target);
self.pending.delete(target);
const result = header[13] ? Number(header[13][0]) : 2;
done(result === 1 ? { ok: true, body: parse(bytes.subarray(8 + length)) } : { ok: false, eresult: result, error: text(header, 14) });
return;
}
if (text(header, 12).startsWith("CompatManager.NotifyStateChanged")) {
self.changes++;
if (typeof window.__BINDING__ === "function") window.__BINDING__(String(self.changes));
}
};
const send = (method, body, auth) => new Promise((resolve, reject) => {
const [id, out] = frame(method, Uint8Array.from(body), auth);
const timer = setTimeout(() => { self.pending.delete(id); reject(new Error(method + " timed out")); }, 10000);
self.pending.set(id, (r) => { clearTimeout(timer); resolve(r); });
self.ws.send(out);
});
const open = async () => {
const info = await SteamClient.WebUITransport.GetTransportInfo();
const ws = new WebSocket("ws://127.0.0.1:" + info.portClientdll + "/transportsocket/");
ws.binaryType = "arraybuffer";
await new Promise((resolve, reject) => { ws.onopen = resolve; ws.onerror = () => reject(new Error("the client's transport refused the connection")); });
ws.onmessage = onMessage;
ws.onclose = () => {
if (self.ws === ws) self.ws = null;
for (const done of self.pending.values()) done({ ok: false, eresult: 0, error: "connection closed" });
self.pending.clear();
};
self.ws = ws;
const body = [];
str(1, info.authKeyClientdll, body);
const r = await send("TransportAuth.Authenticate#1", body, info.authKeyClientdll);
if (!r.ok) { ws.close(); throw new Error("the client's transport refused authentication (" + r.eresult + ")"); }
};
const ready = async () => {
if (self.ws && self.ws.readyState === 1) return;
if (!self.opening) self.opening = open().finally(() => { self.opening = null; });
await self.opening;
};
self.get = async (appid) => {
await ready();
const body = [];
uint(1, appid, body);
const r = await send("CompatManager.GetCompatTools#1", body);
if (!r.ok) throw new Error("GetCompatTools " + appid + " failed (" + r.eresult + " " + r.error + ")");
return { selected: text(r.body, 3), fallback: text(r.body, 4), tools: (r.body[1] || []).map((t) => text(parse(t), 1)) };
};
self.set = async (appid, name) => {
await ready();
const body = [];
uint(1, appid, body);
str(2, name, body);
const r = await send("CompatManager.SpecifyCompatTool#1", body);
if (!r.ok) throw new Error("SpecifyCompatTool " + appid + " " + name + " failed (" + r.eresult + " " + r.error + ")");
return true;
};
self.many = async (apps) => {
const out = {};
for (const app of apps) out[app] = (await self.get(Number(app))).selected;
return out;
};
window.__droiddeckCompat = self;
return "installed";
})()
""".replace("__BINDING__", BINDING)
class Closed(Exception):
pass
def say(message):
print("droiddeck-steam-compat: %s" % message, flush=True)
def connect(port, timeout=5):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(timeout)
try:
sock.bind(("127.0.0.1", 0))
sock.connect(("127.0.0.1", port))
except OSError:
sock.close()
raise
return sock
def http_get(port, path):
sock = connect(port)
try:
sock.sendall(("GET %s HTTP/1.1\r\nHost: 127.0.0.1:%d\r\nConnection: close\r\n\r\n" % (path, port)).encode())
data = b""
length = None
while length is None or len(data) < length:
chunk = sock.recv(65536)
if not chunk:
break
data += chunk
if length is None and b"\r\n\r\n" in data:
head, _, rest = data.partition(b"\r\n\r\n")
for line in head.split(b"\r\n")[1:]:
name, _, value = line.partition(b":")
if name.strip().lower() == b"content-length":
length = int(value.strip())
if length is None:
length = float("inf")
data = rest
if not head.startswith(b"HTTP/1.1 200"):
raise OSError("devtools answered %r" % head[:40])
finally:
sock.close()
return json.loads(data.decode("utf-8", "replace"))
class WebSocket:
def __init__(self, port, path):
self.sock = connect(port)
key = base64.b64encode(os.urandom(16)).decode()
self.sock.sendall(("GET %s HTTP/1.1\r\nHost: 127.0.0.1:%d\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n"
"Sec-WebSocket-Key: %s\r\nSec-WebSocket-Version: 13\r\n\r\n" % (path, port, key)).encode())
self.buffer = b""
while b"\r\n\r\n" not in self.buffer:
self.fill()
head, _, self.buffer = self.buffer.partition(b"\r\n\r\n")
if not head.startswith(b"HTTP/1.1 101"):
raise OSError("devtools refused the websocket: %r" % head[:60])
self.sock.settimeout(None)
def fileno(self):
return self.sock.fileno()
def fill(self):
chunk = self.sock.recv(65536)
if not chunk:
raise Closed("devtools closed the connection")
self.buffer += chunk
def take(self, n):
while len(self.buffer) < n:
self.fill()
data, self.buffer = self.buffer[:n], self.buffer[n:]
return data
def send(self, text, opcode=1):
payload = text.encode() if isinstance(text, str) else text
mask = os.urandom(4)
n = len(payload)
if n < 126:
head = struct.pack("!BB", 0x80 | opcode, 0x80 | n)
elif n < 65536:
head = struct.pack("!BBH", 0x80 | opcode, 0xFE, n)
else:
head = struct.pack("!BBQ", 0x80 | opcode, 0xFF, n)
self.sock.sendall(head + mask + bytes(b ^ mask[i % 4] for i, b in enumerate(payload)))
def pending(self):
return len(self.buffer) > 0
def recv(self):
message = b""
while True:
first, second = self.take(2)
n = second & 0x7F
if n == 126:
n = struct.unpack("!H", self.take(2))[0]
elif n == 127:
n = struct.unpack("!Q", self.take(8))[0]
mask = self.take(4) if second & 0x80 else None
payload = self.take(n)
if mask:
payload = bytes(b ^ mask[i % 4] for i, b in enumerate(payload))
opcode = first & 0x0F
if opcode == 8:
raise Closed("devtools closed the websocket")
if opcode == 9:
self.send(payload, 10)
continue
if opcode == 10:
continue
message += payload
if first & 0x80:
return message.decode("utf-8", "replace")
def close(self):
try:
self.sock.close()
except OSError:
pass
class Page:
def __init__(self, port, path):
self.ws = WebSocket(port, path)
self.next_id = 0
self.events = []
def call(self, method, params=None, timeout=30):
self.next_id += 1
mine = self.next_id
self.ws.send(json.dumps({"id": mine, "method": method, "params": params or {}}))
deadline = time.monotonic() + timeout
while True:
if not self.ws.pending():
left = deadline - time.monotonic()
if left <= 0 or not select.select([self.ws], [], [], left)[0]:
raise TimeoutError("%s timed out" % method)
message = json.loads(self.ws.recv())
if message.get("id") == mine:
if "error" in message:
raise RuntimeError("%s: %s" % (method, message["error"].get("message")))
return message.get("result", {})
if "method" in message:
self.events.append(message)
def evaluate(self, expression, timeout=30):
result = self.call("Runtime.evaluate", {"expression": expression, "awaitPromise": True, "returnByValue": True}, timeout)
if "exceptionDetails" in result:
details = result["exceptionDetails"]
raise RuntimeError((details.get("exception") or {}).get("description") or details.get("text") or "script failed")
return result.get("result", {}).get("value")
def compat(self, call):
return self.evaluate("%s;\nwindow.__droiddeckCompat.%s" % (INSTALL_JS, call))
def drain(self):
while True:
if not self.ws.pending() and not select.select([self.ws], [], [], 0)[0]:
break
message = json.loads(self.ws.recv())
if "method" in message:
self.events.append(message)
events, self.events = self.events, []
return events
def shared_context(port):
for target in http_get(port, "/json/list"):
if target.get("title") == "SharedJSContext" and target.get("webSocketDebuggerUrl"):
path = target["webSocketDebuggerUrl"].split("/", 3)[3]
return "/" + path
return None
class Watcher:
MASK = 0x00000008 | 0x00000080 | 0x00000100 | 0x00000200
def __init__(self, files):
self.files = files
self.fd = -1
try:
libc = ctypes.CDLL(None, use_errno=True)
fd = libc.inotify_init1(os.O_NONBLOCK | os.O_CLOEXEC)
dirs = sorted({os.path.dirname(f) for f in files if os.path.isdir(os.path.dirname(f))})
if fd >= 0 and dirs and all(libc.inotify_add_watch(fd, d.encode(), self.MASK) >= 0 for d in dirs):
self.fd = fd
elif fd >= 0:
os.close(fd)
except (OSError, AttributeError):
self.fd = -1
self.stamp = self.snapshot()
def snapshot(self):
stamp = []
for f in self.files:
for p in (f, os.path.dirname(f)):
try:
stamp.append(os.stat(p).st_mtime_ns)
except OSError:
stamp.append(0)
return stamp
def fileno(self):
return self.fd
def changed(self):
if self.fd < 0:
stamp = self.snapshot()
moved, self.stamp = stamp != self.stamp, stamp
return moved
names = set()
try:
while True:
data = os.read(self.fd, 65536)
if not data:
break
at = 0
while at + 16 <= len(data):
_, _, _, size = struct.unpack_from("iIII", data, at)
names.add(data[at + 16:at + 16 + size].split(b"\0", 1)[0].decode("utf-8", "replace"))
at += 16 + size
except BlockingIOError:
pass
wanted = {os.path.basename(f) for f in self.files}
return any(n in wanted or n.startswith("appmanifest_") for n in names)
class Helper:
def __init__(self, steam_root, port):
self.steam_root = steam_root
self.port = port
self.page = None
self.known = set()
self.installed = None
self.fex_asked = set()
def attach(self):
path = shared_context(self.port)
if path is None:
raise LookupError("no SharedJSContext yet")
page = Page(self.port, path)
try:
if page.evaluate("typeof SteamClient === 'object' && typeof SteamClient.WebUITransport === 'object'") is not True:
raise LookupError("SteamClient is not ready")
page.call("Runtime.addBinding", {"name": BINDING})
page.compat("get(0)")
except Exception:
page.ws.close()
raise
self.page = page
return True
def reconcile(self):
page = self.page
with COMPAT["locked"]():
state_file = COMPAT["state_path"]("state.json")
state = COMPAT["read_json"](state_file)
request = COMPAT["read_json"](COMPAT["state_path"]("request.json"))
catalog = COMPAT["tool_catalog"](self.steam_root)
top = page.compat("get(0)")
self.known = set(top["tools"])
live = [t for t in catalog if t["name"] in self.known]
wanted = COMPAT["requested_tool"](request, catalog)
newer = isinstance(request.get("seq"), int) and request["seq"] > int(state.get("applied", 0) or 0)
waiting = newer and wanted is not None and wanted not in self.known
default, state = COMPAT["choose_default"](top["selected"], state, request, live, apply_request=not waiting)
installed = COMPAT["installed_apps"](self.steam_root)
self.installed = installed
apps = sorted(set(installed) | set(state.get("auto", {})), key=COMPAT["app_order"])
current = dict(page.compat("many(%s)" % json.dumps(apps)) or {})
current["0"] = top["selected"]
native = COMPAT["native_targets"](self.steam_root, installed)
changes, auto = COMPAT["plan_mapping"](current, installed, [t["name"] for t in live], default, COMPAT["followers"](state, current), native)
for app, tool in sorted(changes.items(), key=lambda item: COMPAT["app_order"](item[0])):
page.compat("set(%d, %s)" % (int(app), json.dumps(tool)))
say("%s -> %s" % ("default" if app == "0" else "app %s" % app, tool))
state.update(auto=auto, tools=catalog, live=True)
if waiting:
state["pending"] = wanted
say("%s is new to the running client; it applies when Steam next starts" % wanted)
else:
state.pop("pending", None)
COMPAT["write_json"](state_file, state)
if native or os.path.exists(COMPAT["fex_wanted_marker"]()):
for app in COMPAT["request_fex"](self.steam_root, self.fex_asked):
say("asked the client to install %s for x86 Linux titles and programs" % app)
def request_waiting(self):
request = COMPAT["read_json"](COMPAT["state_path"]("request.json"))
state = COMPAT["read_json"](COMPAT["state_path"]("state.json"))
return isinstance(request.get("seq"), int) and request["seq"] > int(state.get("applied", 0) or 0)
def serve(self, watcher):
with open(COMPAT["state_path"]("live.pid"), "w") as f:
f.write("%d\n" % os.getpid())
say("attached to the client on port %d" % self.port)
self.reconcile()
while True:
sources = [self.page.ws] + ([watcher] if watcher.fileno() >= 0 else [])
ready = [self.page.ws] if self.page.ws.pending() else select.select(sources, [], [], 30 if watcher.fileno() >= 0 else 2)[0]
due = False
if self.page.ws in ready:
due = any(e.get("method") == "Runtime.bindingCalled" and e.get("params", {}).get("name") == BINDING
for e in self.page.drain())
if (watcher in ready or watcher.fileno() < 0) and watcher.changed():
due = self.installed != COMPAT["installed_apps"](self.steam_root) or self.request_waiting()
if due:
time.sleep(0.3)
if watcher.fileno() >= 0 and select.select([watcher], [], [], 0)[0]:
watcher.changed()
self.page.drain()
self.reconcile()
def detach():
try:
os.unlink(COMPAT["state_path"]("live.pid"))
except OSError:
pass
with COMPAT["locked"]():
state_file = COMPAT["state_path"]("state.json")
state = COMPAT["read_json"](state_file)
if state.get("live"):
state["live"] = False
COMPAT["write_json"](state_file, state)
def main():
steam_root = sys.argv[1]
try:
port = int(os.environ.get("BL_CDP_PORT", ""))
except ValueError:
say("no devtools port for this session; the client's mapping is kept through config.vdf")
return 0
os.makedirs(COMPAT["COMPAT_DIR"], exist_ok=True)
watcher = Watcher([COMPAT["state_path"]("request.json")] + [os.path.join(lib, "steamapps", "libraryfolders.vdf") for lib in (steam_root, COMPAT["LIBRARY"] + "-sd")])
helper = Helper(steam_root, port)
failures = 0
last = None
while True:
try:
helper.attach()
failures = 0
last = None
helper.serve(watcher)
except Exception as e:
if helper.page is not None:
say("lost the client (%s); reattaching" % e)
helper.page.ws.close()
helper.page = None
detach()
elif str(e) != last:
say("waiting for the client (%s)" % e)
last = str(e)
failures += 1
time.sleep(min(2 * max(failures, 1), 10))
if __name__ == "__main__":
signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
try:
sys.exit(main())
finally:
detach()
@@ -239,13 +239,13 @@ def watch(steam_root, listing):
status = last_status
if pending and pending != requested and not game_running() and downloads_idle(steam_root):
requested = pending
print("bannerlator-steam-games: restarting the client once to import %s" % pending, flush=True)
print("droiddeck-steam-games: restarting the client once to import %s" % pending, flush=True)
try:
request_restart(steam_root)
except (OSError, subprocess.TimeoutExpired) as error:
print("bannerlator-steam-games: restart unavailable: %s" % error, flush=True)
print("droiddeck-steam-games: restart unavailable: %s" % error, flush=True)
if status != last_status:
print("bannerlator-steam-games: " + status, flush=True)
print("droiddeck-steam-games: " + status, flush=True)
last_status = status
time.sleep(30)
@@ -357,7 +357,7 @@ def route(steam_root, acct, games):
routes[str(game["appid"])] = appid
continue
except OSError as error:
print(f"bannerlator-steam-games: import unavailable ({error})")
print(f"droiddeck-steam-games: import unavailable ({error})")
shortcuts.append(game)
save_json(Path(acct) / "config/.droiddeck-routes.json", routes)
return shortcuts, routes
@@ -6,11 +6,15 @@ cdn=https://client-update.fastly.steamstatic.com
steam_channel=${BL_STEAM_CHANNEL:-steamdeck_publicbeta}
case "$steam_channel" in
publicbeta|steamdeck_publicbeta) ;;
*) echo "bannerlator-steam-install: unsupported Steam channel: $steam_channel" >&2; exit 1 ;;
*) echo "droiddeck-steam-install: unsupported Steam channel: $steam_channel" >&2; exit 1 ;;
esac
manifest=steam_client_${steam_channel}_linuxarm64
steam_root=${XDG_DATA_HOME:-$HOME/.local/share}/Steam
stamp=$steam_root/package/bannerlator-installed
stamp=$steam_root/package/droiddeck-installed
legacy_stamp=$steam_root/package/bannerlator-installed
if [ -f "$legacy_stamp" ] && [ ! -f "$stamp" ]; then
mv -f "$legacy_stamp" "$stamp" || true
fi
[ -x "$steam_root/steamrtarm64/steam" ] && [ -f "$stamp" ] && exit 0
work=$(mktemp -d)
@@ -24,7 +28,7 @@ awk -F'"' '
$2 == "sha2" { sha[block] = $4 }
END { for (b in file) if (b ~ /_all$/ || b ~ /_linuxarm64_linuxarm64$/) print b, file[b], sha[b] }
' "$work/manifest" | sort > "$work/parts"
[ -s "$work/parts" ] || { echo "bannerlator-steam-install: no client components in $manifest" >&2; exit 1; }
[ -s "$work/parts" ] || { echo "droiddeck-steam-install: no client components in $manifest" >&2; exit 1; }
total=$(grep -c . "$work/parts" 2>/dev/null || echo 0)
mkdir -p "$steam_root/package"
@@ -1,12 +1,12 @@
#!/bin/bash
# Starts the Steam client from inside a session (the desktop's menu entries and the per-game
# launchers bannerlator-desktop-games writes):
# bannerlator-steam-launch desktop [appid|steam-url] Steam's desktop UI, optionally a game
# bannerlator-steam-launch bigpicture [appid|steam-url] Big Picture
# launchers droiddeck-desktop-games writes):
# droiddeck-steam-launch desktop [appid|steam-url] Steam's desktop UI, optionally a game
# droiddeck-steam-launch bigpicture [appid|steam-url] Big Picture
#
# On a desktop that draws with the GPU (labwc on vulkan or gles2 - the default since the GPU
# desktop) the client runs right there: a window on the desktop, through the desktop's Xwayland,
# with the same set-up a Steam session gives it (bannerlator-session steam, minus gamescope), and
# with the same set-up a Steam session gives it (droiddeck-session steam, minus gamescope), and
# its games open in windows of their own. A client already running there gets the request instead
# of a second one being started.
#
@@ -35,24 +35,24 @@ if [ "${BL_DESKTOP:-}" = 1 ] && [ -n "${WLR_RENDERER:-}" ] && [ "$WLR_RENDERER"
# One client at a time: the running one takes the request (a second start hands its command
# line over and exits). No game asked for = switch it to the UI that was clicked.
[ -n "$url" ] || { [ "$ui" = bigpicture ] && url=steam://open/bigpicture || url=steam://open/main; }
echo "bannerlator-steam-launch: the client is running on the desktop; handing it $url" | tee -a "$log"
echo "droiddeck-steam-launch: the client is running on the desktop; handing it $url" | tee -a "$log"
exec "$client" "$url"
fi
echo "bannerlator-steam-launch: starting the Steam client on the desktop ($ui${url:+, $url}); log: $log"
echo "droiddeck-steam-launch: starting the Steam client on the desktop ($ui${url:+, $url}); log: $log"
# BL_INSIDE: the session script's Steam start-up without gamescope in front of it. The desktop UI
# unless Big Picture was asked for; never Deck mode (its Quick Access Menu needs gamescope).
export BL_INSIDE=1 BL_STEAMDECK=0
if [ "$ui" = desktop ]; then export BL_STEAM_UI=desktop; else unset BL_STEAM_UI; fi
setsid /usr/local/bin/bannerlator-session steam ${url:+"$url"} >>"$log" 2>&1 < /dev/null &
setsid /usr/local/bin/droiddeck-session steam ${url:+"$url"} >>"$log" 2>&1 < /dev/null &
exit 0
fi
# The software desktop, or outside the desktop: ask the app for a Steam session.
dir=${BL_LAUNCH_DIR:-}
if [ -z "$dir" ] || [ ! -d "$dir" ]; then
echo "bannerlator-steam-launch: not inside a DroidDeck session (BL_LAUNCH_DIR unset)" >&2
echo "droiddeck-steam-launch: not inside a DroidDeck session (BL_LAUNCH_DIR unset)" >&2
exit 69
fi
# Written whole, then renamed: the app reads it on close, never half-written.
printf 'ui=%s\nurl=%s\n' "$ui" "$url" > "$dir/steam-launch.tmp" && mv -f "$dir/steam-launch.tmp" "$dir/steam-launch"
echo "bannerlator-steam-launch: asked the app for Steam ($ui${url:+, $url})"
echo "droiddeck-steam-launch: asked the app for Steam ($ui${url:+, $url})"
@@ -2,7 +2,7 @@
# Registers the card the app installs to (bound at /mnt/droiddeck-sd) as a library folder of the
# native Steam client, so they show as installed there instead of being downloaded a second time.
#
# bannerlator-steam-library <steam root>
# droiddeck-steam-library <steam root>
#
# Ported from WinNative's gamescope runtime (GPL-3.0).
import os
@@ -61,7 +61,7 @@ def register(steam_root, library, label):
os.makedirs(steamapps_dir, exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
f.write(text)
print("bannerlator-steam-library: registered %s with %d game(s)" % (library, len(apps)))
print("droiddeck-steam-library: registered %s with %d game(s)" % (library, len(apps)))
return len(apps)
@@ -85,7 +85,7 @@ def rename(steam_root, old, new):
return
with open(path, "w", encoding="utf-8") as f:
f.write(old_path.sub(lambda m: m.group(1) + new + m.group(2), text))
print("bannerlator-steam-library: moved %s to %s" % (old, new))
print("droiddeck-steam-library: moved %s to %s" % (old, new))
def forget(steam_root, library):
@@ -115,7 +115,7 @@ def forget(steam_root, library):
if text != "\n".join(lines):
with open(path, "w", encoding="utf-8") as f:
f.write(text)
print("bannerlator-steam-library: forgot %s" % library)
print("droiddeck-steam-library: forgot %s" % library)
def relabel(steam_root, target, label):
@@ -151,7 +151,7 @@ def relabel(steam_root, target, label):
if changed:
with open(path, "w", encoding="utf-8") as f:
f.write("\n".join(out))
print("bannerlator-steam-library: labelled %s %r" % (target, label))
print("droiddeck-steam-library: labelled %s %r" % (target, label))
# A library also carries its own libraryfolder.vdf, and the client takes the label from there
# back into the list above - so editing the list alone is undone on the next start. The
# client's own library has no such file, which is why that one kept the new name and the card
@@ -165,7 +165,7 @@ def relabel(steam_root, target, label):
if new != text:
with open(marker, "w", encoding="utf-8") as f:
f.write(new)
print("bannerlator-steam-library: labelled %s in its own marker too" % target)
print("droiddeck-steam-library: labelled %s in its own marker too" % target)
def main():
@@ -1,7 +1,7 @@
#!/bin/bash
# `steam` on the desktop. The client writes a menu entry per installed game that runs
# `steam steam://rungameid/<id>`, and its own launchers say `steam`. On the desktop that goes
# through bannerlator-steam-launch - which runs the client on the desktop itself when it draws with
# through droiddeck-steam-launch - which runs the client on the desktop itself when it draws with
# the GPU, hands the URL to a client already running there, and asks the app for a Steam session
# on a software desktop. Anywhere else it is the client itself.
set -u
@@ -14,6 +14,6 @@ if [ "${BL_DESKTOP:-}" = 1 ]; then
-gamepadui|-bigpicture) ui=bigpicture ;;
esac
done
exec /usr/local/bin/bannerlator-steam-launch "$ui" ${url:+"$url"}
exec /usr/local/bin/droiddeck-steam-launch "$ui" ${url:+"$url"}
fi
exec "${HOME:-/root}/.local/share/Steam/steamrtarm64/steam" "$@"
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
"""Import added games while Steam is closed, preserving user-created shortcuts.
Usage: bannerlator-steam-shortcuts <steam_root> <added-games.json>
Usage: droiddeck-steam-shortcuts <steam_root> <added-games.json>
"""
import importlib.machinery
import importlib.util
@@ -163,7 +163,7 @@ def install_art(acct, games):
os.replace(dst + ".tmp", dst)
copied += 1
except OSError as e:
print("bannerlator-steam-shortcuts: could not copy art %s (%s)" % (name, e))
print("droiddeck-steam-shortcuts: could not copy art %s (%s)" % (name, e))
for name in list(manifest):
if name not in wanted:
try:
@@ -173,7 +173,7 @@ def install_art(acct, games):
with open(manifest_path, "w", encoding="utf-8") as f:
json.dump(wanted, f)
if copied:
print("bannerlator-steam-shortcuts: %d piece(s) of art placed" % copied)
print("droiddeck-steam-shortcuts: %d piece(s) of art placed" % copied)
def main():
@@ -185,11 +185,11 @@ def main():
with open(listing, encoding="utf-8") as f:
games = json.load(f)
except (OSError, ValueError) as e:
print("bannerlator-steam-shortcuts: no list (%s)" % e)
print("droiddeck-steam-shortcuts: no list (%s)" % e)
return 0
acct = account_dir(steam_root)
if not acct:
print("bannerlator-steam-shortcuts: no signed-in account yet; nothing written")
print("droiddeck-steam-shortcuts: no signed-in account yet; nothing written")
return 0
config = os.path.join(acct, "config")
os.makedirs(config, exist_ok=True)
@@ -200,9 +200,9 @@ def main():
with open(path, "rb") as f:
existing = parse(f.read()).get("shortcuts", {})
except (ValueError, IndexError, struct.error) as e:
print("bannerlator-steam-shortcuts: could not read the existing file (%s); keeping it unchanged" % e)
print("droiddeck-steam-shortcuts: could not read the existing file (%s); keeping it unchanged" % e)
return 1
helper = os.path.join(os.path.dirname(os.path.abspath(__file__)), "bannerlator-steam-games")
helper = os.path.join(os.path.dirname(os.path.abspath(__file__)), "droiddeck-steam-games")
loader = importlib.machinery.SourceFileLoader("steam_games", helper)
spec = importlib.util.spec_from_loader(loader.name, loader)
routing = importlib.util.module_from_spec(spec)
@@ -225,25 +225,26 @@ def main():
entry["LaunchOptions"] = previous["LaunchOptions"]
ours.append(entry)
entries = {str(i): v for i, v in enumerate(kept + ours)}
tmp = path + ".bannerlator"
tmp = path + ".droiddeck"
with open(tmp, "wb") as f:
f.write(write({"shortcuts": entries}))
os.replace(tmp, path)
print("bannerlator-steam-shortcuts: %d added game(s) written, %d of the client's own kept" % (len(ours), len(kept)))
print("droiddeck-steam-shortcuts: %d added game(s) written, %d of the client's own kept" % (len(ours), len(kept)))
install_art(acct, [g for g in games if g.get("exe") and g.get("name")])
# Each added game runs under the ARM64 Proton: the compat registrar's own mapping writer.
try:
# No .py suffix on the registrar, so the loader has to be named.
compat_path = "/usr/local/bin/bannerlator-steam-compat"
compat_path = "/usr/local/bin/steam-compatibility"
spec = importlib.util.spec_from_file_location("compat", compat_path, loader=importlib.machinery.SourceFileLoader("compat", compat_path))
compat = importlib.util.module_from_spec(spec)
spec.loader.exec_module(compat)
cfg = os.path.join(steam_root, "config", "config.vdf")
if os.path.isfile(cfg) and (ours or routes):
compat.register_default(cfg, apps=[str(int(g["appid"]) & 0xFFFFFFFF) for g in games if g.get("exe")] + [str(appid) for appid in set(routes.values())])
print("bannerlator-steam-shortcuts: mapped them to %s" % compat.TOOL)
compat.register_default(cfg, apps=[str(int(g["appid"]) & 0xFFFFFFFF) for g in games if g.get("exe")] + [str(appid) for appid in set(routes.values())],
protect=compat.adopt_extras(os.path.join(steam_root, "compatibilitytools.d")))
print("droiddeck-steam-shortcuts: mapped them to %s" % compat.TOOL)
except Exception as e: # noqa: BLE001 - the games still show; the client falls back to the default tool
print("bannerlator-steam-shortcuts: compat mapping skipped (%s)" % e)
print("droiddeck-steam-shortcuts: compat mapping skipped (%s)" % e)
return 0
File diff suppressed because it is too large. Load diff
+653
View File
@@ -0,0 +1,653 @@
/* droiddeck-fsync: futex_waitv(2) in userspace, so Proton's own fsync runs where the app sandbox refuses the syscall.
* Credit: fsync, the futex-based synchronization in Proton, by Elizabeth Figura and Paul Gofman;
* futex_waitv(2) by André Almeida. */
#define _GNU_SOURCE
#include <dlfcn.h>
#include <errno.h>
#include <fcntl.h>
#include <linux/futex.h>
#include <pthread.h>
#include <signal.h>
#include <stdlib.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <time.h>
#include <unistd.h>
#ifndef SYS_futex_waitv
#define SYS_futex_waitv 449
#endif
#define FS_SLOTS 2048u
#define FS_WORDS (FS_SLOTS / 64u)
#define FS_BUCKETS 4096u
#define FS_KEYS 128u
#define FS_NOKEY 0xffffffffu
#define FS_SIZE32 2u
#define FS_PRIVATE 128u
#define FS_MAGIC 0x434e595346444444ull
#define FS_SWEEP_NS 1000000000ull
#define FS_POLL_NS 500000l
#define FS_NAP_MIN 10000000ull
#define FS_NAP_MAX 1000000000ull
#define FS_HEAD_SIZE 4096ul
struct fs_waitv {
uint64_t val;
uint64_t uaddr;
uint32_t flags;
uint32_t reserved;
};
struct fs_bucket {
uint64_t bits[FS_WORDS];
};
struct fs_slot {
uint32_t futex;
uint32_t count;
uint64_t owner;
uint32_t keys[FS_KEYS];
};
struct fs_head {
uint64_t magic;
uint32_t sweeper;
uint32_t pad;
uint64_t sweep_at;
uint64_t used[FS_WORDS];
};
#define FS_BUCKET_OFF FS_HEAD_SIZE
#define FS_SLOT_OFF (FS_BUCKET_OFF + (size_t)FS_BUCKETS * sizeof(struct fs_bucket))
#define FS_REGION (FS_SLOT_OFF + (size_t)FS_SLOTS * sizeof(struct fs_slot))
_Static_assert(sizeof(struct fs_head) <= FS_HEAD_SIZE, "fsync registry head");
struct fs_range {
uintptr_t lo;
uintptr_t hi;
uint64_t off;
};
struct fs_map {
uint32_t n;
struct fs_range r[];
};
static int fs_state = -1;
static char *fs_reg;
static int fs_fd = -1;
static struct fs_map *fs_cur;
static uint32_t fs_pid;
static pthread_mutex_t fs_mx = PTHREAD_MUTEX_INITIALIZER;
static pthread_key_t fs_key;
static pthread_once_t fs_once = PTHREAD_ONCE_INIT;
static __thread int32_t fs_tslot = -1;
static __thread int fs_busy;
static __thread uint32_t fs_tid;
static __thread uint32_t fs_nap_key;
static __thread uint32_t fs_nap_n;
static __thread uint64_t fs_nap_ns;
static __thread sigset_t fs_fork_mask;
static long (*fs_real_syscall)(long, ...);
static int (*fs_real_shm_open)(const char *, int, mode_t);
static int (*fs_real_shm_unlink)(const char *);
static void *(*fs_real_mmap)(void *, size_t, int, int, int, off_t);
static void *(*fs_real_mmap64)(void *, size_t, int, int, int, off64_t);
static int (*fs_real_munmap)(void *, size_t);
static int fs_on(void) {
if (fs_state < 0) {
const char *e = getenv("BL_FSYNC");
fs_state = e && e[0] == '1';
}
return fs_state;
}
static long fs_sys(long n, long a, long b, long c, long d, long e, long f) {
if (!fs_real_syscall) fs_real_syscall = (long (*)(long, ...))dlsym(RTLD_NEXT, "syscall");
return fs_real_syscall(n, a, b, c, d, e, f);
}
static void *fs_anon(size_t size) {
if (!fs_real_mmap) fs_real_mmap = (void *(*)(void *, size_t, int, int, int, off_t))dlsym(RTLD_NEXT, "mmap");
return fs_real_mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
}
static uint32_t fs_self(void) {
if (!fs_tid) fs_tid = (uint32_t)fs_sys(SYS_gettid, 0, 0, 0, 0, 0, 0);
return fs_tid;
}
static uint32_t fs_getpid(void) {
if (!fs_pid) fs_pid = (uint32_t)getpid();
return fs_pid;
}
static uint64_t fs_now(void) {
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
return (uint64_t)ts.tv_sec * 1000000000ull + (uint64_t)ts.tv_nsec;
}
static int fs_dead(uint32_t pid, uint32_t tid) {
int saved = errno, dead;
dead = pid && fs_sys(SYS_tgkill, pid, tid, 0, 0, 0, 0) == -1 && errno == ESRCH;
errno = saved;
return dead;
}
static uint64_t fs_owner(void) {
return (uint64_t)fs_getpid() << 32 | fs_self();
}
static void fs_block(sigset_t *old) {
sigset_t all;
sigfillset(&all);
pthread_sigmask(SIG_SETMASK, &all, old);
}
static inline struct fs_head *fs_head(void) {
return (struct fs_head *)fs_reg;
}
static inline struct fs_bucket *fs_bucket(uint32_t key) {
return (struct fs_bucket *)(fs_reg + FS_BUCKET_OFF) + ((key >> 4) & (FS_BUCKETS - 1));
}
static inline struct fs_slot *fs_slot(uint32_t s) {
return (struct fs_slot *)(fs_reg + FS_SLOT_OFF) + s;
}
static int fs_key_of(uint64_t addr, uint32_t *key) {
struct fs_map *m = __atomic_load_n(&fs_cur, __ATOMIC_ACQUIRE);
uint32_t lo = 0, hi;
if (!m) return 0;
hi = m->n;
while (lo < hi) {
uint32_t mid = (lo + hi) / 2;
if (addr < m->r[mid].lo) {
hi = mid;
} else if (addr >= m->r[mid].hi) {
lo = mid + 1;
} else {
uint64_t k = m->r[mid].off + (addr - m->r[mid].lo);
if (k >= FS_NOKEY) return 0;
*key = (uint32_t)k;
return 1;
}
}
return 0;
}
static int fs_overlaps(uintptr_t lo, uintptr_t hi) {
struct fs_map *m = __atomic_load_n(&fs_cur, __ATOMIC_ACQUIRE);
if (!m) return 0;
for (uint32_t i = 0; i < m->n; i++)
if (m->r[i].lo < hi && m->r[i].hi > lo) return 1;
return 0;
}
static void fs_rebuild_locked(uintptr_t lo, uintptr_t hi, const struct fs_range *add) {
struct fs_map *old, *m;
uint32_t n, k = 0;
int added = 0;
old = fs_cur;
n = old ? old->n : 0;
m = fs_anon(sizeof(*m) + (n + 1) * sizeof(struct fs_range));
if (m == MAP_FAILED) return;
for (uint32_t i = 0; i < n; i++) {
const struct fs_range *r = &old->r[i];
if (r->lo < hi && r->hi > lo) continue;
if (add && !added && add->lo < r->lo) {
m->r[k++] = *add;
added = 1;
}
m->r[k++] = *r;
}
if (add && !added) m->r[k++] = *add;
m->n = k;
__atomic_store_n(&fs_cur, m, __ATOMIC_RELEASE);
}
static void fs_rebuild(uintptr_t lo, uintptr_t hi, const struct fs_range *add) {
sigset_t old;
fs_block(&old);
pthread_mutex_lock(&fs_mx);
fs_rebuild_locked(lo, hi, add);
pthread_mutex_unlock(&fs_mx);
pthread_sigmask(SIG_SETMASK, &old, NULL);
}
static void fs_track(void *p, size_t len, int flags, int fd, off_t off) {
uintptr_t lo = (uintptr_t)p, hi = lo + len;
int saved = errno;
if (fs_fd >= 0 && fd == fs_fd && (flags & MAP_SHARED) && off >= 0) {
struct fs_range r = {lo, hi, (uint64_t)off};
fs_rebuild(lo, hi, &r);
} else if (fs_overlaps(lo, hi)) {
fs_rebuild(lo, hi, NULL);
}
errno = saved;
}
static void fs_unregister(uint32_t s, const uint32_t *keys, uint32_t n) {
uint64_t bit = 1ull << (s & 63);
for (uint32_t i = 0; i < n; i++) {
struct fs_bucket *b;
if (keys[i] == FS_NOKEY) continue;
b = fs_bucket(keys[i]);
__atomic_and_fetch(&b->bits[s >> 6], ~bit, __ATOMIC_SEQ_CST);
}
}
static void fs_release(uint32_t s, uint64_t owner) {
struct fs_slot *sl = fs_slot(s);
uint32_t n, keys[FS_KEYS];
if (!owner || !__atomic_compare_exchange_n(&sl->owner, &owner, 0, 0, __ATOMIC_ACQ_REL, __ATOMIC_RELAXED)) return;
n = __atomic_load_n(&sl->count, __ATOMIC_RELAXED);
if (n > FS_KEYS) n = FS_KEYS;
for (uint32_t i = 0; i < n; i++) keys[i] = __atomic_load_n(&sl->keys[i], __ATOMIC_RELAXED);
fs_unregister(s, keys, n);
__atomic_store_n(&sl->count, 0, __ATOMIC_RELAXED);
__atomic_and_fetch(&fs_head()->used[s >> 6], ~(1ull << (s & 63)), __ATOMIC_RELEASE);
}
static void fs_sweep(int force) {
struct fs_head *h = fs_head();
uint64_t now = fs_now();
uint32_t me = fs_getpid(), cur = 0;
if (!force && now < __atomic_load_n(&h->sweep_at, __ATOMIC_RELAXED)) return;
if (!__atomic_compare_exchange_n(&h->sweeper, &cur, me, 0, __ATOMIC_ACQUIRE, __ATOMIC_RELAXED)) {
if (!fs_dead(cur, cur) || !__atomic_compare_exchange_n(&h->sweeper, &cur, me, 0, __ATOMIC_ACQUIRE, __ATOMIC_RELAXED))
return;
}
__atomic_store_n(&h->sweep_at, now + FS_SWEEP_NS, __ATOMIC_RELAXED);
for (uint32_t w = 0; w < FS_WORDS; w++) {
uint64_t u = __atomic_load_n(&h->used[w], __ATOMIC_ACQUIRE);
while (u) {
uint32_t s = w * 64 + (uint32_t)__builtin_ctzll(u);
struct fs_slot *sl = fs_slot(s);
u &= u - 1;
uint64_t owner = __atomic_load_n(&sl->owner, __ATOMIC_ACQUIRE);
if (fs_dead((uint32_t)(owner >> 32), (uint32_t)owner)) fs_release(s, owner);
}
}
__atomic_store_n(&h->sweeper, 0, __ATOMIC_RELEASE);
}
static int32_t fs_alloc(void) {
struct fs_head *h = fs_head();
fs_sweep(0);
for (int pass = 0; pass < 2; pass++) {
for (uint32_t w = 0; w < FS_WORDS; w++) {
uint64_t u = __atomic_load_n(&h->used[w], __ATOMIC_RELAXED);
while (~u) {
uint32_t b = (uint32_t)__builtin_ctzll(~u);
if (__atomic_compare_exchange_n(&h->used[w], &u, u | (1ull << b), 0, __ATOMIC_ACQ_REL, __ATOMIC_RELAXED)) {
uint32_t s = w * 64 + b;
struct fs_slot *sl = fs_slot(s);
__atomic_store_n(&sl->count, 0, __ATOMIC_RELAXED);
__atomic_store_n(&sl->owner, fs_owner(), __ATOMIC_RELEASE);
return (int32_t)s;
}
}
}
fs_sweep(1);
}
return -1;
}
static void fs_thread_gone(void *p) {
uint32_t s = (uint32_t)((uintptr_t)p - 1);
if (fs_reg && s < FS_SLOTS && fs_tslot == (int32_t)s) {
fs_tslot = -1;
fs_busy = 0;
fs_release(s, fs_owner());
}
}
static void fs_key_init(void) {
pthread_key_create(&fs_key, fs_thread_gone);
}
static void fs_fork_prepare(void) {
fs_block(&fs_fork_mask);
pthread_mutex_lock(&fs_mx);
}
static void fs_fork_parent(void) {
pthread_mutex_unlock(&fs_mx);
pthread_sigmask(SIG_SETMASK, &fs_fork_mask, NULL);
}
static void fs_child(void) {
pthread_mutex_init(&fs_mx, NULL);
pthread_sigmask(SIG_SETMASK, &fs_fork_mask, NULL);
fs_pid = 0;
fs_tid = 0;
fs_busy = 0;
if (fs_tslot >= 0) {
fs_tslot = -1;
pthread_setspecific(fs_key, NULL);
}
}
__attribute__((constructor)) static void fs_init(void) {
if (!fs_real_mmap) fs_real_mmap = (void *(*)(void *, size_t, int, int, int, off_t))dlsym(RTLD_NEXT, "mmap");
if (!fs_real_mmap64) fs_real_mmap64 = (void *(*)(void *, size_t, int, int, int, off64_t))dlsym(RTLD_NEXT, "mmap64");
if (!fs_real_munmap) fs_real_munmap = (int (*)(void *, size_t))dlsym(RTLD_NEXT, "munmap");
if (fs_on()) pthread_atfork(fs_fork_prepare, fs_fork_parent, fs_child);
}
static int32_t fs_take(int *temp) {
if (fs_busy) {
*temp = 1;
return fs_alloc();
}
*temp = 0;
fs_busy = 1;
__atomic_signal_fence(__ATOMIC_SEQ_CST);
if (fs_tslot < 0) {
int32_t s = fs_alloc();
if (s < 0) {
fs_busy = 0;
return -1;
}
pthread_once(&fs_once, fs_key_init);
pthread_setspecific(fs_key, (void *)(uintptr_t)(s + 1));
fs_tslot = s;
}
return fs_tslot;
}
static void fs_give(int32_t s, int temp) {
if (temp) {
fs_release((uint32_t)s, fs_owner());
} else {
__atomic_signal_fence(__ATOMIC_SEQ_CST);
fs_busy = 0;
}
}
static int fs_changed(const struct fs_waitv *v, uint32_t n) {
for (uint32_t i = 0; i < n; i++)
if (__atomic_load_n((uint32_t *)(uintptr_t)v[i].uaddr, __ATOMIC_SEQ_CST) != (uint32_t)v[i].val) return (int)i;
return -1;
}
static int fs_past(const struct timespec *ts, int clk) {
struct timespec now;
if (!ts) return 0;
clock_gettime(clk, &now);
return now.tv_sec > ts->tv_sec || (now.tv_sec == ts->tv_sec && now.tv_nsec >= ts->tv_nsec);
}
static uint64_t fs_clock_ns(int clk) {
struct timespec now;
clock_gettime(clk, &now);
return (uint64_t)now.tv_sec * 1000000000ull + (uint64_t)now.tv_nsec;
}
static uint64_t fs_ts_ns(const struct timespec *ts) {
return (uint64_t)ts->tv_sec * 1000000000ull + (uint64_t)ts->tv_nsec;
}
static long fs_poll(const struct fs_waitv *v, uint32_t n, const struct timespec *ts, int clk) {
int i = fs_changed(v, n);
if (i >= 0) {
errno = EAGAIN;
return -1;
}
for (;;) {
struct timespec nap = {0, FS_POLL_NS};
if (fs_past(ts, clk)) {
errno = ETIMEDOUT;
return -1;
}
if (nanosleep(&nap, NULL) == -1 && errno == EINTR) return -1;
if ((i = fs_changed(v, n)) >= 0) return i;
}
}
static long fs_waitv(const struct fs_waitv *v, uint32_t n, uint32_t flags, const struct timespec *ts, int clk) {
uint32_t keys[FS_KEYS];
struct fs_slot *sl;
struct timespec nap_ts;
const struct timespec *until = ts;
int temp, err = 0, op, napping = 0, wclk = clk, saved = errno;
int32_t s;
uint64_t bit, nap;
if (!v || !n || n > FS_KEYS || flags || (ts && clk != CLOCK_MONOTONIC && clk != CLOCK_REALTIME)) {
errno = EINVAL;
return -1;
}
for (uint32_t i = 0; i < n; i++) {
if ((v[i].flags & ~FS_PRIVATE) != FS_SIZE32 || v[i].reserved || (v[i].uaddr & 3)) {
errno = EINVAL;
return -1;
}
if ((v[i].flags & FS_PRIVATE) || !fs_key_of(v[i].uaddr, &keys[i])) keys[i] = FS_NOKEY;
}
if (!fs_reg || (s = fs_take(&temp)) < 0) return fs_poll(v, n, ts, clk);
sl = fs_slot((uint32_t)s);
bit = 1ull << (s & 63);
for (uint32_t i = 0; i < n; i++) __atomic_store_n(&sl->keys[i], keys[i], __ATOMIC_RELAXED);
__atomic_store_n(&sl->count, n, __ATOMIC_RELAXED);
__atomic_store_n(&sl->futex, 0, __ATOMIC_RELAXED);
for (uint32_t i = 0; i < n; i++) {
struct fs_bucket *b;
if (keys[i] == FS_NOKEY) continue;
b = fs_bucket(keys[i]);
__atomic_or_fetch(&b->bits[s >> 6], bit, __ATOMIC_SEQ_CST);
}
__atomic_thread_fence(__ATOMIC_SEQ_CST);
if (fs_changed(v, n) >= 0) err = EAGAIN;
nap = fs_nap_ns && fs_nap_n == n && fs_nap_key == keys[0] ? fs_nap_ns * 2 : FS_NAP_MIN;
if (nap > FS_NAP_MAX) nap = FS_NAP_MAX;
if (!ts || fs_ts_ns(ts) > fs_clock_ns(clk) + nap) {
uint64_t at = fs_clock_ns(CLOCK_MONOTONIC) + nap;
nap_ts.tv_sec = (time_t)(at / 1000000000ull);
nap_ts.tv_nsec = (long)(at % 1000000000ull);
until = &nap_ts;
wclk = CLOCK_MONOTONIC;
napping = 1;
}
op = FUTEX_WAIT_BITSET | (until && wclk == CLOCK_REALTIME ? FUTEX_CLOCK_REALTIME : 0);
while (!err && !__atomic_load_n(&sl->futex, __ATOMIC_ACQUIRE)) {
if (fs_sys(SYS_futex, (long)&sl->futex, op, 0, (long)until, 0, FUTEX_BITSET_MATCH_ANY) == -1 && errno != EAGAIN
&& !__atomic_load_n(&sl->futex, __ATOMIC_ACQUIRE))
err = errno;
}
fs_unregister((uint32_t)s, keys, n);
if (err && __atomic_exchange_n(&sl->futex, 1, __ATOMIC_ACQ_REL)) err = 0;
fs_give(s, temp);
if (err == ETIMEDOUT && napping) {
fs_nap_key = keys[0];
fs_nap_n = n;
fs_nap_ns = nap;
errno = EINTR;
return -1;
}
if (err != EINTR) fs_nap_ns = 0;
if (err) {
errno = err;
return -1;
}
errno = saved;
s = fs_changed(v, n);
return s < 0 ? 0 : s;
}
static long fs_wake(uint32_t key, int n) {
struct fs_bucket *b = fs_bucket(key);
long woken = 0;
__atomic_thread_fence(__ATOMIC_SEQ_CST);
if (n <= 0) return 0;
for (uint32_t w = 0; w < FS_WORDS; w++) {
uint64_t bits = __atomic_load_n(&b->bits[w], __ATOMIC_SEQ_CST);
while (bits) {
uint32_t s = w * 64 + (uint32_t)__builtin_ctzll(bits), c;
struct fs_slot *sl = fs_slot(s);
bits &= bits - 1;
c = __atomic_load_n(&sl->count, __ATOMIC_ACQUIRE);
if (c > FS_KEYS) c = FS_KEYS;
for (uint32_t i = 0; i < c; i++) {
if (__atomic_load_n(&sl->keys[i], __ATOMIC_RELAXED) != key) continue;
if (!__atomic_exchange_n(&sl->futex, 1, __ATOMIC_SEQ_CST)) {
int saved = errno;
fs_sys(SYS_futex, (long)&sl->futex, FUTEX_WAKE, 1, 0, 0, 0);
errno = saved;
if (++woken >= n) return woken;
}
break;
}
}
}
return woken;
}
void bl_fsync_fds_closed(unsigned int first, unsigned int last) __attribute__((visibility("hidden")));
void bl_fsync_fds_closed(unsigned int first, unsigned int last) {
int fd = __atomic_load_n(&fs_fd, __ATOMIC_ACQUIRE);
if (fd >= 0 && (unsigned int)fd >= first && (unsigned int)fd <= last) __atomic_store_n(&fs_fd, -1, __ATOMIC_RELEASE);
}
int bl_fsync_syscall(long number, const long *args, long *ret) __attribute__((visibility("hidden")));
int bl_fsync_syscall(long number, const long *args, long *ret) {
if (number == SYS_futex_waitv) {
if (!fs_on()) return 0;
if (!args[0] && !args[1]) {
errno = EINVAL;
*ret = -1;
return 1;
}
*ret = fs_waitv((const struct fs_waitv *)args[0], (uint32_t)args[1], (uint32_t)args[2],
(const struct timespec *)args[3], (int)args[4]);
return 1;
}
if (number == SYS_futex && (int)args[1] == FUTEX_WAKE && fs_reg && __atomic_load_n(&fs_cur, __ATOMIC_RELAXED)) {
uint32_t key;
if (!fs_key_of((uint64_t)args[0], &key)) return 0;
*ret = fs_wake(key, (int)args[2]);
return 1;
}
return 0;
}
static int fs_fsync_name(const char *name) {
size_t n;
if (!name) return 0;
if (*name == '/') name++;
n = strlen(name);
return n > 11 && !strncmp(name, "wine-", 5) && !strcmp(name + n - 6, "-fsync");
}
static void fs_attach(const char *name, int fd, int create) {
char path[256];
struct stat st;
void *p;
int rfd;
if (fs_reg) {
fs_fd = fd;
return;
}
if (snprintf(path, sizeof(path), "%s-droiddeck", name) >= (int)sizeof(path)) return;
if (create) fs_real_shm_unlink(path);
rfd = fs_real_shm_open(path, O_RDWR | O_CREAT, 0644);
if (rfd < 0) {
fprintf(stderr, "droiddeck-fsync: cannot open %s: %s\n", path, strerror(errno));
return;
}
if (fstat(rfd, &st) == 0 && (size_t)st.st_size < FS_REGION && ftruncate(rfd, (off_t)FS_REGION) != 0) {
fprintf(stderr, "droiddeck-fsync: cannot size %s: %s\n", path, strerror(errno));
close(rfd);
return;
}
if (!fs_real_mmap) fs_real_mmap = (void *(*)(void *, size_t, int, int, int, off_t))dlsym(RTLD_NEXT, "mmap");
p = fs_real_mmap(NULL, FS_REGION, PROT_READ | PROT_WRITE, MAP_SHARED, rfd, 0);
close(rfd);
if (p == MAP_FAILED) {
fprintf(stderr, "droiddeck-fsync: cannot map %s: %s\n", path, strerror(errno));
return;
}
fs_reg = p;
fs_fd = fd;
if (create) {
__atomic_store_n(&fs_head()->magic, FS_MAGIC, __ATOMIC_RELEASE);
fprintf(stderr, "droiddeck-fsync: up and running.\n");
}
}
int shm_open(const char *name, int oflag, mode_t mode) {
int fd, saved;
if (!fs_real_shm_open) fs_real_shm_open = (int (*)(const char *, int, mode_t))dlsym(RTLD_NEXT, "shm_open");
if (!fs_real_shm_unlink) fs_real_shm_unlink = (int (*)(const char *))dlsym(RTLD_NEXT, "shm_unlink");
if (!fs_real_shm_open) {
errno = ENOSYS;
return -1;
}
fd = fs_real_shm_open(name, oflag, mode);
if (fd >= 0 && fs_on() && fs_fsync_name(name) && fs_real_shm_unlink) {
saved = errno;
fs_attach(name, fd, oflag & O_CREAT);
errno = saved;
}
return fd;
}
int shm_unlink(const char *name) {
int r, saved;
if (!fs_real_shm_unlink) fs_real_shm_unlink = (int (*)(const char *))dlsym(RTLD_NEXT, "shm_unlink");
if (!fs_real_shm_unlink) {
errno = ENOSYS;
return -1;
}
r = fs_real_shm_unlink(name);
if (fs_on() && fs_fsync_name(name)) {
char path[256];
saved = errno;
if (snprintf(path, sizeof(path), "%s-droiddeck", name) < (int)sizeof(path)) fs_real_shm_unlink(path);
errno = saved;
}
return r;
}
void *mmap(void *addr, size_t len, int prot, int flags, int fd, off_t off) {
void *p;
if (!fs_real_mmap) fs_real_mmap = (void *(*)(void *, size_t, int, int, int, off_t))dlsym(RTLD_NEXT, "mmap");
p = fs_real_mmap(addr, len, prot, flags, fd, off);
if (p != MAP_FAILED && __builtin_expect(fs_fd >= 0, 0)) fs_track(p, len, flags, fd, off);
return p;
}
void *mmap64(void *addr, size_t len, int prot, int flags, int fd, off64_t off) {
void *p;
if (!fs_real_mmap64) fs_real_mmap64 = (void *(*)(void *, size_t, int, int, int, off64_t))dlsym(RTLD_NEXT, "mmap64");
p = fs_real_mmap64(addr, len, prot, flags, fd, off);
if (p != MAP_FAILED && __builtin_expect(fs_fd >= 0, 0)) fs_track(p, len, flags, fd, off);
return p;
}
int munmap(void *addr, size_t len) {
uintptr_t lo = (uintptr_t)addr, hi = lo + len;
sigset_t old;
int r, saved;
if (!fs_real_munmap) fs_real_munmap = (int (*)(void *, size_t))dlsym(RTLD_NEXT, "munmap");
if (__builtin_expect(fs_fd < 0, 1) || !fs_overlaps(lo, hi)) return fs_real_munmap(addr, len);
fs_block(&old);
pthread_mutex_lock(&fs_mx);
r = fs_real_munmap(addr, len);
saved = errno;
if (r == 0) fs_rebuild_locked(lo, hi, NULL);
pthread_mutex_unlock(&fs_mx);
pthread_sigmask(SIG_SETMASK, &old, NULL);
errno = saved;
return r;
}
+2 -2
View File
@@ -89,7 +89,7 @@ static void discover_pads(void) {
if (pad->devnode[0] == '\0') {
snprintf(pad->sysname, sizeof(pad->sysname), "event%ld", slot);
snprintf(pad->syspath, sizeof(pad->syspath), "/sys/devices/virtual/input/bannerlator%ld",
snprintf(pad->syspath, sizeof(pad->syspath), "/sys/devices/virtual/input/droiddeck%ld",
slot);
snprintf(pad->devnode, sizeof(pad->devnode), "/dev/input/event%ld", slot);
snprintf(pad->uevent, sizeof(pad->uevent), PAD_UEVENT_FORMAT, (int)slot, (int)slot);
@@ -153,7 +153,7 @@ static void refresh_pad(int node) {
if (read_virtual_uevent(node, uevent, sizeof(uevent))) {
snprintf(pad->sysname, sizeof(pad->sysname), "event%d", node);
snprintf(pad->syspath, sizeof(pad->syspath), "/sys/devices/virtual/input/bannerlator%d", node);
snprintf(pad->syspath, sizeof(pad->syspath), "/sys/devices/virtual/input/droiddeck%d", node);
memcpy(pad->uevent, uevent, sizeof(uevent));
pad->devnum = makedev(13, EVENT_MINOR_BASE + (unsigned int)node);
snprintf(pad->devnode, sizeof(pad->devnode), "/dev/input/event%d", node);
+79 -3
View File
@@ -20,6 +20,8 @@
#include <sys/stat.h>
#include <sys/types.h>
#include <errno.h>
#include <signal.h>
#include <time.h>
#include <unistd.h>
static const char *net_dir(void) {
@@ -123,9 +125,75 @@ int listen(int fd, int backlog) {
return ret;
}
static unsigned local_port(int fd) {
struct sockaddr_storage ss;
socklen_t len = sizeof(ss);
if (getsockname(fd, (struct sockaddr *)&ss, &len) != 0) {
return 0;
}
return addr_port((struct sockaddr *)&ss, len);
}
static void bind_before_loopback_connect(int fd, const struct sockaddr *addr, socklen_t len) {
if (!addr || addr->sa_family != AF_INET || len < sizeof(struct sockaddr_in) ||
((const struct sockaddr_in *)addr)->sin_addr.s_addr != htonl(INADDR_LOOPBACK)) {
return;
}
struct sockaddr_storage ss;
socklen_t slen = sizeof(ss);
if (getsockname(fd, (struct sockaddr *)&ss, &slen) != 0 || ss.ss_family != AF_INET ||
((struct sockaddr_in *)&ss)->sin_port != 0) {
return;
}
struct sockaddr_in local;
memset(&local, 0, sizeof(local));
local.sin_family = AF_INET;
local.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
int saved = errno;
bind(fd, (struct sockaddr *)&local, sizeof(local));
errno = saved;
}
static unsigned guarded_port(void) {
static int port = -1;
if (port < 0) {
const char *value = getenv("BL_CDP_GUARD");
port = value && *value ? atoi(value) : 0;
if (port < 0 || port > 65535) port = 0;
}
return (unsigned)port;
}
static int guest_peer(int fd) {
unsigned peer = peer_port(fd);
if (peer == 0) {
return 0;
}
char path[300];
snprintf(path, sizeof(path), "%s/p%u", net_dir(), peer);
struct stat st;
if (stat(path, &st) != 0 || time(NULL) - st.st_mtime > 30) {
return 0;
}
int in = open(path, O_RDONLY);
if (in < 0) {
return 0;
}
char buf[64];
int n = read(in, buf, sizeof(buf) - 1);
close(in);
if (n <= 0) {
return 0;
}
buf[n] = '\0';
int pid = atoi(buf);
return pid > 0 && (kill(pid, 0) == 0 || errno != ESRCH);
}
int connect(int fd, const struct sockaddr *addr, socklen_t len) {
static connect_fn real;
if (!real) real = (connect_fn)dlsym(RTLD_NEXT, "connect");
bind_before_loopback_connect(fd, addr, len);
int ret = real(fd, addr, len);
if (ret == 0 || errno == EINPROGRESS) {
/* The caller goes on to test for EINPROGRESS, which the bookkeeping must not disturb. */
@@ -139,9 +207,17 @@ int connect(int fd, const struct sockaddr *addr, socklen_t len) {
int accept4(int fd, struct sockaddr *addr, socklen_t *len, int flags) {
static accept4_fn real;
if (!real) real = (accept4_fn)dlsym(RTLD_NEXT, "accept4");
int ret = real(fd, addr, len, flags);
if (ret >= 0) record_port(ret, peer_port(ret));
return ret;
for (;;) {
int ret = real(fd, addr, len, flags);
if (ret < 0) return ret;
unsigned guarded = guarded_port();
if (guarded && local_port(ret) == guarded && !guest_peer(ret)) {
close(ret);
continue;
}
record_port(ret, peer_port(ret));
return ret;
}
}
int accept(int fd, struct sockaddr *addr, socklen_t *len) {
+6 -4
View File
@@ -8,7 +8,7 @@
* address ioctl, and every table under /proc/net. Wine builds its adapter, address, route and
* neighbour tables from exactly those, so a game saw a machine with no adapter, no address, no
* gateway and no MAC, and one that checks its adapters before going online waited for good.
* The app knows the real link from ConnectivityManager and writes it to /etc/bannerlator-net;
* The app knows the real link from ConnectivityManager and writes it to /etc/droiddeck-net;
* whatever the kernel refuses is answered from that file. Anything the kernel does answer wins.
*/
#define _GNU_SOURCE 1
@@ -30,7 +30,7 @@
#include <sys/socket.h>
#include <unistd.h>
#define LINK_FILE "/etc/bannerlator-net"
#define LINK_FILE "/etc/droiddeck-net"
#define LO_NAME "lo"
#define LO_INDEX 1
#define LO_MTU 65536
@@ -352,6 +352,7 @@ static int is_link_query(unsigned long request) {
void bl_kgsl_chip_id_fixup(unsigned long request, void *arg, int rc); /* kgslid.c */
int bl_kgsl_poll(int (*real)(int, unsigned long, void *), int fd, unsigned long request,
void *arg, int *rc) __attribute__((visibility("hidden")));
int bl_ntsync_ioctl(int fd, unsigned long request, void *arg, int *rc) __attribute__((visibility("hidden")));
/* Every DRM and evdev call comes through here, so the lookup is done once. Two threads racing
* to store it store the same pointer. */
@@ -363,8 +364,9 @@ int ioctl(int fd, unsigned long request, ...) {
void *arg = va_arg(ap, void *);
va_end(ap);
if (real == NULL) real = (ioctl_fn) dlsym(RTLD_NEXT, "ioctl");
int rc;
if (bl_ntsync_ioctl(fd, request, arg, &rc)) return rc;
if (real == NULL) real = (ioctl_fn) dlsym(RTLD_NEXT, "ioctl");
if (!bl_kgsl_poll(real, fd, request, arg, &rc)) rc = real(fd, request, arg);
bl_kgsl_chip_id_fixup(request, arg, rc);
if (arg == NULL || !is_link_query(request)) return rc;
@@ -466,7 +468,7 @@ static const char *denied_table(const char *path) {
static FILE *synthesize(const char *table) {
struct link link;
if (!load_link(&link)) return NULL;
int fd = memfd_create("bannerlator-net", MFD_CLOEXEC);
int fd = memfd_create("droiddeck-net", MFD_CLOEXEC);
if (fd < 0) return NULL;
FILE *out = fdopen(fd, "w+");
if (out == NULL) {
File diff suppressed because it is too large. Load diff
+3
View File
@@ -35,6 +35,7 @@
int bl_status_without_tracer(const char *path, int flags) __attribute__((visibility("hidden")));
int bl_writable_retry(int dirfd, const char *path, int flags) __attribute__((visibility("hidden")));
int bl_ntsync_open(const char *path, int flags) __attribute__((visibility("hidden")));
static int retry_without_noatime(int fd, int flags) {
return fd < 0 && errno == EPERM && (flags & O_NOATIME);
@@ -61,6 +62,7 @@ int open(const char *path, int flags, ...) {
return -1;
}
if ((fd = bl_status_without_tracer(path, flags)) >= 0) return fd;
if ((fd = bl_ntsync_open(path, flags)) != -2) return fd;
if (real_open == NULL) real_open = dlsym(RTLD_NEXT, "open");
if (real_open == NULL) {
errno = ENOSYS;
@@ -87,6 +89,7 @@ int openat(int dirfd, const char *path, int flags, ...) {
return -1;
}
if ((fd = bl_status_without_tracer(path, flags)) >= 0) return fd;
if ((fd = bl_ntsync_open(path, flags)) != -2) return fd;
if (real_openat == NULL) real_openat = dlsym(RTLD_NEXT, "openat");
if (real_openat == NULL) {
errno = ENOSYS;
+63
View File
@@ -9,6 +9,7 @@
#define _GNU_SOURCE
#include <dlfcn.h>
#include <errno.h>
#include <fcntl.h>
#include <linux/futex.h>
#include <pthread.h>
#include <stdarg.h>
@@ -18,7 +19,23 @@
#include <sys/uio.h>
#include <unistd.h>
#ifndef SYS_openat2
#define SYS_openat2 437
#endif
#define BL_RESOLVE_BENEATH 0x08
#define BL_RESOLVE_IN_ROOT 0x10
struct bl_open_how {
uint64_t flags;
uint64_t mode;
uint64_t resolve;
};
static long (*real_syscall)(long, ...);
static int openat2_missing;
int bl_fsync_syscall(long number, const long *args, long *ret) __attribute__((visibility("hidden")));
static struct robust_list_head *thread_robust_head(void) {
pthread_mutexattr_t attr;
@@ -54,6 +71,35 @@ static struct robust_list_head *thread_robust_head(void) {
return head;
}
static long openat2_via_openat(const long *args) {
const char *path = (const char *)args[1];
const struct bl_open_how *how = (const struct bl_open_how *)args[2];
if (!path || !how || (size_t)args[3] < sizeof(*how)) {
errno = EINVAL;
return -1;
}
if (how->resolve & BL_RESOLVE_IN_ROOT) {
while (*path == '/') {
path++;
}
if (!*path) {
path = ".";
}
} else if ((how->resolve & BL_RESOLVE_BENEATH) && *path == '/') {
errno = EXDEV;
return -1;
}
return real_syscall(SYS_openat, args[0], (long)path, (long)(int)how->flags, (long)how->mode);
}
static int execveat_is_execve(const long *args) {
const char *path = (const char *)args[1];
if (!path || (args[4] & (AT_EMPTY_PATH | AT_SYMLINK_NOFOLLOW))) {
return 0;
}
return path[0] == '/' || (int)args[0] == AT_FDCWD;
}
long syscall(long number, ...) {
va_list ap;
long args[6];
@@ -72,8 +118,25 @@ long syscall(long number, ...) {
*(size_t *)args[2] = sizeof(*head);
return 0;
}
long handled;
if (bl_fsync_syscall(number, args, &handled)) {
return handled;
}
if (!real_syscall) {
real_syscall = (long (*)(long, ...)) dlsym(RTLD_NEXT, "syscall");
}
if (number == SYS_openat2) {
if (!__atomic_load_n(&openat2_missing, __ATOMIC_RELAXED)) {
long fd = real_syscall(number, args[0], args[1], args[2], args[3]);
if (fd != -1 || errno != ENOSYS) {
return fd;
}
__atomic_store_n(&openat2_missing, 1, __ATOMIC_RELAXED);
}
return openat2_via_openat(args);
}
if (number == SYS_execveat && execveat_is_execve(args)) {
return real_syscall(SYS_execve, args[1], args[2], args[3]);
}
return real_syscall(number, args[0], args[1], args[2], args[3], args[4], args[5]);
}
+3
View File
@@ -0,0 +1,3 @@
URUNTIME_VERSION=v0.8.1
URUNTIME_ASSET=uruntime-appimage-lite-aarch64
URUNTIME_SHA256=8485ae20eb0bfd4eaeac38d7fd7f78e153b391c718cf33ff07879605683be1ea
+1 -1
View File
@@ -2,7 +2,7 @@ import os, sys
paths = ["/etc/os-release", "/root/abs-link", "/root/to-bind", "/root/to-bind/inbind.txt", "/root/rel-to-bind",
"/root/rel-to-bind/inbind.txt", "/root/dotdot", "/root/dotdot/libc.so.6", "/root/dangling", "/root/loop",
"/usr/lib/../lib/libc.so.6", "/usr//lib/./libc.so.6", "/lib", "/lib/libc.so.6", "/bin/env", "/nope", "/nope/x",
"/etc/os-release/x", "/mnt/droiddeck-sd", "/mnt/droiddeck-sd/inbind.txt", "/mnt/droiddeck-sd/../bannerlator",
"/etc/os-release/x", "/mnt/droiddeck-sd", "/mnt/droiddeck-sd/inbind.txt", "/mnt/droiddeck-sd/../droiddeck",
"/dev/null", "/dev/shm", "/dev/shm/inbind.txt", "/dev/random", "/proc/self/exe", "/proc/version", "/proc",
"/sys/class/thermal/thermal_zone28/temp", "/sdcard", "/", "/tmp", "/root/in-root-abs-dir/os-release",
"/data/local/tmp/dd/tmp/inbind.txt", "/data/local/tmp/dd/mini/etc/os-release", "/root/to-bind/missing/x", "/root/to-bind/inbind.txt/x", "/usr/lib/nope/deeper/x.so", "/mnt/droiddeck-sd/nope/x", "/root/dangling/x", "/root/abs-link/x", "/nope/a/b/c", "/sdcard/nope/x"]
+1 -1
View File
@@ -13,7 +13,7 @@ D=/data/local/tmp/dd
V=$1; shift
U=$(id -u)
REL="\\Linux\\DroidDeck\\$(uname -r)\\$(uname -v)\\aarch64\\localdomain\\-1\\"
BINDS="-b /dev -b /proc -b /sys -b /dev/urandom:/dev/random -b /proc/self/fd:/dev/fd -b /proc/self/fd/0:/dev/stdin -b /proc/self/fd/1:/dev/stdout -b /proc/self/fd/2:/dev/stderr -b $D/tmp:/dev/shm -b $D -b /sdcard -b /data/local/tmp/dd/mini/etc/os-release:/proc/version -b $D/mini/etc/os-release:/proc/loadavg -b $D/mini/etc/os-release:/proc/uptime -b /sys/class/thermal/thermal_zone0/temp:/sys/class/thermal/thermal_zone28/temp -b $D/tmp:/mnt/droiddeck-sd -b $D/tmp:/mnt/bannerlator"
BINDS="-b /dev -b /proc -b /sys -b /dev/urandom:/dev/random -b /proc/self/fd:/dev/fd -b /proc/self/fd/0:/dev/stdin -b /proc/self/fd/1:/dev/stdout -b /proc/self/fd/2:/dev/stderr -b $D/tmp:/dev/shm -b $D -b /sdcard -b /data/local/tmp/dd/mini/etc/os-release:/proc/version -b $D/mini/etc/os-release:/proc/loadavg -b $D/mini/etc/os-release:/proc/uptime -b /sys/class/thermal/thermal_zone0/temp:/sys/class/thermal/thermal_zone28/temp -b $D/tmp:/mnt/droiddeck-sd -b $D/tmp:/mnt/droiddeck"
export PROOT_TMP_DIR=$D/tmp
case $V in
patched) export PROOT_LOADER=$D/loader; exec $D/proot --kill-on-exit --kernel-release="$REL" -i $U:$U -r $D/mini -w /root $BINDS /usr/bin/env -i HOME=/root PATH=/usr/bin LANG=C.UTF-8 "$@" ;;
+2 -1
View File
@@ -8,7 +8,8 @@ OVERLAY = Path(__file__).resolve().parents[1] / 'linuxfs/overlay'
def check(apk, overlay=OVERLAY):
sources = list((overlay / 'usr/local/bin').glob('bannerlator-*'))
sources = list((overlay / 'usr/local/bin').glob('droiddeck-*'))
sources += [path for path in [overlay / 'usr/local/bin/steam-compatibility'] if path.is_file()]
sources += [path for path in (overlay / 'usr/bin').rglob('*') if path.is_file()]
errors = []
with ZipFile(apk) as package:
+132
View File
@@ -0,0 +1,132 @@
import os
import shutil
import subprocess
import tempfile
import unittest
from pathlib import Path
RUN = Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin/droiddeck-appimage-run"
KEYS = ("QT_QPA_PLATFORM", "WAYLAND_DISPLAY", "SDL_VIDEODRIVER", "XDG_SESSION_TYPE", "QT_WAYLAND_DISABLE_WINDOWDECORATION", "DISABLE_GAMESCOPE_WSI")
QUICK = ("usr/lib/libQt6Quick.so.6", "usr/lib/libvulkan.so.1", "usr/plugins/platforms/libqwayland-generic.so")
RUN_MODE = {"GAMESCOPE_WAYLAND_DISPLAY": "gamescope-0", "WAYLAND_DISPLAY": "gamescope-0", "DISPLAY": ":0",
"QT_QPA_PLATFORM": "xcb", "SDL_VIDEODRIVER": "x11", "XDG_SESSION_TYPE": "x11"}
DESKTOP = {"WAYLAND_DISPLAY": "wayland-1", "DISPLAY": ":1", "QT_QPA_PLATFORM": "wayland;xcb",
"SDL_VIDEODRIVER": "wayland,x11", "XDG_SESSION_TYPE": "wayland"}
class AppImageRunTest(unittest.TestCase):
def setUp(self):
self.tmp = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.tmp, True)
self.dir = self.tmp / "opennow"
self.out = self.tmp / "env"
app = self.dir / "app"
app.mkdir(parents=True)
run = app / "AppRun"
run.write_text("#!/bin/sh\nfor k in %s; do eval \"echo $k=\\${$k-unset}\"; done > %s\n" % (" ".join(KEYS), self.out))
run.chmod(0o755)
def bundle(self, *paths):
for path in paths:
lib = self.dir / "app" / path
lib.parent.mkdir(parents=True, exist_ok=True)
lib.write_bytes(b"")
def launch(self, session):
env = {k: v for k, v in os.environ.items() if k not in KEYS + ("GAMESCOPE_WAYLAND_DISPLAY", "DISPLAY")}
env.update(session, HOME=str(self.tmp / "home"), DBUS_SESSION_BUS_ADDRESS="unix:path=/dev/null")
result = subprocess.run(["bash", str(RUN), str(self.dir)], env=env, capture_output=True, text=True, timeout=30)
self.assertEqual(0, result.returncode, result.stderr)
return dict(line.split("=", 1) for line in self.out.read_text().splitlines()), result.stdout
def test_a_vulkan_qt_quick_app_draws_on_gamescope_wayland(self):
self.bundle(*QUICK)
env, log = self.launch(RUN_MODE)
self.assertEqual("wayland;xcb", env["QT_QPA_PLATFORM"])
self.assertEqual("gamescope-0", env["WAYLAND_DISPLAY"])
self.assertEqual("wayland", env["SDL_VIDEODRIVER"])
self.assertEqual("wayland", env["XDG_SESSION_TYPE"])
self.assertEqual("1", env["QT_WAYLAND_DISABLE_WINDOWDECORATION"])
self.assertEqual("1", env["DISABLE_GAMESCOPE_WSI"])
self.assertIn("Qt wayland (wayland;xcb)", log)
def test_a_quick_app_that_loads_vulkan_itself_is_found_by_its_binary(self):
self.bundle(QUICK[0], QUICK[2])
binary = self.dir / "app/usr/bin/opennow-qt"
binary.parent.mkdir(parents=True, exist_ok=True)
binary.write_bytes(b"\x7fELF\0_ZN7QWindow17setVulkanInstanceEP15QVulkanInstance\0")
env, _ = self.launch(RUN_MODE)
self.assertEqual("wayland;xcb", env["QT_QPA_PLATFORM"])
self.assertEqual("1", env["DISABLE_GAMESCOPE_WSI"])
def test_other_apps_stay_on_xwayland(self):
self.bundle("usr/lib/libQt6Widgets.so.6", "usr/lib/libvulkan.so.1", "usr/plugins/platforms/libqxcb.so")
env, _ = self.launch(RUN_MODE)
self.assertEqual("xcb", env["QT_QPA_PLATFORM"])
self.assertEqual("x11", env["SDL_VIDEODRIVER"])
self.assertEqual("unset", env["QT_WAYLAND_DISABLE_WINDOWDECORATION"])
self.assertEqual("unset", env["DISABLE_GAMESCOPE_WSI"])
def test_a_quick_app_without_the_wayland_plugin_stays_on_xwayland(self):
self.bundle(*QUICK[:2])
env, log = self.launch(RUN_MODE)
self.assertEqual("xcb", env["QT_QPA_PLATFORM"])
self.assertIn("stays on X11", log)
def test_the_desktop_keeps_its_decorations(self):
self.bundle(*QUICK)
env, _ = self.launch(DESKTOP)
self.assertEqual("wayland;xcb", env["QT_QPA_PLATFORM"])
self.assertEqual("wayland-1", env["WAYLAND_DISPLAY"])
self.assertEqual("wayland", env["SDL_VIDEODRIVER"])
self.assertEqual("unset", env["QT_WAYLAND_DISABLE_WINDOWDECORATION"])
self.assertEqual("unset", env["DISABLE_GAMESCOPE_WSI"])
def test_how_the_program_ended_is_logged(self):
run = self.dir / "app/AppRun"
run.write_text("#!/bin/sh\nexit 3\n")
env = {k: v for k, v in os.environ.items() if k not in KEYS}
env.update(RUN_MODE, HOME=str(self.tmp / "home"), DBUS_SESSION_BUS_ADDRESS="unix:path=/dev/null")
result = subprocess.run(["bash", str(RUN), str(self.dir)], env=env, capture_output=True, text=True, timeout=30)
self.assertEqual(3, result.returncode)
self.assertIn("exited with status 3", result.stdout)
run.write_text("#!/bin/sh\nkill -SEGV $$\n")
result = subprocess.run(["bash", str(RUN), str(self.dir)], env=env, capture_output=True, text=True, timeout=30)
self.assertEqual(139, result.returncode)
self.assertIn("ended by signal 11 (SEGV)", result.stdout)
def test_a_failed_program_shows_the_log_it_wrote(self):
home = self.tmp / "home"
old = home / ".local/share/emulator/log/old.log"
old.parent.mkdir(parents=True)
old.write_text("from an earlier run\n")
os.utime(old, (1, 1))
(self.dir / "app/AppRun").write_text(
"#!/bin/sh\nmkdir -p \"$HOME/.local/share/emulator/log\"\n"
"printf 'Assertion failed: vulkan device\\n\\345\\001\\n' > \"$HOME/.local/share/emulator/log/emulator_log.txt\"\n"
"kill -ILL $$\n")
env = {k: v for k, v in os.environ.items() if k not in KEYS}
env.update(RUN_MODE, HOME=str(home), DBUS_SESSION_BUS_ADDRESS="unix:path=/dev/null")
result = subprocess.run(["bash", str(RUN), str(self.dir)], env=env, capture_output=True, text=True, timeout=30)
self.assertIn("ended by signal 4 (ILL)", result.stdout)
self.assertIn("end of %s" % (home / ".local/share/emulator/log/emulator_log.txt"), result.stdout)
self.assertIn("Assertion failed: vulkan device", result.stdout)
self.assertNotIn("from an earlier run", result.stdout)
(self.dir / "app/AppRun").write_text("#!/bin/sh\necho again > \"$HOME/.local/share/emulator/log/emulator_log.txt\"\n")
result = subprocess.run(["bash", str(RUN), str(self.dir)], env=env, capture_output=True, text=True, timeout=30)
self.assertNotIn("end of", result.stdout)
def test_the_per_app_choice_wins(self):
self.bundle(*QUICK)
(self.dir / "qt").write_text("xcb\n")
env, log = self.launch(DESKTOP)
self.assertEqual("xcb", env["QT_QPA_PLATFORM"])
self.assertIn("Qt xcb (xcb)", log)
(self.dir / "qt").write_text("wayland\n")
(self.dir / "app/usr/lib/libQt6Quick.so.6").unlink()
env, _ = self.launch(RUN_MODE)
self.assertEqual("wayland;xcb", env["QT_QPA_PLATFORM"])
if __name__ == "__main__":
unittest.main()
+1 -1
View File
@@ -6,7 +6,7 @@ import unittest
from unittest.mock import patch
HELPER = Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin/bannerlator-desktop-bookmarks"
HELPER = Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin/droiddeck-desktop-bookmarks"
MODULE = runpy.run_path(str(HELPER))
+515
View File
@@ -0,0 +1,515 @@
import contextlib
import gzip
import io
import json
import os
import runpy
import shutil
import struct
import tempfile
import unittest
from pathlib import Path
from unittest import mock
BIN = Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin"
FEX = runpy.run_path(str(BIN / "droiddeck-fex"))
G = FEX["main"].__globals__
def elf(machine, data=1, size=64):
head = bytearray(size)
head[0:4] = b"\x7fELF"
head[4] = 2
head[5] = data
struct.pack_into("<H" if data == 1 else ">H", head, 18, machine)
return bytes(head)
class FexTestCase(unittest.TestCase):
def setUp(self):
self.tmp = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.tmp, True)
self.home = self.tmp / "home"
self.steam = self.home / ".local/share/Steam"
(self.steam / "steamapps/common").mkdir(parents=True)
state = self.home / ".local/share/droiddeck-fex"
patches = {
"HOME": str(self.home),
"STEAM_ROOT": str(self.steam),
"LIBRARIES": (str(self.tmp / "sd"),),
"STATE": str(state),
"CONFIG_DIR": str(state / "config"),
"DATA_DIR": str(state / "data"),
"CACHE_DIR": str(self.home / ".cache/droiddeck-fex"),
"STATUS_FILE": str(state / "status.json"),
"WANTED": str(self.home / ".config/droiddeck/fex-wanted"),
"PRELOADS": str(self.tmp / "preloads"),
"HOST_PRELOAD": str(self.tmp / "ld.so.preload"),
}
patcher = mock.patch.dict(G, patches)
patcher.start()
self.addCleanup(patcher.stop)
def write(self, path, data=b"", mode=0o644):
path = Path(path)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(data if isinstance(data, bytes) else data.encode())
path.chmod(mode)
return path
def fex_tool(self, library=None, server=True, thunks=True):
tool = Path(library or self.steam) / "steamapps/common/FEX-Emu"
self.write(tool / "usr/bin/FEX", elf(183), 0o755)
if server:
self.write(tool / "usr/bin/FEXServer", elf(183), 0o755)
if thunks:
(tool / "usr/lib/aarch64-linux-gnu/fex-emu/HostThunks").mkdir(parents=True)
(tool / "usr/share/fex-emu/GuestThunks").mkdir(parents=True)
self.write(tool / "usr/share/fex-emu/ThunksDB.json", "{}")
self.write(Path(library or self.steam) / "steamapps/appmanifest_3127680.acf", '"AppState" { "installdir" "FEX-Emu" }')
return tool
def runtime(self, version="3.0.20260805.254768", lines=None):
runtime = self.steam / "steamapps/common/SteamLinuxRuntime_sniper"
platform = runtime / ("sniper_platform_" + version)
files = platform / "files"
self.write(files / "lib/x86_64-linux-gnu/libc.so.6", elf(62))
self.write(files / "ab/cdef-1.bin", b"blob")
self.write(files / "share/ca/real.crt", b"cert")
mtree = lines or [
"#mtree",
". type=dir",
"./bin type=dir",
"./lib64 type=dir",
"./lib64/ld-linux-x86-64.so.2 type=link link=/usr/lib/x86_64-linux-gnu/ld-2.31.so",
"./lib/x86_64-linux-gnu/libc.so.6 type=file mode=644 size=64",
"./bin/\\133 type=file mode=755 size=4 contents=./ab/cdef-1.bin",
"./share/ca/Na\\075me.crt type=link link=real.crt",
"./etc/ssl/cert.pem type=link link=/etc/ssl/certs/ca.pem",
]
with gzip.open(platform / "usr-mtree.txt.gz", "wt") as out:
out.write("\n".join(mtree) + "\n")
self.write(self.steam / "steamapps/appmanifest_1628350.acf", '"AppState" { "installdir" "SteamLinuxRuntime_sniper" }')
return platform
def quiet(self, function, *args):
with contextlib.redirect_stderr(io.StringIO()) as err:
return function(*args), err.getvalue()
class ArchitectureTest(FexTestCase):
def test_elf_machines(self):
self.assertEqual(FEX["elf_arch"](self.write(self.tmp / "a", elf(62))), "x86_64")
self.assertEqual(FEX["elf_arch"](self.write(self.tmp / "b", elf(3))), "i386")
self.assertEqual(FEX["elf_arch"](self.write(self.tmp / "c", elf(183))), "arm64")
self.assertEqual(FEX["elf_arch"](self.write(self.tmp / "d", elf(62, data=2))), "x86_64")
self.assertEqual(FEX["elf_arch"](self.write(self.tmp / "e", elf(8))), "other")
self.assertIsNone(FEX["elf_arch"](self.write(self.tmp / "f", "#!/bin/sh\n")))
self.assertIsNone(FEX["elf_arch"](self.tmp / "missing"))
def test_a_script_follows_the_programs_beside_it(self):
game = self.tmp / "game"
script = self.write(game / "start.sh", "#!/bin/bash\n./bin/game\n")
self.assertFalse(FEX["needs_fex"](str(script)))
self.write(game / "bin/game.x86_64", elf(62), 0o755)
self.assertTrue(FEX["needs_fex"](str(script)))
self.assertTrue(FEX["needs_fex"](str(game)))
self.write(game / "bin/game.arm64", elf(183), 0o755)
self.assertFalse(FEX["needs_fex"](str(script)))
self.assertTrue(FEX["needs_fex"](str(game / "bin/game.x86_64")))
self.assertFalse(FEX["needs_fex"](str(game / "bin/game.arm64")))
def test_the_scan_stops_at_its_depth(self):
game = self.tmp / "deep"
self.write(game / "a/b/c/d/e/game", elf(62), 0o755)
self.assertFalse(FEX["needs_fex"](str(game)))
self.write(game / "a/b/game", elf(62), 0o755)
self.assertTrue(FEX["needs_fex"](str(game)))
def test_appimage_offset_is_the_end_of_the_section_headers(self):
head = bytearray(elf(62))
struct.pack_into("<Q", head, 0x28, 1000)
struct.pack_into("<HH", head, 0x3A, 64, 30)
self.assertEqual(FEX["appimage_offset"](self.write(self.tmp / "x.AppImage", bytes(head))), 1000 + 64 * 30)
head = bytearray(elf(3))
head[4] = 1
struct.pack_into("<I", head, 0x20, 500)
struct.pack_into("<HH", head, 0x2E, 40, 10)
self.assertEqual(FEX["appimage_offset"](self.write(self.tmp / "y.AppImage", bytes(head))), 900)
def test_run_options(self):
self.assertEqual(FEX["parse_run"](["--mode", "on", "--for", "/x", "--", "a", "--mode"]), ("on", "/x", ["a", "--mode"]))
self.assertEqual(FEX["parse_run"](["a"]), ("auto", None, ["a"]))
with self.assertRaises(ValueError):
FEX["parse_run"](["--mode", "maybe", "a"])
with self.assertRaises(ValueError):
FEX["parse_run"](["--bogus", "a"])
class LocateTest(FexTestCase):
def test_tools_are_found_in_any_library(self):
self.assertIsNone(FEX["fex_install"]() if not os.access("/usr/bin/FEX", os.X_OK) else None)
sd = self.tmp / "sd"
(sd / "steamapps/common").mkdir(parents=True)
tool = self.fex_tool(sd)
fex = FEX["fex_install"]()
self.assertEqual((fex["bin"], fex["server"], fex["portable"]), (str(tool / "usr/bin/FEX"), str(tool / "usr/bin/FEXServer"), True))
self.assertTrue(fex["host_thunks"].endswith("HostThunks") and fex["guest_thunks"].endswith("GuestThunks") and fex["thunks_db"])
def test_library_folders_are_read(self):
other = self.tmp / "other"
(other / "steamapps/common").mkdir(parents=True)
self.write(self.steam / "steamapps/libraryfolders.vdf", '"libraryfolders" { "1" { "path" "%s" } }' % other)
tool = self.fex_tool(other)
self.assertEqual(FEX["app_dir"]("3127680"), str(tool))
def test_the_newest_runtime_platform_wins(self):
self.runtime("3.0.20260805.254768")
newer = self.runtime("3.0.20261001.1")
self.assertEqual(FEX["runtime_platform"](), str(newer))
class PrepareTest(FexTestCase):
def test_materialize_links_the_tree_in_place(self):
platform = self.runtime()
files = platform / "files"
self.assertTrue(FEX["materialize"](str(platform)))
self.assertEqual(os.readlink(files / "lib64/ld-linux-x86-64.so.2"), "../lib/x86_64-linux-gnu/ld-2.31.so")
self.assertEqual(os.readlink(files / "bin/["), "../ab/cdef-1.bin")
self.assertEqual((files / "bin/[").read_bytes(), b"blob")
self.assertEqual((files / "share/ca/Na=me.crt").read_bytes(), b"cert")
self.assertEqual(os.readlink(files / "etc/ssl/cert.pem"), "certs/ca.pem")
self.assertEqual(os.readlink(files / "usr"), ".")
self.assertEqual((files / "usr/lib/x86_64-linux-gnu/libc.so.6").read_bytes()[:4], b"\x7fELF")
self.assertFalse(FEX["materialize"](str(platform)))
os.unlink(files / "bin/[")
os.utime(platform / "usr-mtree.txt.gz", ns=(1, 1))
self.assertTrue(FEX["materialize"](str(platform)))
self.assertTrue(os.path.islink(files / "bin/["))
def test_materialize_ignores_paths_outside_the_tree(self):
platform = self.runtime(lines=["./../escape type=link link=x", "/abs type=dir", "./ok type=link link=x"])
FEX["materialize"](str(platform))
self.assertFalse((platform / "escape").exists() or os.path.islink(platform / "escape"))
self.assertTrue(os.path.islink(platform / "files/ok"))
def test_preloads_mirror_the_host_list(self):
root = self.tmp / "root"
for arch in ("x86_64", "i386"):
for name in ("libblsession.so", "libfakeinput.so"):
self.write(self.tmp / "preloads" / arch / name, arch.encode() + name.encode())
self.write(self.tmp / "preloads/x86_64/libfaultreport.so", b"report")
self.write(self.tmp / "preloads/x86_64/libthunkaudit.so", b"audit")
self.write(self.tmp / "preloads/x86_64/libvulkan-thunk.so", b"pin")
self.write(self.tmp / "ld.so.preload", "/usr/local/lib/libblsession.so\n/usr/local/lib/libfakeinput.so\n")
FEX["install_preloads"](str(root))
self.assertEqual((root / "lib/x86_64-linux-gnu/droiddeck/libfaultreport.so").read_bytes(), b"report")
self.assertEqual((root / "lib/x86_64-linux-gnu/droiddeck/libthunkaudit.so").read_bytes(), b"audit")
self.assertEqual((root / "lib/x86_64-linux-gnu/droiddeck/libvulkan-thunk.so").read_bytes(), b"pin")
self.assertEqual((root / "etc/ld.so.preload").read_text(),
"/usr/$LIB/droiddeck/libblsession.so\n/usr/$LIB/droiddeck/libfakeinput.so\n")
self.assertEqual((root / "lib/i386-linux-gnu/droiddeck/libfakeinput.so").read_bytes(), b"i386libfakeinput.so")
self.write(self.tmp / "ld.so.preload", "/usr/local/lib/libblsession.so\n")
FEX["install_preloads"](str(root))
self.assertEqual((root / "etc/ld.so.preload").read_text(), "/usr/$LIB/droiddeck/libblsession.so\n")
def test_config_turns_thunks_on_only_when_complete(self):
FEX["write_config"]({"host_thunks": "/h", "guest_thunks": "/g", "thunks_db": "/db"}, "/root")
config = json.loads((self.home / ".local/share/droiddeck-fex/config/Config.json").read_text())
self.assertEqual(config, {"Config": {"RootFS": "/root", "ThunkHostLibs": "/h", "ThunkGuestLibs": "/g"}, "ThunksDB": {"GL": 1, "Vulkan": 1}})
FEX["write_config"]({"host_thunks": None, "guest_thunks": "/g", "thunks_db": "/db"}, "/root")
config = json.loads((self.home / ".local/share/droiddeck-fex/config/Config.json").read_text())
self.assertEqual(config, {"Config": {"RootFS": "/root"}})
def test_missing_pieces_are_asked_for(self):
with mock.patch.dict(G, {"fex_install": lambda: None, "steam_running": lambda: False}):
fex, said = self.quiet(FEX["prepare"])
self.assertIsNone(fex)
self.assertIn("FEX (Steam app 3127680)", said)
self.assertIn("Steam Linux Runtime 3.0", said)
self.assertTrue(said.startswith("droiddeck-fex: x86 Linux programs need "))
self.assertTrue((self.home / ".config/droiddeck/fex-wanted").is_file())
status = json.loads((self.home / ".local/share/droiddeck-fex/status.json").read_text())
self.assertFalse(status["ready"])
def test_a_ready_install_is_prepared(self):
tool = self.fex_tool()
platform = self.runtime()
self.write(self.home / ".config/droiddeck/fex-wanted", "1\n")
fex, said = self.quiet(FEX["prepare"])
self.assertEqual(fex["bin"], str(tool / "usr/bin/FEX"))
self.assertIn("prepared sniper_platform_", said)
self.assertTrue(os.path.islink(platform / "files/usr"))
status = json.loads((self.home / ".local/share/droiddeck-fex/status.json").read_text())
self.assertEqual((status["ready"], status["runtime"], status["thunks"]), (True, str(platform), True))
self.assertFalse((self.home / ".config/droiddeck/fex-wanted").exists())
self.assertEqual(self.quiet(FEX["prepare"])[1], "")
def test_status_refreshes_what_the_app_reads_once_steam_installed_fex(self):
with mock.patch.dict(G, {"steam_running": lambda: False}):
self.quiet(FEX["prepare"])
status_file = self.home / ".local/share/droiddeck-fex/status.json"
self.assertFalse(json.loads(status_file.read_text())["ready"])
self.fex_tool()
self.runtime()
with contextlib.redirect_stdout(io.StringIO()) as out:
self.assertEqual(0, FEX["main"](["status"]))
self.assertTrue(json.loads(out.getvalue())["ready"])
self.assertTrue(json.loads(status_file.read_text())["ready"])
def test_fex_is_found_at_the_tool_root_too(self):
tool = self.steam / "steamapps/common/FEX-Emu"
self.write(tool / "bin/FEX", elf(183), 0o755)
self.write(tool / "bin/FEXServer", elf(183), 0o755)
self.write(self.steam / "steamapps/appmanifest_3127680.acf", '"AppState" { "installdir" "FEX-Emu" }')
fex = FEX["fex_install"]()
self.assertEqual((fex["bin"], fex["server"], fex["portable"]), (str(tool / "bin/FEX"), str(tool / "bin/FEXServer"), True))
def test_an_installed_tool_without_fex_says_what_it_holds(self):
tool = self.steam / "steamapps/common/FEX-Emu"
self.write(tool / "toolmanifest.vdf", "")
self.write(self.steam / "steamapps/appmanifest_3127680.acf", '"AppState" { "installdir" "FEX-Emu" }')
with mock.patch.dict(G, {"steam_running": lambda: False}), mock.patch("shutil.which", return_value=None), \
mock.patch("os.access", lambda path, mode: False):
fex, said = self.quiet(FEX["prepare"])
self.assertIsNone(fex)
self.assertIn("holds no FEX or FEXInterpreter", said)
self.assertIn("toolmanifest.vdf", said)
def test_a_missing_server_is_not_ready(self):
self.fex_tool(server=False)
self.runtime()
with mock.patch.dict(G, {"steam_running": lambda: False}), mock.patch("shutil.which", return_value=None):
fex, said = self.quiet(FEX["prepare"])
self.assertIsNone(fex)
self.assertIn("FEXServer beside", said)
class ExtractTest(FexTestCase):
def setUp(self):
super().setUp()
self.work = self.tmp / "work"
self.work.mkdir()
self.image = self.write(self.work / "image.AppImage", elf(62) + b"payload", 0o755)
chdir = contextlib.chdir(self.work)
chdir.__enter__()
self.addCleanup(chdir.__exit__, None, None, None)
def uruntime(self, body):
return str(self.write(self.tmp / "uruntime", "#!/bin/sh\n" + body, 0o755))
def test_uruntime_unpacks_an_x86_image_without_fex(self):
tool = self.uruntime('[ "$1" = --appimage-extract ] && [ "$TARGET_APPIMAGE" = "%s" ] || exit 9\nmkdir -p AppDir && touch AppDir/AppRun\n' % self.image)
with mock.patch.dict(G, {"URUNTIME": tool, "fex_install": lambda: None, "steam_running": lambda: False}):
status, said = self.quiet(FEX["extract"], "image.AppImage")
self.assertEqual((status, said), (0, ""))
self.assertEqual(os.readlink("squashfs-root"), "AppDir")
self.assertTrue(Path("squashfs-root/AppRun").is_file())
self.assertTrue((self.home / ".config/droiddeck/fex-wanted").is_file())
self.assertFalse(json.loads((self.home / ".local/share/droiddeck-fex/status.json").read_text())["ready"])
def test_an_arm64_image_does_not_ask_for_fex(self):
self.write(self.image, elf(183) + b"payload", 0o755)
tool = self.uruntime("mkdir -p squashfs-root\n")
with mock.patch.dict(G, {"URUNTIME": tool}):
status, said = self.quiet(FEX["extract"], "image.AppImage")
self.assertEqual((status, said), (0, ""))
self.assertFalse((self.home / ".config/droiddeck/fex-wanted").exists())
def test_nothing_to_unpack_with_says_so(self):
tool = self.uruntime("mkdir -p squashfs-root\nexit 1\n")
with mock.patch.dict(G, {"URUNTIME": tool, "fex_install": lambda: None, "steam_running": lambda: False}), \
mock.patch("shutil.which", return_value=None):
status, said = self.quiet(FEX["extract"], "image.AppImage")
self.assertEqual(status, 1)
self.assertIn("uruntime could not extract it (status 1)", said)
self.assertIn("no AppImage runtime for ARM64, no FEX yet and no unsquashfs", said)
self.assertFalse(os.path.lexists("squashfs-root"))
class LaunchTest(FexTestCase):
def test_the_guest_environment(self):
arm = self.write(self.tmp / "arm.so", elf(183))
x86 = self.write(self.tmp / "x86.so", elf(62))
env = {"LD_PRELOAD": "%s:%s %s" % (arm, x86, self.tmp / "gone.so"),
"LD_LIBRARY_PATH": "/steam/steamrtarm64:/usr/lib/aarch64-linux-gnu:/game/lib",
"FEX_ROOTFS": "/usr/share/guestos/fex-mesa", "FEX_TSOENABLED": "1"}
guest = FEX["guest_environment"]({"portable": True}, env)
self.assertEqual((guest["LD_PRELOAD"], guest["LD_LIBRARY_PATH"], guest["FEX_PORTABLE"]), (str(x86), "/game/lib", "1"))
self.assertNotIn("FEX_ROOTFS", guest)
self.assertEqual(guest["FEX_TSOENABLED"], "1")
self.assertTrue(guest["FEX_APP_CONFIG_LOCATION"].endswith("/config/"))
self.assertEqual((guest["FEX_SILENTLOG"], guest["FEX_OUTPUTLOG"]), ("0", "stderr"))
guest = FEX["guest_environment"]({"portable": False}, {"LD_PRELOAD": str(arm), "FEX_PORTABLE": "1", "FEX_OUTPUTLOG": "/tmp/fex.log"})
self.assertEqual(guest["FEX_OUTPUTLOG"], "/tmp/fex.log")
self.assertNotIn("LD_PRELOAD", guest)
self.assertNotIn("FEX_PORTABLE", guest)
def test_the_runtime_and_thunks_are_passed_to_fex(self):
fex = {"portable": True, "rootfs": "/rt/files", "host_thunks": "/h", "guest_thunks": "/g", "thunks_db": "/db"}
guest = FEX["guest_environment"](fex, {})
self.assertEqual((guest["FEX_ROOTFS"], guest["FEX_THUNKHOSTLIBS"], guest["FEX_THUNKGUESTLIBS"]), ("/rt/files", "/h", "/g"))
self.assertTrue(guest["FEX_THUNKCONFIG"].endswith("/config/ThunksConfig.json"))
guest = FEX["guest_environment"](dict(fex, host_thunks=None), {})
self.assertEqual(guest["FEX_ROOTFS"], "/rt/files")
self.assertNotIn("FEX_THUNKCONFIG", guest)
FEX["write_config"](fex, "/rt/files")
thunks = json.loads((self.home / ".local/share/droiddeck-fex/config/ThunksConfig.json").read_text())
self.assertEqual(thunks, {"ThunksDB": {"GL": 1, "Vulkan": 1}})
def test_native_programs_run_directly(self):
program = self.write(self.tmp / "native", elf(183), 0o755)
with mock.patch("os.execvpe") as native, mock.patch("os.execve") as emulated:
FEX["launch"]([str(program), "--flag"], {"PATH": "/usr/bin"}, "auto")
native.assert_called_once_with(str(program), [str(program), "--flag"], {"PATH": "/usr/bin"})
emulated.assert_not_called()
def test_x86_programs_run_under_fex(self):
tool = self.fex_tool()
self.runtime()
program = self.write(self.tmp / "game.x86_64", elf(62), 0o755)
with mock.patch("os.execvpe") as native, mock.patch("os.execve") as emulated:
self.quiet(FEX["launch"], [str(program), "-w"], {"PATH": "/usr/bin"}, "auto")
native.assert_not_called()
binary, argv, env = emulated.call_args[0]
self.assertEqual((binary, argv), (str(tool / "usr/bin/FEX"), [str(tool / "usr/bin/FEX"), str(program), "-w"]))
self.assertEqual(env["FEX_PORTABLE"], "1")
self.assertNotIn("LD_PRELOAD", env)
self.assertEqual(env["FEX_ROOTFS"], str(self.steam / "steamapps/common/SteamLinuxRuntime_sniper/sniper_platform_3.0.20260805.254768/files"))
self.write(self.tmp / "preloads/x86_64/libfaultreport.so", elf(62))
x86 = self.write(self.tmp / "x86.so", elf(62))
with mock.patch("os.execve") as emulated:
self.quiet(FEX["launch"], [str(program)], {"LD_PRELOAD": str(x86)}, "auto")
self.assertEqual(emulated.call_args[0][2]["LD_PRELOAD"], "/usr/lib/x86_64-linux-gnu/droiddeck/libfaultreport.so:%s" % x86)
old = self.write(self.tmp / "game.i386", elf(3), 0o755)
with mock.patch("os.execve") as emulated:
self.quiet(FEX["launch"], [str(old)], {}, "auto")
self.assertNotIn("LD_PRELOAD", emulated.call_args[0][2])
with mock.patch("os.execvpe") as native, mock.patch("os.execve") as emulated:
self.quiet(FEX["launch"], [str(program)], {}, "off")
native.assert_called_once()
emulated.assert_not_called()
def test_programs_that_bring_a_new_enough_runtime_get_the_vulkan_thunk(self):
tool = self.fex_tool()
self.runtime()
self.write(tool / "usr/share/fex-emu/GuestThunks/libvulkan-guest.so",
elf(62) + b"\0libstdc++.so.6\0GLIBCXX_3.4\0GLIBCXX_3.4.29\0libc.so.6\0GLIBC_2.2.5\0GLIBC_2.34\0")
self.write(self.tmp / "preloads/x86_64/libthunkaudit.so", elf(62))
pin = self.write(self.tmp / "preloads/x86_64/libvulkan-thunk.so", elf(62))
audit = "/usr/lib/x86_64-linux-gnu/droiddeck/libthunkaudit.so"
def app(name, glibc=b"GLIBC_2.34\0GLIBC_2.44", glibcxx=b"GLIBCXX_3.4.29\0GLIBCXX_3.4.36", bundled=True):
run = self.write(self.tmp / name / "AppRun", elf(62), 0o755)
if bundled:
self.write(self.tmp / name / "shared/lib/libc.so.6", elf(62) + b"\0GLIBC_2.2.5\0" + glibc + b"\0")
self.write(self.tmp / name / "shared/lib/libstdc++.so.6",
elf(62) + b"\0GLIBCXX_3.4\0" + glibcxx + b"\0GLIBC_2.34\0")
return run
def started(program, env=None, decide=None):
with mock.patch("os.execve") as emulated:
_, said = self.quiet(FEX["launch"], [str(program)], dict(env or {}), "on", decide)
return emulated.call_args[0][2], said
env, said = started(app("new"))
self.assertEqual(env["LD_AUDIT"], audit)
self.assertIn("thunks on, its own Vulkan loader too", said)
env, _ = started(app("kept"), {"LD_AUDIT": "/opt/audit.so"})
self.assertEqual(env["LD_AUDIT"], audit + ":/opt/audit.so")
env, _ = started(self.write(self.tmp / "elsewhere/run.sh", "#!/bin/sh\n", 0o755), decide=str(self.tmp / "new"))
self.assertEqual(env["LD_AUDIT"], audit)
env, said = started(app("old-cxx", glibcxx=b"GLIBCXX_3.4.28"))
self.assertNotIn("LD_AUDIT", env)
self.assertIn("thunks on)", said)
self.assertNotIn("LD_AUDIT", started(app("old-libc", glibc=b"GLIBC_2.31"))[0])
self.assertNotIn("LD_AUDIT", started(app("runtime-only", bundled=False))[0])
i386 = self.write(self.tmp / "new/game.i386", elf(3), 0o755)
self.assertNotIn("LD_AUDIT", started(i386)[0])
pin.unlink()
self.assertNotIn("LD_AUDIT", started(app("no-pin"))[0])
self.write(pin, elf(62))
(tool / "usr/share/fex-emu/GuestThunks/libvulkan-guest.so").unlink()
self.assertNotIn("LD_AUDIT", started(app("no-thunk"))[0])
def test_steam_verbs(self):
with mock.patch.dict(G, {"launch": mock.Mock(return_value=0)}):
self.assertEqual(FEX["main"](["steam", "getcompatpath", "/x"]), 0)
G["launch"].assert_not_called()
FEX["main"](["steam", "waitforexitandrun", "/game/run.sh", "-a"])
command, env, mode = G["launch"].call_args[0]
self.assertEqual((command, mode), (["/game/run.sh", "-a"], "auto"))
class AppInfoTest(unittest.TestCase):
COMPAT = runpy.run_path(str(BIN / "steam-compatibility"))
@staticmethod
def appinfo(apps, magic=0x07564429):
strings = []
def key(name):
if magic != 0x07564429:
return name.encode() + b"\0"
if name not in strings:
strings.append(name)
return struct.pack("<I", strings.index(name))
def section(items):
out = b""
for name, value in items.items():
if isinstance(value, dict):
out += b"\x00" + key(name) + section(value)
else:
out += b"\x01" + key(name) + value.encode() + b"\0"
return out + b"\x08"
body = b""
for app, (oslist, osarch, kind) in apps.items():
data = section({"appinfo": {"appid": str(app), "common": {"type": kind, "oslist": oslist, "osarch": osarch}}})
header = struct.pack("<IIQ", 2, 0, 0) + b"\0" * 20 + struct.pack("<I", 1) + (b"\0" * 20 if magic >= 0x07564428 else b"")
body += struct.pack("<II", app, len(header) + len(data)) + header + data
body += struct.pack("<I", 0)
if magic != 0x07564429:
return struct.pack("<II", magic, 1) + body
table_at = 16 + len(body)
table = struct.pack("<I", len(strings)) + b"".join(s.encode() + b"\0" for s in strings)
return struct.pack("<IIq", magic, 1, table_at) + body + table
def test_linux_only_titles_are_sent_to_fex(self):
for magic in (0x07564427, 0x07564428, 0x07564429):
with tempfile.TemporaryDirectory() as tmp:
steam = Path(tmp)
(steam / "appcache").mkdir()
(steam / "appcache/appinfo.vdf").write_bytes(self.appinfo({
10: ("windows", "", "Game"), 20: ("linux", "64", "Game"), 30: ("windows,linux,macos", "", "game"),
40: ("linux", "arm64", "game"), 50: ("linux", "", "Demo"), 70: ("linux", "", "Tool"), 80: ("linux", "", "config"),
90: ("linux", "", ""),
}, magic))
self.COMPAT["APPINFO_CACHE"].clear()
targets = self.COMPAT["native_targets"](str(steam), ["10", "20", "30", "40", "50", "70", "80", "90", "1628350", "60"])
self.assertEqual(targets, {"20": "droiddeck-fex", "50": "droiddeck-fex"}, hex(magic))
def test_the_mapping_sends_native_titles_to_fex_and_keeps_picks(self):
tool, fex = self.COMPAT["TOOL"], self.COMPAT["FEX_TOOL"]
changes, auto = self.COMPAT["plan_mapping"](
{"0": tool, "20": "fex", "21": "steamlinuxruntime_sniper", "22": "GE-Proton11-7"},
["20", "21", "22", "23", "24"], [tool, "GE-Proton11-7"], tool, {},
{"20": fex, "21": fex, "22": fex, "23": fex})
self.assertEqual(changes, {"20": fex, "21": fex, "23": fex, "24": tool})
self.assertEqual(auto, {"20": fex, "21": fex, "23": fex, "24": tool})
def test_the_tool_files(self):
with tempfile.TemporaryDirectory() as tmp:
tool = Path(tmp) / "droiddeck-fex"
self.COMPAT["build_fex_tool"](str(tool))
self.assertEqual(sorted(os.listdir(tool)), sorted(self.COMPAT["FEX_FILES"]))
self.assertIn('"commandline" "/droiddeck-fex-tool %verb%"', (tool / "toolmanifest.vdf").read_text())
self.assertTrue(os.access(tool / "droiddeck-fex-tool", os.X_OK))
vdf = (tool / "compatibilitytool.vdf").read_text()
self.assertIn('"from_oslist" "linux"', vdf)
self.assertIn('"to_oslist" "linux"', vdf)
if __name__ == "__main__":
unittest.main()
File diff suppressed because it is too large. Load diff
+346
View File
@@ -0,0 +1,346 @@
import os
from pathlib import Path
import shutil
import subprocess
import tempfile
import unittest
PRELOAD = Path(__file__).resolve().parents[1] / "linuxfs/preload"
PROGRAM = r"""
#define _GNU_SOURCE
#include <errno.h>
#include <fcntl.h>
#include <linux/futex.h>
#include <pthread.h>
#include <signal.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <sys/wait.h>
#include <time.h>
#include <unistd.h>
struct waitv { uint64_t val; uint64_t uaddr; uint32_t flags; uint32_t reserved; };
static char name[64];
static int *page;
static long waitv(int **addrs, int *vals, int n, const struct timespec *ts, int clk) {
struct waitv v[8];
for (int i = 0; i < n; i++) {
v[i].val = (uint32_t)vals[i];
v[i].uaddr = (uintptr_t)addrs[i];
v[i].flags = 2;
v[i].reserved = 0;
}
return syscall(449, v, n, 0, ts, clk);
}
static long waitv_retry(int **addrs, int *vals, int n) {
long r;
do r = waitv(addrs, vals, n, NULL, 0); while (r == -1 && errno == EINTR);
return r;
}
static void wake(int *addr) {
syscall(SYS_futex, addr, FUTEX_WAKE, 0x7fffffff, NULL, NULL, 0);
}
static void deadline(struct timespec *ts, int clk, long ms) {
clock_gettime(clk, ts);
ts->tv_sec += ms / 1000;
ts->tv_nsec += (ms % 1000) * 1000000;
if (ts->tv_nsec >= 1000000000) {
ts->tv_sec++;
ts->tv_nsec -= 1000000000;
}
}
static double now(void) {
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
return ts.tv_sec + ts.tv_nsec / 1e9;
}
static int *map_page(int fd, int index) {
int *p = mmap(NULL, 0x10000, PROT_READ | PROT_WRITE, MAP_SHARED, fd, (off_t)index * 0x10000);
if (p == MAP_FAILED) { perror("mmap"); exit(2); }
return p;
}
#define CHECK(c, ...) do { if (!(c)) { printf("FAIL %d: ", __LINE__); printf(__VA_ARGS__); printf("\n"); exit(1); } } while (0)
static int client(void) {
int fd = shm_open(name, O_RDWR, 0644);
int *p, *addrs[3];
int vals[3] = {0, 0, 0};
long r;
CHECK(fd >= 0, "client open %d", errno);
p = map_page(fd, 1);
addrs[0] = &p[0]; addrs[1] = &p[4]; addrs[2] = &p[8];
__atomic_store_n(&p[12], 1, __ATOMIC_SEQ_CST);
wake(&p[12]);
r = waitv_retry(addrs, vals, 3);
CHECK(r == 2, "client woke with %ld errno %d", r, errno);
__atomic_store_n(&p[16], 1, __ATOMIC_SEQ_CST);
wake(&p[16]);
return 0;
}
static int sleeper(void) {
int fd = shm_open(name, O_RDWR, 0644);
int *p = map_page(fd, 1), *a = &p[20], v = 0;
__atomic_store_n(&p[24], 1, __ATOMIC_SEQ_CST);
wake(&p[24]);
for (;;) waitv(&a, &v, 1, NULL, 0);
return 0;
}
struct sem { int *count; int n; };
static int *sems;
static volatile int taken, given;
#define SEMS 6
#define ROUNDS 20000
static void *consumer(void *arg) {
int got = 0;
while (got < ROUNDS) {
int *addrs[SEMS], vals[SEMS], i;
for (i = 0; i < SEMS; i++) {
int *c = &sems[i * 4], cur = __atomic_load_n(c, __ATOMIC_SEQ_CST);
while (cur > 0) {
if (__atomic_compare_exchange_n(c, &cur, cur - 1, 0, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST)) { got++; goto next; }
}
addrs[i] = c;
vals[i] = 0;
}
waitv(addrs, vals, SEMS, NULL, 0);
next:;
}
__atomic_add_fetch(&taken, got, __ATOMIC_SEQ_CST);
return arg;
}
static int churn_fd;
static volatile int churn_stop;
static void *churn(void *arg) {
while (!churn_stop) munmap(map_page(churn_fd, 1), 0x10000);
return arg;
}
static void *one_wait(void *arg) {
int *q = map_page(churn_fd, 1), *a = &q[64], v = 0;
long r;
do r = waitv(&a, &v, 1, NULL, 0); while (r == -1 && errno == EINTR && !__atomic_load_n(a, __ATOMIC_SEQ_CST));
munmap(q, 0x10000);
return (void *)(intptr_t)(r == -1 && errno == EINTR);
}
static int used_slots(void) {
char reg[96];
int fd, n = 0;
uint64_t *head;
snprintf(reg, sizeof(reg), "%s-droiddeck", name);
fd = shm_open(reg, O_RDONLY, 0);
if (fd < 0) return -1;
head = mmap(NULL, 4096, PROT_READ, MAP_SHARED, fd, 0);
close(fd);
if (head == MAP_FAILED) return -1;
for (int i = 0; i < 32; i++) n += __builtin_popcountll(__atomic_load_n(&head[3 + i], __ATOMIC_SEQ_CST));
munmap(head, 4096);
return n;
}
static void *producer(void *arg) {
unsigned seed = (unsigned)(uintptr_t)arg;
for (int i = 0; i < ROUNDS; i++) {
int *c = &sems[(rand_r(&seed) % SEMS) * 4];
__atomic_add_fetch(c, 1, __ATOMIC_SEQ_CST);
wake(c);
__atomic_add_fetch(&given, 1, __ATOMIC_SEQ_CST);
}
return arg;
}
int main(int argc, char **argv) {
snprintf(name, sizeof(name), "/wine-%x-fsync", (unsigned)getpid() ^ 0x5a5a00);
if (argc > 2) snprintf(name, sizeof(name), "%s", argv[2]);
if (argc > 1 && !strcmp(argv[1], "client")) return client();
if (argc > 1 && !strcmp(argv[1], "sleeper")) return sleeper();
{
long r = syscall(449, NULL, 0, 0, NULL, 0);
CHECK(r == -1 && errno == EINVAL, "probe %ld errno %d", r, errno);
}
int fd = shm_open(name, O_RDWR | O_CREAT | O_EXCL, 0644);
CHECK(fd >= 0, "create %d", errno);
CHECK(ftruncate(fd, 0x20000) == 0, "truncate");
page = map_page(fd, 1);
{
char reg[96];
struct stat st;
snprintf(reg, sizeof(reg), "/dev/shm%s-droiddeck", name);
CHECK(stat(reg, &st) == 0 && st.st_size > 0, "registry %s missing", reg);
}
{
int *a = &page[40], v = 1;
long r = waitv(&a, &v, 1, NULL, 0);
CHECK(r == -1 && errno == EAGAIN, "mismatch %ld errno %d", r, errno);
}
for (int clk = 0; clk < 2; clk++) {
int c = clk ? CLOCK_REALTIME : CLOCK_MONOTONIC, *a = &page[44], v = 0;
struct timespec ts;
double t = now();
long r;
deadline(&ts, c, 150);
do r = waitv(&a, &v, 1, &ts, c); while (r == -1 && errno == EINTR);
t = now() - t;
CHECK(r == -1 && errno == ETIMEDOUT && t > 0.12 && t < 1.5, "timeout clock %d r %ld errno %d after %.3f", c, r, errno, t);
}
{
pid_t pid = fork();
if (!pid) { execl("/proc/self/exe", argv[0], "client", name, (char *)NULL); _exit(3); }
int *ready = &page[12], *done = &page[16], z = 0, st;
double t = now();
while (!__atomic_load_n(ready, __ATOMIC_SEQ_CST) && now() - t < 20) waitv(&ready, &z, 1, NULL, 0);
usleep(20000);
__atomic_store_n(&page[8], 1, __ATOMIC_SEQ_CST);
wake(&page[8]);
while (!__atomic_load_n(done, __ATOMIC_SEQ_CST) && now() - t < 20) waitv(&done, &z, 1, NULL, 0);
CHECK(__atomic_load_n(done, __ATOMIC_SEQ_CST), "client never finished");
waitpid(pid, &st, 0);
CHECK(WIFEXITED(st) && WEXITSTATUS(st) == 0, "client status %d", st);
}
{
int rounds = 2000, z = 0;
double t = now();
pid_t pid = fork();
if (!pid) {
int fd2 = shm_open(name, O_RDWR, 0644);
int *p = map_page(fd2, 1);
for (int i = 0; i < rounds; i++) {
int *ping = &p[48], *pong = &p[52];
while (__atomic_load_n(ping, __ATOMIC_SEQ_CST) == i * 2) { int v = i * 2; waitv(&ping, &v, 1, NULL, 0); }
__atomic_store_n(pong, i * 2 + 2, __ATOMIC_SEQ_CST);
wake(pong);
}
_exit(0);
}
for (int i = 0; i < rounds; i++) {
int *ping = &page[48], *pong = &page[52];
__atomic_store_n(ping, i * 2 + 2, __ATOMIC_SEQ_CST);
wake(ping);
while (__atomic_load_n(pong, __ATOMIC_SEQ_CST) == i * 2) { int v = i * 2; waitv(&pong, &v, 1, NULL, 0); }
}
waitpid(pid, NULL, 0);
printf("pingpong %.2f us/round\n", (now() - t) * 1e6 / rounds);
(void)z;
}
{
pthread_t c[4], p[4];
sems = &page[256];
for (int i = 0; i < 4; i++) pthread_create(&c[i], NULL, consumer, NULL);
for (int i = 0; i < 4; i++) pthread_create(&p[i], NULL, producer, (void *)(uintptr_t)(i + 1));
for (int i = 0; i < 4; i++) pthread_join(p[i], NULL);
for (int i = 0; i < 4; i++) pthread_join(c[i], NULL);
CHECK(taken == 4 * ROUNDS && given == 4 * ROUNDS, "stress taken %d given %d", taken, given);
for (int i = 0; i < SEMS; i++) CHECK(sems[i * 4] == 0, "sem %d left %d", i, sems[i * 4]);
}
for (int round = 0; round < 3; round++) {
pid_t kids[40];
int z = 0;
for (int i = 0; i < 40; i++) {
__atomic_store_n(&page[24], 0, __ATOMIC_SEQ_CST);
kids[i] = fork();
if (!kids[i]) { execl("/proc/self/exe", argv[0], "sleeper", name, (char *)NULL); _exit(3); }
int *r = &page[24];
while (!__atomic_load_n(r, __ATOMIC_SEQ_CST)) waitv(&r, &z, 1, NULL, 0);
}
for (int i = 0; i < 40; i++) { kill(kids[i], SIGKILL); waitpid(kids[i], NULL, 0); }
double t = now();
for (int i = 0; i < 1000; i++) wake(&page[20]);
CHECK(now() - t < 2.0, "wakes after killed sleepers took %.3f", now() - t);
}
{
pthread_t churners[2], t;
int rescued = 0, threads = 3000;
double t0 = now();
churn_fd = fd;
for (int i = 0; i < 2; i++) pthread_create(&churners[i], NULL, churn, NULL);
for (int i = 0; i < threads; i++) {
void *res;
__atomic_store_n(&page[64], 0, __ATOMIC_SEQ_CST);
pthread_create(&t, NULL, one_wait, NULL);
if (i & 1) usleep(50);
__atomic_store_n(&page[64], 1, __ATOMIC_SEQ_CST);
wake(&page[64]);
pthread_join(t, &res);
rescued += (int)(intptr_t)res;
}
churn_stop = 1;
for (int i = 0; i < 2; i++) pthread_join(churners[i], NULL);
printf("thread churn %.2f s, %d of %d waits needed the nap\n", now() - t0, rescued, threads);
CHECK(rescued <= 3 && now() - t0 < 30, "thread churn: %d naps in %.2f s", rescued, now() - t0);
usleep(1100000);
__atomic_store_n(&page[64], 0, __ATOMIC_SEQ_CST);
pthread_create(&t, NULL, one_wait, NULL);
usleep(20000);
__atomic_store_n(&page[64], 1, __ATOMIC_SEQ_CST);
wake(&page[64]);
pthread_join(t, NULL);
CHECK(used_slots() == 1, "registry slots still in use: %d", used_slots());
}
{
char reg[96];
struct stat st;
snprintf(reg, sizeof(reg), "/dev/shm%s-droiddeck", name);
shm_unlink(name);
CHECK(stat(reg, &st) != 0, "registry left behind");
}
printf("fsync emulation ok\n");
return 0;
}
"""
@unittest.skipUnless(shutil.which("cc") and os.path.exists("/dev/shm"), "a C compiler and /dev/shm are needed")
class FsyncPreloadTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.tmp = tempfile.TemporaryDirectory()
base = Path(cls.tmp.name)
cls.lib = base / "libfsync.so"
cls.exe = base / "fsync-test"
source = base / "fsync-test.c"
source.write_text(PROGRAM)
subprocess.run(["cc", "-O2", "-Wall", "-fPIC", "-shared", "-pthread", "-o", str(cls.lib),
str(PRELOAD / "fsync.c"), str(PRELOAD / "robust.c"), "-ldl"], check=True)
subprocess.run(["cc", "-O2", "-Wall", "-pthread", "-o", str(cls.exe), str(source)], check=True)
@classmethod
def tearDownClass(cls):
cls.tmp.cleanup()
def run_program(self, **env):
return subprocess.run([str(self.exe)], env=dict(os.environ, LD_PRELOAD=str(self.lib), **env),
capture_output=True, text=True, timeout=120)
def test_emulated_waits_wake_time_out_and_survive_dead_waiters(self):
result = self.run_program(BL_FSYNC="1")
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertIn("fsync emulation ok", result.stdout)
self.assertIn("droiddeck-fsync: up and running.", result.stderr)
def test_off_without_the_switch(self):
result = self.run_program(BL_FSYNC="0")
self.assertNotIn("droiddeck-fsync", result.stderr)
self.assertNotIn("fsync emulation ok", result.stdout)
if __name__ == "__main__":
unittest.main()
+329 -8
View File
@@ -1,5 +1,6 @@
import json
import os
import shutil
from pathlib import Path
import runpy
import subprocess
@@ -8,8 +9,20 @@ import tempfile
import unittest
BIN = Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin"
MODULE = runpy.run_path(str(BIN / "bannerlator-game-env"))
COMPAT = runpy.run_path(str(BIN / "bannerlator-steam-compat"))
MODULE = runpy.run_path(str(BIN / "droiddeck-game-env"))
COMPAT = runpy.run_path(str(BIN / "steam-compatibility"))
STATE = tempfile.TemporaryDirectory()
COMPAT["main"].__globals__["COMPAT_DIR"] = STATE.name
def mapping_of(text):
tokens = COMPAT["tokenize"](text)
return COMPAT["read_mapping"](tokens, COMPAT["mapping_block"](tokens))
def config_text(entries):
mapping = "".join('"%s" { "name" "%s" "config" "" "priority" "%s" }' % (app, name, "75" if app == "0" else "250") for app, name in entries)
return '"InstallConfigStore" { "Software" { "Valve" { "Steam" { "CompatToolMapping" { %s } } } } }' % mapping
class GameEnvironmentTest(unittest.TestCase):
@@ -94,14 +107,14 @@ class GameEnvironmentTest(unittest.TestCase):
env = {**os.environ, "HOME": tmp, "STEAM_COMPAT_DATA_PATH": "/compatdata/42"}
for value in ("first value", "$(touch " + str(home / "injected") + "); 'literal'=value"):
config.write_text(json.dumps({"version": 1, "shared": {"CUSTOM": value}}))
result = subprocess.check_output([sys.executable, str(BIN / "bannerlator-game-env"), str(probe), "waitforexitandrun", "path with spaces", "a=b"], env=env, text=True)
result = subprocess.check_output([sys.executable, str(BIN / "droiddeck-game-env"), str(probe), "waitforexitandrun", "path with spaces", "a=b"], env=env, text=True)
self.assertEqual(json.loads(result), [value, ["waitforexitandrun", "path with spaces", "a=b"]])
self.assertFalse((home / "injected").exists())
for verb, prefix in (("run", "/compatdata/42"), ("waitforexitandrun", "/compatdata/0")):
output = subprocess.check_output([sys.executable, str(BIN / "bannerlator-game-env"), str(probe), verb], env={**env, "STEAM_COMPAT_DATA_PATH": prefix, "CUSTOM": "original"}, text=True)
output = subprocess.check_output([sys.executable, str(BIN / "droiddeck-game-env"), str(probe), verb], env={**env, "STEAM_COMPAT_DATA_PATH": prefix, "CUSTOM": "original"}, text=True)
self.assertEqual(json.loads(output)[0], "original")
config.write_text("{broken")
result = subprocess.run([sys.executable, str(BIN / "bannerlator-game-env"), str(probe), "waitforexitandrun"], env={**env, "CUSTOM": "original"}, text=True, capture_output=True, check=True)
result = subprocess.run([sys.executable, str(BIN / "droiddeck-game-env"), str(probe), "waitforexitandrun"], env={**env, "CUSTOM": "original"}, text=True, capture_output=True, check=True)
self.assertEqual(json.loads(result.stdout)[0], "original")
def test_generated_valve_and_third_party_launchers_apply_configuration(self):
@@ -118,24 +131,332 @@ class GameEnvironmentTest(unittest.TestCase):
proton.write_text("#!/usr/bin/python3\nimport json, os, sys\nprint(json.dumps([os.environ['CUSTOM'], sys.argv[1:]]))\nsys.exit(7)\n")
proton.chmod(0o755)
(extra / "toolmanifest.vdf").write_text('"manifest" { "commandline" "/proton %verb%" }')
COMPAT["build_tool"](str(tools / COMPAT["TOOL"]))
COMPAT["build_tool"](str(tools / COMPAT["TOOL"]), str(depot))
COMPAT["adopt_extras"](str(tools))
config = home / ".config/droiddeck/game-environment.json"
config.parent.mkdir(parents=True)
config.write_text(json.dumps({"version": 1, "shared": {"CUSTOM": "shared"}, "games": {"42": {"CUSTOM": "specific"}}}))
wrappers = (tools / COMPAT["TOOL"] / COMPAT["LAUNCHER"], extra / COMPAT["EXTRA_WRAPPER"])
for wrapper in wrappers:
wrapper.write_text(wrapper.read_text().replace("/usr/local/bin/bannerlator-game-env", str(BIN / "bannerlator-game-env")))
wrapper.write_text(wrapper.read_text().replace("/usr/local/bin/droiddeck-game-env", str(BIN / "droiddeck-game-env")))
result = subprocess.run([str(wrapper), "waitforexitandrun", "game with spaces.exe"],
env={"PATH": os.defpath, "HOME": tmp, "STEAM_COMPAT_DATA_PATH": "/compatdata/42", "STEAM_COMPAT_CLIENT_INSTALL_PATH": str(steam)},
text=True, capture_output=True)
self.assertEqual(result.returncode, 7, result.stderr)
self.assertEqual(json.loads(result.stdout), ["specific", ["waitforexitandrun", "game with spaces.exe"]])
def test_default_is_compatible_tool_and_labels_mark_it(self):
with tempfile.TemporaryDirectory() as tmp:
home = Path(tmp)
label = home / "label"
label.write_text("Compatible\n")
COMPAT["display_name"].__globals__["LABEL_FILE"] = str(label)
tools = home / "compatibilitytools.d"
extra = tools / "proton-cachyos-11"
extra.mkdir(parents=True)
(extra / "proton").write_text("")
(extra / "toolmanifest.vdf").write_text('"manifest" { "commandline" "/proton %verb%" }')
vdf = tools / COMPAT["TOOL"] / "compatibilitytool.vdf"
COMPAT["build_tool"](str(tools / COMPAT["TOOL"]), None)
self.assertIn('"display_name" "Proton ARM64 (Compatible)"', vdf.read_text())
for source in COMPAT["SOURCES"]:
COMPAT["build_tool"](str(tools / COMPAT["TOOL"]), str(home / "steamapps/common" / source))
self.assertIn('"display_name" "%s ARM64 (Compatible)"' % source.replace(" (ARM64)", ""), vdf.read_text())
protect = COMPAT["adopt_extras"](str(tools))
self.assertIn('"display_name" "proton-cachyos-11 (Compatible)"', (extra / "compatibilitytool.vdf").read_text())
config = home / "config.vdf"
mapping = "".join('"%s" { "name" "%s" "config" "" "priority" "%s" }' % entry for entry in (
("0", "GE-Proton10-25", "75"), ("42", "proton_experimental_arm64", "250"), ("43", "proton-cachyos-11", "250")))
config.write_text('"InstallConfigStore" { "Software" { "Valve" { "Steam" { "CompatToolMapping" { %s } } } } }' % mapping)
COMPAT["register_default"](str(config), ["44"], protect=protect)
tokens = COMPAT["tokenize"](config.read_text())
names = {tokens[i - 1].strip('"'): tokens[i + 2].strip('"') for i in range(1, len(tokens) - 2) if tokens[i] == "{" and tokens[i + 1] == '"name"'}
self.assertEqual(names, {"0": COMPAT["TOOL"], "42": COMPAT["TOOL"], "43": "proton-cachyos-11", "44": COMPAT["TOOL"]})
def test_existing_install_moves_to_new_tool_name(self):
with tempfile.TemporaryDirectory() as tmp:
home = Path(tmp)
(home / "label").write_text("Compatible\n")
COMPAT["main"].__globals__["LABEL_FILE"] = str(home / "label")
steam = home / "Steam"
depot = steam / "steamapps/common" / COMPAT["SOURCES"][0]
(depot / "files/bin-arm64").mkdir(parents=True)
(steam / "steamapps/appmanifest_42.acf").write_text("")
legacy = steam / "compatibilitytools.d" / COMPAT["LEGACY_TOOL"]
legacy.mkdir(parents=True)
for name in COMPAT["OWN_FILES"]:
(legacy / name).write_text("old")
(steam / "config").mkdir()
config = steam / "config/config.vdf"
mapping = "".join('"%s" { "name" "%s" "config" "" "priority" "%s" }' % (app, COMPAT["LEGACY_TOOL"], priority)
for app, priority in (("0", "75"), ("42", "250")))
config.write_text('"InstallConfigStore" { "Software" { "Valve" { "Steam" { "CompatToolMapping" { %s } } } } }' % mapping)
argv = sys.argv
try:
sys.argv = ["steam-compatibility", str(steam)]
COMPAT["main"]()
migrated = config.read_text()
COMPAT["main"]()
finally:
sys.argv = argv
self.assertFalse(legacy.exists())
self.assertIn('"display_name" "Proton Experimental ARM64 (Compatible)"',
(steam / "compatibilitytools.d" / COMPAT["TOOL"] / "compatibilitytool.vdf").read_text())
self.assertNotIn(COMPAT["LEGACY_TOOL"], migrated)
self.assertEqual(migrated.count('"%s"' % COMPAT["TOOL"]), 2)
self.assertEqual(config.read_text(), migrated)
def test_launchers_and_adopted_protons_from_before_the_rename_move_over(self):
with tempfile.TemporaryDirectory() as tmp:
home = Path(tmp)
(home / "label").write_text("Compatible\n")
COMPAT["main"].__globals__["LABEL_FILE"] = str(home / "label")
tools = home / "compatibilitytools.d"
own = tools / COMPAT["TOOL"]
own.mkdir(parents=True)
for name in ("toolmanifest.vdf", "compatibilitytool.vdf", COMPAT["LEGACY_LAUNCHER"]):
(own / name).write_text("old")
spare = tools / COMPAT["TOOL_11"]
spare.mkdir()
for name in ("toolmanifest.vdf", "compatibilitytool.vdf", COMPAT["LEGACY_LAUNCHER"]):
(spare / name).write_text("old")
extra = tools / "GE-Proton11-7"
extra.mkdir()
(extra / "proton").write_text("")
(extra / "toolmanifest.vdf").write_text('"manifest" { "commandline" "/%s %%verb%%" }' % COMPAT["LEGACY_EXTRA_WRAPPER"])
(extra / ("toolmanifest.vdf" + COMPAT["LEGACY_ORIGINAL_SUFFIX"])).write_text('"manifest" { "commandline" "/proton %verb%" }')
(extra / COMPAT["LEGACY_EXTRA_WRAPPER"]).write_text("old")
COMPAT["build_tool"](str(own), None)
COMPAT["remove_tool"](str(spare))
self.assertEqual(COMPAT["adopt_extras"](str(tools)), {"GE-Proton11-7": "GE-Proton11-7"})
self.assertEqual(sorted(os.listdir(own)), sorted(COMPAT["OWN_FILES"]))
self.assertFalse(spare.exists())
self.assertEqual(sorted(os.listdir(extra)), sorted(["proton", "toolmanifest.vdf", "compatibilitytool.vdf",
COMPAT["EXTRA_WRAPPER"], "toolmanifest.vdf" + COMPAT["ORIGINAL_SUFFIX"]]))
self.assertIn("/%s %%verb%%" % COMPAT["EXTRA_WRAPPER"], (extra / "toolmanifest.vdf").read_text())
self.assertIn('"/proton %verb%"', (extra / ("toolmanifest.vdf" + COMPAT["ORIGINAL_SUFFIX"])).read_text())
self.assertIn('"$here/proton"', (extra / COMPAT["EXTRA_WRAPPER"]).read_text())
self.assertIn("GE-Proton11-7", [entry["name"] for entry in COMPAT["tool_catalog"](str(home), None)])
def test_both_proton_wrappers_call_environment_launcher(self):
for script in (COMPAT["LAUNCHER_SH"], COMPAT["EXTRA_WRAPPER_SH"] % "proton"):
subprocess.run(["bash", "-n"], input=script, text=True, check=True)
self.assertIn('exec ${BL_TASKSET:-} /usr/local/bin/bannerlator-game-env', script)
self.assertIn('exec ${BL_TASKSET:-} /usr/local/bin/droiddeck-game-env', script)
def test_wrappers_preload_the_session_library_before_the_input_shim(self):
overlay = "/root/.local/share/Steam/ubuntu12_64/gameoverlayrenderer.so"
with tempfile.TemporaryDirectory() as tmp:
for name in ("libblsession.so", "libfakeinput.so"):
(Path(tmp) / name).write_bytes(b"")
for script in (COMPAT["LAUNCHER_SH"], COMPAT["EXTRA_WRAPPER_SH"] % "proton"):
start = script.index("for fake in ")
block = script[start:script.index("\ndone\n", start) + 6].replace("/usr/local/lib/", tmp + "/")
for inherited, expected in (
(None, ["libblsession.so", "libfakeinput.so"]),
(overlay, ["libblsession.so", "libfakeinput.so", overlay]),
(tmp + "/libfakeinput.so:" + overlay, ["libblsession.so", "libfakeinput.so", overlay]),
(tmp + "/libblsession.so:" + tmp + "/libfakeinput.so", ["libblsession.so", "libfakeinput.so"])):
env = {"PATH": os.defpath}
if inherited is not None:
env["LD_PRELOAD"] = inherited
out = subprocess.run(["bash", "-c", block + 'printf %s "$LD_PRELOAD"'], env=env,
capture_output=True, text=True, check=True).stdout
self.assertEqual([entry.replace(tmp + "/", "") for entry in out.split(":")], expected, inherited)
class ProtonDefaultTest(unittest.TestCase):
TOOL = COMPAT["TOOL"]
CATALOG = [
{"name": COMPAT["TOOL"], "display": "Proton Experimental ARM64", "dir": "Proton Experimental (ARM64)", "valve": True},
{"name": COMPAT["TOOL_11"], "display": "Proton 11.0 ARM64", "dir": "Proton 11.0 (ARM64)", "valve": True},
{"name": "GE-Proton11-7", "display": "GE-Proton11-7", "dir": "GE-Proton11-7", "valve": False},
]
NAMES = [t["name"] for t in CATALOG]
def plan(self, current, installed, default, auto):
return COMPAT["plan_mapping"](current, installed, self.NAMES, default, auto)
def test_new_installs_and_valve_picks_follow_the_default(self):
changes, auto = self.plan({"0": self.TOOL, "10": "proton_experimental", "11": "proton_11_arm64", "12": "GE-Proton11-7", "13": "steamlinuxruntime"},
["10", "11", "12", "13", "14"], "GE-Proton11-7", {})
self.assertEqual(changes, {"0": "GE-Proton11-7", "10": "GE-Proton11-7", "11": "GE-Proton11-7", "14": "GE-Proton11-7"})
self.assertEqual(auto, {"10": "GE-Proton11-7", "11": "GE-Proton11-7", "14": "GE-Proton11-7"})
def test_titles_following_the_default_move_with_it_and_picks_stay(self):
current = {"0": self.TOOL, "10": self.TOOL, "11": "GE-Proton11-7", "12": self.TOOL}
auto = {"10": self.TOOL, "11": self.TOOL, "12": self.TOOL}
changes, auto = self.plan(current, ["10", "11", "12"], COMPAT["TOOL_11"], auto)
self.assertEqual(changes, {"0": COMPAT["TOOL_11"], "10": COMPAT["TOOL_11"], "12": COMPAT["TOOL_11"]})
self.assertEqual(auto, {"10": COMPAT["TOOL_11"], "12": COMPAT["TOOL_11"]})
def test_a_removed_tool_falls_back_to_the_default(self):
changes, auto = self.plan({"0": self.TOOL, "10": "GE-Proton10-1", "11": "droiddeck-proton-12-arm64"}, ["10", "11"], self.TOOL, {"10": "GE-Proton10-1"})
self.assertEqual(changes, {"10": self.TOOL, "11": self.TOOL})
self.assertEqual(auto, {"10": self.TOOL, "11": self.TOOL})
def test_uninstalled_titles_are_forgotten(self):
changes, auto = self.plan({"0": self.TOOL}, [], self.TOOL, {"10": self.TOOL})
self.assertEqual((changes, auto), ({}, {}))
def test_the_last_side_to_change_wins(self):
state = {"default": self.TOOL, "applied": 5}
default, state = COMPAT["choose_default"]("GE-Proton11-7", state, {"seq": 5, "dir": "Proton 11.0 (ARM64)", "valve": True}, self.CATALOG, now=100)
self.assertEqual((default, state["source"], state["seq"]), ("GE-Proton11-7", "steam", 100))
default, state = COMPAT["choose_default"]("GE-Proton11-7", state, {"seq": 6, "dir": "Proton 11.0 (ARM64)", "valve": True}, self.CATALOG, now=200)
self.assertEqual((default, state["source"], state["seq"], state["applied"]), (COMPAT["TOOL_11"], "app", 200, 6))
default, state = COMPAT["choose_default"]("GE-Proton11-7", state, {"seq": 6}, self.CATALOG, adopt_steam=False, now=300)
self.assertEqual((default, state["seq"]), (COMPAT["TOOL_11"], 200))
default, state = COMPAT["choose_default"]("proton_experimental", state, {"seq": 6}, self.CATALOG, now=300)
self.assertEqual(default, COMPAT["TOOL_11"])
def test_a_request_for_a_missing_tool_is_dropped(self):
default, state = COMPAT["choose_default"](self.TOOL, {"default": self.TOOL}, {"seq": 9, "dir": "GE-Proton9-1", "valve": False}, self.CATALOG, now=1)
self.assertEqual((default, state["applied"]), (self.TOOL, 9))
default, state = COMPAT["choose_default"]("GE-Proton11-7", state, {"seq": 9, "dir": "GE-Proton9-1", "valve": False}, self.CATALOG, now=2)
self.assertEqual(default, "GE-Proton11-7")
def test_settle_before_the_client_starts(self):
with tempfile.TemporaryDirectory() as tmp:
config = Path(tmp) / "config.vdf"
config.write_text(config_text([("0", self.TOOL), ("10", "proton_experimental"), ("11", "GE-Proton11-7")]))
(Path(tmp) / "request.json").write_text(json.dumps({"seq": 3, "dir": "Proton 11.0 (ARM64)", "valve": True}))
default = COMPAT["register_default"](str(config), ["10", "11", "12"], catalog=self.CATALOG, directory=tmp, settle=True)
self.assertEqual(default, COMPAT["TOOL_11"])
self.assertEqual(mapping_of(config.read_text()), {"0": COMPAT["TOOL_11"], "10": COMPAT["TOOL_11"], "11": "GE-Proton11-7", "12": COMPAT["TOOL_11"]})
state = json.loads((Path(tmp) / "state.json").read_text())
self.assertEqual((state["applied"], state["source"], state["tools"]), (3, "app", self.CATALOG))
config.write_text(config_text([("0", "GE-Proton11-7"), ("10", COMPAT["TOOL_11"]), ("11", "GE-Proton11-7"), ("12", COMPAT["TOOL_11"])]))
default = COMPAT["register_default"](str(config), ["10", "11", "12"], catalog=self.CATALOG, directory=tmp, settle=True)
self.assertEqual(default, "GE-Proton11-7")
self.assertEqual(mapping_of(config.read_text())["10"], "GE-Proton11-7")
self.assertEqual(json.loads((Path(tmp) / "state.json").read_text())["source"], "steam")
config.write_text(config_text([("0", "proton_experimental")]))
default = COMPAT["register_default"](str(config), ["10"], catalog=self.CATALOG, directory=tmp)
self.assertEqual(mapping_of(config.read_text()), {"0": "GE-Proton11-7", "10": "GE-Proton11-7"})
def test_the_shortcut_writer_keeps_the_chosen_default(self):
with tempfile.TemporaryDirectory() as tmp:
steam = Path(tmp) / "Steam"
for name in COMPAT["SOURCES"]:
(steam / "steamapps/common" / name / "files/bin-arm64").mkdir(parents=True)
(steam / "config").mkdir()
config = steam / "config/config.vdf"
config.write_text(config_text([("0", COMPAT["TOOL_11"]), ("10", COMPAT["TOOL_11"]), ("3044416433", COMPAT["TOOL_11"])]))
(Path(tmp) / "state.json").write_text(json.dumps({"default": COMPAT["TOOL_11"], "auto": {"10": COMPAT["TOOL_11"], "3044416433": COMPAT["TOOL_11"]}}))
COMPAT["register_default"](str(config), apps=["3044416433", "3212965118"], protect={}, directory=tmp)
self.assertEqual(mapping_of(config.read_text()), {"0": COMPAT["TOOL_11"], "10": COMPAT["TOOL_11"], "3044416433": COMPAT["TOOL_11"], "3212965118": COMPAT["TOOL_11"]})
def test_each_valve_depot_gets_its_own_tool(self):
with tempfile.TemporaryDirectory() as tmp:
steam = Path(tmp) / "Steam"
for name in COMPAT["SOURCES"]:
(steam / "steamapps/common" / name / "files/bin-arm64").mkdir(parents=True)
self.assertEqual(COMPAT["find_source"](str(steam), COMPAT["OWN_TOOLS"][1][1]), str(steam / "steamapps/common/Proton 11.0 (ARM64)"))
self.assertEqual(COMPAT["find_source"](str(steam)), str(steam / "steamapps/common/Proton Experimental (ARM64)"))
self.assertEqual([(t["name"], t["dir"]) for t in COMPAT["tool_catalog"](str(steam), {})],
[(COMPAT["TOOL"], "Proton Experimental (ARM64)"), (COMPAT["TOOL_11"], "Proton 11.0 (ARM64)")])
for name, sources in COMPAT["OWN_TOOLS"]:
COMPAT["build_tool"](str(steam / "compatibilitytools.d" / name), COMPAT["find_source"](str(steam), sources), sources, name)
self.assertEqual(COMPAT["adopt_extras"](str(steam / "compatibilitytools.d")), {})
self.assertEqual(sorted(os.listdir(steam / "compatibilitytools.d" / COMPAT["TOOL_11"])), sorted(COMPAT["OWN_FILES"]))
launcher = COMPAT["launcher_sh"](COMPAT["OWN_TOOLS"][1][1])
self.assertIn('for name in "Proton 11.0 (ARM64)"; do', launcher)
self.assertIn("${major%%.*}", launcher)
class DirectAudioPrefixTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
base = Path(self.tmp.name)
self.tool = base / "tool"
(self.tool / "files/bin-arm64").mkdir(parents=True)
wine = self.tool / "files/bin-arm64/wine"
wine.write_text("#!/bin/sh\necho wine-11.0-4c8f2e1 '(Staging)'\n")
wine.chmod(0o755)
self.audio = base / "directaudio"
for arch in ("aarch64-windows", "i386-windows", "aarch64-unix"):
(self.audio / "lib/wine" / arch).mkdir(parents=True)
(self.audio / "lib/wine/aarch64-windows/winedirectaudio.drv").write_bytes(b"arm64x")
(self.audio / "lib/wine/i386-windows/winedirectaudio.drv").write_bytes(b"i386")
self.compat = base / "compat"
self.windows = self.compat / "pfx/drive_c/windows"
for folder in ("system32", "syswow64"):
(self.windows / folder).mkdir(parents=True)
self.reg = self.compat / "pfx/user.reg"
def run_setup(self):
script = COMPAT["BL_DIRECTAUDIO_SETUP"] + '\nbl_directaudio "$1"\necho "WINEDLLPATH=$WINEDLLPATH"\n'
env = dict(os.environ, BL_DIRECTAUDIO=str(self.audio), STEAM_COMPAT_DATA_PATH=str(self.compat))
env.pop("WINEDLLPATH", None)
return subprocess.run(["bash", "-c", script, "bash", str(self.tool)], env=env, capture_output=True, text=True, check=True)
def audio_values(self):
found, current = [], None
for line in self.reg.read_text().splitlines():
if line.startswith("["):
current = line.rsplit("] ", 1)[0] + "]"
elif line.startswith('"Audio"='):
found.append((current, line))
return found
def link(self, folder):
path = self.windows / folder / "winedirectaudio.drv"
return os.readlink(path) if path.is_symlink() else None
def test_the_driver_is_placed_and_selected_the_way_wine_reads_it(self):
self.reg.write_text("WINE REGISTRY Version 2\n;; All keys relative to \\\\User\\\\S-1-5-21-0-0-0-1000\n\n#arch=win64\n")
result = self.run_setup()
self.assertIn("DirectAudio selected", result.stderr)
self.assertIn("WINEDLLPATH=%s/lib/wine" % self.audio, result.stdout)
self.assertEqual(self.link("system32"), str(self.audio / "lib/wine/aarch64-windows/winedirectaudio.drv"))
self.assertEqual(self.link("syswow64"), str(self.audio / "lib/wine/i386-windows/winedirectaudio.drv"))
self.assertEqual((self.windows / "system32/winedirectaudio.drv").read_bytes(), b"arm64x")
self.assertEqual(self.audio_values(), [("[Software\\\\Wine\\\\Drivers]", '"Audio"="directaudio,pulse"')])
before = self.reg.read_text()
self.assertNotIn("DirectAudio selected", self.run_setup().stderr)
self.assertEqual(self.reg.read_text(), before)
self.assertEqual(sorted(os.listdir(self.windows / "system32")), ["winedirectaudio.drv"])
def test_older_selections_are_upgraded(self):
self.reg.write_text('WINE REGISTRY Version 2\n\n[SoftwareWineDrivers] 1790995967\n#time=1dd52e24550e980\n"Audio"="directaudio"\n'
'\n[Software\\\\Wine\\\\Drivers] 1790995986\n#time=1dd52e250b31ec4\n"Audio"="directaudio"\n')
self.assertIn("DirectAudio selected", self.run_setup().stderr)
self.assertEqual(self.audio_values(), [("[SoftwareWineDrivers]", '"Audio"="directaudio"'),
("[Software\\\\Wine\\\\Drivers]", '"Audio"="directaudio"'),
("[Software\\\\Wine\\\\Drivers]", '"Audio"="directaudio,pulse"')])
self.assertNotIn("DirectAudio selected", self.run_setup().stderr)
def test_a_copied_driver_is_replaced_by_the_staged_one(self):
self.reg.write_text("WINE REGISTRY Version 2\n")
(self.windows / "system32/winedirectaudio.drv").write_bytes(b"stale copy")
self.run_setup()
self.assertEqual(self.link("system32"), str(self.audio / "lib/wine/aarch64-windows/winedirectaudio.drv"))
def test_nothing_is_selected_without_the_driver_in_the_prefix(self):
self.reg.write_text("WINE REGISTRY Version 2\n")
shutil.rmtree(self.windows)
self.assertIn("could not be placed", self.run_setup().stderr)
self.assertEqual(self.audio_values(), [])
(self.windows / "system32").mkdir(parents=True)
(self.audio / "lib/wine/aarch64-windows/winedirectaudio.drv").unlink()
self.assertIn("could not be placed", self.run_setup().stderr)
self.assertEqual(self.audio_values(), [])
def test_a_new_prefix_waits_for_the_next_launch(self):
shutil.rmtree(self.compat / "pfx")
self.assertIn("no prefix yet", self.run_setup().stderr)
self.assertFalse(self.reg.exists())
def test_another_wine_major_is_left_alone(self):
(self.tool / "files/bin-arm64/wine").write_text("#!/bin/sh\necho wine-10.0\n")
self.reg.write_text("WINE REGISTRY Version 2\n")
result = self.run_setup()
self.assertIn("left off", result.stderr)
self.assertIn("WINEDLLPATH=\n", result.stdout)
self.assertEqual(self.audio_values(), [])
self.assertIsNone(self.link("system32"))
if __name__ == "__main__":
+1 -1
View File
@@ -14,7 +14,7 @@ try:
except ImportError:
Gio = GLib = None
SCRIPT = Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin/bannerlator-login1"
SCRIPT = Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin/droiddeck-login1"
NAME = "org.freedesktop.login1"
ROOT = "/org/freedesktop/login1"
IFACE = NAME + ".Manager"
+1 -1
View File
@@ -20,7 +20,7 @@ try:
except ImportError:
Gio = GLib = None
SCRIPT = Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin/bannerlator-netmanager"
SCRIPT = Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin/droiddeck-netmanager"
if Gio:
loader = importlib.machinery.SourceFileLoader("netmanager", str(SCRIPT))
spec = importlib.util.spec_from_loader(loader.name, loader)
+52
View File
@@ -0,0 +1,52 @@
import runpy
import shutil
import tempfile
import unittest
from pathlib import Path
COMPAT = runpy.run_path(str(Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin/steam-compatibility"))
class RequestFexTest(unittest.TestCase):
def setUp(self):
self.root = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.root, True)
(self.root / "steamapps").mkdir()
client = self.root / "steamrtarm64/steam"
client.parent.mkdir()
self.log = self.root / "client.log"
client.write_text("#!/bin/sh\necho \"start $1\" >> %s\nsleep 0.3\necho \"end $1\" >> %s\n" % (self.log, self.log))
client.chmod(0o755)
def install(self, app):
(self.root / ("steamapps/appmanifest_%s.acf" % app)).write_text("")
def test_one_install_is_asked_for_at_a_time(self):
asked = set()
self.assertEqual(["3127680"], COMPAT["request_fex"](str(self.root), asked))
self.assertEqual(["start steam://install/3127680", "end steam://install/3127680"], self.log.read_text().splitlines())
self.assertEqual([], COMPAT["request_fex"](str(self.root), asked))
self.install("3127680")
self.assertEqual(["1628350"], COMPAT["request_fex"](str(self.root), asked))
self.assertEqual(["start steam://install/3127680", "end steam://install/3127680",
"start steam://install/1628350", "end steam://install/1628350"],
self.log.read_text().splitlines())
self.install("1628350")
self.assertEqual([], COMPAT["request_fex"](str(self.root), asked))
self.assertEqual({"3127680", "1628350"}, asked)
def test_a_declined_install_is_not_asked_again_in_the_same_session(self):
asked = set()
self.assertEqual(["3127680"], COMPAT["request_fex"](str(self.root), asked))
self.assertEqual([], COMPAT["request_fex"](str(self.root), asked))
self.assertEqual(["3127680"], COMPAT["request_fex"](str(self.root), set()))
def test_only_what_is_missing_is_asked_for_once(self):
self.install("3127680")
asked = set()
self.assertEqual(["1628350"], COMPAT["request_fex"](str(self.root), asked))
self.assertEqual([], COMPAT["request_fex"](str(self.root), asked))
self.assertEqual(["start steam://install/1628350", "end steam://install/1628350"], self.log.read_text().splitlines())
if __name__ == "__main__":
unittest.main()
+2 -2
View File
@@ -17,11 +17,11 @@ class SessionAssetsTest(unittest.TestCase):
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.overlay = self.root / 'overlay'
self.helper = self.overlay / 'usr/local/bin/bannerlator-login1'
self.helper = self.overlay / 'usr/local/bin/droiddeck-login1'
self.helper.parent.mkdir(parents=True)
self.helper.write_bytes(b'current helper')
self.apk = self.root / 'app.apk'
self.name = 'assets/linuxfs/usr/local/bin/bannerlator-login1'
self.name = 'assets/linuxfs/usr/local/bin/droiddeck-login1'
def test_missing_required_helper_rejects_cached_bundle(self):
with ZipFile(self.apk, 'w') as package:
+39
View File
@@ -0,0 +1,39 @@
import re
import shutil
import subprocess
import tempfile
import unittest
from pathlib import Path
SESSION = Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin/droiddeck-session"
def log_redirect():
match = re.search(r"^\s*(exec > >\(.*\) 2>&1)$", SESSION.read_text(), re.M)
return match.group(1)
class SessionLogTest(unittest.TestCase):
def setUp(self):
self.tmp = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.tmp, True)
self.log = self.tmp / "session.log"
def run_session(self, body, cap=None):
script = "BL_LOG=%s\n%s%s\n%s\n" % (self.log, "BL_LOG_CAP=%d\n" % cap if cap else "", log_redirect(), body)
subprocess.run(["bash", "-c", script], timeout=30, check=False)
def test_the_last_lines_survive_the_writer_being_killed(self):
self.run_session('for i in $(seq 1 60); do echo "line $i"; done\nsleep 0.5\n'
'for p in $(pgrep -P $$); do kill -9 $(pgrep -P $p) $p 2>/dev/null; done')
lines = self.log.read_text().splitlines()
self.assertEqual(60, len(lines))
self.assertEqual("line 60", lines[-1])
def test_the_log_keeps_its_first_bytes_and_drains_the_rest(self):
self.run_session("printf 'abcdefghij\\nklmnop\\n'\nsleep 0.5", cap=5)
self.assertEqual("abcde", self.log.read_text())
if __name__ == "__main__":
unittest.main()
+218
View File
@@ -0,0 +1,218 @@
import json
import os
from pathlib import Path
import runpy
import shutil
import subprocess
import tempfile
import unittest
BIN = Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin"
LIVE = runpy.run_path(str(BIN / "droiddeck-steam-compat"))
COMPAT = LIVE["COMPAT"]
TOOL = COMPAT["TOOL"]
TOOL_11 = COMPAT["TOOL_11"]
FAKE_STEAM = r"""
const dec = new TextDecoder(), enc = new TextEncoder();
function readVarint(b, at) { let r = 0n, s = 0n, x; do { x = b[at.i++]; r |= BigInt(x & 0x7f) << s; s += 7n; } while (x & 0x80); return r; }
function fields(b) {
const f = {}, at = { i: 0 };
while (at.i < b.length) {
const k = Number(readVarint(b, at)), n = k >> 3, w = k & 7; let v;
if (w === 0) v = readVarint(b, at);
else if (w === 1) { v = 0n; for (let j = 0; j < 8; j++) v |= BigInt(b[at.i + j]) << BigInt(8 * j); at.i += 8; }
else if (w === 2) { const len = Number(readVarint(b, at)); v = b.slice(at.i, at.i + len); at.i += len; }
else throw new Error("wire " + w);
(f[n] = f[n] || []).push(v);
}
return f;
}
function vint(n, out) { n = BigInt(n); do { let x = Number(n & 0x7fn); n >>= 7n; if (n) x |= 0x80; out.push(x); } while (n); }
function lenField(n, bytes, out) { vint(n * 8 + 2, out); vint(bytes.length, out); for (const x of bytes) out.push(x); }
function reply(eresult, jobid, body, name) {
const h = [];
if (jobid !== null) { vint(11 * 8 + 1, h); let n = jobid; for (let i = 0; i < 8; i++) { h.push(Number(n & 0xffn)); n >>= 8n; } }
if (name) lenField(12, enc.encode(name), h);
vint(13 * 8, h); vint(eresult, h);
const out = new Uint8Array(8 + h.length + body.length), dv = new DataView(out.buffer);
dv.setUint32(0, (0x80000000 | 147) >>> 0, true); dv.setUint32(4, h.length, true);
out.set(h, 8); out.set(body, 8 + h.length);
return out.buffer;
}
const state = { mapping: JSON.parse(process.argv[2]), known: JSON.parse(process.argv[3]), log: [], authed: false, notify: [] };
class FakeSocket {
constructor(url) { this.url = url; this.readyState = 0; state.socket = this; setTimeout(() => { this.readyState = 1; this.onopen(); }, 0); }
send(buffer) {
const b = new Uint8Array(buffer), dv = new DataView(buffer.buffer || buffer);
if (dv.getUint32(0, true) !== ((0x80000000 | 146) >>> 0)) throw new Error("bad emsg");
const hl = dv.getUint32(4, true), h = fields(b.slice(8, 8 + hl)), body = fields(b.slice(8 + hl));
const method = dec.decode(h[12][0]), jobid = h[10][0];
let out = [], result = 1;
if (method === "TransportAuth.Authenticate#1") {
state.authed = dec.decode(h[40][0]) === "secret" && dec.decode(body[1][0]) === "secret";
result = state.authed ? 1 : 5;
} else if (!state.authed) result = 5;
else if (method === "CompatManager.GetCompatTools#1") {
const app = String(body[1] ? body[1][0] : 0n);
for (const name of state.known) { const t = []; lenField(1, enc.encode(name), t); lenField(2, enc.encode(name + " shown"), t); lenField(1, t, out); }
lenField(3, enc.encode(state.mapping[app] || ""), out);
lenField(4, enc.encode("proton_experimental"), out);
} else if (method === "CompatManager.SpecifyCompatTool#1") {
const app = String(body[1] ? body[1][0] : 0n), name = body[2] ? dec.decode(body[2][0]) : "";
state.mapping[app] = name; state.log.push(app + "=" + name);
setTimeout(() => this.onmessage({ data: reply(1, null, new Uint8Array(0), "CompatManager.NotifyStateChanged#1") }), 0);
} else result = 2;
setTimeout(() => this.onmessage({ data: reply(result, jobid, Uint8Array.from(out)) }), 0);
}
close() {}
}
globalThis.window = globalThis;
globalThis.WebSocket = FakeSocket;
globalThis.SteamClient = { WebUITransport: { GetTransportInfo: async () => ({ portClientdll: 4242, authKeyClientdll: "secret" }) } };
window.droiddeckCompatChanged = (n) => state.notify.push(n);
const install = require("fs").readFileSync(process.argv[4], "utf8");
(async () => {
if (eval(install) !== "installed" || eval(install) !== "present") throw new Error("install is not idempotent");
const c = window.__droiddeckCompat;
const before = await c.get(0);
await c.set(0, "droiddeck-proton-11-arm64");
await c.set(42, "");
const after = await c.many(["0", "42", "43"]);
await new Promise((r) => setTimeout(r, 10));
console.log(JSON.stringify({ before, after, log: state.log, notify: state.notify, url: state.socket.url }));
})().catch((e) => { console.error(e.stack || String(e)); process.exit(1); });
"""
class FakePage:
def __init__(self, mapping, known):
self.mapping = dict(mapping)
self.known = list(known)
self.sets = []
def compat(self, call):
name, _, args = call.partition("(")
args = json.loads("[%s]" % args[:-1])
if name == "get":
return {"selected": self.mapping.get(str(args[0]), ""), "fallback": "proton_experimental", "tools": self.known}
if name == "many":
return {app: self.mapping.get(app, "") for app in args[0]}
if name == "set":
self.mapping[str(args[0])] = args[1]
self.sets.append((str(args[0]), args[1]))
return True
raise AssertionError(call)
@unittest.skipUnless(shutil.which("node"), "node is not installed")
class PageScriptTest(unittest.TestCase):
def test_transport_framing_auth_and_notifications(self):
with tempfile.TemporaryDirectory() as tmp:
script = Path(tmp) / "install.js"
script.write_text(LIVE["INSTALL_JS"])
harness = Path(tmp) / "harness.js"
harness.write_text(FAKE_STEAM)
result = subprocess.run(["node", str(harness), json.dumps({"0": TOOL, "42": "proton_experimental"}),
json.dumps([TOOL, TOOL_11, "proton_experimental"]), str(script)],
capture_output=True, text=True, timeout=60)
self.assertEqual(result.returncode, 0, result.stderr)
out = json.loads(result.stdout)
self.assertEqual(out["url"], "ws://127.0.0.1:4242/transportsocket/")
self.assertEqual(out["before"], {"selected": TOOL, "fallback": "proton_experimental", "tools": [TOOL, TOOL_11, "proton_experimental"]})
self.assertEqual(out["after"], {"0": TOOL_11, "42": "", "43": ""})
self.assertEqual(out["log"], ["0=" + TOOL_11, "42="])
self.assertEqual(out["notify"], ["1", "2"])
class DevtoolsHttpTest(unittest.TestCase):
def test_a_kept_alive_answer_is_read_to_its_length(self):
import socket
import threading
server = socket.socket()
server.bind(("127.0.0.1", 0))
server.listen(1)
port = server.getsockname()[1]
body = json.dumps([{"title": "other"}, {"title": "SharedJSContext", "webSocketDebuggerUrl": "ws://127.0.0.1:%d/devtools/page/AB" % port}]).encode()
done = threading.Event()
def answer():
conn, _ = server.accept()
conn.recv(4096)
conn.sendall(b"HTTP/1.1 200 OK\r\nContent-Length: %d\r\n\r\n" % len(body) + body)
done.wait(10)
conn.close()
server.close()
thread = threading.Thread(target=answer)
thread.start()
try:
self.assertEqual(LIVE["shared_context"](port), "/devtools/page/AB")
finally:
done.set()
thread.join()
class ReconcileTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
base = Path(self.tmp.name)
self.steam = base / "Steam"
for name in COMPAT["SOURCES"]:
(self.steam / "steamapps/common" / name / "files/bin-arm64").mkdir(parents=True)
for app in ("10", "11"):
(self.steam / ("steamapps/appmanifest_%s.acf" % app)).write_text("")
self.state = base / "compat"
self.state.mkdir()
COMPAT["main"].__globals__["COMPAT_DIR"] = str(self.state)
COMPAT["main"].__globals__["LABEL_FILE"] = str(base / "no-label")
self.addCleanup(COMPAT["main"].__globals__.__setitem__, "COMPAT_DIR", COMPAT["COMPAT_DIR"])
self.helper = LIVE["Helper"](str(self.steam), 1)
def reconcile(self, page):
self.helper.page = page
self.helper.reconcile()
return json.loads((self.state / "state.json").read_text())
def test_new_install_gets_the_default_live(self):
page = FakePage({"0": TOOL, "10": TOOL, "11": "proton_experimental"}, [TOOL, TOOL_11])
state = self.reconcile(page)
self.assertEqual(page.sets, [("11", TOOL)])
self.assertEqual(state["auto"], {"10": TOOL, "11": TOOL})
self.assertTrue(state["live"])
self.assertEqual(self.helper.installed, ["10", "11"])
def test_app_request_moves_the_default_and_followers(self):
page = FakePage({"0": TOOL, "10": TOOL, "11": "GE-Proton11-7"}, [TOOL, TOOL_11])
(self.state / "state.json").write_text(json.dumps({"default": TOOL, "auto": {"10": TOOL}}))
(self.state / "request.json").write_text(json.dumps({"seq": 7, "valve": True, "dir": "Proton 11.0 (ARM64)"}))
state = self.reconcile(page)
self.assertEqual(page.sets, [("0", TOOL_11), ("10", TOOL_11)])
self.assertEqual((state["default"], state["source"], state["applied"]), (TOOL_11, "app", 7))
self.assertEqual(page.mapping["11"], "GE-Proton11-7")
def test_a_steam_menu_change_is_mirrored(self):
page = FakePage({"0": TOOL_11, "10": TOOL}, [TOOL, TOOL_11])
(self.state / "state.json").write_text(json.dumps({"default": TOOL, "auto": {"10": TOOL}}))
state = self.reconcile(page)
self.assertEqual((state["default"], state["source"]), (TOOL_11, "steam"))
self.assertEqual(page.sets, [("10", TOOL_11), ("11", TOOL_11)])
def test_a_tool_the_client_has_not_loaded_waits_for_a_restart(self):
page = FakePage({"0": TOOL, "10": TOOL, "11": TOOL}, [TOOL])
(self.state / "request.json").write_text(json.dumps({"seq": 3, "valve": True, "dir": "Proton 11.0 (ARM64)"}))
state = self.reconcile(page)
self.assertEqual(page.sets, [])
self.assertEqual((state["pending"], state.get("applied", 0)), (TOOL_11, 0))
self.assertTrue(self.helper.request_waiting())
def test_file_mode_defers_to_the_live_helper(self):
(self.state / "live.pid").write_text("%d\n" % os.getpid())
self.assertTrue(COMPAT["live_helper_running"]())
LIVE["detach"]()
self.assertFalse(COMPAT["live_helper_running"]())
if __name__ == "__main__":
unittest.main()
+5 -5
View File
@@ -18,9 +18,9 @@ def load(name):
return module
imports = load('bannerlator-steam-games')
shortcuts = load('bannerlator-steam-shortcuts')
library = load('bannerlator-steam-library')
imports = load('droiddeck-steam-games')
shortcuts = load('droiddeck-steam-shortcuts')
library = load('droiddeck-steam-library')
class ImportsTest(unittest.TestCase):
@@ -289,7 +289,7 @@ class ImportsTest(unittest.TestCase):
path.write_bytes(original)
listing = self.root / 'games.json'
listing.write_text(json.dumps([self.game]))
run = subprocess.run(['python3', str(BIN / 'bannerlator-steam-shortcuts'), str(self.steam), str(listing)], capture_output=True)
run = subprocess.run(['python3', str(BIN / 'droiddeck-steam-shortcuts'), str(self.steam), str(listing)], capture_output=True)
self.assertEqual(1, run.returncode)
self.assertEqual(original, path.read_bytes())
@@ -300,7 +300,7 @@ class ImportsTest(unittest.TestCase):
path.write_bytes(shortcuts.write({'shortcuts': {'0': manual, '1': shortcuts.shortcut(self.game)}}))
listing = self.root / 'games.json'
listing.write_text(json.dumps([self.game]))
subprocess.run(['python3', str(BIN / 'bannerlator-steam-shortcuts'), str(self.steam), str(listing)], check=True, capture_output=True)
subprocess.run(['python3', str(BIN / 'droiddeck-steam-shortcuts'), str(self.steam), str(listing)], check=True, capture_output=True)
result = shortcuts.parse(path.read_bytes())['shortcuts']
self.assertEqual([manual], list(result.values()))
self.assertEqual({str(self.game['appid']): 42}, json.loads((path.parent / '.droiddeck-routes.json').read_text()))
+3 -3
View File
@@ -8,7 +8,7 @@ import unittest
import zipfile
SCRIPT = Path(__file__).resolve().parents[1] / 'linuxfs/overlay/usr/local/bin/bannerlator-steam-install'
SCRIPT = Path(__file__).resolve().parents[1] / 'linuxfs/overlay/usr/local/bin/droiddeck-steam-install'
class SteamInstallTest(unittest.TestCase):
@@ -64,7 +64,7 @@ cp "$TEST_CDN/${2##*/}" "$4"
result = self.install(channel)
self.assertEqual(0, result.returncode, result.stderr)
self.assertEqual(channel + '\n', (self.steam / 'package/beta').read_text())
self.assertEqual('123456\n', (self.steam / 'package/bannerlator-installed').read_text())
self.assertEqual('123456\n', (self.steam / 'package/droiddeck-installed').read_text())
client = self.steam / 'steamrtarm64/steam'
self.assertTrue(os.access(client, os.X_OK))
self.assertEqual(channel, client.read_text())
@@ -99,7 +99,7 @@ cp "$TEST_CDN/${2##*/}" "$4"
original = archive.read_bytes()
archive.write_bytes(b'corrupt download')
self.assertNotEqual(0, self.install('publicbeta').returncode)
self.assertFalse((self.steam / 'package/bannerlator-installed').exists())
self.assertFalse((self.steam / 'package/droiddeck-installed').exists())
archive.write_bytes(original)
self.assertEqual(0, self.install('publicbeta').returncode)
self.assertEqual('publicbeta\n', (self.steam / 'package/beta').read_text())
+1 -1
View File
@@ -5,7 +5,7 @@ import unittest
BIN = Path(__file__).resolve().parents[1] / 'linuxfs/overlay/usr/local/bin'
MODULE = runpy.run_path(str(BIN / 'bannerlator-steam-ui-scale'))
MODULE = runpy.run_path(str(BIN / 'droiddeck-steam-ui-scale'))
initialize = MODULE['initialize']
seed_text = MODULE['seed_text']
parse = MODULE['parse']