Files
DroidDeck/tools/linuxfs/overlay/usr/local/bin/droiddeck-seed-redists
T
MaxsTechReview eadebc46d5 Performance: idle front end, proot stops, launch-time passes and log collection (#529)
* Front end: draw the capsule wall without clipping it every frame

The tilted wall clipped each capsule to a rounded rect and recomposed its columns on every drift
step, and a rotated rounded clip is rasterised in software on the render threads: the idle main
page held about 1.7 cores. The art is now loaded at the capsule's size with its corners already
rounded, each column draws its capsules itself in one layer moved by the drift, and the drift
advances at most every 15 ms. The idle page takes about a third of a core.

* libfakeinput: let calls on descriptors it does not serve skip its lock

Every read, write, ioctl, close and poll of every process in a Proton session went through the
controller lock and the three map lookups, Wine's wineserver pipes and the game's files alike.
A bitmap of the descriptors the maps hold, set before one is handed out and cleared when it is
closed, lets everything else go straight to the kernel.

* Pad motion: ask the display for its rotation four times a second

The rotation was looked up for every sensor reading, every 2 ms, and cost more than the rest of
the reading; it is now refreshed every 250 ms, and the readings no longer allocate pairs.

* Session: keep the library passes out of a game's launch

The redist seeder stamped each prefix with the checksum of the whole set of names, which differs
between the session's pass and the game launch's (the game's own names are added), so every
launch re-appended the markers to every prefix in the library while the game loaded. Each
prefix's marker now lists the names it was given and only the missing ones are added.

The beat's seed and compatibility passes, and the Lossless Scaling ownership probe (which loads
steamclient.so), also wait while a wineserver runs: a launch rewrites the manifest and made them
due at exactly that moment. The game's own launch seeds its prefix regardless.

* Agent: keep the focus watcher's X connection open while nothing changes

The connect timeout stayed on the socket, so with no property change the read timed out, and the
watcher dropped the connection, reconnected and took a fresh snapshot every few seconds.

* Game environment: start Cyberpunk 2077 without REDlauncher

The prelauncher opens REDlauncher, a Qt WebEngine window that hands over to the game half a
minute later and then stays up beside it, well over a core busy for as long as the game runs; its
own --launcher-skip started the game twice. The game's executable is started directly instead.

* netif: answer Wine's network driver from a reading of the link

Mortal Kombat X has Wine's network driver enumerate every interface, address and route table
hundreds of times a second. Each enumeration read /etc/droiddeck-net several times, asked the
kernel for its interface dump again (a netlink socket bound under proot, refused each time) and
looked the real functions up with dlsym. The link file is now read at most once a second, a
refused dump is not asked for again within that second, and the real functions are looked up
once.

* Fast path: create memfds without a stop in proot

proot looks at a memfd's name only to refuse the few that are then executed or locked; every other
memfd_create is now issued from the trampoline. The session's /proc/net and status copies made
one per read, a stop each. Without memfd in the kernel the call still goes to proot's ashmem.

* Logs: run each redaction rule only on a line that could match it

A session's end copies some 70 MB of the Steam client's logs and puts every line through all
twenty-odd rules, most of a minute of a core on the phone after every session. Each rule now runs
only on a line holding what it cannot match without: its keyword, the five separators of a MAC,
the four dashes of a GUID, a long enough hex or token run, the learned account or address text.
Lines with non-ASCII text still meet every case-insensitive rule. Over 895,000 lines of real
client logs, with the device's accounts and addresses learned, the output is identical and the
pass takes a twentieth of the time.
2026-10-10 19:00:33 -04:00

173 lines
8.5 KiB
Bash
Executable File

#!/bin/bash
# Marks Steam's shared redistributables as already run in every Proton prefix, and in the
# template Proton makes new prefixes from.
#
# A game whose depot lists _CommonRedist makes the client run the real installer inside the
# prefix on first launch. Those are x86 bundles, so they run under FEX, and the 2019 one leaves
# a task behind every time: its main thread exits while a helper stays blocked reading a pipe it
# also holds the write end of, so the process never reaps and the client keeps waiting on it.
#
# The client skips a "Run Process" block whose HasRunKey is already set, which is the same escape
# WineUtils.seedVcRedistRegistryIfDllsPresent() takes for the app's own containers. The names and
# versions come out of the install scripts rather than a list kept here, because Valve revises
# them; a prefix already carrying the same set is left alone.
#
# Games also carry install scripts of their own (<game>/_CommonRedist/vcredist/2022/installscript.vdf
# and the like), often with another version in the value name; those are read too. And the VC++
# 2015-2022 runtime is recorded as installed the way Microsoft's installers look for it: on ARM64
# Proton the x64 bundle otherwise stops with "Microsoft Visual C++ 2015-2022 Redistributable (arm64)
# is required", while Proton's own msvcp140/vcruntime140 already serve the game.
set -u
steam_root=${1:?steam root}
LIBRARIES=${DROIDDECK_LIBRARIES-/mnt/droiddeck /mnt/droiddeck-sd}
redist="$steam_root/steamapps/common/Steamworks Shared/_CommonRedist"
# Every process an install script would run, as the registry key and value the client writes once
# it has run it. Names sit one level inside "Run Process", the key they share a level further in.
read_scripts() { xargs -0 -r awk '
# A block whose HasRunKey is set is one process; its name is the key the block opened under, at
# whatever depth - an install script nests it under "Run Process", the client'"'"'s evaluator script
# for a game wraps the same blocks once more.
FNR == 1 { depth = 0; pending = "" }
/^[[:space:]]*\{/ { depth++; name[depth] = pending; key[depth] = ""; pending = ""; next }
/^[[:space:]]*\}/ {
if (depth > 0 && key[depth] != "" && name[depth] != "")
printf "%s\t%s\n", key[depth], name[depth]
if (depth > 0) depth--
next
}
{
n = split($0, part, "\"")
if (n >= 5 && tolower(part[2]) == "hasrunkey") { key[depth] = part[4]; next }
if (n == 3) pending = part[2]
}
'; }
pairs=$(find "$redist" -name installscript.vdf -print0 2>/dev/null | sort -z | read_scripts)
# The client downloads the scripts with the first game that wants them, which is too late for that
# game; until then the set they held when this was written stands in. (WinNative's list.)
[ -n "$pairs" ] || pairs=$(sed 's/^\([^ ]*\) /HKEY_LOCAL_MACHINE\\\\Software\\\\Valve\\\\Steam\\\\Apps\\\\CommonRedist\\\\\1\t/' <<'KNOWN'
DirectX\\Jun2010 dxsetup
.NET\\4.8 4.8
vcredist\\2008 x86 SP1
vcredist\\2008 x64 SP1
vcredist\\2010 x86
vcredist\\2010 x64
vcredist\\2013 x86 12.0.30501
vcredist\\2013 x64 12.0.30501
vcredist\\2015 x86 Update 3 14.0.24215.0
vcredist\\2015 x64 Update 3 14.0.24215.0
vcredist\\2017 x86 14.10.25008.0
vcredist\\2017 x64 14.10.25008.0
vcredist\\2019 x86 14.28.29334.0
vcredist\\2019 x64 14.28.29334.0
vcredist\\2022 x86 14.51.36247.0
vcredist\\2022 x64 14.51.36247.0
KNOWN
)
# The games' own scripts. Finding them walks every game folder, so the list is kept until a game is
# installed, updated or removed (its appmanifest changes) - this runs every quarter minute.
cache="$steam_root/.droiddeck-game-scripts"
libraries="$steam_root"
for library in $LIBRARIES; do libraries="$libraries
$library"; done
libraries=$(printf '%s\n%s\n' "$libraries" "$(sed -n 's/^[[:space:]]*"path"[[:space:]]*"\(.*\)"[[:space:]]*$/\1/p' "$steam_root/steamapps/libraryfolders.vdf" 2>/dev/null)" | awk 'NF && !seen[$0]++')
sig=$(while IFS= read -r library; do ls -l --time-style=+%s "$library"/steamapps/appmanifest_*.acf 2>/dev/null; done <<<"$libraries" | cksum)
if [ "$(head -n 1 "$cache" 2>/dev/null)" != "$sig" ]; then
{
printf '%s\n' "$sig"
while IFS= read -r library; do
[ -d "$library/steamapps/common" ] || continue
find "$library/steamapps/common" -mindepth 2 -maxdepth 5 -iname 'installscript*.vdf' \
-not -path '*/Steamworks Shared/*' 2>/dev/null
done <<<"$libraries" | sort -u
} >"$cache.tmp" && mv -f "$cache.tmp" "$cache"
fi
game_pairs=$(tail -n +2 "$cache" 2>/dev/null | tr '\n' '\0' | read_scripts)
# DROIDDECK_SEED_SCRIPTS: more scripts, one path a line - droiddeck-game-env passes the client's
# evaluator script for the game about to run its installers, read before the client looks.
if [ -n "${DROIDDECK_SEED_SCRIPTS:-}" ]; then
game_pairs=$(printf '%s\n%s\n' "$game_pairs" "$(printf '%s\n' "$DROIDDECK_SEED_SCRIPTS" | tr '\n' '\0' | read_scripts)")
fi
[ -n "$(printf '%s' "$game_pairs" | tr -d '[:space:]')" ] && pairs=$(printf '%s\n%s\n' "$pairs" "$game_pairs" | sort -u | sed '/^$/d')
# The VC++ 2015-2022 runtime as its installers check for it, for each architecture and both registry
# views. The version is the newest the scripts above know, so any 2015-2022 bundle finds itself
# already installed and exits without asking for anything.
VCRT_MAJOR=14 VCRT_MINOR=51 VCRT_BUILD=36247
vcrt() {
local view arch
for view in 'Software' 'Software\\Wow6432Node'; do
for arch in x86 x64 arm64; do
printf '\n[%s\\\\Microsoft\\\\VisualStudio\\\\14.0\\\\VC\\\\Runtimes\\\\%s] %d\n' "$view" "$arch" "$1"
printf '"Bld"=dword:%08x\n"Installed"=dword:00000001\n"Major"=dword:%08x\n"Minor"=dword:%08x\n"Rbld"=dword:00000000\n' \
"$VCRT_BUILD" "$VCRT_MAJOR" "$VCRT_MINOR"
printf '"Version"="v%d.%d.%d.00"\n' "$VCRT_MAJOR" "$VCRT_MINOR" "$VCRT_BUILD"
done
done
}
# Each prefix's marker lists the names it was given, so it is handed only what it lacks. One stamp
# for the whole set flipped between the session's set and the evaluator's (which adds the game's
# own names) on every launch, and every prefix was appended to again, inside the launch.
want=()
declare -A wanted=()
while IFS= read -r line; do
[[ $line == *$'\t'* ]] && [ -z "${wanted[$line]:-}" ] && wanted[$line]=1 && want+=("$line")
done <<<"$pairs"
want+=("vcrt"$'\t'"$VCRT_MAJOR.$VCRT_MINOR.$VCRT_BUILD")
printf -v now '%(%s)T' -1
block_for() { printf '%s\n' "$@" | awk -F'\t' -v now="$now" '
{
path = $1
sub(/^HKEY_LOCAL_MACHINE\\\\/, "", path)
if (path == $1)
next
printf "\n[%s] %d\n\"%s\"=dword:00000001\n", path, now, $2
# The client writes these from 32-bit steam.exe, so its own copy lands under Wow6432Node;
# both spellings are seeded because the reader picks by its own word size.
# Joined rather than sub()stituted: gawk and mawk read backslashes in a replacement differently.
if (substr(path, 1, 10) != "Software\\\\")
next
printf "\n[%s] %d\n\"%s\"=dword:00000001\n", "Software\\\\Wow6432Node\\\\" substr(path, 11), now, $2
}
'
local line
for line; do [[ $line == vcrt$'\t'* ]] && vcrt "$now"; done
}
# Proton builds a new prefix from its template, and the client runs the install scripts in that
# prefix moments later, so the template carries the markers too. Its date is kept: Proton reads a
# changed one as a new template and upgrades every prefix made from the old.
for reg in "$steam_root"/steamapps/common/Proton*/files/share/default_pfx*/system.reg \
/mnt/droiddeck-sd/steamapps/common/Proton*/files/share/default_pfx*/system.reg \
/root/Games/*/steamapps/common/Proton*/files/share/default_pfx*/system.reg \
"$steam_root"/compatibilitytools.d/*/files/share/default_pfx*/system.reg \
"$steam_root"/steamapps/compatdata/*/pfx/system.reg \
/mnt/droiddeck-sd/steamapps/compatdata/*/pfx/system.reg \
/root/Games/*/steamapps/compatdata/*/pfx/system.reg; do
[ -f "$reg" ] || continue
seeded=${reg%/*}/.droiddeck-redists
legacy=${reg%/*}/.bannerlator-redists
if [ -f "$legacy" ] && [ ! -f "$seeded" ]; then
mv -f "$legacy" "$seeded"
fi
declare -A had=()
if [ -f "$seeded" ]; then
while IFS= read -r line; do [[ $line == *$'\t'* ]] && had[$line]=1; done <"$seeded"
fi
missing=()
for line in "${want[@]}"; do [ -n "${had[$line]:-}" ] || missing+=("$line"); done
[ ${#missing[@]} -eq 0 ] && continue
dated=$(mktemp) || continue
touch -r "$reg" "$dated"
if printf '%s\n' "$(block_for "${missing[@]}")" >>"$reg"; then
printf '%s\n' "${!had[@]}" "${missing[@]}" >"$seeded"
echo "== seeded redist markers into $reg"
fi
touch -r "$dated" "$reg"
rm -f "$dated"
done