From 5c07c122feb8e1f9c64881a0bfb4b36227df3f51 Mon Sep 17 00:00:00 2001 From: MaxsTechReview Date: Sun, 18 Oct 2026 20:38:48 -0400 Subject: [PATCH] Steam x86-64 client: menus come up, sign-in stays put, games listed once (#1096) * Steam x86-64 client: menus come up, sign-in stays put, games listed once Switching the Steam client from arm64 to x86-64 left the client on a blank screen behind "Downloading the Steam client update 100%", and each switch signed one of the clients out. - The x86-64 client is Valve's steamrt64 platform, but it was installed and synced from the ubuntu12 manifest, which lags it (1792108788 against 1792192922). The client was therefore never the arm64 client's build, the arm64 web helper was never used, and the client then updated itself in place after the helper had been chosen. Install, sync and the build check now read the steamrt64 manifest. - The x86 web helper's GPU process loads libpci, which exits when /sys/bus/pci/devices lists a device whose files the app may not read (a RedMagic's PCIe root ports). Every GPU process exited with status 1, even in software, until Chromium gave up and the menus stayed blank. An empty folder is bound over /sys/bus/pci/devices on such devices. - Steam refuses one client's login token from the other ("Invalid Password" for a token the arm64 client signed in with minutes before), so copying the newer token at every start signed the other client out. Each client now keeps its own token; a client with no accounts only gets the other's account list. - Games could be listed twice: an added game whose folder a Steam import already covers is now shown as that Steam game only, a client switch drops the old client's imported entries, an import stays one when its ownership cannot be confirmed instead of gaining a shortcut beside it, and only the client's own library folders count as an install. * Steam x86-64 client: share the arm64 sign-in again The x86-64 client refused the arm64 client's login token because it did not know the account's SteamID: it takes it from Accounts in config/config.vdf, which this script creates without one. It logged on as [U:1:0] with a valid token, Steam answered "Invalid Password", and the client dropped the token and AutoLoginUser, so every switch asked for the password. The newer sign-in is copied between the clients again, each start adds the accounts either client knows to the other's config.vdf, and an AutoLoginUser or RememberPassword one client lacks is filled in. Device: the x86 client signs in with the arm64 token on a fresh install and on an install left signed out, and the arm64 client stays signed in. --- README.md | 2 +- .../com/droiddeck/launcher/MainActivity.kt | 5 +- .../droiddeck/launcher/frontend/Library.kt | 16 ++- .../launcher/runtime/LinuxRuntime.java | 12 ++ .../launcher/frontend/LibraryArchTest.kt | 13 ++ docs/development/x86-64-steam-client.md | 37 +++-- .../usr/local/bin/droiddeck-steam-games | 15 ++- .../overlay/usr/local/bin/droiddeck-steam-x64 | 127 +++++++++++++----- tools/tests/test_steam_games.py | 18 +++ tools/tests/test_steam_x64_signin.py | 78 +++++++++++ 10 files changed, 278 insertions(+), 45 deletions(-) create mode 100644 tools/tests/test_steam_x64_signin.py diff --git a/README.md b/README.md index d699d4419b..1660171ebd 100644 --- a/README.md +++ b/README.md @@ -47,7 +47,7 @@ Compatibility and performance vary by device; hardware validation is limited. De ## Credits and licence -GPL-3.0. Runtime, shim, input, and controller work build on WinNative and Bannerlator (maxjivi05). Epic Games and GOG support build on [Phobos’s](https://github.com/phobos665) adaptation of [Legendary](https://github.com/legendary-gl/legendary) and [gogdl](https://github.com/Heroic-Games-Launcher/heroic-gogdl), respectively. LSFG frame generation is from the work of Camille LaVey and the [Eden](https://eden-emu.dev) emulator project, following [lsfg-vk](https://github.com/PancakeTAS/lsfg-vk), ported to WinNative and DroidDeck by [@maxjivi05](https://github.com/maxjivi05); it needs your own copy of [Lossless Scaling](https://store.steampowered.com/app/993090/) and ships none of its shaders. x86 AppImages, and any whose own runtime cannot unpack them, are unpacked with [uruntime](https://github.com/VHSgunzo/uruntime) by VHSgunzo (MIT), shipped unmodified with its licence. The Steam client's audio goes through [@The412Banner](https://github.com/The412Banner)'s [DirectAudio](https://github.com/The412Banner/directaudio) PulseAudio sink (LGPL-2.1-or-later), built from `tools/directaudio/sink` with its request size kept to its ring so clients are not asked to queue 100 ms ahead. mmap ring-buffer controller input was developed by ribbit384 ([WinNative #345](https://github.com/WinNative-Emu/WinNative/pull/345)). See [LICENSE](LICENSE). Steam and Proton belong to Valve Corporation; this project is not affiliated with Valve. +GPL-3.0. Runtime, shim, input, and controller work build on WinNative and Bannerlator (maxjivi05). Epic Games and GOG support build on [Phobos’s](https://github.com/phobos665) adaptation of [Legendary](https://github.com/legendary-gl/legendary) and [gogdl](https://github.com/Heroic-Games-Launcher/heroic-gogdl), respectively. LSFG frame generation is from the work of Camille LaVey and the [Eden](https://eden-emu.dev) emulator project, following [lsfg-vk](https://github.com/PancakeTAS/lsfg-vk), ported to WinNative and DroidDeck by [@maxjivi05](https://github.com/maxjivi05); it needs your own copy of [Lossless Scaling](https://store.steampowered.com/app/993090/) and ships none of its shaders. x86 AppImages, and any whose own runtime cannot unpack them, are unpacked with [uruntime](https://github.com/VHSgunzo/uruntime) by VHSgunzo (MIT), shipped unmodified with its licence. The Steam client's audio goes through [@The412Banner](https://github.com/The412Banner)'s [DirectAudio](https://github.com/The412Banner/directaudio) PulseAudio sink (LGPL-2.1-or-later), built from `tools/directaudio/sink` with its request size kept to its ring so clients are not asked to queue 100 ms ahead. mmap ring-buffer controller input was developed by ribbit384 ([WinNative #345](https://github.com/WinNative-Emu/WinNative/pull/345)). Why the x86-64 Steam client refused the arm64 client's sign-in was checked by reading Steam's saved token with the `ConnectCache` format from [deadlock-api-ingest](https://github.com/deadlock-api/deadlock-api-ingest). See [LICENSE](LICENSE). Steam and Proton belong to Valve Corporation; this project is not affiliated with Valve. XR glasses support builds on WinNative's VITURE glasses work. VITURE protocol details were checked against [XRLinuxDriver](https://github.com/wheaney/XRLinuxDriver) by [@wheaney](https://github.com/wheaney), including the VITURE protocol header it ships, [ar-drivers-rs](https://github.com/badicsalex/ar-drivers-rs) by [@badicsalex](https://github.com/badicsalex) and the [go-macos/iokit VITURE package](https://pkg.go.dev/github.com/go-macos/iokit/viture); the optional VITURE SDK belongs to VITURE and is not included. RayNeo support adapts [@informalTechCode](https://github.com/informalTechCode)'s [RayNeo-Air-Series-3DoF-Scaffold](https://github.com/informalTechCode/RayNeo-Air-Series-3DoF-Scaffold) (Apache-2.0) and references [@verncat](https://github.com/verncat)'s [RayNeo-Air-3S-Pro-OpenVR](https://github.com/verncat/RayNeo-Air-3S-Pro-OpenVR) (MIT) and [Monado](https://gitlab.freedesktop.org/monado/monado) (BSL-1.0); see [docs/third-party/rayneo](docs/third-party/rayneo/README.md). XREAL 1S support references [XRLinuxDriver](https://github.com/wheaney/XRLinuxDriver) and [xrealOneDeviceKit](https://github.com/wheaney/xrealOneDeviceKit) by @wheaney, [xreal_one_driver](https://github.com/rohitsangwan01/xreal_one_driver) by [@rohitsangwan01](https://github.com/rohitsangwan01) (MIT) and [xr-tools](https://github.com/0xcaff/xr-tools) by [@0xcaff](https://github.com/0xcaff). The REDMAGIC glasses-display findings draw on [@mekhontsev](https://github.com/mekhontsev)'s [MagicDesk investigation](https://github.com/mekhontsev/magicdesk/blob/main/docs/xr-resolution-fix.md) and [@Coding-BR](https://github.com/Coding-BR)'s [decompiled Nubia kernel](https://github.com/Coding-BR/android_kernel_nubia_sm8850_qwjujube). VITURE, RayNeo, XREAL and REDMAGIC are trademarks of their owners; this project is not affiliated with them. diff --git a/app/src/main/java/com/droiddeck/launcher/MainActivity.kt b/app/src/main/java/com/droiddeck/launcher/MainActivity.kt index 2330dfb23c..2796927140 100644 --- a/app/src/main/java/com/droiddeck/launcher/MainActivity.kt +++ b/app/src/main/java/com/droiddeck/launcher/MainActivity.kt @@ -1882,7 +1882,7 @@ class MainActivity : ComponentActivity() { val listX64 = SessionPrefs.activeSteamClientArch(this) == "x86_64" if (listX64 != steamGamesX64) { steamGamesX64 = listX64 - steamGames = steamGames.filterNot(Library::fromSteamClient) + steamGames = steamGames.filterNot { Library.fromSteamClient(it) || (it.library == Library.ADDED && it.gameId in 1..0xFFFFFFFFL) } } if (!busy && LinuxRuntimeInstaller.isBusy()) followRuntimeOperation { LinuxRuntimeInstaller.attach(it) } if (showComponents) drivers.refreshDrivers() @@ -1916,12 +1916,13 @@ class MainActivity : ComponentActivity() { ui.post { if (scanGeneration == libraryScanGeneration) emulatorList = emus } if (ready) { val quick = runCatching { Library.steamGames(this) }.getOrNull() + val claimed = quick?.let { runCatching { Library.claimedFolders(it) }.getOrDefault(emptySet()) }.orEmpty() if (quick != null) ui.post { if (scanGeneration == libraryScanGeneration && steamGames.isNotEmpty()) { val before = steamGames.map { it.gameId }.toSet() val fresh = quick.filter { it.gameId !in before } if (fresh.isNotEmpty()) { - steamGames = (quick + steamGames.filter { it.library == Library.ADDED }).distinctBy { it.gameId } + steamGames = (quick + steamGames.filter { it.library == Library.ADDED && it.gameFiles?.absolutePath.let { path -> path == null || path !in claimed } }).distinctBy { it.gameId } fresh.forEach { com.droiddeck.launcher.frontend.AutoComponents.queueGame(this, it, asked = false) } } } diff --git a/app/src/main/java/com/droiddeck/launcher/frontend/Library.kt b/app/src/main/java/com/droiddeck/launcher/frontend/Library.kt index 743fea5d9e..1242d9add1 100644 --- a/app/src/main/java/com/droiddeck/launcher/frontend/Library.kt +++ b/app/src/main/java/com/droiddeck/launcher/frontend/Library.kt @@ -178,9 +178,23 @@ object Library { var steam = steamGames(context, strictRead, arch) val targets = steam.filter(::fromSteamClient).map { SteamGrid.target(it, context) } + list.map { SteamGrid.target(it, context) } if (runCatching { SteamGrid.sync(context, targets) }.getOrDefault(false)) steam = steamGames(context, strictRead, arch) - return (steam + list.map { g -> addedGame(context, g, roots) }).distinctBy { it.gameId } + val claimed = claimedFolders(steam) + val own = list.filter { g -> + g.steamAppId != null || claimed.isEmpty() || + (runCatching { g.folder.canonicalPath }.getOrDefault(g.folder.absolutePath) !in claimed && + AddedGames.guestPath(context, g.folder).let { it == null || it !in claimed }) + } + return (steam + own.map { g -> addedGame(context, g, roots) }).distinctBy { it.gameId } } + fun claimedFolders(steam: List): Set = steam.filter(::fromSteamClient).flatMap { g -> + val dir = g.gameFiles ?: return@flatMap emptyList() + listOfNotNull( + runCatching { dir.canonicalPath }.getOrNull(), + runCatching { java.nio.file.Files.readSymbolicLink(dir.toPath()).toString() }.getOrNull(), + ) + }.toSet() + fun addedGame(context: Context, g: AddedGames.Game, roots: List = steamLibraries(context).map { it.first }): SteamGame = AddedGameArt.resolve(context, g).let { art -> SteamGame( diff --git a/app/src/main/java/com/droiddeck/launcher/runtime/LinuxRuntime.java b/app/src/main/java/com/droiddeck/launcher/runtime/LinuxRuntime.java index f7f68d6da3..c729850fcd 100644 --- a/app/src/main/java/com/droiddeck/launcher/runtime/LinuxRuntime.java +++ b/app/src/main/java/com/droiddeck/launcher/runtime/LinuxRuntime.java @@ -214,6 +214,9 @@ public final class LinuxRuntime { bind(cmd, fake.getPath() + ":" + entry[1]); } } + if (unreadablePciDevice()) { + bind(cmd, new File(root, "etc/droiddeck/empty").getPath() + ":/sys/bus/pci/devices"); + } bindGpuNode(context, cmd); bindAdrenoStats(cmd, extraBinds); bindCpuTemps(cmd, root); @@ -225,6 +228,15 @@ public final class LinuxRuntime { return specs; } + private static boolean unreadablePciDevice() { + File[] devices = new File("/sys/bus/pci/devices").listFiles(); + if (devices == null) return false; + for (File device : devices) { + if (!new File(device, "vendor").canRead()) return true; + } + return false; + } + private static void bindGpuNode(Context context, List cmd) { StructStat st; try { diff --git a/app/src/test/java/com/droiddeck/launcher/frontend/LibraryArchTest.kt b/app/src/test/java/com/droiddeck/launcher/frontend/LibraryArchTest.kt index d3a230a813..8ecd411b0c 100644 --- a/app/src/test/java/com/droiddeck/launcher/frontend/LibraryArchTest.kt +++ b/app/src/test/java/com/droiddeck/launcher/frontend/LibraryArchTest.kt @@ -22,4 +22,17 @@ class LibraryArchTest { assertEquals(listOf("Team Fortress 2"), Library.launchableGames(app, added = emptyList(), arch = "x86_64").map { it.name }) assertEquals(emptyList(), Library.launchableGames(app, added = emptyList(), arch = "arm64").map { it.name }) } + + @Test fun anAddedGameSteamAlreadyListsFromItsFolderIsShownOnce() { + SessionPrefs.setGameStorage(app, SessionPrefs.GAME_STORAGE_OFF, "") + val folder = File(app.filesDir, "games/Example").apply { mkdirs() } + val exe = File(folder, "Example.exe").apply { writeText("game") } + val apps = File(SessionPrefs.steamRoot(app, "arm64"), "steamapps").apply { mkdirs() } + java.nio.file.Files.createSymbolicLink(File(apps, "common").apply { mkdirs() }.toPath().resolve("DroidDeck-42"), folder.toPath()) + File(apps, "appmanifest_42.acf").writeText("\"AppState\"\n{\n\t\"appid\"\t\t\"42\"\n\t\"name\"\t\t\"Example\"\n\t\"StateFlags\"\t\t\"4\"\n\t\"installdir\"\t\t\"DroidDeck-42\"\n}\n") + val added = AddedGames.Game(folder, "Example", exe, "/g/Example.exe", "/g", 0x87654321L, (0x87654321L shl 32) or 0x02000000L, emptyList()) + val other = File(app.filesDir, "games/Other").apply { mkdirs() } + val second = AddedGames.Game(other, "Other", File(other, "Other.exe"), "/g/Other.exe", "/g", 0x81234567L, (0x81234567L shl 32) or 0x02000000L, emptyList()) + assertEquals(listOf("Example" to "internal", "Other" to Library.ADDED), Library.launchableGames(app, added = listOf(added, second), arch = "arm64").map { it.name to it.library }) + } } diff --git a/docs/development/x86-64-steam-client.md b/docs/development/x86-64-steam-client.md index 25e25a700c..f11923fe7d 100644 --- a/docs/development/x86-64-steam-client.md +++ b/docs/development/x86-64-steam-client.md @@ -39,6 +39,11 @@ arm64 client down first. standalone launcher honours up to five times, as `steam.sh` does). - Before each start `x64_sync_client` brings the client to Valve's current build, downloading only the changed parts (9 of 28 on a device). If that fails, the client updates itself. +- The client is the `steamrt64` platform (`steamrt64/steam`, the manifest it checks for updates), so + the install, the sync and the web-helper build check read `steam_client__steamrt64`. The + `ubuntu12` manifest lags it (device: 1792108788 against 1792192922): an install from it was always + one build behind the arm64 client, so the arm64 web helper was never used, and the client updated + itself in place at its first start (506 MB) after the helper had already been chosen. - `.steam-enable-steamrt64-client` in the client folder opts into the SteamRT3 build (`steamrt64/steam`). Without it the client re-executes itself in the scout environment through `steam.sh`, whose start-up checks need what proot cannot give; the session stalled before the @@ -119,6 +124,11 @@ The client's web helper draws its whole UI. There are two: backend through FEX's GL thunk to zink and Turnip. On ANGLE's Vulkan backend the x86 compositor thread was the bottleneck (16 fps against 60). Never `--in-process-gpu`, the preload's default: a GPU crash there took the whole helper down (102 restarts). `BL_X64_CEF_GPU=0` keeps it in software. + Its GPU process loads libpci, which exits when `/sys/bus/pci/devices` lists a device whose files + the app may not read (a RedMagic's PCIe root ports): every GPU process exited with status 1 in about + 50 ms, software ones too, until Chromium gave up ("GPU process isn't usable") and the menus stayed + blank. `LinuxRuntime` binds an empty folder over `/sys/bus/pci/devices` on such devices; device: + 0 GPU process exits, ANGLE on zink and Turnip. The session shim (`preload/net.c`) turns `steamrt64/steamwebhelper.sh`'s exec of `./steamwebhelper` into `BL_X64_ARM_WEBHELPER`. Nothing in the client's folder is edited: the client checksums its files @@ -217,7 +227,7 @@ alone would then read as an installed client. - **What a session log shows.** `FEX ready (, )` names the build and where it came from (`catalog`, `built-in`, or `kept` when a newer one did not install, with the reason a line above); - `x86-64 rootfs:` names the image; `x86-64 client:` the client's ubuntu12 and steamrt64 builds at start + `x86-64 rootfs:` names the image; `x86-64 client:` the client's steamrt64 build at start (it may update itself after). The rootfs download reports every tenth of its size as a step. When the web helper's GPU process keeps exiting until Chromium gives up on the GPU (the menus then never appear), one line says so with the exit codes, followed by x86 `glxinfo -B` and `vulkaninfo --summary` through @@ -301,15 +311,24 @@ alone would then read as an installed client. in 51 s. - **Sign-in.** Before either client starts, the newer sign-in (`config/loginusers.vdf`, the login token in `local.vdf`, `AutoLoginUser` and `RememberPassword` in `registry.vdf`) is copied to the other, so - signing in or switching accounts in one counts for both. Steam renews its token every few days and - refuses an old copy (a 3-day-old one asked for the password), so the newest is copied at every start, - never while either client runs, since each rewrites these files. A client that refused its token - rewrites `local.vdf` with an empty `ConnectCache`: newest, but copied over the other it would sign - that one out too (seen after the x86 client's "Invalid Password"), so it is never the source. The copy keeps - the time stamp, so both match until one client signs in again. The rest of `registry.vdf` - (language, skin) stays each client's. + signing in or switching accounts in one counts for both; never while either client runs, since each + rewrites these files. A client that refused its token rewrites `local.vdf` with an empty + `ConnectCache`, so it is never the source. The copy keeps the time stamp, so both match until one + client signs in again. The rest of `registry.vdf` (language, skin) stays each client's. +- **Account IDs.** A client logs on with the token only once it knows the account's SteamID, which it + takes from `Accounts` in `config/config.vdf`. The x86 client's `config.vdf` is made by this script + without one, so it logged on as `[U:1:0]` with the arm64 token and Steam answered "Invalid Password" + (and the client then dropped the token and `AutoLoginUser`): every switch asked for the password. + Each start adds the accounts either client knows to the other's `Accounts`, and an `AutoLoginUser` or + `RememberPassword` one client has and the other lacks. Device: the x86 client signed in with the + arm64 token (`RecvMsgClientLogOnResponse ... 'OK'`). - **App side.** The library, art picker, grid sync and boot movie follow the running client, else the - selected one; a changed client applies from the next session. Offline Mode is written to both + selected one; a changed client applies from the next session. A game is listed once per client: an added game + whose folder a Steam install already covers (its `DroidDeck-` import) is shown as that Steam + game only, and a client switch drops the old client's imported entries with its Steam games. In + Steam, an import stays one when its ownership cannot be confirmed that session instead of gaining a + shortcut beside it, and only the client's own library folders count as an install (the arm64 card + library made the x86 client skip a game's import and its shortcut). Offline Mode is written to both clients' `loginusers.vdf`; until the x86 client's first start its sign-in is the arm64 one. The log redactor learns account names from both clients. The startup movie is staged into the x86 client's config only once its first start has created it; `--prepare` links the arm64 `config/uioverrides` diff --git a/tools/linuxfs/overlay/usr/local/bin/droiddeck-steam-games b/tools/linuxfs/overlay/usr/local/bin/droiddeck-steam-games index 4bfbb87ede..c710ff0acc 100755 --- a/tools/linuxfs/overlay/usr/local/bin/droiddeck-steam-games +++ b/tools/linuxfs/overlay/usr/local/bin/droiddeck-steam-games @@ -291,7 +291,7 @@ def libraries(steam_root): except OSError: text = "" paths = [path.replace("\\\\", "\\") for path in re.findall(r'"path"\s+"((?:[^"\\]|\\.)*)"', text)] - return list(dict.fromkeys([Path(steam_root), Path(LIBRARY)] + [Path(path) for path in paths])) + return list(dict.fromkeys([Path(steam_root)] + [Path(path) for path in paths])) def ours(manifest, appid): @@ -311,6 +311,16 @@ def unpark(steamapps): os.replace(manifest, live) +def seeded(steamapps, game, appid): + target = steamapps / "common" / f"DroidDeck-{appid}" + manifest = steamapps / f"appmanifest_{appid}.acf" + try: + return (target.is_symlink() and target.is_dir() and manifest.is_file() and ours(manifest, appid) + and os.readlink(target) in (game.get("folder", ""), legacy_path(game.get("folder", "")))) + except OSError: + return False + + def park(steamapps): for target in sorted((steamapps / "common").glob("DroidDeck-*")): appid = target.name.removeprefix("DroidDeck-") @@ -428,6 +438,9 @@ def route(steam_root, acct, games): continue except OSError as error: print(f"droiddeck-steam-games: import unavailable ({error})") + elif isinstance(appid, int) and 0 < appid < 0x80000000 and seeded(steamapps, game, appid): + routes[str(game["appid"])] = appid + continue shortcuts.append(game) try: park(steamapps) diff --git a/tools/linuxfs/overlay/usr/local/bin/droiddeck-steam-x64 b/tools/linuxfs/overlay/usr/local/bin/droiddeck-steam-x64 index 2c27c9db4d..e52a4e9858 100755 --- a/tools/linuxfs/overlay/usr/local/bin/droiddeck-steam-x64 +++ b/tools/linuxfs/overlay/usr/local/bin/droiddeck-steam-x64 @@ -8,7 +8,7 @@ ROOTFS_SHA256=2854b06d3ff1b8f6e526135bfb6dd5b7b30ab3ab73e79ae933a3d9fed959a178 CDN=https://client-update.fastly.steamstatic.com # Must match the -clientbeta droiddeck-session starts this client with (steam_channel). X64_CHANNEL=steamdeck_publicbeta -MANIFEST=steam_client_${X64_CHANNEL}_ubuntu12 +MANIFEST=steam_client_${X64_CHANNEL}_steamrt64 REAL_HOME=${HOME:-/root} X64_HOME=${BL_X64_HOME:-$REAL_HOME/.droiddeck-x64} @@ -37,17 +37,17 @@ x64_client_parts() { /^\t"/ && NF >= 3 && $4 == "" { block = $2 } $2 == "file" { file[block] = $4 } $2 == "sha2" { sha[block] = $4 } - END { for (b in file) if (b ~ /_all$/ || b ~ /_ubuntu12$/) print b, file[b], sha[b] } + END { for (b in file) if (b ~ /_all$/ || b ~ /_steamrt64$/) print b, file[b], sha[b] } ' "$1" | sort } x64_sync_client() { local chan cur version work total i block file sum old chan=$(cat "$STEAM_X64/package/beta" 2>/dev/null || echo "$X64_CHANNEL") - old=$STEAM_X64/package/steam_client_${chan}_ubuntu12.manifest + old=$STEAM_X64/package/steam_client_${chan}_steamrt64.manifest cur=$(awk -F'"' '$2 == "version" { print $4; exit }' "$old" 2>/dev/null || true) work=$(mktemp -d) - if ! curl -fsSL "$CDN/steam_client_${chan}_ubuntu12" -o "$work/manifest"; then + if ! curl -fsSL "$CDN/steam_client_${chan}_steamrt64" -o "$work/manifest"; then rm -rf "$work"; return 0 fi version=$(awk -F'"' '$2 == "version" { print $4; exit }' "$work/manifest") @@ -471,7 +471,7 @@ if [ ! -x "$STEAM_X64/steamrt64/steam" ] || [ ! -f "$STEAM_X64/package/droiddeck /^\t"/ && NF >= 3 && $4 == "" { block = $2 } $2 == "file" { file[block] = $4 } $2 == "sha2" { sha[block] = $4 } - END { for (b in file) if (b ~ /_all$/ || b ~ /_ubuntu12$/) print b, file[b], sha[b] } + END { for (b in file) if (b ~ /_all$/ || b ~ /_steamrt64$/) print b, file[b], sha[b] } ' "$work/manifest" | sort > "$work/parts" [ -s "$work/parts" ] || { step "no client components in $MANIFEST"; exit 1; } mkdir -p "$STEAM_X64/package" @@ -910,7 +910,7 @@ x64_exec() { [ -x "$ARM_STEAM/steamrtarm64/steamwebhelper" ]; then local chan x86_build arm_build chan=$(cat "$STEAM_X64/package/beta" 2>/dev/null || echo "$X64_CHANNEL") - x86_build=$(awk -F'"' '$2 == "version" { print $4; exit }' "$STEAM_X64/package/steam_client_${chan}_ubuntu12.manifest" 2>/dev/null || true) + x86_build=$(awk -F'"' '$2 == "version" { print $4; exit }' "$STEAM_X64/package/steam_client_${chan}_steamrt64.manifest" 2>/dev/null || true) arm_build=$(awk -F'"' '$2 == "version" { print $4; exit }' "$ARM_STEAM/package/steam_client_${chan}_linuxarm64.manifest" 2>/dev/null || true) if [ -n "$x86_build" ] && [ "$x86_build" = "$arm_build" ]; then export BL_X64_ARM_WEBHELPER=/usr/local/bin/droiddeck-x64-arm-webhelper @@ -937,7 +937,7 @@ x64_exec() { touch "$STEAM_X64/logs/bootstrap_log.txt" ( tail -n0 -F --pid=$$ "$STEAM_X64/logs/bootstrap_log.txt" 2>/dev/null | grep --line-buffered -oE 'Downloading update \([0-9,]+ of [0-9,]+ KB\)' | sed -u 's/,//g' >&2 ) & - echo "== x86-64 client: ubuntu12 $(x64_manifest_version ubuntu12), steamrt64 $(x64_manifest_version steamrt64) (the client may update itself)" >&2 + echo "== x86-64 client: steamrt64 $(x64_manifest_version steamrt64) (the client may update itself)" >&2 x64_webhelper_gpu && { x64_gpu_watch >&2 & } exec "${BL_FEX:-/usr/local/bin/FEX}" "$STEAM_X64/steamrt64/steam" "$@" } @@ -952,6 +952,76 @@ arm, x64, arm_reg, x64_reg = sys.argv[1:5] FILES = ("config/loginusers.vdf", "local.vdf") +def read(path): + try: + return open(path, encoding="utf-8", errors="surrogateescape").read() + except OSError: + return None + + +def write(path, text): + open(path, "w", encoding="utf-8", errors="surrogateescape").write(text) + + +def block(text, name): + m = re.search(r'"%s"\s*\{' % name, text) + if not m: + return None + depth, k = 1, m.end() + while depth and k < len(text): + depth += text[k] == "{" + depth -= text[k] == "}" + k += 1 + return m, k + + +def accounts(root): + text = read(os.path.join(root, "config/config.vdf")) or "" + found = block(text, "Accounts") + if not found: + return {} + return dict(re.findall(r'"([^"]+)"\s*\{\s*"SteamID"\s*"(\d+)"', text[found[0].end():found[1]])) + + +def add_accounts(root, wanted): + path = os.path.join(root, "config/config.vdf") + text = read(path) + if text is None: + return + have = accounts(root) + missing = {name: sid for name, sid in wanted.items() if name not in have} + if not missing: + return + entries = "".join('\n\t\t\t\t\t"%s"\n\t\t\t\t\t{\n\t\t\t\t\t\t"SteamID"\t\t"%s"\n\t\t\t\t\t}' % item for item in missing.items()) + found = block(text, "Accounts") + if found: + text = text[:found[0].end()] + entries + text[found[0].end():] + else: + steam = re.search(r'"Valve"\s*\{\s*"Steam"\s*\{', text) + if not steam: + return + text = text[:steam.end()] + '\n\t\t\t\t"Accounts"\n\t\t\t\t{' + entries + '\n\t\t\t\t}' + text[steam.end():] + write(path, text) + + +def set_keys(src_reg, dst_reg, force): + st, dt = read(src_reg), read(dst_reg) + if st is None or dt is None: + return + for key in ("AutoLoginUser", "RememberPassword"): + v = re.search(r'"%s"\s*"([^"]*)"' % key, st) + if not v: + continue + if re.search(r'"%s"\s*"' % key, dt): + if force: + dt = re.sub(r'("%s"\s*")[^"]*(")' % key, lambda m: m.group(1) + v.group(1) + m.group(2), dt, count=1) + continue + hkcu = re.search(r'"HKCU"\s*\{\s*"Software"\s*\{\s*"Valve"\s*\{\s*"Steam"\s*\{\n', dt) + if hkcu: + dt = dt[:hkcu.end()] + '\t\t\t\t\t"%s"\t\t"%s"\n' % (key, v.group(1)) + dt[hkcu.end():] + write(dst_reg, dt) + + def stamp(root): if not all(os.path.isfile(os.path.join(root, f)) for f in FILES): return None @@ -963,31 +1033,26 @@ def stamp(root): a, x = stamp(arm), stamp(x64) -if a is None and x is None: - sys.exit(0) -if a is not None and x is not None and abs(a - x) < 1: - sys.exit(0) -if x is None or (a is not None and a > x): - src, dst, sreg, dreg, who = arm, x64, arm_reg, x64_reg, "arm64" -else: - src, dst, sreg, dreg, who = x64, arm, x64_reg, arm_reg, "x86-64" -for f in FILES: - os.makedirs(os.path.dirname(os.path.join(dst, f)), exist_ok=True) - # copy2 keeps the time stamp, so both match until one client signs in again. - shutil.copy2(os.path.join(src, f), os.path.join(dst, f)) -if os.path.isfile(sreg): - st = open(sreg, encoding="utf-8", errors="surrogateescape").read() - if not os.path.isfile(dreg): - os.makedirs(os.path.dirname(dreg), exist_ok=True) - shutil.copy2(sreg, dreg) +if a is not None or x is not None: + if x is None or (a is not None and a > x): + src, dst, sreg, dreg, who = arm, x64, arm_reg, x64_reg, "arm64" else: - dt = open(dreg, encoding="utf-8", errors="surrogateescape").read() - for key in ("AutoLoginUser", "RememberPassword"): - v = re.search(r'"%s"\s*"([^"]*)"' % key, st) - if v and re.search(r'"%s"\s*"' % key, dt): - dt = re.sub(r'("%s"\s*")[^"]*(")' % key, lambda m: m.group(1) + v.group(1) + m.group(2), dt, count=1) - open(dreg, "w", encoding="utf-8", errors="surrogateescape").write(dt) -print("droiddeck-steam-x64: sign-in shared from the %s client (the newer one)" % who) + src, dst, sreg, dreg, who = x64, arm, x64_reg, arm_reg, "x86-64" + copy = a is None or x is None or abs(a - x) >= 1 + if copy: + for f in FILES: + os.makedirs(os.path.dirname(os.path.join(dst, f)), exist_ok=True) + # copy2 keeps the time stamp, so both match until one client signs in again. + shutil.copy2(os.path.join(src, f), os.path.join(dst, f)) + if os.path.isfile(sreg) and not os.path.isfile(dreg): + os.makedirs(os.path.dirname(dreg), exist_ok=True) + shutil.copy2(sreg, dreg) + set_keys(sreg, dreg, copy) + set_keys(dreg, sreg, False) + add_accounts(dst, accounts(src)) + add_accounts(src, accounts(dst)) + if copy: + print("droiddeck-steam-x64: sign-in shared from the %s client (the newer one)" % who) PY } diff --git a/tools/tests/test_steam_games.py b/tools/tests/test_steam_games.py index b451261647..7ad6e53003 100644 --- a/tools/tests/test_steam_games.py +++ b/tools/tests/test_steam_games.py @@ -311,6 +311,24 @@ class ImportsTest(unittest.TestCase): self.assertFalse((self.steam / 'steamapps/appmanifest_42.acf').exists()) self.assertFalse(os.path.lexists(self.steam / 'steamapps/common/DroidDeck-42')) + def test_imported_game_stays_imported_when_ownership_is_unconfirmed(self): + self.snapshot() + imports.route(self.steam, self.acct, [self.game]) + for owned, account in [(False, '123'), (True, '456')]: + self.snapshot(owned, account) + self.assertEqual(([], {str(self.game['appid']): 42}), imports.route(self.steam, self.acct, [self.game])) + self.assertTrue((self.steam / 'steamapps/appmanifest_42.acf').is_file()) + + def test_unlisted_card_library_is_not_another_clients_install(self): + card = self.root / 'card' + (card / 'steamapps').mkdir(parents=True) + (card / 'steamapps/appmanifest_42.acf').write_text('the other client installed it here') + self.snapshot() + with patch.object(imports, 'LIBRARY', str(card)): + self.assertEqual(([], {str(self.game['appid']): 42}), imports.route(self.steam, self.acct, [self.game])) + self.assertTrue((self.steam / 'steamapps/appmanifest_42.acf').is_file()) + self.assertEqual(self.folder, (self.steam / 'steamapps/common/DroidDeck-42').resolve()) + def test_replaced_game_does_not_inherit_cached_ownership(self): self.snapshot() self.exe.write_bytes(b'a different game') diff --git a/tools/tests/test_steam_x64_signin.py b/tools/tests/test_steam_x64_signin.py new file mode 100644 index 0000000000..c294108e8c --- /dev/null +++ b/tools/tests/test_steam_x64_signin.py @@ -0,0 +1,78 @@ +from pathlib import Path +import os +import subprocess +import tempfile +import unittest + + +SCRIPT = Path(__file__).resolve().parents[1] / 'linuxfs/overlay/usr/local/bin/droiddeck-steam-x64' + +REGISTRY = ('"Registry"\n{\n\t"HKCU"\n\t{\n\t\t"Software"\n\t\t{\n\t\t\t"Valve"\n\t\t\t{\n\t\t\t\t"Steam"\n\t\t\t\t{\n' + '\t\t\t\t\t"language"\t\t"english"\n%s\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n}\n') +CONFIG = ('"InstallConfigStore"\n{\n\t"Software"\n\t{\n\t\t"Valve"\n\t\t{\n\t\t\t"Steam"\n\t\t\t{\n%s' + '\t\t\t\t"CompatToolMapping"\n\t\t\t\t{\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n}\n') +ACCOUNTS = ('\t\t\t\t"Accounts"\n\t\t\t\t{\n\t\t\t\t\t"someone"\n\t\t\t\t\t{\n' + '\t\t\t\t\t\t"SteamID"\t\t"76561198000000001"\n\t\t\t\t\t}\n\t\t\t\t}\n') + + +def local(token): + entry = '\t\t"abc1"\t\t"%s"\n' % token if token else '' + return '"MachineUserConfigStore"\n{\n\t"ConnectCache"\n\t{\n%s\t}\n}\n' % entry + + +class X64SignInTest(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory() + self.addCleanup(self.temp.cleanup) + self.home = Path(self.temp.name) + self.arm = self.home / '.local/share/Steam' + self.x64 = self.home / '.droiddeck-x64/.local/share/Steam' + for root in (self.arm, self.x64): + (root / 'config').mkdir(parents=True) + (self.home / '.steam').mkdir() + (self.home / '.droiddeck-x64/.steam').mkdir(parents=True) + self.write(self.arm / 'local.vdf', local('00ff')) + self.write(self.arm / 'config/loginusers.vdf', '"users"\n{\n}\n') + self.write(self.arm / 'config/config.vdf', CONFIG % ACCOUNTS) + self.write(self.home / '.steam/registry.vdf', REGISTRY % '\t\t\t\t\t"AutoLoginUser"\t\t"someone"\n') + self.write(self.x64 / 'local.vdf', local(None)) + self.write(self.x64 / 'config/loginusers.vdf', '"users"\n{\n}\n') + self.write(self.x64 / 'config/config.vdf', CONFIG % '') + self.write(self.home / '.droiddeck-x64/.steam/registry.vdf', REGISTRY % '') + + def write(self, path, text): + path.write_text(text) + + def sync(self): + env = dict(os.environ, HOME=str(self.home)) + env.pop('XDG_DATA_HOME', None) + env.pop('BL_X64_HOME', None) + return subprocess.run(['bash', str(SCRIPT), '--sync-login'], env=env, capture_output=True, text=True, check=True) + + def test_a_refused_x64_sign_in_gets_the_token_account_and_auto_login(self): + self.assertIn('sign-in shared from the arm64 client', self.sync().stdout) + self.assertEqual(local('00ff'), (self.x64 / 'local.vdf').read_text()) + self.assertIn('"SteamID"\t\t"76561198000000001"', (self.x64 / 'config/config.vdf').read_text()) + self.assertIn('"AutoLoginUser"\t\t"someone"', (self.home / '.droiddeck-x64/.steam/registry.vdf').read_text()) + + def test_an_earlier_copy_still_gains_the_account_and_nothing_repeats(self): + self.sync() + self.write(self.x64 / 'config/config.vdf', CONFIG % '') + self.write(self.home / '.droiddeck-x64/.steam/registry.vdf', REGISTRY % '') + self.assertNotIn('sign-in shared', self.sync().stdout) + self.assertIn('"76561198000000001"', (self.x64 / 'config/config.vdf').read_text()) + self.assertIn('"AutoLoginUser"', (self.home / '.droiddeck-x64/.steam/registry.vdf').read_text()) + self.sync() + for root in (self.arm, self.x64): + self.assertEqual(1, (root / 'config/config.vdf').read_text().count('"SteamID"')) + self.assertEqual(1, (self.home / '.droiddeck-x64/.steam/registry.vdf').read_text().count('AutoLoginUser')) + + def test_a_newer_x64_sign_in_moves_to_arm64(self): + self.write(self.x64 / 'local.vdf', local('11ee')) + os.utime(self.arm / 'local.vdf', (1, 1)) + self.assertIn('from the x86-64 client', self.sync().stdout) + self.assertEqual(local('11ee'), (self.arm / 'local.vdf').read_text()) + + +if __name__ == '__main__': + unittest.main()