From 239ba482916527c3b8115f41886a0200a96dda20 Mon Sep 17 00:00:00 2001 From: The412Banner <205237651+The412Banner@users.noreply.github.com> Date: Tue, 6 Oct 2026 10:46:27 -0400 Subject: [PATCH] Windows components for Steam and added games; no more redist installer prompts (#287) * Put a game's Windows components into its Proton prefix at launch droiddeck-wincomponents copies the DLLs of the components picked for a game into compatdata//pfx before Proton starts and hands their overrides to WINEDLLOVERRIDES, so the registry is never touched. It re-checks every launch (an older Proton deletes the files it once placed), takes a component's files out again when it is turned off, and forgets Proton's config_info then so Proton puts its own builtins back. * Windows components for added games: picker, packages and their catalog Settings > Games lists a Windows components row under each added game. Its dialog switches components on per game; the first switch-on for any game downloads the package (a catalog of its own beside desktop.json, installed with the desktop package installer to /opt/droiddeck/wincomponents/). The picks are kept in the app and published to the runtime at every session start for droiddeck-wincomponents to apply. tools/wincomponents/package.py builds the packages: Bannerlator's eight wincomponents archives re-rooted, and OpenAL from OpenAL Soft's release (soft_oal.dll placed as OpenAL32.dll, 32- and 64-bit). * Windows components: list the installed ones when the catalog cannot be reached Offline the dialog only looked up ids it already knew - the catalog's and the game's picks - so a component downloaded earlier never showed. * Windows components: Bannerlator's list with Recommended, on the game's launch settings The picker moves from Settings to a Windows components card in an added game's launch settings. It offers Bannerlator's whole catalog (components.json on winlator-contents): what the game's folder points at first (Bannerlator's DependencyDetector, mapped to the DLL-copy twin where there is one, oalinst -> oalinst_dll), then every component that installs here, then the ones that need a Windows installer, greyed until DroidDeck can run one. A component installs by running its catalog steps into the runtime once: archives downloaded (one download per archive for a batch), DLLs found by name as Bannerlator does and copied to /opt/droiddeck/wincomponents/, overrides and bundled components written to component.json. The launch script now takes catalog names (xaudio2.7), expands bundles (directmusic), places files one level down (drivers/gm.dls) and keeps an override's own mode (dsound=n). Drops the hand-made packages and their script. * Windows components: Steam games too, and a pad that can drive the list The card shows on a Steam title's launch settings as well, keyed by its appid - the compatdata folder droiddeck-game-env reads - and Recommended scans the game's install folder. With a pad: Done takes focus while the catalog loads and the first component once it is there (the list was not drawn yet when the dialog asked for focus, so nothing had it); switches stay enabled during a download and a press then is ignored (a disabled switch dropped the pad's focus); LB/RB jump between Recommended, All, the installers and Done. * Redist markers from games' own install scripts; VC++ runtime as installed The client still started a game's own _CommonRedist installer - its script sits in the game's folder, often with another version in the value name, and only Steamworks Shared's were read. On ARM64 Proton the x64 VC++ 2015-2022 bundle then stops with "Microsoft Visual C++ 2015-2022 Redistributable (arm64) is required" (reported on an AYN Odin 2 Portal Max). - Read installscript*.vdf from every installed game in each library as well; the list is cached until an appmanifest changes, since this runs every quarter minute. - Record the VC++ 2015-2022 runtime as installed where its installers look (VisualStudio\14.0\VC\Runtimes\{x86,x64,arm64}, both registry views), so any bundle - the client's or a game's own - finds it and exits; Proton's msvcp140/vcruntime140 serve the game. - Join the Wow6432Node path instead of sub(): mawk and gawk read the replacement's backslashes differently (gawk, the runtime's, was right). * Mark a game's install-script processes as run just before the client evaluates them An Odin 3 bundle (0.3.1, Proton Experimental ARM64) shows the client running its install-script evaluator on the first launch of three new titles (409710, 2161700, 3404260) while their Steamworks Shared redistributables (VC++ 2010/2012/2019/2022, DirectX) were still downloading - 2161700 at 22:03:22, the shared app finished at 22:03:29 - so the periodic seeder had nothing to mark yet and the installers ran. The client writes the exact steps to legacycompat/evaluatorscript_.vdf and starts iscriptevaluator.exe through our launcher. droiddeck-game-env now hands that script to droiddeck-seed-redists first, which marks every process in it in the template and the prefixes before the evaluator reads them. The seeder's reader no longer assumes a depth: a block with a HasRunKey is a process named by the key it opened under, wherever it sits. * Windows components: keep a file that was not Proton's, put it back when turned off A component's DLL replaced whatever had that name in the prefix, and turning it off deleted ours, so a file a real installer (or the user) had put there was gone for good. Before replacing a file that is neither Proton's own (a symlink into Proton, or a Wine builtin/placeholder DLL) nor an earlier copy of ours, keep it under compatdata//.droiddeck-wincomponents-backup and move it back when the component is turned off. The launch note says how many went back. * Windows components: recommend what Steam installs, say what the game already has - Steam titles: the appmanifest's SharedDepots (Steamworks Shared, app 228980) list the VC++ 2005-2022 and DirectX redists Steam installs with the game; they live outside the game folder, so the folder scan never saw them. They join Recommended as "Steam installs VC++ 2010 with this game" (SteamRedists). - Bannerlator's PrefixInstalledDetector, on compatdata//pfx: OpenAL32, Mono, PhysX and Games for Windows Live that are already there from elsewhere show "Already in this game". Files droiddeck-wincomponents placed itself do not count. - The game page passes the prefix and, for Steam titles, the appid. * Windows components: friendly names The list showed catalog keys (oalinst_dll, vcredist2010_dll, dotnet452, d3dcompiler_47). Bannerlator's catalog has no display name and some of its descriptions do not fit the entry, so WinComponentNames gives one: families by pattern (Visual C++ , .NET Framework x.y[.z], XAudio, D3DCompiler, MSXML, Wine Mono, DirectMusic parts), the rest from a table, the key itself for anything new. The key stays in the row's detail for logs and bug reports; both lists sort by the name, and the game page's card names the picks the same way. --------- Co-authored-by: The412Banner --- app/build.gradle | 2 + .../launcher/frontend/DependencyDetector.kt | 253 ++++++++++++++ .../frontend/PrefixInstalledDetector.kt | 66 ++++ .../launcher/frontend/SteamRedists.kt | 46 +++ .../launcher/session/SessionFiles.kt | 2 + .../launcher/session/WinComponentNames.kt | 54 +++ .../launcher/session/WinComponents.kt | 326 ++++++++++++++++++ .../droiddeck/launcher/ui/FrontEndGames.kt | 4 +- .../droiddeck/launcher/ui/FrontEndSetup.kt | 24 +- .../launcher/ui/GameEnvironmentEditor.kt | 2 +- .../launcher/ui/ModeSettingsDialog.kt | 14 +- .../launcher/ui/WinComponentsEditor.kt | 261 ++++++++++++++ app/src/main/res/values/strings.xml | 22 ++ .../frontend/WinComponentDetectorsTest.kt | 56 +++ .../launcher/session/WinComponentNamesTest.kt | 17 + .../overlay/usr/local/bin/droiddeck-game-env | 43 +++ .../usr/local/bin/droiddeck-seed-redists | 82 ++++- .../usr/local/bin/droiddeck-wincomponents | 241 +++++++++++++ tools/tests/test_game_environment.py | 19 + tools/tests/test_seed_redists.py | 96 ++++++ tools/tests/test_wincomponents.py | 170 +++++++++ 21 files changed, 1777 insertions(+), 23 deletions(-) create mode 100644 app/src/main/java/com/droiddeck/launcher/frontend/DependencyDetector.kt create mode 100644 app/src/main/java/com/droiddeck/launcher/frontend/PrefixInstalledDetector.kt create mode 100644 app/src/main/java/com/droiddeck/launcher/frontend/SteamRedists.kt create mode 100644 app/src/main/java/com/droiddeck/launcher/session/WinComponentNames.kt create mode 100644 app/src/main/java/com/droiddeck/launcher/session/WinComponents.kt create mode 100644 app/src/main/java/com/droiddeck/launcher/ui/WinComponentsEditor.kt create mode 100644 app/src/test/java/com/droiddeck/launcher/frontend/WinComponentDetectorsTest.kt create mode 100644 app/src/test/java/com/droiddeck/launcher/session/WinComponentNamesTest.kt create mode 100755 tools/linuxfs/overlay/usr/local/bin/droiddeck-wincomponents create mode 100644 tools/tests/test_seed_redists.py create mode 100644 tools/tests/test_wincomponents.py diff --git a/app/build.gradle b/app/build.gradle index d92138f395..db23dd8ba8 100644 --- a/app/build.gradle +++ b/app/build.gradle @@ -190,6 +190,8 @@ dependencies { // The runtime tarball is a zstd-compressed tar. implementation 'com.github.luben:zstd-jni:1.5.2-3@aar' implementation 'org.apache.commons:commons-compress:1.27.1' + // Bannerlator's Windows components come as .tar.xz (WinComponents). + implementation 'org.tukaani:xz:1.10' // The UI: every menu, dialog and overlay is Compose. The session's SurfaceView and the // on-screen pad are Views underneath it, which is what they are in every Compose app. implementation platform('androidx.compose:compose-bom:2024.02.00') diff --git a/app/src/main/java/com/droiddeck/launcher/frontend/DependencyDetector.kt b/app/src/main/java/com/droiddeck/launcher/frontend/DependencyDetector.kt new file mode 100644 index 0000000000..d14538d17f --- /dev/null +++ b/app/src/main/java/com/droiddeck/launcher/frontend/DependencyDetector.kt @@ -0,0 +1,253 @@ +package com.droiddeck.launcher.frontend + +import java.io.File +import java.util.Locale + +/** + * Best-effort detection of the Windows redistributables a game bundles, by scanning its install + * folder for the redist installers Steam/Epic/GOG ship (`_CommonRedist/`, `_Redist/`, + * `Redistributables/`, loose `vcredist_x64.exe`, …) and mapping them to entries in our component + * catalog (WinComponents, Bannerlator's components.json `name` keys). The result drives the + * Recommended part of a game's Windows components; the user still switches each one on. + * Bannerlator's detector, unchanged but for this note and its package. + * + * Wine-aware: keeps the redists that actually matter under Wine/DXVK (VC++ runtimes, .NET, PhysX, + * OpenAL, XACT, XNA, GFWL) and SKIPS the ones DXVK/VKD3D already provide (DX12 Agility SDK) or that + * aren't our components (anti-cheat, Rockstar/Social-Club/launcher setups). Never throws; returns an + * empty list on anything unexpected. Pure filesystem — no network, no Android deps (unit-testable). + */ +object DependencyDetector { + + /** How a recommendation was found — drives the UI's "recommended vs optional" split. */ + enum class Kind { + /** The game bundles the redist installer (in _CommonRedist etc.) — meant to be installed. */ + BUNDLED, + /** A runtime DLL the game ships loose — usually already works from the game folder; install + * only if the game misbehaves. */ + SHIPPED, + /** Steam installs it with the game from Steamworks Shared ([SteamRedists]). */ + STEAM, + } + + /** A recommended component: the catalog [componentName] to install, a display [label], the on-disk + * redist that triggered it ([reason]), and [kind] (bundled installer vs shipped DLL). */ + data class Recommendation(val componentName: String, val label: String, val reason: String, val kind: Kind = Kind.BUNDLED) + + private const val MAX_DEPTH = 5 + private const val MAX_ENTRIES = 8000 // cost guard against giant game trees + + /** A loose VC++ installer with no year in its path — resolved to "latest" only if no specific + * vcredistYYYY was found anywhere (else it's just a duplicate of the versioned one). */ + private const val VCREDIST_LOOSE = "__vcredist_loose" + + /** Directory names that are redist containers worth descending into. */ + private val REDIST_DIR_RE = Regex( + """(?i)^(_?commonredist|_?redist(ributables)?|__?installer|vcredist|vc_redist|directx|direct_x|physx|openal|dotnet(fx)?|\.net|xna|xact|redist|support)$""" + ) + + /** Skip whole subtrees that are never our components. */ + private val SKIP_DIR_RE = Regex("""(?i)^(d3d12.*|.*agility.*|dxvk|vkd3d|battleye|easyanticheat|eac|anticheat)$""") + + private val YEAR_TO_VCREDIST = mapOf( + "2005" to "vcredist2005", "2008" to "vcredist2008", "2010" to "vcredist2010", + "2012" to "vcredist2012", "2013" to "vcredist2013", "2015" to "vcredist2015", + "2017" to "vcredist2015", "2019" to "vcredist2019", "2022" to "vcredist2022", + ) + + private val LABELS = mapOf( + "vcredist2005" to "Visual C++ 2005", "vcredist2008" to "Visual C++ 2008", + "vcredist2010" to "Visual C++ 2010", "vcredist2012" to "Visual C++ 2012", + "vcredist2013" to "Visual C++ 2013", "vcredist2015" to "Visual C++ 2015-2017", + "vcredist2019" to "Visual C++ 2019", "vcredist2022" to "Visual C++ 2015-2022", + "physx" to "NVIDIA PhysX", "oalinst" to "OpenAL", "xact" to "DirectX XACT audio", + "xact_x64" to "DirectX XACT audio (x64)", "xna40" to "XNA Framework 4.0", + "xna31" to "XNA Framework 3.1", "d3dx9" to "DirectX 9 (d3dx9)", "d3dx11" to "DirectX 11 (d3dx11)", + "XLiveRedist" to "Games for Windows Live", "dotnet40" to ".NET Framework 4.0", + "dotnet45" to ".NET Framework 4.5", "dotnet452" to ".NET Framework 4.5.2", + "dotnet46" to ".NET Framework 4.6", "dotnet472" to ".NET Framework 4.7.2", + "dotnet48" to ".NET Framework 4.8", "dotnet35" to ".NET Framework 3.5", + "dotnetcoredesktop6" to ".NET Desktop 6", "dotnetcoredesktop7" to ".NET Desktop 7", + "dotnetcoredesktop8" to ".NET Desktop 8", + ) + + /** Preferred install order (runtimes others may depend on first). */ + private val INSTALL_ORDER = listOf( + "vcredist2005", "vcredist2008", "vcredist2010", "vcredist2012", "vcredist2013", + "vcredist2015", "vcredist2019", "vcredist2022", + "dotnet35", "dotnet40", "dotnet45", "dotnet452", "dotnet46", "dotnet472", "dotnet48", + "dotnetcoredesktop6", "dotnetcoredesktop7", "dotnetcoredesktop8", + "d3dx9", "d3dx11", "xact", "xact_x64", "xna31", "xna40", "physx", "oalinst", "XLiveRedist", + ) + + /** + * Detect from the .exe the user picked, resolving the game root first: walk up from the exe's + * folder (bounded) to the nearest ancestor that holds a redist container (`_CommonRedist`, …) at + * its top level, and scan from there — so a nested exe (`…/Game/bin/x.exe`) still finds a root-level + * `_CommonRedist`, without over-reaching into a shared `steamapps/common` (which has no redist + * container of its own). Falls back to scanning the exe's own folder. + */ + fun detectForExe(exeFile: File): List { + val start = exeFile.absoluteFile.parentFile ?: return emptyList() + var d: File? = start + var depth = 0 + while (d != null && depth < 4) { + val hasRedistContainer = try { + d.listFiles()?.any { it.isDirectory && REDIST_DIR_RE.matches(it.name.lowercase(Locale.US)) } == true + } catch (_: Throwable) { false } + if (hasRedistContainer) return detect(d) + d = d.parentFile + depth++ + } + return detect(start) + } + + /** + * Scan [gameDir] and return the recommended components, de-duplicated (first reason wins), in a + * stable install-friendly order. Empty when nothing is detected or [gameDir] isn't a directory. + */ + fun detect(gameDir: File): List { + if (!gameDir.isDirectory) return emptyList() + val hits = LinkedHashMap() // componentName -> first recommendation + var visited = 0 + + fun consider(f: File, isDir: Boolean, insideRedist: Boolean) { + val name = f.name.lowercase(Locale.US) + val path = f.path.replace('\\', '/').lowercase(Locale.US) + // Bundled = in a redist container or an installer (exe/msi/cab); shipped = a loose runtime DLL. + val kind = if (insideRedist || isDir || name.endsWith(".exe") || name.endsWith(".msi") || name.endsWith(".cab")) + Kind.BUNDLED else Kind.SHIPPED + for (comp in classify(path, name, isDir)) { + val existing = hits[comp] + if (existing == null) { + hits[comp] = Recommendation(comp, LABELS[comp] ?: comp, relativeReason(gameDir, f), kind) + } else if (existing.kind == Kind.SHIPPED && kind == Kind.BUNDLED) { + hits[comp] = existing.copy(kind = Kind.BUNDLED) // a bundled installer outranks a shipped DLL + } + } + } + + fun walk(dir: File, depth: Int, insideRedist: Boolean) { + if (depth > MAX_DEPTH || visited > MAX_ENTRIES) return + val entries = dir.listFiles() ?: return + for (f in entries) { + if (visited++ > MAX_ENTRIES) return + val lower = f.name.lowercase(Locale.US) + if (f.isDirectory) { + if (SKIP_DIR_RE.matches(lower)) continue + val isRedistDir = insideRedist || REDIST_DIR_RE.matches(lower) + if (isRedistDir) consider(f, isDir = true, insideRedist = true) + // Aggressive coverage: descend redist containers fully, and otherwise sweep the game + // tree a few levels deep so runtime libraries the game ships (in bin/, etc.) are seen. + if (isRedistDir || depth < 3) walk(f, depth + 1, isRedistDir) + } else { + // Classify installers + runtime libraries (.dll/.exe/.msi) anywhere in the scanned + // tree; other files are ignored. Recommendations are advisory, so cast a wide net. + val ext = lower.substringAfterLast('.', "") + if (insideRedist || ext == "dll" || ext == "exe" || ext == "msi") consider(f, isDir = false, insideRedist = insideRedist) + } + } + } + + return try { + walk(gameDir, 0, false) + resolveLooseVcredist(hits) + hits.values.sortedBy { INSTALL_ORDER.indexOf(it.componentName).let { i -> if (i < 0) 999 else i } } + } catch (_: Throwable) { + emptyList() + } + } + + /** A loose VC++ installer resolves to the latest 14.x only if no specific vcredistYYYY was found. */ + private fun resolveLooseVcredist(hits: LinkedHashMap) { + val loose = hits.remove(VCREDIST_LOOSE) ?: return + val hasSpecific = hits.keys.any { it in YEAR_TO_VCREDIST.values } + if (!hasSpecific) hits["vcredist2022"] = loose.copy(componentName = "vcredist2022", label = LABELS["vcredist2022"]!!) + } + + private fun relativeReason(gameDir: File, f: File): String { + val g = gameDir.path.replace('\\', '/') + val p = f.path.replace('\\', '/') + return if (p.startsWith("$g/")) p.substring(g.length + 1) else f.name + } + + /** + * Map a redist file/dir (its full lowercased [path], leaf [name], and whether it [isDir]) to + * catalog component name(s). Empty for things we deliberately skip. + */ + private fun classify(path: String, name: String, isDir: Boolean): List { + // Hard skips first — DX12 is provided by DXVK/VKD3D; these aren't our components. + if (path.contains("d3d12") || path.contains("agility") || + path.contains("battleye") || path.contains("anticheat") || path.contains("easyanticheat") || + name.contains("socialclub") || name.contains("social-club") || + name.contains("rockstar") || name == "setup.exe" + ) return emptyList() + + // Visual C++ runtimes — most important. Year comes from the path context. + if (path.contains("vcredist") || path.contains("vc_redist") || name.startsWith("vcredist") || name.startsWith("vc_redist")) { + val year = YEAR_TO_VCREDIST.keys.firstOrNull { path.contains(it) } + return when { + year != null -> listOf(YEAR_TO_VCREDIST.getValue(year)) + isDir -> emptyList() // bare "vcredist" container — wait for the year subfolder / installer + else -> listOf(VCREDIST_LOOSE) // loose installer, no year → resolved post-walk + } + } + // VC++ runtime DLL the game ships (msvcp140.dll → 2015-2022, msvcp120.dll → 2013, …). + VCRUNTIME_DLL.entries.firstOrNull { name.startsWith(it.key) }?.let { return listOf(it.value) } + if (path.contains("openal") || name.startsWith("oalinst") || name == "openal32.dll" || name == "wrap_oal.dll") return listOf("oalinst") + if (path.contains("physx") || name.startsWith("physx")) return listOf("physx") + if (path.contains("xact") || name.startsWith("xactengine") || name.startsWith("x3daudio")) return listOf("xact", "xact_x64") + // Games for Windows Live — the redist folder, OR a GFWL library the game ships (xlive.dll, + // Codemasters' dbxLive32/64, gfwl*): GFWL titles need XLiveRedist to run even when they don't + // bundle the redist itself. + if (path.contains("xliveredist") || name.contains("xlive") || name.startsWith("gfwl") || name.startsWith("dbxlive")) { + return listOf("XLiveRedist") + } + if (path.contains("xna")) return listOf(if (path.contains("3.1") || path.contains("3_1") || path.contains("v31")) "xna31" else "xna40") + // .NET — Framework and Core/Desktop runtimes. + if (path.contains("dotnet") || path.contains(".net") || name.startsWith("ndp") || name.startsWith("dotnetfx") || name.contains("windowsdesktop-runtime")) { + val comp = dotnetComponent("$path $name") + return when { + comp != null -> listOf(comp) + isDir -> emptyList() // bare ".NET" container — wait for the versioned installer + else -> listOf("dotnet48") // unlabeled .NET Framework installer file → sensible default + } + } + // DirectX 11 helper DLLs the game ships (d3dx11_43.dll). + if (D3DX11_RE.matches(name)) return listOf("d3dx11") + // DirectX 9 helper DLLs / d3dcompiler / June-2010 DX redist (DXVK provides core d3d9 but NOT + // these helper DLLs). + if (name.startsWith("dxsetup") || name.startsWith("dxwebsetup") || + D3DX9_RE.matches(name) || D3DCOMPILER_RE.matches(name) || + (path.contains("directx") && name.endsWith(".exe")) + ) return listOf("d3dx9") + return emptyList() + } + + /** Runtime DLL prefix → the VC++ redistributable that provides it (checked with startsWith). */ + private val VCRUNTIME_DLL = linkedMapOf( + "vcruntime140" to "vcredist2022", "msvcp140" to "vcredist2022", "msvcr140" to "vcredist2022", "concrt140" to "vcredist2022", + "msvcp120" to "vcredist2013", "msvcr120" to "vcredist2013", + "msvcp110" to "vcredist2012", "msvcr110" to "vcredist2012", + "msvcp100" to "vcredist2010", "msvcr100" to "vcredist2010", + "msvcp90" to "vcredist2008", "msvcr90" to "vcredist2008", + "msvcp80" to "vcredist2005", "msvcr80" to "vcredist2005", + ) + private val D3DX9_RE = Regex("""d3dx9_\d+\.dll""") + private val D3DX11_RE = Regex("""d3dx11_\d+\.dll""") + private val D3DCOMPILER_RE = Regex("""d3dcompiler_\d+\.dll""") + + /** Resolve a .NET component from installer/path text, or null when no version is discernible. */ + private fun dotnetComponent(s: String): String? = when { + s.contains("desktop-runtime-8") || Regex("""\b8\.\d""").containsMatchIn(s) -> "dotnetcoredesktop8" + s.contains("desktop-runtime-7") || Regex("""\b7\.\d""").containsMatchIn(s) -> "dotnetcoredesktop7" + s.contains("desktop-runtime-6") || Regex("""\b6\.\d""").containsMatchIn(s) -> "dotnetcoredesktop6" + s.contains("ndp48") || s.contains("fx48") || s.contains("4.8") -> "dotnet48" + s.contains("ndp472") || s.contains("4.7.2") -> "dotnet472" + s.contains("ndp46") || s.contains("4.6") -> "dotnet46" + s.contains("ndp452") || s.contains("4.5.2") -> "dotnet452" + s.contains("dotnetfx45") || s.contains("ndp45") || s.contains("4.5") -> "dotnet45" + s.contains("dotnetfx40") || s.contains("ndp40") || s.contains("4.0") -> "dotnet40" + s.contains("dotnetfx35") || s.contains("3.5") -> "dotnet35" + else -> null // no version signal (e.g. a bare "DotNet" container dir) + } +} diff --git a/app/src/main/java/com/droiddeck/launcher/frontend/PrefixInstalledDetector.kt b/app/src/main/java/com/droiddeck/launcher/frontend/PrefixInstalledDetector.kt new file mode 100644 index 0000000000..7f5ca5b0b3 --- /dev/null +++ b/app/src/main/java/com/droiddeck/launcher/frontend/PrefixInstalledDetector.kt @@ -0,0 +1,66 @@ +package com.droiddeck.launcher.frontend + +import org.json.JSONObject +import java.io.File + +/** + * Which catalog components a game's prefix already has from elsewhere - a real installer, the game, + * the user - so the list can say so instead of recommending them again. Bannerlator's detector, + * on DroidDeck's compatdata/: only markers absent from a fresh Proton prefix count (OpenAL32, + * Mono, PhysX, Games for Windows Live); msvcp140, d3dx9 and the like ship with Proton and cannot + * tell. Files DroidDeck put there itself (droiddeck-wincomponents' state) are not counted. + * Pure filesystem reads; never throws. + */ +object PrefixInstalledDetector { + private const val STATE = ".droiddeck-wincomponents.json" + + /** Detector names (oalinst, mono, physx, XLiveRedist) present in [compat]/pfx. */ + fun detect(compat: File?): Set = try { + val pfx = compat?.let { File(it, "pfx") } + if (pfx == null || !pfx.isDirectory) emptySet() else { + val ours = ours(compat) + val out = HashSet() + val openal = listOf("system32", "syswow64").map { "drive_c/windows/$it/openal32.dll" } + if (openal.any { it !in ours && fileIgnoringCase(pfx, it) }) out += "oalinst" + if (File(pfx, "drive_c/windows/mono").isDirectory) out += "mono" + if (hasPhysX(pfx)) out += "physx" + if (registryContains(pfx, "Games for Windows")) out += "XLiveRedist" + out + } + } catch (_: Throwable) { + emptySet() + } + + private fun ours(compat: File): Set = try { + val files = JSONObject(File(compat, STATE).readText()).optJSONObject("files") + files?.keys()?.asSequence()?.map { it.lowercase() }?.toSet().orEmpty() + } catch (_: Exception) { + emptySet() + } + + /** Windows names ignore case; the prefix's folder may spell them either way. */ + private fun fileIgnoringCase(pfx: File, rel: String): Boolean { + val target = File(pfx, rel) + return target.parentFile?.listFiles()?.any { it.isFile && it.name.equals(target.name, ignoreCase = true) } == true + } + + private fun hasPhysX(pfx: File): Boolean { + for (pf in arrayOf("drive_c/Program Files", "drive_c/Program Files (x86)")) { + val base = File(pfx, pf) + if (File(base, "NVIDIA Corporation/PhysX").isDirectory) return true + if (base.listFiles()?.any { it.isDirectory && it.name.contains("PhysX", ignoreCase = true) } == true) return true + } + return false + } + + private fun registryContains(pfx: File, needle: String): Boolean = + arrayOf("system.reg", "user.reg").any { name -> + val f = File(pfx, name) + try { + f.isFile && f.length() < 32L * 1024 * 1024 && + f.bufferedReader().use { r -> r.lineSequence().any { it.contains(needle, ignoreCase = true) } } + } catch (_: Throwable) { + false + } + } +} diff --git a/app/src/main/java/com/droiddeck/launcher/frontend/SteamRedists.kt b/app/src/main/java/com/droiddeck/launcher/frontend/SteamRedists.kt new file mode 100644 index 0000000000..f656df784c --- /dev/null +++ b/app/src/main/java/com/droiddeck/launcher/frontend/SteamRedists.kt @@ -0,0 +1,46 @@ +package com.droiddeck.launcher.frontend + +import java.io.File + +/** + * What Steam itself installs alongside a game: the Steamworks Shared redistributables (app 228980) + * listed under "SharedDepots" in the game's appmanifest. Those live in Steamworks Shared rather than + * the game's folder, so the folder scan ([DependencyDetector]) cannot see them; this reads Steam's + * list instead. Depots are matched to Bannerlator's catalog names; unknown ones are left out. + */ +object SteamRedists { + /** Steamworks Shared depot -> (catalog component, what Steam calls it). 2015-2019 runtimes are one family. */ + val DEPOTS = linkedMapOf( + "228981" to ("vcredist2005_dll" to "VC++ 2005"), + "228982" to ("vcredist2008_dll" to "VC++ 2008"), + "228983" to ("vcredist2010_dll" to "VC++ 2010"), + "228984" to ("vcredist2012_dll" to "VC++ 2012"), + "228985" to ("vcredist2013_dll" to "VC++ 2013"), + "228986" to ("vcredist2015_dll" to "VC++ 2015"), + "228987" to ("vcredist2019_dll" to "VC++ 2017"), + "228988" to ("vcredist2019_dll" to "VC++ 2019"), + "228989" to ("vcredist2022_dll" to "VC++ 2022"), + "228990" to ("d3dx9" to "DirectX (June 2010)"), + ) + private val BLOCK = Regex("\"SharedDepots\"\\s*\\{([^}]*)\\}", RegexOption.IGNORE_CASE) + private val ENTRY = Regex("\"(\\d+)\"\\s+\"\\d+\"") + + /** The game's appmanifest beside its install folder (steamapps/common/ -> steamapps/). */ + fun manifest(gameDir: File?, appId: Int): File? { + val steamapps = gameDir?.parentFile?.parentFile ?: return null + return File(steamapps, "appmanifest_$appId.acf").takeIf { it.isFile } + } + + fun detect(gameDir: File?, appId: Int): List { + val text = try { + manifest(gameDir, appId)?.takeIf { it.length() < 1L shl 20 }?.readText() ?: return emptyList() + } catch (_: Exception) { + return emptyList() + } + val block = BLOCK.find(text)?.groupValues?.get(1) ?: return emptyList() + return ENTRY.findAll(block).mapNotNull { DEPOTS[it.groupValues[1]] } + .distinctBy { it.first } + .map { (component, label) -> DependencyDetector.Recommendation(component, label, label, DependencyDetector.Kind.STEAM) } + .toList() + } +} diff --git a/app/src/main/java/com/droiddeck/launcher/session/SessionFiles.kt b/app/src/main/java/com/droiddeck/launcher/session/SessionFiles.kt index d666d1a6c9..3414627450 100644 --- a/app/src/main/java/com/droiddeck/launcher/session/SessionFiles.kt +++ b/app/src/main/java/com/droiddeck/launcher/session/SessionFiles.kt @@ -28,8 +28,10 @@ object SessionFiles { */ fun stage(context: Context, root: File) { GameEnvironmentStore.publish(context) + WinComponents.publish(context) val files = arrayOf( "usr/local/bin/droiddeck-game-env" to "usr/local/bin/droiddeck-game-env", + "usr/local/bin/droiddeck-wincomponents" to "usr/local/bin/droiddeck-wincomponents", "usr/local/bin/droiddeck-esync" to "usr/local/bin/droiddeck-esync", "usr/local/bin/droiddeck-steam-compat" to "usr/local/bin/droiddeck-steam-compat", "usr/local/bin/droiddeck-fex" to "usr/local/bin/droiddeck-fex", diff --git a/app/src/main/java/com/droiddeck/launcher/session/WinComponentNames.kt b/app/src/main/java/com/droiddeck/launcher/session/WinComponentNames.kt new file mode 100644 index 0000000000..5a8d38f5ea --- /dev/null +++ b/app/src/main/java/com/droiddeck/launcher/session/WinComponentNames.kt @@ -0,0 +1,54 @@ +package com.droiddeck.launcher.session + +/** + * What to call a catalog component on screen. Bannerlator's catalog keys (vcredist2010_dll, + * oalinst_dll, d3dcompiler_47...) have no display name, and some descriptions do not fit the entry, + * so names come from here: families by pattern, the rest from a table, the key itself otherwise. + * The "_dll" twin of an installer (the direct DLL copy) shares the installer's name. + */ +object WinComponentNames { + private val TABLE = mapOf( + "oalinst" to "OpenAL", "XLiveRedist" to "Games for Windows Live", "gfw" to "Games for Windows Live", + "physx" to "NVIDIA PhysX", "xact" to "DirectX XACT audio", "xact_x64" to "DirectX XACT audio (64-bit)", + "xinput" to "XInput", "xna31" to "XNA Framework 3.1", "xna40" to "XNA Framework 4.0", + "d3dx9" to "DirectX 9 (d3dx9)", "d3dx11" to "DirectX 11 (d3dx11)", + "directmusic" to "DirectMusic", "directplay" to "DirectPlay", "directshow" to "DirectShow", + "dsound" to "DirectSound", "dx8vb" to "DirectX 8 for Visual Basic", "mediafoundation" to "Media Foundation", + "quartz" to "DirectShow (quartz)", "amstream" to "DirectShow streams (amstream)", "devenum" to "DirectShow devices (devenum)", + "cnc-ddraw" to "cnc-ddraw (DirectDraw)", "gdiplus" to "GDI+", "gecko" to "Wine Gecko", "mono" to "Wine Mono", + "vcredist6" to "Visual C++ 6", "vcredist6sp6" to "Visual C++ 6 SP6", + "vcredist2015" to "Visual C++ 2015", "vcredist2019" to "Visual C++ 2015-2019", "vcredist2022" to "Visual C++ 2015-2022", + "vbrun6" to "Visual Basic 6 runtime", "mfc40" to "MFC 4.0", "mfc42" to "MFC 4.2", + "dotnetcore3" to ".NET Core 3", "dotnetcoredesktop3" to ".NET Desktop 3", "dotnetcoredesktop6" to ".NET Desktop 6", + "dotnetcoredesktop7" to ".NET Desktop 7", "dotnetcoredesktop8" to ".NET Desktop 8", "dotnet50" to ".NET 5", + "webview2" to "Microsoft WebView2", "powershell" to "PowerShell", "powershell_core" to "PowerShell Core", + "K-Lite" to "K-Lite Codec Pack", "VulkanRT" to "Vulkan Runtime", "ffdshow" to "ffdshow codecs", + "lavfilters702" to "LAV Filters 0.70.2", "lavfilters741" to "LAV Filters 0.74.1", "quicktime72" to "QuickTime 7.2", + "cjkfonts" to "CJK fonts", "jet40" to "Jet 4.0 database", "mdac28" to "MDAC 2.8", "wsh57" to "Windows Script Host 5.7", + "riched20" to "RichEdit 2.0", "msftedit" to "RichEdit 4.1 (msftedit)", "sqlite3" to "SQLite 3", + "win7" to "Windows 7 mode", "winXP" to "Windows XP mode", "gmdls" to "General MIDI sounds (gm.dls)", + ) + private val VCREDIST = Regex("vcredist(\\d{4})") + private val DOTNET = Regex("dotnet(\\d)(\\d)(\\d?)(sp\\d)?") + private val D3DCOMPILER = Regex("d3dcompiler_(\\d+)") + private val MSXML = Regex("msxml(\\d)") + private val MONO = Regex("mono-([\\d.]+)") + private val XAUDIO = Regex("xaudio([\\d.]+)") + + fun of(id: String): String { + val base = id.removeSuffix("_dll") + TABLE[base]?.let { return it } + VCREDIST.matchEntire(base)?.let { return "Visual C++ " + it.groupValues[1] } + DOTNET.matchEntire(base)?.let { m -> + val (major, minor, patch, sp) = m.destructured + return ".NET Framework $major.$minor" + (if (patch.isNotEmpty()) ".$patch" else "") + + (if (sp.isNotEmpty()) " " + sp.uppercase() else "") + } + D3DCOMPILER.matchEntire(base)?.let { return "D3DCompiler " + it.groupValues[1] } + MSXML.matchEntire(base)?.let { return "MSXML " + it.groupValues[1] } + MONO.matchEntire(base)?.let { return "Wine Mono " + it.groupValues[1] } + XAUDIO.matchEntire(base)?.let { return "XAudio " + it.groupValues[1] } + if (base.startsWith("dm") || base == "dswave" || base == "dsdmo") return "DirectMusic ($base)" + return id + } +} diff --git a/app/src/main/java/com/droiddeck/launcher/session/WinComponents.kt b/app/src/main/java/com/droiddeck/launcher/session/WinComponents.kt new file mode 100644 index 0000000000..21618d28e2 --- /dev/null +++ b/app/src/main/java/com/droiddeck/launcher/session/WinComponents.kt @@ -0,0 +1,326 @@ +package com.droiddeck.launcher.session + +import android.content.Context +import android.util.AtomicFile +import android.util.Log +import com.droiddeck.launcher.core.ArchivePaths +import com.droiddeck.launcher.core.Downloader +import com.droiddeck.launcher.core.FileUtils +import com.droiddeck.launcher.runtime.LinuxRuntime +import org.apache.commons.compress.archivers.tar.TarArchiveInputStream +import org.apache.commons.compress.compressors.xz.XZCompressorInputStream +import org.json.JSONArray +import org.json.JSONObject +import java.io.BufferedInputStream +import java.io.File +import java.io.FileInputStream +import java.io.FileOutputStream +import java.security.MessageDigest +import java.util.zip.ZipInputStream + +/** + * Windows components (OpenAL, d3dx9, XAudio, VC++ DLLs...) for a game's Proton prefix, picked per + * game, from Bannerlator's catalog (components.json on winlator-contents, Bottles' manifest format). + * + * Installing one runs its steps once, into the runtime: the archives it names are downloaded and + * its DLLs copied to /opt/droiddeck/wincomponents//{system32,syswow64}, with component.json + * holding its DLL overrides and the components it bundles. The picks go to the runtime as + * ~/.config/droiddeck/wincomponents.json, and droiddeck-wincomponents copies the picked components + * into the game's prefix at every launch (see that script for why every launch). + * + * Only components made of file copies and overrides install this way. One that runs a Windows + * installer (install_exe/install_msi), unpacks a cabinet or edits the prefix's registry is listed + * with the reason it cannot be installed yet. + */ +object WinComponents { + private const val TAG = "WinComponents" + const val CATALOG_URL = "https://raw.githubusercontent.com/The412Banner/winlator-contents/main/components.json" + private const val SELECTION = "wincomponents.json" + private const val GUEST_SELECTION = "root/.config/droiddeck/wincomponents.json" + private const val STORE = "opt/droiddeck/wincomponents" + private val ID = Regex("[A-Za-z0-9][A-Za-z0-9._-]{0,63}") + // register_dll is left out on purpose: the DLLs it names are ones Wine ships and has registered + // at the same path, so the native file at that path is the one the registration loads. + private val FILE_STEPS = setOf("download_archive", "archive_extract", "copy_dll", "copy_file", "override_dll", "register_dll") + private val INSTALLER_STEPS = setOf("install_exe", "install_msi") + + fun validId(id: String): Boolean = ID.matches(id) && ".." !in id + + class Step(val action: String, val json: JSONObject) { + fun str(key: String): String = json.optString(key, "") + } + + class Component( + val name: String, val description: String, val provider: String, val status: String, + val dependencies: List, val steps: List, + ) + + enum class Support { READY, NEEDS_INSTALLER, UNSUPPORTED } + + fun fetch(): List? { + val body = Downloader.downloadString(CATALOG_URL) ?: return null + return try { + val arr = JSONObject(body).getJSONArray("components") + (0 until arr.length()).mapNotNull { i -> + val o = arr.getJSONObject(i) + val name = o.optString("name") + if (!validId(name)) return@mapNotNull null + val steps = o.optJSONArray("steps") ?: JSONArray() + val deps = o.optJSONArray("dependencies") ?: JSONArray() + Component( + name, o.optString("description"), o.optString("provider"), o.optString("status"), + (0 until deps.length()).map { deps.getString(it) }.filter { validId(it) }, + (0 until steps.length()).map { steps.getJSONObject(it).let { s -> Step(s.optString("action"), s) } }, + ) + } + } catch (e: Exception) { + Log.w(TAG, "catalog: $e"); null + } + } + + /** Whether [c] can be installed here, its bundled components included. */ + fun support(c: Component, all: Map, depth: Int = 0): Support { + if (c.status != "ready" || depth > 8) return Support.UNSUPPORTED + val own = when { + c.steps.any { it.action in INSTALLER_STEPS } -> Support.NEEDS_INSTALLER + c.steps.all { it.action in FILE_STEPS && fileStepOk(it) } -> Support.READY + else -> Support.UNSUPPORTED + } + if (own != Support.READY) return own + return c.dependencies.map { all[it]?.let { d -> support(d, all, depth + 1) } ?: Support.UNSUPPORTED } + .maxOrNull() ?: Support.READY + } + + private fun fileStepOk(step: Step): Boolean = when (step.action) { + "download_archive", "archive_extract" -> step.str("url").let { url -> + !url.startsWith("http") || url.startsWith("https://github.com/") && + url.substringBefore('?').let { it.endsWith(".tar.xz") || it.endsWith(".zip") } + } + "copy_dll", "copy_file" -> ".." !in step.str("dest") + else -> true + } + + /** The installed version of a component, or null. */ + fun installed(context: Context, id: String): String? = runCatching { + JSONObject(File(LinuxRuntime.rootDir(context), "$STORE/$id/component.json").readText()).optString("version", "?") + }.getOrNull() + + /** Every component in the runtime, by id. */ + fun installedIds(context: Context): List = + File(LinuxRuntime.rootDir(context), STORE).listFiles { f -> f.isDirectory && validId(f.name) } + .orEmpty().map { it.name }.filter { installed(context, it) != null }.sorted() + + /** + * Installs [c] and the components it bundles that are not installed yet. [onProgress] gets a + * line for the step and 0..100 (or -1). Returns null on success, else a message. + */ + fun install(context: Context, c: Component, all: Map, onProgress: (String, Int) -> Unit): String? { + if (support(c, all) != Support.READY) return "${c.name} cannot be installed here yet" + val order = ArrayList() + fun visit(x: Component, depth: Int) { + if (depth > 8 || order.any { it.name == x.name }) return + x.dependencies.mapNotNull { all[it] }.forEach { visit(it, depth + 1) } + if (x.name == c.name || installed(context, x.name) == null) order.add(x) + } + visit(c, 0) + // One download per archive however many components copy out of it (DirectMusic's eleven + // parts all come from dxnt, 100 MB). + val work = File(context.cacheDir, "wincomponents").apply { FileUtils.delete(this); mkdirs() } + val archives = HashMap() + return try { + for (x in order) { + onProgress(x.name, -1) + installOne(context, x, work, archives, onProgress)?.let { return "${x.name}: $it" } + } + null + } catch (e: Exception) { + Log.e(TAG, "install ${c.name}", e) + e.message ?: "Install failed" + } finally { + FileUtils.delete(work) + } + } + + private fun installOne(context: Context, c: Component, work: File, archives: HashMap, onProgress: (String, Int) -> Unit): String? { + val root = LinuxRuntime.rootDir(context) + if (!root.isDirectory) return "The Linux runtime is not installed" + val staging = File(root, "$STORE/.${c.name}.new").apply { FileUtils.delete(this); mkdirs() } + val overrides = ArrayList() + var source: File? = null + for (step in c.steps) when (step.action) { + "download_archive", "archive_extract" -> { + val url = step.str("url") + if (!url.startsWith("http")) continue + source = archives[url] ?: run { + val name = url.substringBefore('?').substringAfterLast('/') + val file = File(work, "${archives.size}-$name") + val ok = Downloader.downloadFile(url, file, false) { f -> onProgress("${c.name}: $name", if (f < 0) -1 else Math.round(f * 100f)) } + if (!ok) return "download failed: $name" + step.str("file_checksum").takeIf { it.length == 32 }?.let { md5 -> + if (!digest(file, "MD5").equals(md5, ignoreCase = true)) return "checksum mismatch: $name" + } + onProgress("${c.name}: unpacking $name", -1) + val dir = File(work, "${archives.size}-x").apply { mkdirs() } + if (!extract(file, dir)) return "could not unpack $name" + file.delete() + dir.also { archives[url] = it } + } + } + "copy_dll", "copy_file" -> { + val from = source ?: return "a copy step before any archive" + val dest = step.str("dest") + val (arch, base) = when (dest.substringBefore('/').lowercase()) { + "win64", "system32" -> "win64" to "system32" + "win32", "syswow64" -> "win32" to "syswow64" + else -> null to "system32" + } + val sub = dest.substringAfter('/', "") + copyMatching(from, step.str("file_name"), File(staging, if (sub.isEmpty()) base else "$base/$sub"), arch) + } + "override_dll" -> { + fun add(dll: String, type: String) { + if (dll.isEmpty()) return + overrides += when (type.ifEmpty { "native,builtin" }) { + "native,builtin" -> dll + "native" -> "$dll=n" + "builtin" -> "$dll=b" + "builtin,native" -> "$dll=b,n" + else -> dll + } + } + add(step.str("dll"), step.str("type")) + step.json.optJSONArray("bundle")?.let { b -> + for (i in 0 until b.length()) b.getJSONObject(i).let { add(it.optString("value"), it.optString("data")) } + } + } + "register_dll" -> Unit + else -> return "unsupported step ${step.action}" + } + val version = Regex("""\b(\d+\.\d+(\.\d+)*)\b""").find(c.description)?.value ?: "catalog" + FileUtils.writeString(File(staging, "component.json"), JSONObject() + .put("id", c.name).put("version", version) + .put("overrides", JSONArray(overrides.distinct())) + .put("requires", JSONArray(c.dependencies)).toString(1)) + val target = File(root, "$STORE/${c.name}") + FileUtils.delete(target) + if (!staging.renameTo(target)) return "could not place the files" + return null + } + + private fun digest(file: File, algorithm: String): String { + val md = MessageDigest.getInstance(algorithm) + FileInputStream(file).use { input -> + val buffer = ByteArray(1 shl 16) + while (true) { val n = input.read(buffer); if (n < 0) break; md.update(buffer, 0, n) } + } + return md.digest().joinToString("") { "%02x".format(it) } + } + + private fun extract(archive: File, dest: File): Boolean = try { + if (archive.name.endsWith(".zip", ignoreCase = true)) ZipInputStream(BufferedInputStream(FileInputStream(archive))).use { zip -> + var entry = zip.nextEntry + while (entry != null) { + val out = ArchivePaths.inside(dest, entry.name) + if (out != null && !entry.isDirectory) { + out.parentFile?.mkdirs() + FileOutputStream(out).use { FileUtils.copy(zip, it) } + } + entry = zip.nextEntry + } + } else TarArchiveInputStream(XZCompressorInputStream(BufferedInputStream(FileInputStream(archive), 1 shl 16))).use { tar -> + var entry = tar.nextEntry + while (entry != null) { + val out = ArchivePaths.inside(dest, entry.name) + if (out != null && entry.isFile) { + out.parentFile?.mkdirs() + FileOutputStream(out).use { FileUtils.copy(tar, it) } + } + entry = tar.nextEntry + } + } + true + } catch (e: Exception) { + Log.w(TAG, "extract ${archive.name}", e); false + } + + /** + * Bannerlator's rule: the manifest's own temp paths are Bottles', so the files are found by name + * anywhere in the unpacked tree, preferring the ones under the matching win32/win64 folder. + */ + private fun copyMatching(srcRoot: File, pattern: String, dest: File, arch: String?) { + if (pattern.isEmpty()) return + val rx = Regex("^" + pattern.split("*").joinToString(".*") { Regex.escape(it) } + "$", RegexOption.IGNORE_CASE) + var matches = srcRoot.walkTopDown().filter { it.isFile && rx.matches(it.name) }.toList() + if (arch != null) { + val inArch = matches.filter { it.path.contains("/$arch/", true) } + val other = if (arch == "win64") "win32" else "win64" + matches = inArch.ifEmpty { matches.filterNot { it.path.contains("/$other/", true) } } + } + dest.mkdirs() + matches.forEach { f -> f.copyTo(File(dest, f.name.lowercase()), overwrite = true) } + } + + /** Removes a component from the runtime and from every game's picks. */ + @Synchronized + fun uninstall(context: Context, id: String) { + require(validId(id)) + FileUtils.delete(File(LinuxRuntime.rootDir(context), "$STORE/$id")) + save(context, read(context).mapValues { (_, ids) -> ids - id }.filterValues { it.isNotEmpty() }) + } + + /** Each game's picks, by the appid its prefix is named after. */ + @Synchronized + fun read(context: Context): Map> = try { + val json = JSONObject(AtomicFile(File(context.filesDir, SELECTION)).readFully().toString(Charsets.UTF_8)) + val games = json.optJSONObject("games") ?: JSONObject() + games.keys().asSequence().associateWith { app -> + val ids = games.getJSONArray(app) + (0 until ids.length()).map { ids.getString(it) }.filter { validId(it) } + } + } catch (_: java.io.FileNotFoundException) { + emptyMap() + } + + fun picks(context: Context, appKey: String): List = read(context)[appKey].orEmpty() + + @Synchronized + fun setPicks(context: Context, appKey: String, ids: List) { + require(ids.all { validId(it) }) + val all = read(context).toMutableMap() + if (ids.isEmpty()) all.remove(appKey) else all[appKey] = ids.distinct() + save(context, all) + } + + private fun save(context: Context, games: Map>) { + val text = JSONObject().put("version", 1).put("games", JSONObject().apply { + games.forEach { (app, ids) -> put(app, JSONArray(ids)) } + }).toString() + write(File(context.filesDir, SELECTION), text) + publish(context, text) + } + + /** + * Writes the picks where the launch script reads them; SessionFiles calls this at every session + * start too, so a runtime reinstalled since keeps them. An empty selection still writes a file, + * which is how a component turned off for every game is taken out of their prefixes. + */ + @Synchronized + fun publish(context: Context, text: String? = null) { + val body = text ?: runCatching { AtomicFile(File(context.filesDir, SELECTION)).readFully().toString(Charsets.UTF_8) }.getOrNull() ?: return + val root = LinuxRuntime.rootDir(context) + if (root.isDirectory) write(File(root, GUEST_SELECTION), body) + } + + private fun write(file: File, text: String) { + check(file.parentFile!!.isDirectory || file.parentFile!!.mkdirs()) + val atomic = AtomicFile(file) + val output = atomic.startWrite() + try { + output.write(text.toByteArray(Charsets.UTF_8)) + atomic.finishWrite(output) + } catch (error: Exception) { + atomic.failWrite(output) + throw error + } + } +} diff --git a/app/src/main/java/com/droiddeck/launcher/ui/FrontEndGames.kt b/app/src/main/java/com/droiddeck/launcher/ui/FrontEndGames.kt index 3074257aaa..01bd1ff0a3 100644 --- a/app/src/main/java/com/droiddeck/launcher/ui/FrontEndGames.kt +++ b/app/src/main/java/com/droiddeck/launcher/ui/FrontEndGames.kt @@ -90,7 +90,7 @@ internal fun GamesPage(s: FrontEndState, a: FrontEndActions, selected: String, o } } Rise(1) { SectionTitle(stringResource(R.string.games_launch_settings), null) } - Rise(2) { LaunchSettings(s, a, host) } + Rise(2) { LaunchSettings(s, a, host, current) } Rise(3) { GameFileFolderActions(s, a) } } return @@ -109,7 +109,7 @@ internal fun GamesPage(s: FrontEndState, a: FrontEndActions, selected: String, o GameActions(current, s, a) } SectionTitle(stringResource(R.string.games_launch_settings), null) - LaunchSettings(s, a, host) + LaunchSettings(s, a, host, current) GameFileFolderActions(s, a) } } diff --git a/app/src/main/java/com/droiddeck/launcher/ui/FrontEndSetup.kt b/app/src/main/java/com/droiddeck/launcher/ui/FrontEndSetup.kt index e6b1777f2d..52c3207003 100644 --- a/app/src/main/java/com/droiddeck/launcher/ui/FrontEndSetup.kt +++ b/app/src/main/java/com/droiddeck/launcher/ui/FrontEndSetup.kt @@ -421,10 +421,20 @@ private fun FrameGenMenu(s: FrontEndState, a: FrontEndActions, host: MenuHost) { * app-wide settings - each card opens the same page or menu Setup does. */ @Composable -internal fun LaunchSettings(s: FrontEndState, a: FrontEndActions, host: MenuHost) { +internal fun LaunchSettings(s: FrontEndState, a: FrontEndActions, host: MenuHost, game: com.droiddeck.launcher.frontend.Library.SteamGame? = null) { // Three across, two on a narrow page; each row's cards share one height. val columns = if (LocalNarrowPane.current) 2 else 3 val controller = a.controller + val context = androidx.compose.ui.platform.LocalContext.current + val inputMode = androidx.compose.ui.platform.LocalInputModeManager.current + // The game's prefix is compatdata/, as droiddeck-game-env reads it: a Steam title's appid, an + // added game's shortcut appid (unsigned). Recommended scans the game's own folder either way. + val wincompKey = game?.let { + if (it.library == com.droiddeck.launcher.frontend.Library.ADDED) Integer.toUnsignedString(it.appId) + else it.appId.takeIf { id -> id > 0 }?.toString() + } + var wincompOpen by remember(wincompKey) { mutableStateOf(null) } + val wincompPicks = remember(wincompKey, wincompOpen) { wincompKey?.let { com.droiddeck.launcher.session.WinComponents.picks(context, it) }.orEmpty() } val cards = buildList<@Composable (Modifier) -> Unit> { add { m -> SettingCard(stringResource(R.string.setup_card_components), stringResource(R.string.setup_card_components_hint), "card:components", m) { a.onComponents(true) } } add { m -> @@ -436,7 +446,19 @@ internal fun LaunchSettings(s: FrontEndState, a: FrontEndActions, host: MenuHost } } if (controller != null) add { m -> SettingCard(stringResource(R.string.setup_card_controls), stringResource(R.string.setup_card_controls_hint), "card:controls", m, controller.onMapping) } + if (wincompKey != null) add { m -> + SettingCard( + stringResource(R.string.wincomp_title), + if (wincompPicks.isEmpty()) stringResource(R.string.wincomp_card_none) else wincompPicks.joinToString(", ") { com.droiddeck.launcher.session.WinComponentNames.of(it) }, + "card:wincomp", m, + ) { wincompOpen = inputMode.inputMode == androidx.compose.ui.input.InputMode.Keyboard } + } } + val opened = wincompOpen + if (opened != null && wincompKey != null && game != null) WinComponentsDialog( + wincompKey, game.name, game.gameFiles, opened, compat = game.protonPrefix, + steamAppId = game.appId.takeIf { game.library != com.droiddeck.launcher.frontend.Library.ADDED }, + ) { wincompOpen = null } Column(verticalArrangement = Arrangement.spacedBy(10.dp), modifier = Modifier.fillMaxWidth().padding(bottom = 16.dp)) { for (row in cards.chunked(columns)) { Row(horizontalArrangement = Arrangement.spacedBy(10.dp), modifier = Modifier.fillMaxWidth().height(IntrinsicSize.Min)) { diff --git a/app/src/main/java/com/droiddeck/launcher/ui/GameEnvironmentEditor.kt b/app/src/main/java/com/droiddeck/launcher/ui/GameEnvironmentEditor.kt index db4f6f0f12..3ca96d6c0b 100644 --- a/app/src/main/java/com/droiddeck/launcher/ui/GameEnvironmentEditor.kt +++ b/app/src/main/java/com/droiddeck/launcher/ui/GameEnvironmentEditor.kt @@ -70,7 +70,7 @@ fun GameEnvironmentRow(modifier: Modifier = Modifier, showHint: Boolean = true) /** With a pad driving, focus [target] once the dialog is laid out, and keep the dialog in pad mode. */ @OptIn(androidx.compose.ui.ExperimentalComposeUiApi::class) @Composable -private fun PadFocus(byPad: Boolean, target: FocusRequester) { +internal fun PadFocus(byPad: Boolean, target: FocusRequester) { val inputMode = LocalInputModeManager.current LaunchedEffect(Unit) { if (!byPad) return@LaunchedEffect diff --git a/app/src/main/java/com/droiddeck/launcher/ui/ModeSettingsDialog.kt b/app/src/main/java/com/droiddeck/launcher/ui/ModeSettingsDialog.kt index 25efcb37f8..2f7ea2586c 100644 --- a/app/src/main/java/com/droiddeck/launcher/ui/ModeSettingsDialog.kt +++ b/app/src/main/java/com/droiddeck/launcher/ui/ModeSettingsDialog.kt @@ -435,12 +435,14 @@ fun ModeSettingsPage(s: ModeSettings, a: ModeSettingsActions) { stringResource(R.string.mode_added_art_hint), s.addedGamesArt, onChange = a.onAddedGamesArt, ) - for (g in s.addedGames) ChoiceRow( - host, "added:" + g.folderPath, g.folderName, if (s.addedGamesDirs.size > 1) stringResource(R.string.mode_added_launches_in, g.exeName, g.folderPath.substringBeforeLast('/').substringAfterLast('/')) else stringResource(R.string.mode_added_launches, g.exeName), - g.candidates + ("__pick__" to stringResource(R.string.mode_added_choose)), g.exePath, - note = stringResource(R.string.mode_added_exe_note), - onPick = { path -> if (path == "__pick__") a.onPickAddedGameExe(g.folderPath) else a.onAddedGameExe(g.folderPath, path) }, - ) + for (g in s.addedGames) { + ChoiceRow( + host, "added:" + g.folderPath, g.folderName, if (s.addedGamesDirs.size > 1) stringResource(R.string.mode_added_launches_in, g.exeName, g.folderPath.substringBeforeLast('/').substringAfterLast('/')) else stringResource(R.string.mode_added_launches, g.exeName), + g.candidates + ("__pick__" to stringResource(R.string.mode_added_choose)), g.exePath, + note = stringResource(R.string.mode_added_exe_note), + onPick = { path -> if (path == "__pick__") a.onPickAddedGameExe(g.folderPath) else a.onAddedGameExe(g.folderPath, path) }, + ) + } } if (steam && tab == ModeSettingsTab.GAMES && s.fexPreset != null) SettingsGroup(stringResource(R.string.game_settings_title)) { if (s.syncBackend != null) SettingsRow( diff --git a/app/src/main/java/com/droiddeck/launcher/ui/WinComponentsEditor.kt b/app/src/main/java/com/droiddeck/launcher/ui/WinComponentsEditor.kt new file mode 100644 index 0000000000..85c22c2ee1 --- /dev/null +++ b/app/src/main/java/com/droiddeck/launcher/ui/WinComponentsEditor.kt @@ -0,0 +1,261 @@ +package com.droiddeck.launcher.ui + +import androidx.compose.foundation.background +import androidx.compose.foundation.border +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Box +import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.Spacer +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.heightIn +import androidx.compose.foundation.layout.padding +import androidx.compose.material3.LinearProgressIndicator +import androidx.compose.material3.MaterialTheme +import androidx.compose.material3.Text +import androidx.compose.runtime.Composable +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.rememberCoroutineScope +import androidx.compose.runtime.setValue +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.draw.alpha +import androidx.compose.ui.draw.clip +import androidx.compose.ui.focus.FocusRequester +import androidx.compose.ui.focus.focusRequester +import androidx.compose.ui.platform.LocalContext +import androidx.compose.ui.res.stringResource +import androidx.compose.ui.text.font.FontWeight +import androidx.compose.ui.unit.dp +import androidx.compose.ui.unit.sp +import com.droiddeck.launcher.R +import com.droiddeck.launcher.frontend.DependencyDetector +import com.droiddeck.launcher.frontend.PrefixInstalledDetector +import com.droiddeck.launcher.frontend.SteamRedists +import com.droiddeck.launcher.session.WinComponentNames +import com.droiddeck.launcher.session.WinComponents +import com.droiddeck.launcher.session.WinComponents.Support +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.launch +import androidx.compose.runtime.withFrameNanos +import kotlinx.coroutines.withContext +import java.io.File + +// Windows components for one added game, Bannerlator's list: what the game's folder suggests first, +// then everything that installs here, then the ones that need a Windows installer (not yet). Each +// one is a switch, downloaded the first time it is turned on for any game; the launch puts the +// picked ones into the game's prefix (droiddeck-wincomponents). + +/** Bannerlator's detector names installers; here the DLL-copy twin is the one that installs. */ +private fun installable(name: String, all: Map): String { + val dll = when (name) { + "oalinst" -> "oalinst_dll" + else -> "${name}_dll" + } + val twin = all[dll] + return if (twin != null && WinComponents.support(twin, all) == Support.READY) dll else name +} + +@Composable +internal fun WinComponentsDialog( + appKey: String, gameName: String, gameDir: File?, byPad: Boolean, + /** compatdata/: what the prefix already has. [steamAppId]: a Steam title, whose appmanifest lists Steam's own redists. */ + compat: File? = null, steamAppId: Int? = null, + onClose: () -> Unit, +) { + val context = LocalContext.current + val coroutine = rememberCoroutineScope() + val shown = rememberShown(onClose) + val close = { shown.targetState = false } + // null while the catalog loads; empty when it could not be had. + var catalog by remember { mutableStateOf?>(null) } + var offline by remember { mutableStateOf(false) } + var recommended by remember { mutableStateOf(emptyList()) } + var installed by remember { mutableStateOf(emptySet()) } + // Detector names (oalinst, physx...) already in the prefix from elsewhere. + var present by remember { mutableStateOf(emptySet()) } + var picks by remember { mutableStateOf(WinComponents.picks(context, appKey)) } + var progress by remember { mutableStateOf?>(null) } + var error by remember { mutableStateOf(null) } + var showWaiting by remember { mutableStateOf(false) } + // With a pad: Done holds focus while the list loads, then the first component takes it; LB and RB + // jump between the sections, since the full list runs past sixty switches. + val recFocus = remember { FocusRequester() } + val allFocus = remember { FocusRequester() } + val waitFocus = remember { FocusRequester() } + val doneFocus = remember { FocusRequester() } + + LaunchedEffect(Unit) { + val (entries, found) = withContext(Dispatchers.IO) { + // The folder's own installers first, then what Steam installs with the game. + val own = gameDir?.let { DependencyDetector.detect(it) } ?: emptyList() + val steam = steamAppId?.let { SteamRedists.detect(gameDir, it) } ?: emptyList() + WinComponents.fetch() to (own + steam) + } + offline = entries == null + catalog = entries.orEmpty().associateBy { it.name } + recommended = found + present = withContext(Dispatchers.IO) { PrefixInstalledDetector.detect(compat) } + installed = withContext(Dispatchers.IO) { WinComponents.installedIds(context).toSet() } + } + fun setPicks(next: List) { + picks = next + coroutine.launch(Dispatchers.IO) { WinComponents.setPicks(context, appKey, next) } + } + fun toggle(id: String, on: Boolean) { + // Switches stay enabled during a download - a disabled one drops the pad's focus - so a press + // then is ignored here. + if (progress != null) return + error = null + if (!on) { setPicks(picks - id); return } + val all = catalog.orEmpty() + val c = all[id] + if (id in installed || c == null) { setPicks(picks + id); return } + progress = id to -1 + coroutine.launch { + val problem = withContext(Dispatchers.IO) { + WinComponents.install(context, c, all) { stage, percent -> progress = stage to percent } + } + progress = null + installed = withContext(Dispatchers.IO) { WinComponents.installedIds(context).toSet() } + if (problem != null) error = problem else setPicks(picks + id) + } + } + + val all = catalog + val here = if (all == null) emptySet() else present.map { installable(it, all) }.toSet() + fun supportOf(id: String): Support = + all?.get(id)?.let { WinComponents.support(it, all) } ?: if (id in installed) Support.READY else Support.UNSUPPORTED + val recIds = if (all == null) emptyList() else recommended.distinctBy { installable(it.componentName, all) } + val ready = all.orEmpty().values.filter { WinComponents.support(it, all.orEmpty()) == Support.READY }.map { it.name } + val extra = (installed + picks).filter { all?.containsKey(it) != true } + val list = (ready + extra).distinct().sortedBy { WinComponentNames.of(it).lowercase() } + val waiting = all.orEmpty().values.filter { WinComponents.support(it, all.orEmpty()) == Support.NEEDS_INSTALLER } + .map { it.name }.sortedBy { WinComponentNames.of(it).lowercase() } + val hasRec = all != null && recIds.any { supportOf(installable(it.componentName, all)) == Support.READY } + val sections = listOfNotNull(recFocus.takeIf { hasRec }, allFocus.takeIf { list.isNotEmpty() }, + waitFocus.takeIf { waiting.isNotEmpty() }, doneFocus) + var section by remember { mutableStateOf(0) } + fun jump(step: Int) { + section = (section + step + sections.size) % sections.size + runCatching { sections[section].requestFocus() } + } + LaunchedEffect(all != null) { + if (!byPad || all == null) return@LaunchedEffect + repeat(2) { withFrameNanos { } } + section = 0 + runCatching { sections.first().requestFocus() } + } + + AppDialog(shown, close, "winComponents", wide = true, modifier = Modifier.bumpers({ jump(-1) }, { jump(1) })) { + PadFocus(byPad, doneFocus) + DialogHeader(gameName, stringResource(R.string.wincomp_title)) + Small(stringResource(R.string.wincomp_applies)) + if (byPad) Small(stringResource(R.string.wincomp_pad_hint)) + if (all == null) Small(stringResource(R.string.wincomp_loading)) + if (offline) Small(stringResource(R.string.wincomp_offline), error = true) + @Composable + fun Item(id: String, reason: String?, focus: FocusRequester?) { + val c = all?.get(id) + val support = supportOf(id) + val status = when { + id in here && id !in picks -> stringResource(R.string.wincomp_already_here) + id in installed -> stringResource(R.string.wincomp_downloaded) + support == Support.NEEDS_INSTALLER -> stringResource(R.string.wincomp_needs_installer) + support == Support.UNSUPPORTED -> stringResource(R.string.wincomp_unsupported) + else -> null + } + // The catalog key stays in the detail: it is what a log or a bug report names. + val detail = listOfNotNull(reason, c?.description?.takeIf { it.isNotEmpty() }, status, id).joinToString(" · ") + val usable = support == Support.READY + ComponentRow( + WinComponentNames.of(id), detail, checked = id in picks, enabled = usable || id in picks, + dim = !usable, modifier = if (usable && focus != null) Modifier.focusRequester(focus) else Modifier, + ) { on -> toggle(id, on) } + } + /** The section's requester goes to its first switch that can take focus. */ + fun firstUsable(ids: List, focus: FocusRequester): (String) -> FocusRequester? { + val first = ids.firstOrNull { supportOf(it) == Support.READY } + return { id -> focus.takeIf { id == first } } + } + + if (all != null && recIds.isNotEmpty()) { + Section(stringResource(R.string.wincomp_recommended)) + val focusOf = firstUsable(recIds.map { installable(it.componentName, all) }, recFocus) + Panel { + recIds.forEachIndexed { i, rec -> + if (i > 0) Divider() + val reason = stringResource( + when (rec.kind) { + DependencyDetector.Kind.BUNDLED -> R.string.wincomp_found_bundled + DependencyDetector.Kind.SHIPPED -> R.string.wincomp_found_shipped + DependencyDetector.Kind.STEAM -> R.string.wincomp_found_steam + }, rec.reason, + ) + val id = installable(rec.componentName, all) + Item(id, reason, focusOf(id)) + } + } + } else if (all != null && gameDir != null) Small(stringResource(R.string.wincomp_no_recommendation)) + + if (list.isNotEmpty()) { + Section(stringResource(R.string.wincomp_all, list.size)) + val focusOf = firstUsable(list, allFocus) + Panel { list.forEachIndexed { i, id -> if (i > 0) Divider(); Item(id, null, focusOf(id)) } } + } + if (waiting.isNotEmpty()) { + Row(verticalAlignment = Alignment.CenterVertically) { + Section(stringResource(R.string.wincomp_installers, waiting.size)) + Spacer(Modifier.weight(1f)) + SecondaryButton( + stringResource(if (showWaiting) R.string.wincomp_hide else R.string.wincomp_show), compact = true, + modifier = Modifier.focusRequester(waitFocus), + ) { showWaiting = !showWaiting } + } + Small(stringResource(R.string.wincomp_installers_note)) + if (showWaiting) Panel { waiting.forEachIndexed { i, id -> if (i > 0) Divider(); Item(id, null, null) } } + } + + progress?.let { (stage, percent) -> + Small(if (percent >= 0) "$stage · $percent%" else stage) + if (percent >= 0) LinearProgressIndicator(progress = { percent / 100f }, modifier = Modifier.fillMaxWidth()) + else LinearProgressIndicator(Modifier.fillMaxWidth()) + } + error?.let { Small(it, error = true) } + Row(verticalAlignment = Alignment.CenterVertically, modifier = Modifier.padding(top = 4.dp)) { + Small(stringResource(R.string.wincomp_next_launch)) + Spacer(Modifier.weight(1f)) + PrimaryButton(stringResource(R.string.game_env_done), enabled = progress == null, modifier = Modifier.focusRequester(doneFocus), onClick = close) + } + } +} + +@Composable +private fun Section(title: String) = + Text(title.uppercase(), fontSize = 12.sp, fontWeight = FontWeight.SemiBold, letterSpacing = 1.5.sp, color = MaterialTheme.colorScheme.onSurfaceVariant) + +@Composable +private fun Panel(content: @Composable androidx.compose.foundation.layout.ColumnScope.() -> Unit) = + Column(Modifier.fillMaxWidth().clip(Shape14).background(MaterialTheme.colorScheme.surface).border(1.dp, LocalPalette.current.line, Shape14), content = content) + +@Composable +private fun Divider() = + Box(Modifier.fillMaxWidth().padding(horizontal = 14.dp).heightIn(min = 1.dp, max = 1.dp).background(LocalPalette.current.line)) + +@Composable +private fun ComponentRow(name: String, detail: String, checked: Boolean, enabled: Boolean, dim: Boolean, modifier: Modifier, onChange: (Boolean) -> Unit) { + val colors = MaterialTheme.colorScheme + Row( + verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(12.dp), + modifier = Modifier.fillMaxWidth().padding(horizontal = 14.dp, vertical = 10.dp).alpha(if (dim) 0.55f else 1f), + ) { + Column(Modifier.weight(1f)) { + Text(name, fontSize = 15.sp, fontWeight = FontWeight.SemiBold, color = colors.onBackground) + if (detail.isNotEmpty()) Text(detail, fontSize = 12.sp, lineHeight = 16.sp, color = colors.onSurfaceVariant) + } + ToggleSwitch(checked, enabled, name, modifier, onChange) + } +} diff --git a/app/src/main/res/values/strings.xml b/app/src/main/res/values/strings.xml index 539e6ba92f..4e3d596e37 100644 --- a/app/src/main/res/values/strings.xml +++ b/app/src/main/res/values/strings.xml @@ -307,6 +307,28 @@ Launches %1$s · in %2$s Choose another file… Defaults to the .exe named after the folder, or else the largest. + Windows components + LB / RB jump between the sections. + None + Windows DLLs Proton does not include, from Bannerlator\'s component list. Turn on the ones this game needs: they go into the game\'s own Windows folder, never its files, and stay whichever Proton it runs on. + Could not reach the component list. Components already downloaded can still be switched. + Recommended for this game + The game ships %1$s + The game has %1$s (often works as is) + Steam installs %1$s with this game + Already in this game + Nothing in the game\'s folder points at a component. + All components (%1$d) + Need a Windows installer (%1$d) + These run a real installer inside the prefix (.NET, PhysX, XNA, the full Visual C++ installers…). DroidDeck cannot do that yet; it is coming next. + Show + Hide + Needs a Windows installer (coming) + Not supported here yet + Loading the components… + Downloaded + %1$s download + Takes effect the next time the game starts. Stretch games to fill the screen Keeps self-resizing games full screen. Turn off if a game shows up small. Applies next session. Stretch 16:9 games diff --git a/app/src/test/java/com/droiddeck/launcher/frontend/WinComponentDetectorsTest.kt b/app/src/test/java/com/droiddeck/launcher/frontend/WinComponentDetectorsTest.kt new file mode 100644 index 0000000000..fbf077910d --- /dev/null +++ b/app/src/test/java/com/droiddeck/launcher/frontend/WinComponentDetectorsTest.kt @@ -0,0 +1,56 @@ +package com.droiddeck.launcher.frontend + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertTrue +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.annotation.Config +import java.io.File + +@RunWith(RobolectricTestRunner::class) +@Config(sdk = [33]) +class WinComponentDetectorsTest { + @get:Rule val tmp = TemporaryFolder() + + @Test fun steamsSharedDepotsBecomeRecommendations() { + val steamapps = tmp.newFolder("steamapps") + val game = File(steamapps, "common/BioShock Remastered").apply { mkdirs() } + File(steamapps, "appmanifest_409710.acf").writeText( + """ + "AppState" + { + "appid" "409710" + "SharedDepots" + { + "228983" "228980" + "228984" "228980" + "228988" "228980" + "228987" "228980" + "228990" "228980" + "999999" "228980" + } + } + """.trimIndent(), + ) + val found = SteamRedists.detect(game, 409710) + assertEquals(listOf("vcredist2010_dll", "vcredist2012_dll", "vcredist2019_dll", "d3dx9"), found.map { it.componentName }) + assertTrue(found.all { it.kind == DependencyDetector.Kind.STEAM }) + assertEquals("VC++ 2010", found.first().reason) + assertEquals(emptyList(), SteamRedists.detect(game, 1)) + } + + @Test fun thePrefixCountsOnlyWhatDroidDeckDidNotPutThere() { + val compat = tmp.newFolder("compatdata", "409710") + val pfx = File(compat, "pfx") + File(pfx, "drive_c/windows/syswow64").mkdirs() + File(pfx, "drive_c/windows/syswow64/OpenAL32.dll").writeText("x") + File(pfx, "system.reg").writeText("[Software\\\\Microsoft\\\\Games for Windows Live]\n") + assertEquals(setOf("oalinst", "XLiveRedist"), PrefixInstalledDetector.detect(compat)) + File(compat, ".droiddeck-wincomponents.json").writeText("""{"files": {"drive_c/windows/syswow64/openal32.dll": "oalinst_dll"}}""") + assertEquals(setOf("XLiveRedist"), PrefixInstalledDetector.detect(compat)) + assertEquals(emptySet(), PrefixInstalledDetector.detect(File(tmp.root, "missing"))) + } +} diff --git a/app/src/test/java/com/droiddeck/launcher/session/WinComponentNamesTest.kt b/app/src/test/java/com/droiddeck/launcher/session/WinComponentNamesTest.kt new file mode 100644 index 0000000000..889515919d --- /dev/null +++ b/app/src/test/java/com/droiddeck/launcher/session/WinComponentNamesTest.kt @@ -0,0 +1,17 @@ +package com.droiddeck.launcher.session + +import org.junit.Assert.assertEquals +import org.junit.Test + +class WinComponentNamesTest { + @Test fun catalogKeysReadAsPeopleKnowThem() { + val names = mapOf( + "oalinst_dll" to "OpenAL", "vcredist2010_dll" to "Visual C++ 2010", "vcredist2022" to "Visual C++ 2015-2022", + "dotnet48" to ".NET Framework 4.8", "dotnet452" to ".NET Framework 4.5.2", "dotnet20sp1" to ".NET Framework 2.0 SP1", + "d3dcompiler_47" to "D3DCompiler 47", "xaudio2.7" to "XAudio 2.7", "mono-10.4.1" to "Wine Mono 10.4.1", + "msxml6" to "MSXML 6", "dmsynth" to "DirectMusic (dmsynth)", "XLiveRedist" to "Games for Windows Live", + "something-new" to "something-new", + ) + for ((id, name) in names) assertEquals(id, name, WinComponentNames.of(id)) + } +} diff --git a/tools/linuxfs/overlay/usr/local/bin/droiddeck-game-env b/tools/linuxfs/overlay/usr/local/bin/droiddeck-game-env index 529b525453..e8f5605a4d 100755 --- a/tools/linuxfs/overlay/usr/local/bin/droiddeck-game-env +++ b/tools/linuxfs/overlay/usr/local/bin/droiddeck-game-env @@ -3,10 +3,12 @@ import json import os import re import runpy +import subprocess import sys from pathlib import Path SYNC_KEYS = ("BL_SYNC_FALLBACK", "BL_FSYNC_FIRST", "BL_SYNC_MIN_NOFILE", "PROTON_NO_ESYNC", "PROTON_NO_FSYNC", "PROTON_NO_NTSYNC", "WINEESYNC") +SEED_REDISTS = "/usr/local/bin/droiddeck-seed-redists" PACK_KEYS = ("WINESERVER", "BL_SYNC_PACK", "WINEESYNC", "PROTON_NO_ESYNC", "PROTON_NO_FSYNC") @@ -117,6 +119,21 @@ def engine_fixes(exe, args): return env, extra, notes +def win_components(compat, app_id): + """The game's Windows components put into its prefix; returns their WINEDLLOVERRIDES entries.""" + if not compat: + return "" + try: + module = runpy.run_path(str(Path(__file__).resolve().parent / "droiddeck-wincomponents")) + names, notes = module["apply"](compat.rstrip("/"), app_id) + for note in notes: + print("droiddeck-game-env: " + note, file=sys.stderr) + return module["overrides_value"](names) + except Exception as error: + print("droiddeck-game-env: Windows components not applied (%s)" % error, file=sys.stderr) + return "" + + def apply_config(env, config, app_id, fixes=None): if config.get("version") != 1: raise ValueError("unsupported environment configuration") @@ -220,11 +237,33 @@ def force_ssbs(command, env): return env +EVALUATOR_SCRIPT = re.compile(r"(?:^|[\\/])evaluatorscript_[0-9]+\.vdf$", re.IGNORECASE) + + +def mark_install_scripts(command, env): + """The client is about to run a game's install scripts (iscriptevaluator.exe): mark every process + in them as already run first, so no redistributable installer starts - on first launch the + shared ones may still be downloading, and on ARM64 Proton the VC++ ones stop with an error.""" + if not any(arg.lower().replace("\\", "/").endswith("iscriptevaluator.exe") for arg in command): + return + root = env.get("STEAM_COMPAT_CLIENT_INSTALL_PATH") or str(Path.home() / ".local/share/Steam") + scripts = [str(Path(root) / arg.replace("\\", "/")) for arg in command if EVALUATOR_SCRIPT.search(arg)] + scripts = [path for path in scripts if os.path.isfile(path)] + if not scripts: + return + try: + subprocess.run([SEED_REDISTS, root], env=dict(env, DROIDDECK_SEED_SCRIPTS="\n".join(scripts)), + stdout=sys.stderr, timeout=60, check=False) + except (OSError, subprocess.SubprocessError) as error: + print("droiddeck-game-env: install scripts not marked (%s)" % error, file=sys.stderr) + + def main(): command = sys.argv[1:] if not command: return 2 env = os.environ.copy() + mark_install_scripts(command, env) app_id = game_id(env) if app_id: waiting = len(command) > 1 and command[1] == "waitforexitandrun" @@ -234,6 +273,10 @@ def main(): for note in notes: print("droiddeck-game-env: " + note, file=sys.stderr) command += extra + components = win_components(env.get("STEAM_COMPAT_DATA_PATH", ""), app_id) + if components: + own = fixes.get("WINEDLLOVERRIDES") + fixes["WINEDLLOVERRIDES"] = components + ";" + own if own else components path = Path.home() / ".config/droiddeck/game-environment.json" try: with path.open(encoding="utf-8") as source: diff --git a/tools/linuxfs/overlay/usr/local/bin/droiddeck-seed-redists b/tools/linuxfs/overlay/usr/local/bin/droiddeck-seed-redists index 9007596a82..62202230b8 100755 --- a/tools/linuxfs/overlay/usr/local/bin/droiddeck-seed-redists +++ b/tools/linuxfs/overlay/usr/local/bin/droiddeck-seed-redists @@ -11,28 +11,39 @@ # WineUtils.seedVcRedistRegistryIfDllsPresent() takes for the app's own containers. The names and # versions come out of the install scripts rather than a list kept here, because Valve revises # them; a prefix already carrying the same set is left alone. +# +# Games also carry install scripts of their own (/_CommonRedist/vcredist/2022/installscript.vdf +# and the like), often with another version in the value name; those are read too. And the VC++ +# 2015-2022 runtime is recorded as installed the way Microsoft's installers look for it: on ARM64 +# Proton the x64 bundle otherwise stops with "Microsoft Visual C++ 2015-2022 Redistributable (arm64) +# is required", while Proton's own msvcp140/vcruntime140 already serve the game. set -u steam_root=${1:?steam root} +LIBRARIES=${DROIDDECK_LIBRARIES-/mnt/droiddeck /mnt/droiddeck-sd} redist="$steam_root/steamapps/common/Steamworks Shared/_CommonRedist" # Every process an install script would run, as the registry key and value the client writes once # it has run it. Names sit one level inside "Run Process", the key they share a level further in. -pairs=$(find "$redist" -name installscript.vdf -print0 2>/dev/null | sort -z | xargs -0 -r awk ' - FNR == 1 { depth = 0; name = ""; key = "" } - /^[[:space:]]*\{/ { depth++; next } +read_scripts() { xargs -0 -r awk ' + # A block whose HasRunKey is set is one process; its name is the key the block opened under, at + # whatever depth - an install script nests it under "Run Process", the client'"'"'s evaluator script + # for a game wraps the same blocks once more. + FNR == 1 { depth = 0; pending = "" } + /^[[:space:]]*\{/ { depth++; name[depth] = pending; key[depth] = ""; pending = ""; next } /^[[:space:]]*\}/ { - if (depth == 3 && name != "" && key != "") - printf "%s\t%s\n", key, name - depth-- + if (depth > 0 && key[depth] != "" && name[depth] != "") + printf "%s\t%s\n", key[depth], name[depth] + if (depth > 0) depth-- next } { n = split($0, part, "\"") - if (n >= 5 && tolower(part[2]) == "hasrunkey") { key = part[4]; next } - if (n >= 3 && depth == 2) { name = part[2]; key = "" } + if (n >= 5 && tolower(part[2]) == "hasrunkey") { key[depth] = part[4]; next } + if (n == 3) pending = part[2] } -') +'; } +pairs=$(find "$redist" -name installscript.vdf -print0 2>/dev/null | sort -z | read_scripts) # The client downloads the scripts with the first game that wants them, which is too late for that # game; until then the set they held when this was written stands in. (WinNative's list.) [ -n "$pairs" ] || pairs=$(sed 's/^\([^ ]*\) /HKEY_LOCAL_MACHINE\\\\Software\\\\Valve\\\\Steam\\\\Apps\\\\CommonRedist\\\\\1\t/' <<'KNOWN' @@ -54,9 +65,51 @@ vcredist\\2022 x86 14.51.36247.0 vcredist\\2022 x64 14.51.36247.0 KNOWN ) + +# The games' own scripts. Finding them walks every game folder, so the list is kept until a game is +# installed, updated or removed (its appmanifest changes) - this runs every quarter minute. +cache="$steam_root/.droiddeck-game-scripts" +libraries="$steam_root" +for library in $LIBRARIES; do libraries="$libraries +$library"; done +sig=$(while IFS= read -r library; do ls -l --time-style=+%s "$library"/steamapps/appmanifest_*.acf 2>/dev/null; done <<<"$libraries" | cksum) +if [ "$(head -n 1 "$cache" 2>/dev/null)" != "$sig" ]; then + { + printf '%s\n' "$sig" + while IFS= read -r library; do + [ -d "$library/steamapps/common" ] || continue + find "$library/steamapps/common" -mindepth 2 -maxdepth 5 -iname 'installscript*.vdf' \ + -not -path '*/Steamworks Shared/*' 2>/dev/null + done <<<"$libraries" | sort -u + } >"$cache.tmp" && mv -f "$cache.tmp" "$cache" +fi +game_pairs=$(tail -n +2 "$cache" 2>/dev/null | tr '\n' '\0' | read_scripts) +# DROIDDECK_SEED_SCRIPTS: more scripts, one path a line - droiddeck-game-env passes the client's +# evaluator script for the game about to run its installers, read before the client looks. +if [ -n "${DROIDDECK_SEED_SCRIPTS:-}" ]; then + game_pairs=$(printf '%s\n%s\n' "$game_pairs" "$(printf '%s\n' "$DROIDDECK_SEED_SCRIPTS" | tr '\n' '\0' | read_scripts)") +fi +[ -n "$(printf '%s' "$game_pairs" | tr -d '[:space:]')" ] && pairs=$(printf '%s\n%s\n' "$pairs" "$game_pairs" | sort -u | sed '/^$/d') + +# The VC++ 2015-2022 runtime as its installers check for it, for each architecture and both registry +# views. The version is the newest the scripts above know, so any 2015-2022 bundle finds itself +# already installed and exits without asking for anything. +VCRT_MAJOR=14 VCRT_MINOR=51 VCRT_BUILD=36247 +vcrt() { + local view arch + for view in 'Software' 'Software\\Wow6432Node'; do + for arch in x86 x64 arm64; do + printf '\n[%s\\\\Microsoft\\\\VisualStudio\\\\14.0\\\\VC\\\\Runtimes\\\\%s] %d\n' "$view" "$arch" "$1" + printf '"Bld"=dword:%08x\n"Installed"=dword:00000001\n"Major"=dword:%08x\n"Minor"=dword:%08x\n"Rbld"=dword:00000000\n' \ + "$VCRT_BUILD" "$VCRT_MAJOR" "$VCRT_MINOR" + printf '"Version"="v%d.%d.%d.00"\n' "$VCRT_MAJOR" "$VCRT_MINOR" "$VCRT_BUILD" + done + done +} + # The stamp is the set of names, taken before the block below dates its keys: stamped from the # dated block it never matched, and every prefix had the whole block appended again on every beat. -stamp=$(printf '%s' "$pairs" | cksum) +stamp=$(printf '%s vcrt %s.%s.%s' "$pairs" "$VCRT_MAJOR" "$VCRT_MINOR" "$VCRT_BUILD" | cksum) block=$(printf '%s\n' "$pairs" | awk -F'\t' -v now="$(date +%s)" ' { @@ -67,11 +120,14 @@ block=$(printf '%s\n' "$pairs" | awk -F'\t' -v now="$(date +%s)" ' printf "\n[%s] %d\n\"%s\"=dword:00000001\n", path, now, $2 # The client writes these from 32-bit steam.exe, so its own copy lands under Wow6432Node; # both spellings are seeded because the reader picks by its own word size. - sub(/^Software\\\\/, "Software\\\\\\\\Wow6432Node\\\\\\\\", path) - printf "\n[%s] %d\n\"%s\"=dword:00000001\n", path, now, $2 + # Joined rather than sub()stituted: gawk and mawk read backslashes in a replacement differently. + if (substr(path, 1, 10) != "Software\\\\") + next + printf "\n[%s] %d\n\"%s\"=dword:00000001\n", "Software\\\\Wow6432Node\\\\" substr(path, 11), now, $2 } ') -[ -n "$block" ] || exit 0 +block="$block +$(vcrt "$(date +%s)")" # Proton builds a new prefix from its template, and the client runs the install scripts in that # prefix moments later, so the template carries the markers too. Its date is kept: Proton reads a diff --git a/tools/linuxfs/overlay/usr/local/bin/droiddeck-wincomponents b/tools/linuxfs/overlay/usr/local/bin/droiddeck-wincomponents new file mode 100755 index 0000000000..828ad57471 --- /dev/null +++ b/tools/linuxfs/overlay/usr/local/bin/droiddeck-wincomponents @@ -0,0 +1,241 @@ +#!/usr/bin/python3 +"""Windows components (OpenAL, d3dx, XAudio, ...) for a game's Proton prefix. + +The app installs each component under STORE/: its files in system32/ (64-bit) and syswow64/ +(32-bit) as Windows lays them out, and component.json naming the overrides that make Wine load +them instead of its own ("name" for native-then-builtin, or "name=n" with Wine's own letters) and +the components it bundles ("requires"). The ids are Bannerlator's catalog names (xaudio2.7, +oalinst_dll...). The user picks components per game; the picks are in SELECTION as +{"version": 1, "games": {"": ["oalinst_dll", ...]}}. + +droiddeck-game-env calls apply() before every game launch. It copies what is missing or changed +into compatdata//pfx and returns the overrides for WINEDLLOVERRIDES, so nothing is written +to the prefix's registry and Wine never has to be running. + +Proton leaves the files alone afterwards: its builtin refresh (update_builtin_libs) replaces only +files that are its own builtins, and a new prefix is built around files already there (copy_pfx +skips them). The one case that removes them is a launch on an older Proton than the prefix's, +which deletes every file Proton once placed - and the next launch's apply() puts ours back. +""" +import json +import os +import re +import shutil +import sys +from pathlib import Path + +STORE = Path("/opt/droiddeck/wincomponents") +SELECTION = Path.home() / ".config/droiddeck/wincomponents.json" +# What this prefix holds from us, so a component the user turns off is taken out again. +STATE = ".droiddeck-wincomponents.json" +# A file ours replaced that was not Proton's (a real installer's, the user's own), put back when the +# component is turned off. Proton's builtins need no copy: it restores those itself. +BACKUP = ".droiddeck-wincomponents-backup" +BUILTIN_MARKS = (b"Wine builtin DLL", b"Wine placeholder DLL") +ARCH_DIRS = ("system32", "syswow64") +ID = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,63}") + + +def valid_id(value): + return isinstance(value, str) and ID.fullmatch(value) is not None and ".." not in value + + +def wanted(app_id, selection=SELECTION): + try: + with open(selection, encoding="utf-8") as source: + config = json.load(source) + except FileNotFoundError: + return [] + if config.get("version") != 1: + raise ValueError("unsupported components selection") + ids = config.get("games", {}).get(app_id, []) + if not isinstance(ids, list) or not all(valid_id(i) for i in ids): + raise ValueError("invalid components selection") + return ids + + +def component(store, cid): + """The component's overrides, its files as (relative path, source) and the ids it bundles, or None if not installed.""" + root = Path(store) / cid + try: + with open(root / "component.json", encoding="utf-8") as source: + meta = json.load(source) + except (FileNotFoundError, ValueError): + return None + files = [] + for arch in ARCH_DIRS: + folder = root / arch + if folder.is_dir(): + # A few go one level down (gmdls: system32/drivers/gm.dls). + for entry in sorted(folder.rglob("*")): + if entry.is_file() and not entry.is_symlink(): + rel = entry.relative_to(folder).as_posix().lower() + files.append(("drive_c/windows/%s/%s" % (arch, rel), entry)) + requires = [r for r in meta.get("requires", []) if valid_id(r)] + return [str(o) for o in meta.get("overrides", [])], files, requires + + +def expand(store, ids): + """The picks with everything they bundle, each once, a bundle's parts before it.""" + out, seen = [], set() + + def visit(cid, depth): + if cid in seen or depth > 8: + return + seen.add(cid) + found = component(store, cid) + for dep in (found[2] if found else []): + visit(dep, depth + 1) + out.append(cid) + + for cid in ids: + visit(cid, 0) + return out + + +def existing(path): + """The file at path as the prefix spells it: Windows names ignore case, the filesystem may not.""" + folder = path.parent + try: + with os.scandir(folder) as entries: + for entry in entries: + if entry.name.lower() == path.name: + return folder / entry.name + except OSError: + pass + return None + + +def same(src, dst): + try: + if dst.is_symlink(): + return False + a, b = src.stat(), dst.stat() + except OSError: + return False + return a.st_size == b.st_size and int(a.st_mtime) == int(b.st_mtime) + + +def builtin(path): + """Proton's own: a symlink into its files, or a Wine DLL that says so in its DOS stub.""" + if path.is_symlink(): + return True + try: + with open(path, "rb") as source: + head = source.read(512) + except OSError: + return False + return any(mark in head for mark in BUILTIN_MARKS) + + +def place(src, target, backup=None): + """Copies src over target unless it is already there; a file of someone else's goes to backup first.""" + target.parent.mkdir(parents=True, exist_ok=True) + found = existing(target) + if found is not None and same(src, found): + return False + if found is not None and backup is not None and not backup.exists() and not builtin(found): + backup.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(found, backup) + final = found or target + temp = final.with_name(".%s.droiddeck" % final.name) + shutil.copy2(src, temp) + # Over a Proton builtin this swaps its symlink for our file. + os.replace(temp, final) + return True + + +def load_state(compat): + try: + with open(compat / STATE, encoding="utf-8") as source: + state = json.load(source) + return state if isinstance(state.get("files"), dict) else {"files": {}} + except (FileNotFoundError, ValueError, AttributeError): + return {"files": {}} + + +def save_state(compat, state): + temp = compat / (STATE + ".tmp") + with open(temp, "w", encoding="utf-8") as out: + json.dump(state, out, indent=1, sort_keys=True) + os.replace(temp, compat / STATE) + + +def apply(compat_path, app_id, selection=SELECTION, store=STORE): + """Brings the prefix in line with the game's picks. Returns (overrides, notes).""" + compat = Path(compat_path) + pfx = compat / "pfx" + ids = wanted(app_id, selection) + state = load_state(compat) + if not ids and not state["files"]: + return [], [] + notes, overrides, keep = [], [], {} + copied = 0 + installed = [] + for cid in expand(store, ids): + found = component(store, cid) + if found is None: + notes.append("%s is not installed; skipped" % cid) + continue + installed.append(cid) + names, files, _ = found + overrides += names + for rel, src in files: + # A file this prefix already has from us is ours to replace; anything else is kept. + if place(src, pfx / rel, None if rel in state["files"] else compat / BACKUP / rel): + copied += 1 + keep[rel] = cid + removed = restored = 0 + for rel, cid in state["files"].items(): + if rel in keep: + continue + path = existing(pfx / rel) + saved = compat / BACKUP / rel + if saved.is_file(): + target = path or (pfx / rel) + os.replace(saved, target) + restored += 1 + elif path is not None and not path.is_symlink(): + path.unlink() + removed += 1 + if removed: + # Proton refreshes its builtins only when its config changes; forgetting the config puts + # its own DLLs back where ours were on the next launch - this one. + try: + (compat / "config_info").unlink() + except FileNotFoundError: + pass + compat.mkdir(parents=True, exist_ok=True) + save_state(compat, {"files": keep, "components": installed}) + if copied or removed or restored: + note = "components %s: %d file(s) placed, %d removed" % (",".join(ids) or "none", copied, removed) + notes.append(note + (", %d put back as they were" % restored if restored else "")) + return overrides, notes + + +def overrides_value(names): + """WINEDLLOVERRIDES entries: native-then-builtin unless the component says otherwise; the last word on a DLL wins.""" + modes = {} + for entry in names: + name, _, mode = entry.partition("=") + name = name.strip().lower() + if name and re.fullmatch(r"[a-z0-9._-]+", name) and re.fullmatch(r"[nb,]*", mode): + modes.pop(name, None) + modes[name] = mode if "=" in entry else "n,b" + return ";".join("%s=%s" % item for item in modes.items()) + + +def main(): + # droiddeck-wincomponents : what a launch would do, for a shell. + if len(sys.argv) != 3: + print("usage: droiddeck-wincomponents ", file=sys.stderr) + return 2 + names, notes = apply(sys.argv[1], sys.argv[2]) + for note in notes: + print(note) + print("WINEDLLOVERRIDES=" + overrides_value(names)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/tests/test_game_environment.py b/tools/tests/test_game_environment.py index 5d1826a33d..f2fa6d41f4 100644 --- a/tools/tests/test_game_environment.py +++ b/tools/tests/test_game_environment.py @@ -26,6 +26,25 @@ def config_text(entries): class GameEnvironmentTest(unittest.TestCase): + def test_the_install_script_evaluator_gets_its_scripts_marked_first(self): + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + script = root / "legacycompat/evaluatorscript_409710.vdf" + script.parent.mkdir(parents=True) + script.write_text("{}") + calls = [] + run = MODULE["mark_install_scripts"].__globals__["subprocess"].run + MODULE["mark_install_scripts"].__globals__["subprocess"].run = lambda args, **kw: calls.append((args, kw["env"])) + try: + env = {"STEAM_COMPAT_CLIENT_INSTALL_PATH": str(root)} + MODULE["mark_install_scripts"](["/proton", "run", str(root / "legacycompat/iscriptevaluator.exe"), "legacycompat\\evaluatorscript_409710.vdf"], env) + MODULE["mark_install_scripts"](["/proton", "waitforexitandrun", "/game/Game.exe"], env) + finally: + MODULE["mark_install_scripts"].__globals__["subprocess"].run = run + self.assertEqual(len(calls), 1) + self.assertEqual(calls[0][0][1], str(root)) + self.assertEqual(calls[0][1]["DROIDDECK_SEED_SCRIPTS"], str(script)) + def test_profile_precedence_and_unset(self): env = {"KEEP": "inherited", "REMOVE": "inherited", "CUSTOM": "launch option"} config = {"version": 1, "shared": {"CUSTOM": "shared"}, "games": {"42": {"REMOVE": None, "CUSTOM": "game", "EMPTY": ""}}} diff --git a/tools/tests/test_seed_redists.py b/tools/tests/test_seed_redists.py new file mode 100644 index 0000000000..abc77ab07f --- /dev/null +++ b/tools/tests/test_seed_redists.py @@ -0,0 +1,96 @@ +import os +import shutil +import subprocess +import tempfile +import unittest +from pathlib import Path + +SCRIPT = Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin/droiddeck-seed-redists" + +GAME_SCRIPT = r'''"InstallScript" +{ + "Run Process" + { + "VCRedist2022" + { + "HasRunKey" "HKEY_LOCAL_MACHINE\\Software\\Valve\\Steam\\Apps\\CommonRedist\\vcredist\\2022" + "process 1" "%INSTALLDIR%\\_CommonRedist\\vcredist\\2022\\VC_redist.x64.exe" + "command 1" "/q /norestart" + "NoCleanUp" "1" + } + } +} +''' + + +class SeedRedistsTest(unittest.TestCase): + def setUp(self): + self.tmp = Path(tempfile.mkdtemp()) + self.addCleanup(shutil.rmtree, self.tmp, True) + self.steam = self.tmp / "Steam" + self.reg = self.steam / "steamapps/compatdata/1234/pfx/system.reg" + self.reg.parent.mkdir(parents=True) + self.reg.write_text("WINE REGISTRY Version 2\n") + + def game(self, name="Some Game", script=GAME_SCRIPT, library=None): + root = Path(library or self.steam) + path = root / "steamapps/common" / name / "_CommonRedist/vcredist/2022/installscript.vdf" + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(script) + manifest = root / "steamapps" / ("appmanifest_%d.acf" % (abs(hash(name)) % 100000)) + manifest.write_text('"AppState" { "installdir" "%s" }' % name) + return path + + def run_seed(self, libraries=""): + env = dict(os.environ, DROIDDECK_LIBRARIES=libraries) + return subprocess.run(["bash", str(SCRIPT), str(self.steam)], env=env, capture_output=True, text=True, check=True) + + def test_a_games_own_install_script_is_marked_as_run(self): + self.game() + self.run_seed() + reg = self.reg.read_text() + self.assertIn("[Software\\\\Valve\\\\Steam\\\\Apps\\\\CommonRedist\\\\vcredist\\\\2022]", reg) + self.assertIn("[Software\\\\Wow6432Node\\\\Valve\\\\Steam\\\\Apps\\\\CommonRedist\\\\vcredist\\\\2022]", reg) + self.assertIn('"VCRedist2022"=dword:00000001', reg) + + def test_games_in_other_libraries_are_read(self): + sd = self.tmp / "sd" + self.game(library=sd) + self.run_seed(str(sd)) + self.assertIn('"VCRedist2022"=dword:00000001', self.reg.read_text()) + + def test_the_vc_runtime_is_recorded_for_every_architecture_and_view(self): + self.run_seed() + reg = self.reg.read_text() + for view in ("Software", "Software\\\\Wow6432Node"): + for arch in ("x86", "x64", "arm64"): + self.assertIn("[%s\\\\Microsoft\\\\VisualStudio\\\\14.0\\\\VC\\\\Runtimes\\\\%s]" % (view, arch), reg) + self.assertIn('"Installed"=dword:00000001', reg) + self.assertIn('"Major"=dword:0000000e', reg) + self.assertIn('"Version"="v14.51.36247.00"', reg) + + def test_a_seeded_prefix_is_left_alone_until_a_game_changes(self): + self.run_seed() + once = self.reg.read_text() + self.run_seed() + self.assertEqual(once, self.reg.read_text()) + self.game() + self.run_seed() + self.assertIn('"VCRedist2022"=dword:00000001', self.reg.read_text()) + + def test_an_evaluator_script_nested_deeper_is_read(self): + script = self.tmp / "legacycompat/evaluatorscript_409710.vdf" + script.parent.mkdir(parents=True) + script.write_text('"evaluatorscript"\n{\n\t"1"\n' + "\n".join("\t" + line for line in GAME_SCRIPT.replace("VCRedist2022", "x64 14.99.1.0").splitlines()) + "\n}\n") + env = dict(os.environ, DROIDDECK_LIBRARIES="", DROIDDECK_SEED_SCRIPTS=str(script)) + subprocess.run(["bash", str(SCRIPT), str(self.steam)], env=env, capture_output=True, text=True, check=True) + self.assertIn('"x64 14.99.1.0"=dword:00000001', self.reg.read_text()) + + def test_the_prefix_date_is_kept(self): + os.utime(self.reg, (1_000_000_000, 1_000_000_000)) + self.run_seed() + self.assertEqual(int(self.reg.stat().st_mtime), 1_000_000_000) + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/tests/test_wincomponents.py b/tools/tests/test_wincomponents.py new file mode 100644 index 0000000000..9564882905 --- /dev/null +++ b/tools/tests/test_wincomponents.py @@ -0,0 +1,170 @@ +import json +import os +import runpy +import tempfile +import unittest +from pathlib import Path +from unittest import mock + +BIN = Path(__file__).resolve().parents[1] / "linuxfs/overlay/usr/local/bin" +MODULE = runpy.run_path(str(BIN / "droiddeck-wincomponents")) +GAME_ENV = runpy.run_path(str(BIN / "droiddeck-game-env")) + + +class WinComponentsTest(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + root = Path(self.tmp.name) + self.store = root / "store" + self.compat = root / "compatdata/2571976725" + self.selection = root / "wincomponents.json" + self.pfx = self.compat / "pfx/drive_c/windows" + + def tearDown(self): + self.tmp.cleanup() + + def component(self, cid, overrides, files): + folder = self.store / cid + for rel, data in files.items(): + path = folder / rel + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(data) + (folder / "component.json").write_text(json.dumps({"id": cid, "overrides": overrides})) + + def pick(self, *ids): + self.selection.write_text(json.dumps({"version": 1, "games": {"2571976725": list(ids)}})) + + def apply(self): + return MODULE["apply"](self.compat, "2571976725", self.selection, self.store) + + def test_places_files_before_the_prefix_exists_and_returns_overrides(self): + self.component("openal", ["openal32"], {"syswow64/OpenAL32.dll": b"x86", "system32/OpenAL32.dll": b"x64"}) + self.pick("openal") + names, notes = self.apply() + self.assertEqual(names, ["openal32"]) + self.assertEqual((self.pfx / "syswow64/openal32.dll").read_bytes(), b"x86") + self.assertEqual((self.pfx / "system32/openal32.dll").read_bytes(), b"x64") + self.assertIn("2 file(s) placed", notes[-1]) + self.assertEqual(MODULE["overrides_value"](names + names), "openal32=n,b") + + def test_second_launch_copies_nothing(self): + self.component("openal", ["openal32"], {"syswow64/OpenAL32.dll": b"x86"}) + self.pick("openal") + self.apply() + _, notes = self.apply() + self.assertEqual(notes, []) + + def test_replaces_a_proton_builtin_symlink_and_keeps_the_prefix_spelling(self): + self.component("xaudio", ["xaudio2_7"], {"syswow64/xaudio2_7.dll": b"native"}) + builtin = Path(self.tmp.name) / "proton/lib/wine/i386-windows/xaudio2_7.dll" + builtin.parent.mkdir(parents=True) + builtin.write_bytes(b"builtin") + (self.pfx / "syswow64").mkdir(parents=True) + (self.pfx / "syswow64/XAudio2_7.dll").symlink_to(builtin) + self.pick("xaudio") + self.apply() + placed = self.pfx / "syswow64/XAudio2_7.dll" + self.assertFalse(placed.is_symlink()) + self.assertEqual(placed.read_bytes(), b"native") + self.assertEqual(builtin.read_bytes(), b"builtin") + self.assertEqual(os.listdir(self.pfx / "syswow64"), ["XAudio2_7.dll"]) + + def test_restores_a_file_an_older_proton_deleted(self): + self.component("openal", ["openal32"], {"syswow64/OpenAL32.dll": b"x86"}) + self.pick("openal") + self.apply() + (self.pfx / "syswow64/openal32.dll").unlink() + _, notes = self.apply() + self.assertTrue((self.pfx / "syswow64/openal32.dll").is_file()) + self.assertIn("1 file(s) placed", notes[-1]) + + def test_turning_a_component_off_removes_its_files_and_forgets_protons_config(self): + self.component("openal", ["openal32"], {"syswow64/OpenAL32.dll": b"x86"}) + self.pick("openal") + self.apply() + (self.compat / "config_info").write_text("cached") + self.pick() + names, notes = self.apply() + self.assertEqual(names, []) + self.assertFalse((self.pfx / "syswow64/openal32.dll").exists()) + self.assertFalse((self.compat / "config_info").exists()) + self.assertIn("1 removed", notes[-1]) + + def test_a_file_of_someone_elses_is_kept_and_put_back_when_turned_off(self): + self.component("vc", ["msvcp100"], {"syswow64/msvcp100.dll": b"ours"}) + installer = self.pfx / "syswow64/MSVCP100.dll" + installer.parent.mkdir(parents=True) + installer.write_bytes(b"from a real installer") + self.pick("vc") + self.apply() + self.assertEqual(installer.read_bytes(), b"ours") + self.apply() # a second launch keeps the first backup, not ours + self.pick() + names, notes = self.apply() + self.assertEqual(installer.read_bytes(), b"from a real installer") + self.assertIn("1 put back as they were", notes[-1]) + self.assertFalse((self.compat / ".droiddeck-wincomponents-backup/drive_c/windows/syswow64/msvcp100.dll").exists()) + + def test_protons_own_dlls_and_our_earlier_copy_get_no_backup(self): + self.component("vc", ["msvcp100"], {"syswow64/msvcp100.dll": b"ours v1", "system32/msvcp100.dll": b"ours 64"}) + builtin = self.pfx / "system32/msvcp100.dll" + builtin.parent.mkdir(parents=True) + builtin.write_bytes(b"MZ" + b"\0" * 62 + b"Wine builtin DLL") + self.pick("vc") + self.apply() + (self.store / "vc/syswow64/msvcp100.dll").write_bytes(b"ours v2") + os.utime(self.store / "vc/syswow64/msvcp100.dll", (1, 1)) + self.apply() + self.assertFalse((self.compat / ".droiddeck-wincomponents-backup").exists()) + + def test_missing_component_is_skipped(self): + self.pick("physx") + names, notes = self.apply() + self.assertEqual(names, []) + self.assertEqual(notes, ["physx is not installed; skipped"]) + + def test_no_selection_touches_nothing(self): + self.assertEqual(self.apply(), ([], [])) + self.assertFalse(self.compat.exists()) + + def test_bad_selection_is_refused(self): + self.selection.write_text(json.dumps({"version": 1, "games": {"2571976725": ["a/../etc"]}})) + with self.assertRaises(ValueError): + self.apply() + + def test_game_env_puts_component_overrides_before_the_games_own(self): + self.component("openal", ["openal32"], {"syswow64/OpenAL32.dll": b"x86"}) + self.pick("openal") + globals_ = GAME_ENV["win_components"].__globals__ + real = runpy.run_path(str(BIN / "droiddeck-wincomponents")) + real["apply"].__defaults__ = (self.selection, self.store) + with mock.patch.dict(globals_, {"runpy": mock.Mock(run_path=lambda _path: real)}): + self.assertEqual(GAME_ENV["win_components"](str(self.compat) + "/", "2571976725"), "openal32=n,b") + self.assertEqual(GAME_ENV["win_components"]("", "2571976725"), "") + + def test_bundle_brings_its_parts_and_catalog_names_with_dots_work(self): + self.component("dmband", ["dmband"], {"syswow64/dmband.dll": b"band"}) + self.component("dsound", ["dsound=n"], {"syswow64/dsound.dll": b"ds"}) + meta = self.store / "directmusic" + meta.mkdir(parents=True) + (meta / "component.json").write_text(json.dumps({"id": "directmusic", "overrides": [], "requires": ["dmband", "dsound"]})) + self.component("xaudio2.7", ["xaudio2_7"], {"system32/xaudio2_7.dll": b"xa"}) + self.pick("directmusic", "xaudio2.7") + names, _ = self.apply() + self.assertEqual(MODULE["overrides_value"](names), "dmband=n,b;dsound=n;xaudio2_7=n,b") + self.assertEqual((self.pfx / "syswow64/dmband.dll").read_bytes(), b"band") + self.assertEqual(json.loads((self.compat / ".droiddeck-wincomponents.json").read_text())["components"], + ["dmband", "dsound", "directmusic", "xaudio2.7"]) + + def test_files_one_level_down_are_placed(self): + self.component("gmdls", [], {"system32/drivers/GM.DLS": b"dls"}) + self.pick("gmdls") + self.apply() + self.assertEqual((self.pfx / "system32/drivers/gm.dls").read_bytes(), b"dls") + + def test_overrides_value_last_word_wins_and_refuses_junk(self): + self.assertEqual(MODULE["overrides_value"](["a", "b=n", "a=b", "bad;name", "c=x"]), "b=n;a=b") + + +if __name__ == "__main__": + unittest.main()