From 06e108a72637cd926b2570ee2d77765edd44d206 Mon Sep 17 00:00:00 2001 From: The412Banner <205237651+The412Banner@users.noreply.github.com> Date: Fri, 9 Oct 2026 12:36:05 -0400 Subject: [PATCH] Windows components: lay out installer .exe wrappers without running them (#480) * Windows components: lay out installer .exe wrappers without running them The components page could only take .msi packages apart; most catalog entries ship an installer .exe, so VC++ runtimes, .NET Framework 4.0-4.8 and the .NET Core/5+ runtimes sat under 'Need a Windows installer'. droiddeck-msi-install now opens those wrappers and lays out the packages inside the same way: - WiX bundles (Visual C++ 2012-2022, .NET Core/5+ desktop runtimes): the .wixburn section names the containers, each a cabinet found by its header (signing moves them); payloads get the names the bundle manifest lists, so each package finds its cab1.cab beside it. ARM64 packages are skipped, x86 and x64 both laid out. - Self-extracting cabinets (Visual C++ 2005-2010): cabextract, external cabinet beside the package (vc_red.cab). - Self-extracting 7-Zip archives (.NET Framework 4.0-4.8): 7-Zip, now bundled with the msitools (Arch Linux ARM 7zip 26.04: 7z + its 7z.so plugin, licences beside the others). The complete product (netfx_Full_*) wins over the 4.0 parts and the per-Windows copies. - An .exe that is its own setup program (NSIS, InnoSetup) reports that it holds no package. - --unpack opens an archive for the app: DirectX's redistributable for XACT's cabinet picks. The app executes the remaining catalog steps offline: set_windows (HKCU\Software\Wine Version), set_register_key, get_from_cab (through --unpack) and 'uninstall Wine Mono' (the mscoree override beside it is what matters), and reads url/file_name the catalog nests under 'environment' on a few .NET entries. Setup-program installers stay in the waiting section with a note that says what they are. Verified on this branch's engine with msitools, cabextract and 7z: VC++ 2010 x64 (19 files, 61 values), VC++ 2022 x64 bundle (26 files, 66 values), .NET 4.0 (1053 files, 6056 values, Framework64\v4.0.30319 with clr.dll and mscorlib.dll, NDP\v4 keys) and .NET 4.8 (1633 files, 522 GAC assemblies, 6495 values). Not yet run on a device. * Windows components: the pressed row greys out and shows its own two-step progress bar Pressing a switch now greys that row at once and runs a thin bar across it: the download 0..100, then the install 0..100, with a caption naming the file or the stage (a dependency's name in front while that installs first). Turning a component off greys the row the same way until the pick is saved. The page-wide bar at the bottom is gone; Done waits on the busy row. droiddeck-msi-install reports how far along it is ('progress 37% placing files'): reading, each cabinet, every 5% of files placed, the registry, done - and a wrapper maps each package into its share, so a bundle's two packages run 10..55 and 55..100. The app parses the percent and passes a typed Progress (component, stage, phase, percent) up to the editor. * Windows components: the ones turned on sort to the top of the list In 'All components' the picks for this game come first, in their usual alphabetical order among themselves; turning one off returns it to its place. The row moves once the install has finished and the pick is saved, not while its progress bar is running. * Windows components: an 'Installed for this game' section under the recommendations What is turned on for this game gets its own section between Recommended and All components, alphabetical; the recommendations stay where they are (a recommended pick stays there). A component in the section leaves All components, and goes back to its place when turned off. LB/RB jump through the new section too. --- .github/workflows/build.yml | 7 +- .../launcher/session/SessionFiles.kt | 4 +- .../launcher/session/WinComponents.kt | 216 +++++++++++-- .../launcher/ui/WinComponentsEditor.kt | 109 +++++-- app/src/main/res/values/strings.xml | 8 +- .../session/WinComponentsSupportTest.kt | 44 ++- .../usr/local/bin/droiddeck-msi-install | 306 ++++++++++++++++-- tools/msitools/licenses/7zip-License | 151 +++++++++ tools/msitools/licenses/7zip-unRarLicense | 41 +++ tools/msitools/licenses/NOTICE | 5 +- tools/msitools/packages.txt | 4 +- 11 files changed, 803 insertions(+), 92 deletions(-) create mode 100644 tools/msitools/licenses/7zip-License create mode 100644 tools/msitools/licenses/7zip-unRarLicense diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 80e3497d38..87df262956 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -254,10 +254,13 @@ jobs: done install -m644 msi-pkgs/usr/bin/msiinfo "$T/msiinfo" install -m644 msi-pkgs/usr/bin/cabextract "$T/cabextract" + # 7-Zip (the full build with its format plugin) opens installer .exe wrappers. + install -m644 msi-pkgs/usr/lib/7zip/7z "$T/7z" + install -m644 msi-pkgs/usr/lib/7zip/7z.so "$T/7z.so" for so in libmsi-1.0.so.0 libgsf-1.so.114 libgcab-1.0.so.0; do cp -L "msi-pkgs/usr/lib/$so" "$T/$so" done - for f in NOTICE GPL-2 GPL-3 LGPL-2.1; do + for f in NOTICE GPL-2 GPL-3 LGPL-2.1 7zip-License 7zip-unRarLicense; do install -Dm644 "tools/msitools/licenses/$f" "$D/usr/local/share/licenses/droiddeck-msitools/$f" done rm -rf msi-pkgs @@ -391,7 +394,7 @@ jobs: test -f "$D/usr/local/bin/droiddeck-session" test -f "$D/usr/local/bin/droiddeck-proton-extra" test -f "$D/usr/local/bin/droiddeck-msi-install" - for f in msiinfo cabextract libmsi-1.0.so.0 libgsf-1.so.114 libgcab-1.0.so.0; do + for f in msiinfo cabextract 7z 7z.so libmsi-1.0.so.0 libgsf-1.so.114 libgcab-1.0.so.0; do test -f "$D/usr/local/lib/droiddeck-msitools/$f" || { echo "::error::droiddeck-msitools/$f missing"; exit 1; } done diff --git a/app/src/main/java/com/droiddeck/launcher/session/SessionFiles.kt b/app/src/main/java/com/droiddeck/launcher/session/SessionFiles.kt index e61a83413b..04dc792606 100644 --- a/app/src/main/java/com/droiddeck/launcher/session/SessionFiles.kt +++ b/app/src/main/java/com/droiddeck/launcher/session/SessionFiles.kt @@ -117,9 +117,9 @@ object SessionFiles { "usr/local/lib/mangoapp/libtracefs.so.1", ).map { it to it } // What the Windows components installer reads .msi packages with (tools/msitools). - val msitools = listOf("msiinfo", "cabextract", "libmsi-1.0.so.0", "libgsf-1.so.114", "libgcab-1.0.so.0") + val msitools = listOf("msiinfo", "cabextract", "7z", "7z.so", "libmsi-1.0.so.0", "libgsf-1.so.114", "libgcab-1.0.so.0") .map { "usr/local/lib/droiddeck-msitools/$it" } + - listOf("NOTICE", "GPL-2", "GPL-3", "LGPL-2.1").map { "usr/local/share/licenses/droiddeck-msitools/$it" } + listOf("NOTICE", "GPL-2", "GPL-3", "LGPL-2.1", "7zip-License", "7zip-unRarLicense").map { "usr/local/share/licenses/droiddeck-msitools/$it" } val fexPreloads = listOf("x86_64", "i386").flatMap { arch -> listOf("libblsession.so", "libfakeinput.so").map { "$arch/$it" to "usr/local/lib/droiddeck-fex/$arch/$it" } } + listOf("libfaultreport.so", "libthunkaudit.so", "libvulkan-thunk.so").map { "x86_64/$it" to "usr/local/lib/droiddeck-fex/x86_64/$it" } diff --git a/app/src/main/java/com/droiddeck/launcher/session/WinComponents.kt b/app/src/main/java/com/droiddeck/launcher/session/WinComponents.kt index d344127535..850ef5059c 100644 --- a/app/src/main/java/com/droiddeck/launcher/session/WinComponents.kt +++ b/app/src/main/java/com/droiddeck/launcher/session/WinComponents.kt @@ -47,16 +47,35 @@ object WinComponents { // at the same path, so the native file at that path is the one the registration loads. private val FILE_STEPS = setOf("download_archive", "archive_extract", "copy_dll", "copy_file", "override_dll", "register_dll") private val INSTALLER_STEPS = setOf("install_exe", "install_msi") + /** + * Steps done here without any installer running: a Windows version or registry value for the + * prefix (written by droiddeck-wincomponents with the rest), files picked out of a cabinet, and + * "uninstall Wine Mono", which the mscoree override beside it already does for the game. + */ + private val OFFLINE_STEPS = setOf("set_windows", "set_register_key", "get_from_cab", "uninstall") + /** + * Installers that are their own setup program (NSIS, InnoSetup, IExpress scripts): nothing + * inside them is a package to lay out, so they wait for a way to run them. + */ + private val RUNS_OWN_SETUP = setOf( + "K-Lite", "ffdshow", "lavfilters702", "lavfilters741", "quicktime72", "dirac", "webview2", "aairruntime", + "gfw", "ie8_kb2936068", "art2kmin", "art2k7min", "vcredist6sp6", "VulkanRT", "jet40", "mdac28", "oalinst", + "dotnet20", "dotnet20sp1", "dotnet35", "dotnet35sp1", + ) private const val MSI_INSTALL = "/usr/local/bin/droiddeck-msi-install" - /** Where a downloaded .msi waits for the installer, inside the runtime so it can read it. */ + /** Where a downloaded package or installer waits for the installer, inside the runtime so it can read it. */ private const val MSI_CACHE = "$STORE/.packages" /** Catalog entries Proton already provides newer copies of, which an install would only shadow. */ - private val PROTON_PROVIDES = setOf("gecko") + private val PROTON_PROVIDES = setOf("gecko", "mono") + /** The keys the catalog nests under "environment" on a few .NET entries. */ + private val NESTED_KEYS = setOf("url", "file_name", "file_checksum", "file_size") fun validId(id: String): Boolean = ID.matches(id) && ".." !in id class Step(val action: String, val json: JSONObject) { - fun str(key: String): String = json.optString(key, "") + fun str(key: String): String = json.optString(key, "").ifEmpty { + if (key in NESTED_KEYS) json.optJSONObject("environment")?.optString(key, "").orEmpty() else "" + } } class Component( @@ -66,10 +85,27 @@ object WinComponents { enum class Support { READY, NEEDS_INSTALLER, UNSUPPORTED } - /** A step that installs a Windows Installer package, which droiddeck-msi-install does here. */ - private fun isMsi(step: Step): Boolean = step.action in INSTALLER_STEPS && step.str("url").startsWith("https://") && + /** The two halves of an install the page shows one after the other, each 0..100. */ + enum class Phase { DOWNLOAD, INSTALL } + + /** Progress: the component being worked on, what it is doing, the phase, 0..100 or -1 when unknown. */ + class Progress(val component: String, val stage: String, val phase: Phase, val percent: Int) + + /** The percent a droiddeck-msi-install progress line carries ("progress 37% placing files"), or -1. */ + private val PERCENT_LINE = Regex("""^(\d{1,3})% (.*)$""") + private fun engineLine(text: String): Pair = + PERCENT_LINE.find(text)?.let { it.groupValues[2] to it.groupValues[1].toInt().coerceIn(0, 100) } ?: (text to -1) + + /** + * A step that installs a Windows Installer package, or an installer .exe that is a wrapper + * around packages (a WiX bundle, a self-extracting cabinet or 7-Zip archive), which + * droiddeck-msi-install lays out here without running it. + */ + private fun isPackage(step: Step): Boolean = step.action in INSTALLER_STEPS && step.str("url").startsWith("https://") && (step.action == "install_msi" || listOf(step.str("file_name"), step.str("url").substringBefore('?')) - .any { it.endsWith(".msi", ignoreCase = true) }) + .any { it.endsWith(".msi", ignoreCase = true) || it.endsWith(".exe", ignoreCase = true) }) + + private fun protonProvides(name: String): Boolean = name in PROTON_PROVIDES || name.startsWith("mono-") fun fetch(): List? { val body = Downloader.downloadString(CATALOG_URL) ?: return null @@ -94,13 +130,14 @@ object WinComponents { /** Whether [c] can be installed here, its bundled components included. */ fun support(c: Component, all: Map, depth: Int = 0): Support { - if (c.status != "ready" || depth > 8 || c.name in PROTON_PROVIDES) return Support.UNSUPPORTED + if (c.status != "ready" || depth > 8 || protonProvides(c.name)) return Support.UNSUPPORTED val installers = c.steps.filter { it.action in INSTALLER_STEPS } + fun offline(step: Step) = step.action == "delete_dlls" || step.action in OFFLINE_STEPS || step.action in FILE_STEPS && fileStepOk(step) val own = when { - installers.isNotEmpty() && installers.all { isMsi(it) } && - c.steps.all { it in installers || it.action == "delete_dlls" || it.action in FILE_STEPS && fileStepOk(it) } -> Support.READY + installers.isNotEmpty() && c.name !in RUNS_OWN_SETUP && installers.all { isPackage(it) } && + c.steps.all { it in installers || offline(it) } -> Support.READY installers.isNotEmpty() -> Support.NEEDS_INSTALLER - c.steps.all { it.action in FILE_STEPS && fileStepOk(it) } -> Support.READY + c.steps.all { offline(it) } -> Support.READY else -> Support.UNSUPPORTED } if (own != Support.READY) return own @@ -109,9 +146,10 @@ object WinComponents { } private fun fileStepOk(step: Step): Boolean = when (step.action) { + // An .exe or .cab archive (DirectX's redistributable) is opened by the runtime's tools. "download_archive", "archive_extract" -> step.str("url").let { url -> !url.startsWith("http") || url.startsWith("https://github.com/") && - url.substringBefore('?').let { it.endsWith(".tar.xz") || it.endsWith(".zip") } + url.substringBefore('?').lowercase().let { it.endsWith(".tar.xz") || it.endsWith(".zip") || it.endsWith(".exe") || it.endsWith(".cab") } } "copy_dll", "copy_file" -> ".." !in step.str("dest") else -> true @@ -131,7 +169,7 @@ object WinComponents { * Installs [c] and the components it bundles that are not installed yet. [onProgress] gets a * line for the step and 0..100 (or -1). Returns null on success, else a message. */ - fun install(context: Context, c: Component, all: Map, onProgress: (String, Int) -> Unit): String? { + fun install(context: Context, c: Component, all: Map, onProgress: (Progress) -> Unit): String? { if (support(c, all) != Support.READY) return "${c.name} cannot be installed here yet" val order = ArrayList() fun visit(x: Component, depth: Int) { @@ -146,7 +184,7 @@ object WinComponents { val archives = HashMap() return try { for (x in order) { - onProgress(x.name, -1) + onProgress(Progress(x.name, x.name, Phase.DOWNLOAD, -1)) installOne(context, x, work, archives, onProgress)?.let { return "${x.name}: $it" } } null @@ -158,18 +196,27 @@ object WinComponents { } } - private fun installOne(context: Context, c: Component, work: File, archives: HashMap, onProgress: (String, Int) -> Unit): String? { + private fun installOne(context: Context, c: Component, work: File, archives: HashMap, onProgress: (Progress) -> Unit): String? { val root = LinuxRuntime.rootDir(context) if (!root.isDirectory) return "The Linux runtime is not installed" val staging = File(root, "$STORE/.${c.name}.new").apply { FileUtils.delete(this); mkdirs() } val overrides = ArrayList() var source: File? = null var msi: JSONObject? = null + // The registry values the steps themselves set (a Windows version, a key), beside the packages'. + val registry = ArrayList() + // Archives the runtime's tools opened (installer .exe, cabinets), by the name the catalog + // calls them, and the folders they were opened into; cleaned up at the end. + val runtimeArchives = HashMap() + val runtimeDirs = ArrayList() + val cabs = HashMap() + val temp = File(root, "$MSI_CACHE/.t-${c.name}").apply { FileUtils.delete(this) } + try { for (step in c.steps) when (step.action) { // A component may have several packages (PowerShell's 32- and 64-bit): each adds to the // same folder, the first one names the component. "install_msi", "install_exe" -> { - val (result, problem) = installMsi(context, c, step, staging, onProgress) + val (result, problem) = installPackage(context, c, step, staging, onProgress) if (result == null) return problem msi = msi?.apply { val notes = optJSONArray("notes") ?: JSONArray().also { put("notes", it) } @@ -183,19 +230,84 @@ object WinComponents { if (!url.startsWith("http")) continue source = archives[url] ?: run { val name = url.substringBefore('?').substringAfterLast('/') - val file = File(work, "${archives.size}-$name") - val ok = Downloader.downloadFile(url, file, false) { f -> onProgress("${c.name}: $name", if (f < 0) -1 else Math.round(f * 100f)) } + // A zip or tar.xz is opened here; an .exe or .cab by the runtime's tools, so it + // is downloaded where the runtime can read it and kept for get_from_cab steps. + val byRuntime = name.lowercase().let { it.endsWith(".exe") || it.endsWith(".cab") } + val file = if (byRuntime) File(root, "$MSI_CACHE/${archives.size}-$name").apply { parentFile?.mkdirs() } + else File(work, "${archives.size}-$name") + val ok = Downloader.downloadFile(url, file, false) { f -> onProgress(Progress(c.name, name, Phase.DOWNLOAD, if (f < 0) -1 else Math.round(f * 100f))) } if (!ok) return "download failed: $name" step.str("file_checksum").takeIf { it.length == 32 }?.let { md5 -> if (!digest(file, "MD5").equals(md5, ignoreCase = true)) return "checksum mismatch: $name" } - onProgress("${c.name}: unpacking $name", -1) - val dir = File(work, "${archives.size}-x").apply { mkdirs() } - if (!extract(file, dir)) return "could not unpack $name" - file.delete() + onProgress(Progress(c.name, "unpacking $name", Phase.INSTALL, -1)) + val dir = if (byRuntime) { + runtimeArchives[step.str("file_name").ifEmpty { name }] = file + runtimeDirs += file + unpackInRuntime(context, c, file, "${archives.size}-x", onProgress)?.also { runtimeDirs += it } + ?: return "could not unpack $name" + } else { + File(work, "${archives.size}-x").apply { mkdirs() }.also { if (!extract(file, it)) return "could not unpack $name" } + .also { file.delete() } + } dir.also { archives[url] = it } } } + // Files picked out of a cabinet: one the catalog downloaded (DirectX's redistributable, + // a self-extracting cabinet), or cabinets an earlier pick put under temp/. + "get_from_cab" -> { + val pattern = step.str("source") + val containers = runtimeArchives[pattern]?.let { listOf(it) } ?: run { + val folder = File(temp, pattern.substringBeforeLast('/', "").replace('\\', '/')) + val rx = globRegex(pattern.substringAfterLast('/')) + folder.listFiles { f -> f.isFile && rx.matches(f.name) }?.sortedBy { it.name }.orEmpty() + } + if (containers.isEmpty()) return "nothing is called $pattern" + val dest = step.str("dest").replace('\\', '/') + for (container in containers) { + val unpacked = cabs.getOrPut(container.path) { + unpackInRuntime(context, c, container, "c${cabs.size}", onProgress)?.also { runtimeDirs += it } + ?: return "could not open ${container.name}" + } + if (dest.startsWith("temp/", ignoreCase = true)) { + copyMatching(unpacked, step.str("file_name"), File(temp, dest.substring(5).trimEnd('/')), null) + } else { + val (arch, base) = when (dest.substringBefore('/').lowercase()) { + "win64", "system32" -> "win64" to "system32" + "win32", "syswow64" -> "win32" to "syswow64" + else -> null to "system32" + } + val sub = dest.substringAfter('/', "").trimEnd('/') + copyMatching(unpacked, step.str("file_name"), File(staging, if (sub.isEmpty()) base else "$base/$sub"), arch) + } + } + } + // The Windows version a game sees: HKCU\Software\Wine "Version", the last word wins. + "set_windows" -> step.str("version").takeIf { it.isNotEmpty() }?.let { version -> + registry.removeAll { it.optString("key") == "Software\\Wine" && it.optString("name") == "Version" } + registry += JSONObject().put("hive", "HKCU").put("key", "Software\\Wine").put("name", "Version").put("type", "sz").put("data", version) + } + "set_register_key" -> { + val full = step.str("key").replace("\\\\", "\\").trim('\\') + val hive = when (full.substringBefore('\\').uppercase()) { + "HKLM", "HKEY_LOCAL_MACHINE" -> "HKLM" + "HKCU", "HKEY_CURRENT_USER" -> "HKCU" + else -> return "a registry key outside HKLM and HKCU: $full" + } + val key = full.substringAfter('\\', "") + if (key.isEmpty()) return "a registry value without a key: $full" + val data = step.str("data") + val value = JSONObject().put("hive", hive).put("key", key).put("name", step.str("value")) + when (step.str("type").uppercase()) { + "REG_DWORD" -> value.put("type", "dword").put("data", data.trim().let { it.removePrefix("0x").toLongOrNull(if (it.startsWith("0x")) 16 else 10) ?: 0L }) + "REG_EXPAND_SZ" -> value.put("type", "expand_sz").put("data", data) + else -> value.put("type", "sz").put("data", data) + } + registry += value + } + // "Uninstall Wine Mono" before a real .NET Framework: the mscoree override the catalog + // sets beside it is what takes Mono out of the game's way; nothing to remove here. + "uninstall" -> Unit "copy_dll", "copy_file" -> { val from = source ?: return "a copy step before any archive" val dest = step.str("dest") @@ -226,6 +338,17 @@ object WinComponents { "register_dll" -> Unit else -> return "unsupported step ${step.action}" } + } finally { + runtimeDirs.forEach { FileUtils.delete(it) } + FileUtils.delete(temp) + } + if (registry.isNotEmpty()) { + // The packages' values are already in registry.json; the steps' own go after them. + val file = File(staging, "registry.json") + val values = runCatching { JSONObject(file.readText()).optJSONArray("values") }.getOrNull() ?: JSONArray() + registry.forEach { values.put(it) } + FileUtils.writeString(file, JSONObject().put("version", 1).put("values", values).toString(1)) + } val version = msi?.optString("version")?.takeIf { it.isNotEmpty() } ?: Regex("""\b(\d+\.\d+(\.\d+)*)\b""").find(c.description)?.value ?: "catalog" val meta = JSONObject() @@ -240,31 +363,64 @@ object WinComponents { return null } + /** The regex for a catalog file pattern: * matches anything, case does not matter. */ + private fun globRegex(pattern: String) = + Regex("^" + pattern.split("*").joinToString(".*") { Regex.escape(it) } + "$", RegexOption.IGNORE_CASE) + /** - * Downloads the component's .msi into the runtime and has droiddeck-msi-install lay it out in - * [staging]. Returns the installer's result (product, version, counts, notes), or null and why. - * Everything the installer prints also goes to Download/DroidDeck/tools. + * Has the runtime's tools open [archive] (an installer .exe, a cabinet, a self-extracting + * one) into a folder of the package cache called [tag]; that folder, or null and a log line. */ - private fun installMsi(context: Context, c: Component, step: Step, staging: File, onProgress: (String, Int) -> Unit): Pair { + private fun unpackInRuntime(context: Context, c: Component, archive: File, tag: String, onProgress: (Progress) -> Unit): File? { + val root = LinuxRuntime.rootDir(context) + val cache = File(root, MSI_CACHE) + val dir = File(cache, ".x-$tag").apply { FileUtils.delete(this); mkdirs() } + val inside = archive.relativeTo(root).path + var problem: String? = null + var ok = false + val status = GuestCommand.run(context, listOf(MSI_INSTALL, "--unpack", "/$inside", "/$MSI_CACHE/${dir.name}"), + logName = "wincomponents-${c.name}") { line -> + when { + line.startsWith("progress ") -> engineLine(line.removePrefix("progress ")).let { (text, _) -> onProgress(Progress(c.name, text, Phase.INSTALL, -1)) } + line.startsWith("result ") -> ok = true + line.startsWith("error ") -> problem = line.removePrefix("error ") + } + } + if (status != 0 || !ok) { + Log.w(TAG, "unpack ${archive.name}: ${problem ?: "status $status"}") + FileUtils.delete(dir) + return null + } + return dir + } + + /** + * Downloads the component's package (.msi) or installer (.exe) into the runtime and has + * droiddeck-msi-install lay it out in [staging]. Returns the installer's result (product, + * version, counts, notes), or null and why. Everything the installer prints also goes to + * Download/DroidDeck/tools. + */ + private fun installPackage(context: Context, c: Component, step: Step, staging: File, onProgress: (Progress) -> Unit): Pair { val root = LinuxRuntime.rootDir(context) val url = step.str("url") val name = url.substringBefore('?').substringAfterLast('/') val cache = File(root, MSI_CACHE).apply { mkdirs() } - val file = File(cache, "${c.name}.msi") + val ext = if (listOf(step.str("file_name"), name).any { it.endsWith(".exe", ignoreCase = true) }) "exe" else "msi" + val file = File(cache, "${c.name}-${c.steps.indexOf(step)}.$ext") try { - if (!Downloader.downloadFile(url, file, false) { f -> onProgress("${c.name}: $name", if (f < 0) -1 else Math.round(f * 100f)) }) { + if (!Downloader.downloadFile(url, file, false) { f -> onProgress(Progress(c.name, name, Phase.DOWNLOAD, if (f < 0) -1 else Math.round(f * 100f))) }) { return null to "download failed: $name" } step.str("file_checksum").takeIf { it.length == 32 }?.let { md5 -> if (!digest(file, "MD5").equals(md5, ignoreCase = true)) return null to "checksum mismatch: $name" } - onProgress("${c.name}: installing", -1) + onProgress(Progress(c.name, "installing", Phase.INSTALL, 0)) var result: JSONObject? = null var problem: String? = null val status = GuestCommand.run(context, listOf(MSI_INSTALL, "/$MSI_CACHE/${file.name}", "/$STORE/${staging.name}"), logName = "wincomponents-${c.name}") { line -> when { - line.startsWith("progress ") -> onProgress("${c.name}: ${line.removePrefix("progress ")}", -1) + line.startsWith("progress ") -> engineLine(line.removePrefix("progress ")).let { (text, pct) -> onProgress(Progress(c.name, text, Phase.INSTALL, pct)) } line.startsWith("result ") -> result = runCatching { JSONObject(line.removePrefix("result ")) }.getOrNull() line.startsWith("error ") -> problem = line.removePrefix("error ") } @@ -273,7 +429,7 @@ object WinComponents { if (status != 0 || done == null) return null to (problem ?: "the installer stopped (status $status)") return done to "" } catch (e: Exception) { - Log.e(TAG, "msi ${c.name}", e) + Log.e(TAG, "package ${c.name}", e) return null to (e.message ?: "the installer failed") } finally { file.delete() diff --git a/app/src/main/java/com/droiddeck/launcher/ui/WinComponentsEditor.kt b/app/src/main/java/com/droiddeck/launcher/ui/WinComponentsEditor.kt index 9538f345e7..25e9539277 100644 --- a/app/src/main/java/com/droiddeck/launcher/ui/WinComponentsEditor.kt +++ b/app/src/main/java/com/droiddeck/launcher/ui/WinComponentsEditor.kt @@ -82,7 +82,12 @@ internal fun WinComponentsDialog( // Detector names (oalinst, physx...) already in the prefix from elsewhere. var present by remember { mutableStateOf(emptySet()) } var picks by remember { mutableStateOf(WinComponents.picks(context, appKey)) } - var progress by remember { mutableStateOf?>(null) } + // The row being worked on: its id, the latest progress (a dependency's while that installs + // first), and whether it is turning off rather than on. The row greys out and shows the bar + // the moment the switch is pressed, and only that row; the rest of the page stays as it is. + var busyId by remember { mutableStateOf(null) } + var progress by remember { mutableStateOf(null) } + var reverting by remember { mutableStateOf(false) } var error by remember { mutableStateOf(null) } // The component the error is about, so it shows on its own row and not only at the bottom. var failed by remember { mutableStateOf(null) } @@ -91,6 +96,7 @@ internal fun WinComponentsDialog( // pad, so the d-pad always has somewhere to start; LB and RB jump between the sections, since // the full list runs past sixty switches. val recFocus = remember { FocusRequester() } + val activeFocus = remember { FocusRequester() } val allFocus = remember { FocusRequester() } val waitFocus = remember { FocusRequester() } val doneFocus = remember { FocusRequester() } @@ -116,22 +122,35 @@ internal fun WinComponentsDialog( } fun toggle(id: String, on: Boolean) { // Switches stay enabled during a download - a disabled one drops the pad's focus - so a press - // then is ignored here. - if (progress != null) return + // then is ignored here; the busy row is greyed instead. + if (busyId != null) return error = null failed = null - if (!on) { setPicks(picks - id); return } + if (!on) { + // Turning off is a saved pick, quick, but the row greys out until it is written so the + // press is seen to land; the files leave the prefix at the game's next launch. + busyId = id; reverting = true; progress = null + val next = picks - id + picks = next + coroutine.launch { + withContext(Dispatchers.IO) { WinComponents.setPicks(context, appKey, next) } + busyId = null; reverting = false + } + return + } val all = catalog.orEmpty() val c = all[id] if (id in installed || c == null) { setPicks(picks + id); return } - progress = id to -1 + busyId = id; reverting = false + progress = WinComponents.Progress(id, "", WinComponents.Phase.DOWNLOAD, -1) coroutine.launch { val problem = withContext(Dispatchers.IO) { - WinComponents.install(context, c, all) { stage, percent -> progress = stage to percent } + WinComponents.install(context, c, all) { p -> progress = p } } progress = null installed = withContext(Dispatchers.IO) { WinComponents.installedIds(context).toSet() } if (problem != null) { error = problem; failed = id } else setPicks(picks + id) + busyId = null } } @@ -142,12 +161,18 @@ internal fun WinComponentsDialog( val recIds = if (all == null) emptyList() else recommended.distinctBy { installable(it.componentName, all) } val ready = all.orEmpty().values.filter { WinComponents.support(it, all.orEmpty()) == Support.READY }.map { it.name } val extra = (installed + picks).filter { all?.containsKey(it) != true } - val list = (ready + extra).distinct().sortedBy { WinComponentNames.of(it).lowercase() } + // What is turned on for this game gets its own section under the recommendations, which stay + // where they are; the rest of the list is everything else, so a component turned off goes back + // to its place in the alphabet. + val recSet = recIds.map { installable(it.componentName, all.orEmpty()) }.toSet() + val active = picks.filter { it !in recSet && (all?.containsKey(it) == true || it in installed) }.distinct() + .sortedBy { WinComponentNames.of(it).lowercase() } + val list = (ready + extra).distinct().filter { it !in active }.sortedBy { WinComponentNames.of(it).lowercase() } val waiting = all.orEmpty().values.filter { WinComponents.support(it, all.orEmpty()) == Support.NEEDS_INSTALLER } .map { it.name }.sortedBy { WinComponentNames.of(it).lowercase() } val hasRec = all != null && recIds.any { supportOf(installable(it.componentName, all)) == Support.READY } - val sections = listOfNotNull(recFocus.takeIf { hasRec }, allFocus.takeIf { list.isNotEmpty() }, - waitFocus.takeIf { waiting.isNotEmpty() }, doneFocus) + val sections = listOfNotNull(recFocus.takeIf { hasRec }, activeFocus.takeIf { active.isNotEmpty() }, + allFocus.takeIf { list.isNotEmpty() }, waitFocus.takeIf { waiting.isNotEmpty() }, doneFocus) var section by remember { mutableStateOf(0) } fun jump(step: Int) { section = (section + step + sections.size) % sections.size @@ -164,7 +189,7 @@ internal fun WinComponentsDialog( withFrameNanos { } runCatching { target.requestFocus() } withFrameNanos { } - if (switchFocused || (target !== recFocus && target !== allFocus)) return@LaunchedEffect + if (switchFocused || (target !== recFocus && target !== activeFocus && target !== allFocus)) return@LaunchedEffect } } @@ -189,9 +214,24 @@ internal fun WinComponentsDialog( // The catalog key stays in the detail: it is what a log or a bug report names. val detail = listOfNotNull(reason, c?.description?.takeIf { it.isNotEmpty() }, status, id).joinToString(" · ") val usable = support == Support.READY + val busy = id == busyId + val caption = when { + !busy -> null + reverting -> stringResource(R.string.wincomp_reverting) + else -> progress?.let { p -> + val stage = if (p.component != id && p.stage.isNotEmpty()) "${WinComponentNames.of(p.component)}: ${p.stage}" else p.stage + val text = when (p.phase) { + WinComponents.Phase.DOWNLOAD -> stringResource(R.string.wincomp_downloading, stage) + WinComponents.Phase.INSTALL -> stringResource(R.string.wincomp_installing, stage) + } + if (p.percent >= 0) "$text · ${p.percent}%" else text + } + } ComponentRow( WinComponentNames.of(id), detail, checked = id in picks, enabled = usable || id in picks, - dim = !usable, modifier = (if (usable && focus != null) Modifier.focusRequester(focus) else Modifier) + dim = !usable || busy, busy = busy, caption = caption, + percent = if (busy && !reverting) progress?.percent ?: -1 else -1, + modifier = (if (usable && focus != null) Modifier.focusRequester(focus) else Modifier) .onFocusChanged { if (it.isFocused) switchFocused = true }, ) { on -> toggle(id, on) } } @@ -220,6 +260,11 @@ internal fun WinComponentsDialog( } } else if (all != null && gameDir != null) Small(stringResource(R.string.wincomp_no_recommendation)) + if (active.isNotEmpty()) { + Section(stringResource(R.string.wincomp_active, active.size)) + val focusOf = firstUsable(active, activeFocus) + Panel { active.forEachIndexed { i, id -> if (i > 0) Divider(); Item(id, null, focusOf(id)) } } + } if (list.isNotEmpty()) { Section(stringResource(R.string.wincomp_all, list.size)) val focusOf = firstUsable(list, allFocus) @@ -238,16 +283,11 @@ internal fun WinComponentsDialog( if (showWaiting) Panel { waiting.forEachIndexed { i, id -> if (i > 0) Divider(); Item(id, null, null) } } } - progress?.let { (stage, percent) -> - Small(if (percent >= 0) "$stage · $percent%" else stage) - if (percent >= 0) LinearProgressIndicator(progress = { percent / 100f }, modifier = Modifier.fillMaxWidth()) - else LinearProgressIndicator(Modifier.fillMaxWidth()) - } error?.let { Small(it, error = true) } Row(verticalAlignment = Alignment.CenterVertically, modifier = Modifier.padding(top = 4.dp)) { Small(stringResource(R.string.wincomp_next_launch)) Spacer(Modifier.weight(1f)) - PrimaryButton(stringResource(R.string.game_env_done), enabled = progress == null, modifier = Modifier.focusRequester(doneFocus), onClick = close) + PrimaryButton(stringResource(R.string.game_env_done), enabled = busyId == null, modifier = Modifier.focusRequester(doneFocus), onClick = close) } } } @@ -264,17 +304,34 @@ private fun Panel(content: @Composable androidx.compose.foundation.layout.Column private fun Divider() = Box(Modifier.fillMaxWidth().padding(horizontal = 14.dp).heightIn(min = 1.dp, max = 1.dp).background(LocalPalette.current.line)) +/** + * One component: name, detail and its switch; while [busy] the row is greyed and a thin bar runs + * across it with [caption] above (the download, then the install, each 0..100 as [percent], or + * indeterminate at -1). The switch stays enabled while busy so a pad's focus does not drop; the + * press is ignored by the caller instead. + */ @Composable -private fun ComponentRow(name: String, detail: String, checked: Boolean, enabled: Boolean, dim: Boolean, modifier: Modifier, onChange: (Boolean) -> Unit) { +private fun ComponentRow( + name: String, detail: String, checked: Boolean, enabled: Boolean, dim: Boolean, busy: Boolean = false, + caption: String? = null, percent: Int = -1, modifier: Modifier, onChange: (Boolean) -> Unit, +) { val colors = MaterialTheme.colorScheme - Row( - verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(12.dp), - modifier = Modifier.fillMaxWidth().padding(horizontal = 14.dp, vertical = 10.dp).alpha(if (dim) 0.55f else 1f), - ) { - Column(Modifier.weight(1f)) { - Text(name, fontSize = 15.sp, fontWeight = FontWeight.SemiBold, color = colors.onBackground) - if (detail.isNotEmpty()) Text(detail, fontSize = 12.sp, lineHeight = 16.sp, color = colors.onSurfaceVariant) + Column(Modifier.fillMaxWidth().padding(horizontal = 14.dp, vertical = 10.dp)) { + Row( + verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(12.dp), + modifier = Modifier.fillMaxWidth().alpha(if (dim) 0.55f else 1f), + ) { + Column(Modifier.weight(1f)) { + Text(name, fontSize = 15.sp, fontWeight = FontWeight.SemiBold, color = colors.onBackground) + if (detail.isNotEmpty()) Text(detail, fontSize = 12.sp, lineHeight = 16.sp, color = colors.onSurfaceVariant) + } + ToggleSwitch(checked, enabled, name, modifier.alpha(if (busy) 0.5f else 1f), onChange) + } + if (busy) { + if (!caption.isNullOrEmpty()) Text(caption, fontSize = 12.sp, lineHeight = 16.sp, color = colors.onSurfaceVariant, modifier = Modifier.padding(top = 4.dp)) + val bar = Modifier.fillMaxWidth().padding(top = 6.dp).heightIn(min = 4.dp, max = 4.dp) + if (percent >= 0) LinearProgressIndicator(progress = { percent / 100f }, modifier = bar) + else LinearProgressIndicator(modifier = bar) } - ToggleSwitch(checked, enabled, name, modifier, onChange) } } diff --git a/app/src/main/res/values/strings.xml b/app/src/main/res/values/strings.xml index b12ca68dba..bc495bc9cf 100644 --- a/app/src/main/res/values/strings.xml +++ b/app/src/main/res/values/strings.xml @@ -326,16 +326,20 @@ Steam installs %1$s with this game Already in this game Nothing in the game\'s folder points at a component. + Installed for this game (%1$d) All components (%1$d) Need a Windows installer (%1$d) - These run a real installer inside the prefix (.NET, PhysX, XNA, the full Visual C++ installers…). DroidDeck cannot do that yet; it is coming next. + These are their own setup programs (codec packs, browsers, old runtimes) with no package inside to take apart. DroidDeck cannot run a Windows setup program yet; it is coming next. Show Hide - Needs a Windows installer (coming) + Runs its own setup program (coming) Not supported here yet Loading the components… Could not install: %1$s Downloaded + Downloading %1$s + Installing: %1$s + Turning off… %1$s download Takes effect the next time the game starts. Stretch 16:9 games diff --git a/app/src/test/java/com/droiddeck/launcher/session/WinComponentsSupportTest.kt b/app/src/test/java/com/droiddeck/launcher/session/WinComponentsSupportTest.kt index 2e43be1ce9..12a7c84441 100644 --- a/app/src/test/java/com/droiddeck/launcher/session/WinComponentsSupportTest.kt +++ b/app/src/test/java/com/droiddeck/launcher/session/WinComponentsSupportTest.kt @@ -32,15 +32,49 @@ class WinComponentsSupportTest { step("install_msi", "url" to "$base/ps-x86.msi"), step("install_msi", "url" to "$base/ps-x64.msi")))) } - @Test fun otherInstallersStillWait() { - assertEquals(Support.NEEDS_INSTALLER, support(component("dotnet48", step("install_exe", "url" to "$base/ndp48.exe")))) - // An uninstall step first (wine-mono) is not something done here. - assertEquals(Support.NEEDS_INSTALLER, support(component("mono", step("uninstall"), - step("install_msi", "url" to "$base/mono.msi")))) + @Test fun installerWrappersInstallHere() { + // A WiX bundle, a self-extracting cabinet or 7-Zip archive: the packages inside are laid out. + assertEquals(Support.READY, support(component("vcredist2022", + step("install_exe", "url" to "$base/vcredist2022__VC_redist.x86.exe", "file_name" to "VC_redist.x86.exe"), + step("install_exe", "url" to "$base/vcredist2022__VC_redist.x64.exe", "file_name" to "VC_redist.x64.exe"), + step("override_dll", "dll" to "vcruntime140", "type" to "native,builtin")))) + // .NET Framework: "uninstall Wine Mono", the Windows version, the installer, a registry key. + assertEquals(Support.READY, support(component("dotnet40", + step("uninstall", "file_name" to "Wine Mono"), step("set_windows", "version" to "win7"), + step("install_exe", "url" to "https://download.microsoft.com/x/dotNetFx40_Full_x86_x64.exe", "file_name" to "dotNetFx40_Full_x86_x64.exe"), + step("set_windows", "version" to "win10"), + step("set_register_key", "key" to "HKLM\\\\Software\\\\Microsoft\\\\NET Framework Setup\\\\NDP\\\\v4\\\\Full", "value" to "Install", "data" to "0001", "type" to "REG_DWORD"), + step("override_dll", "dll" to "mscoree", "type" to "native")))) + } + + @Test fun nestedCatalogKeysAreRead() { + // A few .NET entries keep url and file_name under "environment". + val nested = WinComponents.Step("install_exe", JSONObject().put("environment", JSONObject() + .put("WINEDLLOVERRIDES", "fusion=b").put("file_name", "ndp48-x86-x64-allos-enu.exe") + .put("url", "https://download.visualstudio.microsoft.com/x/ndp48-x86-x64-allos-enu.exe"))) + assertEquals("ndp48-x86-x64-allos-enu.exe", nested.str("file_name")) + assertEquals(Support.READY, support(component("dotnet48", nested))) + } + + @Test fun cabinetPicksInstallHere() { + assertEquals(Support.READY, support(component("xact", + step("download_archive", "url" to "$base/directx_Jun2010_redist.exe", "file_name" to "directx_Jun2010_redist.exe"), + step("get_from_cab", "source" to "directx_Jun2010_redist.exe", "file_name" to "*XACT_x86*.cab", "dest" to "temp/XACT_x86/"), + step("get_from_cab", "source" to "XACT_x86/*.cab", "file_name" to "xactengine*.dll", "dest" to "win32/"), + step("override_dll", "dll" to "xactengine3_7", "type" to "native,builtin"), + step("register_dll", "dlls" to "")))) + } + + @Test fun setupProgramsStillWait() { + // NSIS / InnoSetup installers hold no package to lay out. + assertEquals(Support.NEEDS_INSTALLER, support(component("K-Lite", + step("install_exe", "url" to "$base/K-Lite.exe", "file_name" to "K-Lite_1960.exe")))) assertEquals(Support.NEEDS_INSTALLER, support(component("plain", step("install_msi", "url" to "http://example.com/a.msi")))) } @Test fun whatProtonShipsIsNotOffered() { assertEquals(Support.UNSUPPORTED, support(component("gecko", step("install_msi", "url" to "$base/gecko.msi")))) + assertEquals(Support.UNSUPPORTED, support(component("mono", step("uninstall"), step("install_msi", "url" to "$base/mono.msi")))) + assertEquals(Support.UNSUPPORTED, support(component("mono-10.4.1", step("install_msi", "url" to "$base/mono.msi")))) } } diff --git a/tools/linuxfs/overlay/usr/local/bin/droiddeck-msi-install b/tools/linuxfs/overlay/usr/local/bin/droiddeck-msi-install index 3df6861736..98b1af030e 100755 --- a/tools/linuxfs/overlay/usr/local/bin/droiddeck-msi-install +++ b/tools/linuxfs/overlay/usr/local/bin/droiddeck-msi-install @@ -1,7 +1,8 @@ #!/usr/bin/python3 """Installs a Windows Installer package (.msi) the way Wine's own msiexec would, without running it. -droiddeck-msi-install +droiddeck-msi-install +droiddeck-msi-install --unpack A .msi is a database: which files go to which folder, which registry values to write, which .NET and side-by-side assemblies to register. This reads those tables (msiinfo), unpacks the package's @@ -11,15 +12,26 @@ for droiddeck-wincomponents, which puts it into a game's prefix at launch: /drive_c/... the files, at the paths they get on 64-bit Windows /registry.json the registry values, in the view (32/64-bit) Windows uses +An installer .exe is opened the same way when it is a wrapper around packages: a WiX bundle +(Visual C++ 2012 and later, the .NET Core and 5+ runtimes: the packages sit in a cabinet the +bundle's own header locates, named in its manifest), a self-extracting cabinet (Visual C++ 2005 +to 2010) or a self-extracting 7-Zip archive (.NET Framework 4.0 to 4.8). The packages inside are +laid out one after the other into the same component dir; the 32- and 64-bit ones both, an ARM64 +one never. An .exe that is its own setup program (NSIS, InnoSetup) holds no package: an error says so. + What a package does by running its own code (custom actions such as DXSETUP, self-registering DLLs) cannot be done this way; each is named in the notes, so nothing is left out silently. +--unpack opens an archive or installer for the caller to take files out of (cabextract for a +cabinet or self-extracting cabinet, 7-Zip for the rest) and prints the files it found. + Prints "progress " lines while it works and ends with "result ": the counts and notes. """ import json import os import re import shutil +import struct import subprocess import sys import tempfile @@ -28,6 +40,11 @@ from pathlib import Path TOOLS = Path(os.environ.get("DROIDDECK_MSITOOLS", "/usr/local/lib/droiddeck-msitools")) MSIINFO = os.environ.get("DROIDDECK_MSIINFO", str(TOOLS / "msiinfo")) CABEXTRACT = os.environ.get("DROIDDECK_CABEXTRACT", str(TOOLS / "cabextract")) +SEVENZIP = os.environ.get("DROIDDECK_7Z", str(TOOLS / "7z")) +MSI_MAGIC = b"\xd0\xcf\x11\xe0\xa1\xb1\x1a\xe1" +BURN_MAGIC = 0x00F14300 +# A bundle's manifest names ARM64 packages too; a game's prefix wants the x86 and x64 ones. +SKIP_PLATFORMS = ("arm64", "arm") def tool_env(): @@ -89,8 +106,21 @@ class MsiError(Exception): pass -def progress(text): - print("progress " + text, flush=True) +def progress(text, fraction=None): + """A progress line: "progress ", or "progress NN% " when how far along is known.""" + if fraction is None: + print("progress " + text, flush=True) + else: + print("progress %d%% %s" % (max(0, min(100, int(fraction * 100))), text), flush=True) + + +class Span: + """The slice of the whole job one package's progress maps into (a wrapper runs several).""" + def __init__(self, lo=0.0, hi=1.0): + self.lo, self.hi = lo, hi + + def report(self, fraction, text): + progress(text, self.lo + (self.hi - self.lo) * max(0.0, min(1.0, fraction))) # ---------------------------------------------------------------- reading the package @@ -339,9 +369,10 @@ def clr_version(path): class Installer: - def __init__(self, msi, out, package=None): + def __init__(self, msi, out, package=None, span=None): self.pkg = package or Package(msi) self.out = Path(out) + self.span = span or Span() self.notes = [] self.registry = [] self.files = {} # File key -> rel path it was installed to @@ -539,23 +570,30 @@ class Installer: def unpack(self, work): """Every cabinet in the package, unpacked; returns File key -> unpacked file.""" found = {} - for index, row in enumerate(self.pkg.table("Media")): + media = [row for row in self.pkg.table("Media") if row.get("Cabinet", "")] + for index, row in enumerate(media): cabinet = row.get("Cabinet", "") - if not cabinet: - continue - if not cabinet.startswith("#"): - self.notes.append("external cabinet %s is not part of the package; its files are missing" % cabinet) - continue - progress("unpacking %s" % cabinet[1:]) - cab = Path(work) / ("media%d.cab" % index) - self.pkg.stream(cabinet[1:], cab) + self.span.report(0.05 + 0.3 * index / max(1, len(media)), "unpacking %s" % cabinet.lstrip("#")) dest = Path(work) / ("media%d" % index) dest.mkdir() + if cabinet.startswith("#"): + cab = Path(work) / ("media%d.cab" % index) + self.pkg.stream(cabinet[1:], cab) + own = True + else: + # An external cabinet: a file beside the package, as an installer wrapper lays it out + # (vc_red.cab beside vc_red.msi, netfx_Full.mzz beside netfx_Full_x64.msi). + cab = sibling(Path(self.pkg.path), cabinet) + if cab is None: + self.notes.append("external cabinet %s is not part of the package; its files are missing" % cabinet) + continue + own = False result = subprocess.run([CABEXTRACT, "-q", "-d", str(dest), str(cab)], capture_output=True, timeout=1800, env=tool_env()) if result.returncode != 0: raise MsiError("cabextract: %s" % result.stderr.decode("utf-8", "replace").strip()[:300]) - cab.unlink() + if own: + cab.unlink() for entry in dest.rglob("*"): if entry.is_file(): found.setdefault(entry.relative_to(dest).as_posix(), entry) @@ -604,9 +642,13 @@ class Installer: if rel: self.files[key] = rel assembly_folders = set() - for key, rel in destinations.items(): - if not rel: - continue + todo = [(key, rel) for key, rel in destinations.items() if rel] + last = -1 + for done, (key, rel) in enumerate(todo): + step = done * 20 // max(1, len(todo)) + if step != last: + last = step + self.span.report(0.35 + 0.55 * done / max(1, len(todo)), "placing files (%d of %d)" % (done, len(todo))) source = unpacked.get(key) if source is None: missing += 1 @@ -728,13 +770,14 @@ class Installer: self.notes.append("%d %s" % (count, what)) def install(self): - progress("reading %s" % (self.pkg.summary.get("subject") or Path(self.pkg.path).name)) + self.span.report(0.0, "reading %s" % (self.pkg.summary.get("subject") or Path(self.pkg.path).name)) self.run_set_actions() self.select() with tempfile.TemporaryDirectory(dir=str(self.out.parent), prefix=".msi-") as work: unpacked = self.unpack(work) - progress("placing files") + self.span.report(0.35, "placing files") self.install_files(unpacked) + self.span.report(0.9, "writing the registry values") self.install_registry() self.check() if not self.counts["files"] and not self.registry: @@ -748,18 +791,235 @@ class Installer: pass with open(self.out / "registry.json", "w", encoding="utf-8") as out: json.dump({"version": 1, "values": values + self.registry}, out, indent=1) + self.span.report(1.0, "done") return {"product": self.props.get("ProductName", ""), "version": self.props.get("ProductVersion", ""), "counts": self.counts, "notes": self.notes} +# ---------------------------------------------------------------- installer wrappers + +def sibling(path, name): + """A file next to [path] called [name], whatever its case; None when there is none.""" + folder = path.parent + candidate = folder / name + if candidate.is_file(): + return candidate + lower = name.lower() + for entry in folder.iterdir(): + if entry.is_file() and entry.name.lower() == lower: + return entry + return None + + +def kind_of(path): + """What a file is by its first bytes: msi, exe, cab or other.""" + with open(path, "rb") as source: + head = source.read(8) + if head.startswith(MSI_MAGIC): + return "msi" + if head[:2] == b"MZ": + return "exe" + if head[:4] == b"MSCF": + return "cab" + return "other" + + +def run_tool(args, timeout=1800): + try: + return subprocess.run(args, capture_output=True, timeout=timeout, env=tool_env()) + except OSError as error: + raise MsiError("%s is not available (%s)" % (Path(args[0]).name, error)) + + +def burn_containers(data): + """A WiX bundle's containers as (offset, size) pairs: the .wixburn section of the stub names + their sizes; each is a cabinet laid out after the stub, found by its header since signing can + move them. None when the file is no bundle.""" + try: + pe = struct.unpack_from(" len(data): + return None + if struct.unpack_from(" packages/vcRuntimeMinimum_amd64/vc_runtimeMinimum_x64.msi).""" + data = path.read_bytes() + names = {} + for index, (start, size) in enumerate(containers): + progress("unpacking the bundle, part %d of %d" % (index + 1, len(containers)), 0.02 + 0.08 * index / max(1, len(containers))) + cab = dest / ("container%d.cab" % index) + cab.write_bytes(data[start:start + size]) + folder = dest / ("container%d" % index) + folder.mkdir() + result = run_tool([CABEXTRACT, "-q", "-d", str(folder), str(cab)]) + cab.unlink() + if result.returncode != 0: + raise MsiError("cabextract: %s" % result.stderr.decode("utf-8", "replace").strip()[:300]) + if index == 0: + manifest = folder / "0" + if manifest.is_file(): + text = manifest.read_text("utf-8", "replace") + for file_path, source in re.findall(r']*?\bFilePath="([^"]+)"[^>]*?\bSourcePath="([^"]+)"', text): + names[source] = file_path + continue + for entry in list(folder.iterdir()): + wanted = names.get(entry.name) + if not wanted: + continue + rel = Path(*[part for part in wanted.replace("\\", "/").split("/") if part not in ("", ".", "..")]) + target = folder / rel + target.parent.mkdir(parents=True, exist_ok=True) + entry.rename(target) + + +def unpack_installer(path, dest): + """Opens an installer or archive into [dest]: a bundle by its header, a cabinet (or + self-extracting one) with cabextract, anything else with 7-Zip.""" + kind = kind_of(path) + if kind == "exe": + containers = burn_containers(path.read_bytes()) + if containers: + unpack_burn(path, dest, containers) + return "bundle" + if kind == "cab" or kind == "exe" and b"MSCF" in path.read_bytes(): + result = run_tool([CABEXTRACT, "-q", "-d", str(dest), str(path)]) + if result.returncode == 0 and any(dest.rglob("*")): + return "cabinet" + for stale in list(dest.iterdir()): + shutil.rmtree(stale) if stale.is_dir() else stale.unlink() + progress("unpacking %s" % path.name, 0.02) + result = run_tool([SEVENZIP, "x", "-y", "-bd", "-bso0", "-bsp0", "-o" + str(dest), str(path)]) + if result.returncode not in (0, 1) or not any(dest.rglob("*")): + raise MsiError("could not open %s: %s" % (path.name, (result.stderr or result.stdout).decode("utf-8", "replace").strip()[:200] + or "nothing inside it can be unpacked")) + return "archive" + + +def packages_in(root): + """The Windows Installer packages an unpacked installer holds, shallowest first, by name.""" + found = [] + for entry in sorted(root.rglob("*")): + if entry.is_file() and kind_of(entry) == "msi": + found.append(entry) + found.sort(key=lambda entry: (len(entry.relative_to(root).parts), entry.name.lower())) + return found + + +def choose_packages(found, root): + """Which packages make up the install: the ones at the top of the installer (a bundle keeps + each in its own folder, so there every level counts), for the x86 and x64 platforms. The .NET + Framework 4.5+ installers carry a complete product (netfx_Full_*) beside the 4.0 parts they + patch (netfx_Core_*, netfx_Extended_*) and per-Windows-version copies in NetFx* folders: the + complete product wins, the copies are left out.""" + if not found: + return [] + depth = min(len(entry.relative_to(root).parts) for entry in found) + bundle = (root / "container1").is_dir() + top = [entry for entry in found if bundle or len(entry.relative_to(root).parts) == depth] + names = [entry.name.lower() for entry in top] + if any("full" in name for name in names) and any("_core" in name or "_extended" in name for name in names): + top = [entry for entry in top if not ("_core" in entry.name.lower() or "_extended" in entry.name.lower())] + chosen = [] + for entry in top: + template = Package(entry).summary.get("template", "").split(";")[0].strip().lower() + if template in SKIP_PLATFORMS: + continue + chosen.append(entry) + return chosen + + +def install_wrapper(path, out): + """Lays out every package inside an installer wrapper; the result is the sum of them.""" + total = {"product": "", "version": "", "counts": {"files": 0, "assemblies": 0, "registry": 0}, "notes": [], "packages": []} + with tempfile.TemporaryDirectory(dir=str(out.parent), prefix=".unpack-") as work: + dest = Path(work) / "x" + dest.mkdir() + how = unpack_installer(path, dest) + packages = choose_packages(packages_in(dest), dest) + if not packages: + raise MsiError("%s holds no installer package: it is its own setup program, which cannot be run here" % path.name) + for index, package in enumerate(packages): + lo, hi = 0.1 + 0.9 * index / len(packages), 0.1 + 0.9 * (index + 1) / len(packages) + progress("package %d of %d: %s" % (index + 1, len(packages), package.name), lo) + try: + result = Installer(str(package), out, span=Span(lo, hi)).install() + except MsiError as error: + # A package with nothing for this prefix (a patch for another Windows) is skipped, + # unless it is the only one; anything else is the whole installer's failure. + if "installs nothing" in str(error) and len(packages) > 1: + total["notes"].append("%s: %s" % (package.name, error)) + continue + raise + total["packages"].append(package.name) + if not total["product"]: + total["product"], total["version"] = result["product"], result["version"] + for key in total["counts"]: + total["counts"][key] += result["counts"].get(key, 0) + for note in result["notes"]: + if note not in total["notes"]: + total["notes"].append(note) + total["how"] = how + return total + + +def unpack_only(path, dest): + dest.mkdir(parents=True, exist_ok=True) + how = unpack_installer(path, dest) + files = sorted(str(entry.relative_to(dest)) for entry in dest.rglob("*") if entry.is_file()) + return {"how": how, "files": len(files), "names": files[:2000]} + + def main(): - if len(sys.argv) != 3: - print("usage: droiddeck-msi-install ", file=sys.stderr) + args = sys.argv[1:] + if len(args) == 3 and args[0] == "--unpack": + try: + result = unpack_only(Path(args[1]), Path(args[2])) + except MsiError as error: + print("error " + str(error), flush=True) + return 1 + print("result " + json.dumps(result), flush=True) + return 0 + if len(args) != 2: + print("usage: droiddeck-msi-install \n" + " droiddeck-msi-install --unpack ", file=sys.stderr) return 2 - out = Path(sys.argv[2]) + source, out = Path(args[0]), Path(args[1]) out.mkdir(parents=True, exist_ok=True) try: - result = Installer(sys.argv[1], out).install() + if kind_of(source) == "msi": + result = Installer(str(source), out).install() + else: + result = install_wrapper(source, out) except MsiError as error: print("error " + str(error), flush=True) return 1 diff --git a/tools/msitools/licenses/7zip-License b/tools/msitools/licenses/7zip-License new file mode 100644 index 0000000000..9c301dc1da --- /dev/null +++ b/tools/msitools/licenses/7zip-License @@ -0,0 +1,151 @@ + 7-Zip source code + ~~~~~~~~~~~~~~~~~ + License for use and distribution + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + 7-Zip Copyright (C) 1999-2026 Igor Pavlov. + + The licenses for files are: + + - CPP/7zip/Compress/Rar* files: the "GNU LGPL" with "unRAR license restriction" + - CPP/7zip/Compress/LzfseDecoder.cpp: the "BSD 3-clause License" + - C/ZstdDec.c: the "BSD 3-clause License" + - C/Xxh64.c: the "BSD 2-clause License" + - Some files are "public domain" files, if "public domain" status is stated in source file. + - the "GNU LGPL" for all other files. If there is no license information in + some source file, that file is under the "GNU LGPL". + + The "GNU LGPL" with "unRAR license restriction" means that you must follow both + "GNU LGPL" rules and "unRAR license restriction" rules. + + + + +GNU LGPL information +-------------------- + + This library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + This library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with this library; if not, + you can get a copy of the GNU Lesser General Public License from + http://www.gnu.org/ + + + + +BSD 3-clause License in 7-Zip code +---------------------------------- + + The "BSD 3-clause License" is used for the following code in 7z.dll + 1) LZFSE data decompression. + CPP/7zip/Compress/LzfseDecoder.cpp. + That code was derived from the code in the "LZFSE compression library" developed by Apple Inc, + that also uses the "BSD 3-clause License". + 2) ZSTD data decompression. + C/ZstdDec.c + that code was developed using original zstd decoder code as reference code. + The original zstd decoder code was developed by Facebook Inc, + that also uses the "BSD 3-clause License". + + Copyright (c) 2015-2016, Apple Inc. All rights reserved. + Copyright (c) Facebook, Inc. All rights reserved. + Copyright (c) 2023-2026 Igor Pavlov. + +Text of the "BSD 3-clause License" +---------------------------------- + +Redistribution and use in source and binary forms, with or without modification, +are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +3. Neither the name of the copyright holder nor the names of its contributors may + be used to endorse or promote products derived from this software without + specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR +ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +--- + + + + +BSD 2-clause License in 7-Zip code +---------------------------------- + + The "BSD 2-clause License" is used for the XXH64 code in 7-Zip. + C/Xxh64.c + + XXH64 code in 7-Zip was derived from the original XXH64 code developed by Yann Collet. + + Copyright (c) 2012-2021 Yann Collet. + Copyright (c) 2023-2026 Igor Pavlov. + +Text of the "BSD 2-clause License" +---------------------------------- + +Redistribution and use in source and binary forms, with or without modification, +are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR +ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +--- + + + + +unRAR license restriction +------------------------- + +The decompression engine for RAR archives was developed using source +code of unRAR program. +All copyrights to original unRAR code are owned by Alexander Roshal. + +The license for original unRAR code has the following restriction: + + The unRAR sources cannot be used to re-create the RAR compression algorithm, + which is proprietary. Distribution of modified unRAR sources in separate form + or as a part of other software is permitted, provided that it is clearly + stated in the documentation and source comments that the code may + not be used to develop a RAR (WinRAR) compatible archiver. + +-- diff --git a/tools/msitools/licenses/7zip-unRarLicense b/tools/msitools/licenses/7zip-unRarLicense new file mode 100644 index 0000000000..5f78b728da --- /dev/null +++ b/tools/msitools/licenses/7zip-unRarLicense @@ -0,0 +1,41 @@ + ****** ***** ****** unRAR - free utility for RAR archives + ** ** ** ** ** ** ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ****** ******* ****** License for use and distribution of + ** ** ** ** ** ** ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ** ** ** ** ** ** FREE portable version + ~~~~~~~~~~~~~~~~~~~~~ + + The source code of unRAR utility is freeware. This means: + + 1. All copyrights to RAR and the utility unRAR are exclusively + owned by the author - Alexander Roshal. + + 2. The unRAR sources may be used in any software to handle RAR + archives without limitations free of charge, but cannot be used + to re-create the RAR compression algorithm, which is proprietary. + Distribution of modified unRAR sources in separate form or as a + part of other software is permitted, provided that it is clearly + stated in the documentation and source comments that the code may + not be used to develop a RAR (WinRAR) compatible archiver. + + 3. The unRAR utility may be freely distributed. No person or company + may charge a fee for the distribution of unRAR without written + permission from the copyright holder. + + 4. THE RAR ARCHIVER AND THE UNRAR UTILITY ARE DISTRIBUTED "AS IS". + NO WARRANTY OF ANY KIND IS EXPRESSED OR IMPLIED. YOU USE AT + YOUR OWN RISK. THE AUTHOR WILL NOT BE LIABLE FOR DATA LOSS, + DAMAGES, LOSS OF PROFITS OR ANY OTHER KIND OF LOSS WHILE USING + OR MISUSING THIS SOFTWARE. + + 5. Installing and using the unRAR utility signifies acceptance of + these terms and conditions of the license. + + 6. If you don't agree with terms of the license you must remove + unRAR files from your storage devices and cease to use the + utility. + + Thank you for your interest in RAR and unRAR. + + + Alexander L. Roshal \ No newline at end of file diff --git a/tools/msitools/licenses/NOTICE b/tools/msitools/licenses/NOTICE index ee62a42c51..5c625a0106 100644 --- a/tools/msitools/licenses/NOTICE +++ b/tools/msitools/licenses/NOTICE @@ -9,6 +9,9 @@ builds, unchanged: https://gitlab.gnome.org/GNOME/gcab cabextract cabextract 1.11 GPL-3.0-or-later https://www.cabextract.org.uk/ + 7z, 7z.so 7-Zip 26.04 LGPL-2.1-or-later, with the unRAR restriction + (7zip-License, 7zip-unRarLicense); https://7-zip.org/ -The packages and their sources: https://github.com/Droid-Deck/DroidDeck/releases/tag/msitools-0.106-r1, +The packages and their sources: https://github.com/Droid-Deck/DroidDeck/releases/tag/msitools-0.106-r1 +(7-Zip from Arch Linux ARM's mirror, see tools/msitools/packages.txt), https://github.com/archlinuxarm/PKGBUILDs. The licence texts are beside this file. diff --git a/tools/msitools/packages.txt b/tools/msitools/packages.txt index 5a04b4c626..8329e68a7f 100644 --- a/tools/msitools/packages.txt +++ b/tools/msitools/packages.txt @@ -1,6 +1,7 @@ # What droiddeck-msi-install runs: msitools' msiinfo reads a Windows Installer package's tables # and streams, cabextract unpacks its cabinets (every compression a cabinet can use, which gcab -# alone cannot). Arch Linux ARM's aarch64 packages, hosted unchanged on this repo's +# alone cannot), 7-Zip (7z + its 7z.so format plugin) opens the installer .exe wrappers around +# packages (self-extracting 7-Zip and cabinet files, WiX bundles). Arch Linux ARM's aarch64 packages, hosted unchanged on this repo's # msitools-0.106-r1 release because the mirror keeps only the newest of each. CI fetches each, # checks the sha256 and stages the files below into the apk under usr/local/lib/droiddeck-msitools; # SessionFiles copies them into the runtime at every launch. Everything else they need (glib, gio, @@ -10,3 +11,4 @@ 4028f44d83b2956ba2931d32aee8cceecbd6481e2febf86a3b6bfed69ef403d5 https://github.com/Droid-Deck/DroidDeck/releases/download/msitools-0.106-r1/libgsf-1.14.60-1-aarch64.pkg.tar.xz 3d7bd103313cb36d6cb703a5e583363e56ff255f105bcc0727916d195967976d https://github.com/Droid-Deck/DroidDeck/releases/download/msitools-0.106-r1/gcab-1.6-2-aarch64.pkg.tar.xz 87fe375a22de1519982234ce99f909ae13c06500784baaa1e8d7a0c0be0d14ee https://github.com/Droid-Deck/DroidDeck/releases/download/msitools-0.106-r1/cabextract-1.11-3-aarch64.pkg.tar.xz +e3bc83ac5960a4363f7ce05eb1d50b03d60d627a4a29f9b02c8e0466a5381b1f http://mirror.archlinuxarm.org/aarch64/extra/7zip-26.04-1-aarch64.pkg.tar.xz